Method for identifying neonatal identity information based on footprint and related equipment

Abstract

The invention provides a method for identifying neonatal identity information based on footprints and related equipment, which can quickly and accurately identify the neonatal identity information. The method comprises the following steps: determining a sole video frame set corresponding to a target neonate, wherein the target neonate is a neonate of which identity information is to be identified; detecting the sole video frame set to judge whether a footprint video frame set meeting a preset requirement exists in the sole video frame set or not; if yes, preprocessing each footprint video frame in the footprint video frame set to obtain a target footprint image set; inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set; and determining identity information of the target newborn according to the footprint depth feature vector corresponding to each footprint image.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method for recognizing identity information of a newborn based on footprints and related equipment. Background Art

[0002] In the medical system, the safe, accurate recording and management of newborn information is one of the core tasks of the daily work of the hospital, and it plays an irreplaceable role in safeguarding the rights and interests of mothers and infants and promoting trust between doctors and patients. For a long time, many hospitals in China have adopted the traditional method of collecting newborn footprints as a preliminary means of identification. The original intention of this practice is to prevent babies from being misidentified or taken away by mistake in the hospital, and to ensure that every newborn can receive correct and timely medical care.

[0003] However, traditional methods of collecting footprints, such as using ink rubbings, have many limitations. The skin of newborns is extremely delicate, and the footprints are shallow and easily affected by external factors, such as the amount of ink in the ink, the strength of the collector, and the movements of the newborn itself, which may affect the clarity of the footprints. In addition, because the papillary patterns on the feet of newborns are shallow and dense, and the contrast between the convex and concave lines is small, what is actually collected is often wrinkles rather than standard footprints, which further reduces its reliability as a basis for identity recognition. More importantly, footprints will change rapidly as the newborn grows, and do not have the stability for long-term recognition, which limits their application in identity management. Summary of the invention

[0004] The embodiment of the present invention provides a method for identifying the identity information of a newborn based on footprints and related equipment, which can quickly and accurately identify the identity information of a newborn.

[0005] A first aspect of the present invention provides a method for identifying identity information of a newborn based on foot prints, comprising:

[0006] Determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified;

[0007] Detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set;

[0008] If yes, preprocessing each footprint video frame in the footprint video frame set to obtain a target footprint image set;

[0009] Inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set;

[0010] The identity information of the target newborn is determined according to the footprint depth feature vector corresponding to each footprint image.

[0011] A second aspect of the present invention provides a device for identifying identity information of a newborn based on foot prints, comprising:

[0012] A determination module, used to determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified;

[0013] A detection module, used for detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set;

[0014] a preprocessing module, configured to preprocess each footprint video frame in the foot print video frame set to obtain a target footprint image set when there is a footprint video frame set that meets the preset requirement in the foot sole video frame set;

[0015] A feature recognition module, used for inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set;

[0016] The identity recognition module is used to determine the identity information of the target newborn according to the footprint depth feature vector corresponding to each footprint image.

[0017] In a possible design, the detection module is specifically used for:

[0018] Eliminate video frames that do not include the target newborn's footprints from the sole video frame set to obtain an initial sole video frame set;

[0019] Determine whether there are N consecutive video frames in the initial sole video frame set, each of which satisfies the requirement that the footprint is not blocked, the footprint meets the recognition requirement, and the toes of the footprint are completely visible, where N is an integer greater than or equal to 5;

[0020] If yes, the continuous N video frames are determined as the footprint video frame set;

[0021] If not, a prompt message is issued, wherein the prompt message is used to instruct the user to re-photograph the footprints of the target newborn.

[0022] In a possible design, the feature recognition module is further used for:

[0023] Acquire a footprint image of each of the multiple newborns, wherein the footprint image includes a positive sample image whose shooting angle meets the preset requirements and a negative sample image whose shooting angle does not meet the preset requirements, and the footprint image includes a left foot footprint image and a right foot footprint image;

[0024] Annotating the foot print images of training newborns to obtain an initial sample set, wherein the training labeled newborn is any one of the multiple newborns;

[0025] Preprocessing each initial sample in the initial sample set to obtain a target training sample set;

[0026] An initial footprint recognition model is constructed, and the initial footprint recognition model is iteratively trained based on the target training sample set until a preset iteration termination condition is reached, and the initial footprint recognition model when the preset iteration termination condition is reached is determined as the footprint recognition model.

[0027] In a possible design, the feature recognition module annotates the training newborn's foot print image to obtain an initial sample set including:

[0028] Annotating the training newborn's footprint image based on the SAM tool to obtain the overall outline of the training newborn's initial footprint;

[0029] Adjusting the overall contour of the initial footprint to obtain the overall contour of the target footprint;

[0030] Determine the minimum circumscribed rotation rectangle of the training newborn's footprint according to the overall contour of the target footprint, wherein the minimum circumscribed rotation rectangle of the footprint includes a center point, a length, a width, and a rotation angle;

[0031] Annotating the key points and visible states of the toes of the training newborn contained in the minimum circumscribed rotation rectangle of the footprint to obtain an initial annotated footprint image;

[0032] The attributes of the training newborn's footprints in the initial labeled footprint image are labeled to obtain the initial sample set, wherein the attributes of the footprints include whether the footprints are blocked, the image shooting quality, the number of the training newborn, and the footprint position.

[0033] In a possible design, the feature recognition module preprocesses each initial sample in the initial sample set to obtain a target training sample set including:

[0034] Adjusting the angle of the training newborn's footprint in the target initial sample according to the rotation angle of the minimum circumscribed rotation matrix of the footprint to obtain a first training sample;

[0035] cropping the footprints of the training newborn in the first training sample to obtain a footprint image of the training newborn;

[0036] Determining a training combination of the foot print images for training the newborn according to the foot print positions;

[0037] Extracting the footprint pattern of each footprint image in the footprint image training set;

[0038] Determining a texture contour area corresponding to the footprint pattern;

[0039] The texture contour area is added to the corresponding foot print image to obtain the target training sample set.

[0040] In a possible design, the identity recognition module is specifically used for:

[0041] Calculate the similarity between the footprint depth feature vector and each footprint feature in the newborn footprint feature database to obtain a similarity set;

[0042] Determine the target footprint feature in the similarity set that is greater than a similarity threshold;

[0043] The target identity information corresponding to the target footprint feature in the newborn footprint feature database is determined as the identity information of the target newborn.

[0044] In a possible design, the identity recognition module is also used for:

[0045] If the target footprint feature does not exist in the similarity set, an input prompt message is issued;

[0046] Acquire the identity information of the target newborn input by the user according to the input prompt information;

[0047] The identity information of the target newborn is associated with the footprint depth feature vector and stored in the newborn footprint feature database.

[0048] The third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the processor is used to implement the steps of the method for identifying the identity information of a newborn based on footprints as described in the first aspect when executing a computer management program stored in the memory.

[0049] A fourth aspect of the present invention provides a computer-readable storage medium having a computer management program stored thereon, which, when executed by a processor, implements the steps of the method for identifying the identity information of a newborn based on footprints as described in the first aspect above.

[0050] In summary, it can be seen that in the embodiment provided by the present invention, the image of the newborn's foot print is quickly captured by high-definition photography, and the foot print features in the image are automatically extracted using a deep learning model, and the extracted foot print features are compared with the foot print features in the newborn feature database, thereby determining the identity information of the newborn, thereby achieving rapid identity confirmation and safety management. This non-contact collection method not only avoids the discomfort and hygiene problems that may be caused by traditional rubbing methods, but also greatly improves the collection efficiency and accuracy. . BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a process for identifying newborn identity information based on footprints provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a model framework of a footprint detection model provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of the network structure of a footprint detection model provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the training process of the footprint recognition model provided by an embodiment of the present invention;

[0055] Figure 5 A virtual structural diagram of a device for identifying newborn identity information based on footprints provided by an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of the hardware structure of a device for identifying newborn identity information based on footprints provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of an electronic device according to an embodiment of the present invention;

[0058] Figure 8 A schematic diagram of an embodiment of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0060] In the following description, specific embodiments of the present invention will be described with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and computer execution referred to herein includes operations by a computer processing unit of electronic signals representing data in a structured form. This operation converts the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to testers in the field. The data structure maintained by the data is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present invention are described in the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.

[0061] The principles of the present invention operate with many other general or special purpose computing, communication environments or configurations. Examples of well-known computing systems, environments and configurations suitable for use with the present invention may include, but are not limited to, handheld phones, personal computers, servers, multiprocessor systems, microcomputer-based systems, mainframe computers, and distributed computing environments, including any of the above systems or devices.

[0062] The terms "first", "second", and "third" in the present invention are used to distinguish different objects rather than to describe a specific order. In addition, the terms "include", "have", and any variations thereof are intended to cover non-exclusive inclusions.

[0063] Newborn foot print recognition technology has gradually attracted attention due to its uniqueness and advantages. Compared with fingerprints and facial features, newborn foot prints are unique at birth, and because the feet are usually exposed, they are easy to collect immediately. More importantly, the stability and recognition accuracy of foot prints are much higher than traditional footprints, making them an ideal choice for newborn identification.

[0064] The present invention deeply integrates computer vision and deep learning technology to develop an efficient and accurate newborn footprint recognition algorithm. The algorithm quickly captures the newborn footprint image by high-definition photography, and uses a deep learning model to automatically extract and compare the footprint features in the image, thereby realizing rapid identity confirmation and safety management. This non-contact collection method not only avoids the discomfort and hygiene problems that may be caused by the traditional rubbing method, but also greatly improves the collection efficiency and accuracy. At the same time, the application of newborn footprint recognition technology will also bring many benefits to hospitals. First, it can help hospitals manage newborn medical records more effectively, avoid information confusion or errors, and improve the efficiency and quality of medical services. Secondly, through footprint recognition technology, hospitals can more conveniently track the growth and health of newborns, provide doctors with valuable reference data, and help formulate personalized medical and nursing plans. Finally, footprint recognition technology can also prevent the occurrence of infant abduction or identity confusion to a certain extent, reduce conflicts between doctors and patients, and enhance the trust between doctors and patients.

[0065] The method for identifying the identity information of a newborn based on footprints provided by the present invention is described below from the perspective of a device for identifying the identity information of a newborn based on footprints. The device for identifying the identity information of a newborn based on footprints can be a terminal device (the terminal device includes but is not limited to a mobile phone, a computer, a tablet, etc.), or a server, or a service unit in a server, without specific limitation.

[0066] See also Figure 1 , Figure 1 A flow chart of a method for identifying newborn identity information based on footprints provided by an embodiment of the present invention, the method for identifying newborn identity information based on footprints includes:

[0067] 101. Determine a set of foot sole video frames corresponding to the target newborn.

[0068] In this embodiment, the device for identifying the identity information of a newborn based on footprints can shoot a video of the soles of the feet of a target newborn through a camera device, and then obtain a sole video corresponding to the target newborn. The target newborn is a newborn whose identity information is to be identified. When shooting the video, the left foot of the target newborn can be shot separately, or the right foot of the newborn can be shot separately. Of course, the left and right feet can also be shot at the same time. In addition, after obtaining the sole video, the sole video is skipped and saved as an image to obtain a set of sole video frames corresponding to the target newborn.

[0069] 102 . Detect the sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the sole video frame set. If so, execute step 103 .

[0070] In this embodiment, after obtaining a set of sole video frames, the device for identifying the identity information of a newborn based on footprints can detect each sole video frame in the set of sole video frames. The purpose of the detection is to determine whether there is a set of footprint video frames that meets preset requirements in the set of sole video frames. Specifically, the device for identifying the identity information of a newborn based on footprints can first eliminate video frames that do not include footprints of the target newborn in the set of sole video frames to obtain an initial set of sole video frames; determine whether there are N consecutive video frames in the initial set of sole video frames, each of which satisfies the requirement that there is no obstruction of the footprint, the footprint meets the identification requirement, and the toe of the footprint is completely visible, wherein N is an integer greater than or equal to 5; if there are N consecutive video frames in the set of sole video frames, each of which has no obstruction of the footprint, the footprint meets the identification requirement, and the toe of the footprint is completely visible, then the N consecutive video frames are determined as the set of footprint video frames, and step 103 is executed; otherwise, a prompt message is issued, and the prompt message is used to instruct the user to re-shoot the sole of the target newborn.

[0071] It should be noted that when the sole video frame set is detected to determine whether there is a foot print video frame set that meets the preset requirements in the continuous sole video frame set, the video frames can be identified by a pre-trained foot print detection model to identify the foot print video frame set that meets the preset requirements. Figure 2 , Figure 2 The model framework diagram of the footprint detection model provided by the embodiment of the present invention is shown in FIG. 1 . In this embodiment, fcos is used for expansion, and the training process of the entire model is as follows:

[0072] Anchor points are preset according to certain rules. Because footprint detection mainly focuses on large targets, a larger receptive field area is used for detection in the setting of anchor points. Specifically, it is laid out by downsampling the original image 8 times. Then, in the actual reasoning process, these anchor points will be regressed to the offset and classification of the nearest positive sample, as well as other tasks.

[0073] Regarding the regression of the anchor points, since the present invention is more concerned with the outline of the footprint rather than the area of ​​the footprint, the key point regression method is used for regression, and 128 points are used to fit the sole contour. Currently, this task is only learned as an auxiliary task. In footprint recognition, it has been fully verified that auxiliary tasks can enhance the accuracy of core tasks. During the training process, occlusion classification, image quality classification, and whether the key points of the toes are visible are all trained using a binary classification method. The advantage of doing them together here is that these tasks are highly correlated and can promote each other. Moreover, it can reduce time consumption. The network structure of the footprint detection model uses mobilenetV2. The network structure Figure 3 shown.

[0074] 103. Preprocess each footprint video frame in the footprint video frame set to obtain a target footprint image set.

[0075] In this embodiment, after obtaining a set of footprint video frames, the device for identifying the identity information of a newborn based on footprints can pre-process each footprint video frame in the set of footprint video frames, first determine the overall contour of the footprint in the footprint video frame, and determine the minimum circumscribed rotation matrix of the footprint based on the overall contour of the footprint, the minimum circumscribed rotation rectangle of the footprint including a center point, a length, a width, and a rotation angle, then adjust the angle of the footprint of the target newborn in the footprint video frame based on the rotation angle of the minimum circumscribed rotation matrix of the footprint, and crop the footprint of the target newborn in the adjusted footprint video frame to obtain a footprint cropped image, and use a high contrast retention algorithm to extract the footprint texture in the footprint cropped image, and calculate the contour area of ​​the texture in the footprint cropped image through a dynamic threshold, and finally superimpose the texture lines on the footprint cropped image through linear superposition to obtain a set of target footprint images.

[0076] 104. Input the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set.

[0077] In this embodiment, the device for identifying the identity information of a newborn based on footprints inputs a target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set. The footprint depth feature vector is generally represented by a vector of length 1024. That is, the device for identifying the identity information of a newborn based on footprints can pre-train a footprint recognition model for identifying footprint images. When it is necessary to identify the footprint depth feature vector, the footprint image can be input into the footprint recognition model to obtain the footprint depth feature vector corresponding to the footprint image. For ease of understanding, the following is combined with Figure 4 The following is an explanation of the training process of the footprint recognition model:

[0078] Step 1, obtaining a footprint image of each of a plurality of newborns, wherein the footprint image includes a positive sample image whose shooting angle meets a preset requirement and a negative sample image whose shooting angle does not meet the preset requirement, and the footprint image includes a left foot footprint image and a right foot footprint image;

[0079] In this step, the footprints of multiple newborns (for example, 1,000, of course, it can also be other values, there is no specific limitation, and the following examples are all explained by taking 1,000 newborns as an example) are collected. Specifically, each collection uses a camera to shoot a video of the sole of the newborn's foot. During the shooting process, it is necessary to ensure that the position facing the newborn's footprint is captured (that is, the positive sample image whose shooting angle meets the preset requirements). At the same time, some pictures with bad angles are also deliberately taken (that is, the negative sample images whose shooting angles do not meet the preset requirements). These pictures with bad angles are mainly used as negative samples for learning. The left and right feet of each newborn need to be photographed. After the shooting is completed, the video of each newborn among the multiple newborns is saved as a picture by skipping frames, and then the footprint image of each newborn is obtained.

[0080] Step 2: labeling the foot print images of the training newborns to obtain an initial sample set, where the training labeled newborn is any one of the multiple newborns;

[0081] In this step, the device for identifying the identity information of newborns based on footprints can traverse multiple newborns, and annotate the footprint images of training newborns based on the SAM tool to obtain the overall outline of the initial footprints of the training newborns; adjust the overall outline of the initial footprints to obtain the overall outline of the target footprints; determine the minimum circumscribed rotation rectangle of the footprints of the training newborn according to the overall outline of the target footprints, and the minimum circumscribed rotation rectangle of the footprints includes a center point, length, width and rotation angle; annotate the key point positions and visible states of the toes of the training newborns contained in the minimum circumscribed rotation rectangle of the footprints to obtain an initial annotated footprint image; annotate the footprint attributes of the training newborns in the initial annotated footprint image to obtain an initial sample set, and the footprint attributes include whether the footprints are blocked, the image shooting quality, the number of the training newborns and the footprint positions.

[0082] That is, the annotation of footprint images mainly includes four parts:

[0083] 1. The overall outline of the footprint is marked mainly through the SAM tool for initial marking, and the overall outline of the footprint after initial marking is displayed to the user. The user performs manual fine-tuning to obtain the overall outline of the footprint;

[0084] 2. The marking of the minimum circumscribed rotation rectangle of the footprint. The marking here mainly includes the center point, length, width and rotation angle of the minimum circumscribed rotation rectangle of the footprint;

[0085] 3. Marking of key positions and visible status of toes. The main purpose of marking here is to mark out the five toes and mark whether the toes are clearly visible;

[0086] 4. The attributes of the footprints are marked, including whether the footprints are blocked (blocked or not), the shooting quality (the shooting quality is mainly reflected by the resolution of the video), the number of the newborn, and the location of the footprints (left foot, right foot);

[0087] Finally, after the annotations in the above four parts, the initial sample set used for model training can be obtained.

[0088] Step 3: preprocess each initial sample in the initial sample set to obtain a target training sample set;

[0089] In this step, after obtaining the initial sample, the device for identifying the identity information of the newborn based on the footprint may preprocess each initial sample in the initial sample set, wherein the preprocessing includes but is not limited to footprint alignment and footprint enhancement. The footprint alignment and footprint enhancement are described below:

[0090] Adjusting the angle of the footprint of the training newborn in the target initial sample according to the rotation angle of the minimum circumscribed rotation matrix of the footprint to obtain a first training sample;

[0091] cropping the footprints of the training newborn in the first training sample to obtain a footprint image of the training newborn;

[0092] Determine a training combination of foot print images for training newborns according to the foot print positions;

[0093] Extracting the footprint pattern of each footprint image in the footprint image training set;

[0094] Determine the texture contour area corresponding to the foot pattern;

[0095] The texture contour area is added to the corresponding foot print image to obtain the target training sample set.

[0096] That is, the device for identifying the identity information of newborns based on footprints adjusts the footprints to a position where the toes face upward according to the rotation angle of the minimum circumscribed rotation matrix of the footprints marked in step 2. After the rotation, the circumscribed rectangular frame of the footprints is a regular rectangle. The footprint image is cropped out by the minimum circumscribed rotation matrix of the footprints marked in step 2 to obtain a cropped footprint image. The numbering range of the current 1,000 newborns is [0, 1000]. For the annotation of the left and right feet, the annotations are recombined. First, the left foot of the footprint of newborn No. 0 is numbered 0 and the right foot is numbered 1. The left foot of newborn No. 1 is numbered 2 and the right foot is 3, and so on. In this way, the footprint numbering is adjusted from [0-1000] to [0-2000] (the purpose of the adjustment is that even the footprints of the same newborn are different and need to be studied as different individuals). The training combination of the left and right foot footprint images of each newborn can be determined by the footprint position;

[0097] Afterwards, in order to increase the generalization of model detection, the footprints will be cut out through the marked footprint outlines, and then pasted into various Beijing environments. At the same time, since the footprints of different newborns have different depths, in order to make the model more integrated with the learning patterns, the device based on the footprint recognition of newborn identity information will enhance the texture of the newborn's footprints. First, the high-contrast retention algorithm is used to extract the footprint patterns, and then the contour area of ​​the texture is calculated through the dynamic threshold. Finally, the texture lines are superimposed on the original footprint image by linear superposition to obtain the target training sample set.

[0098] Step 4: construct an initial footprint recognition model, and iteratively train the initial footprint recognition model based on the target training sample set until a preset iteration termination condition is reached, and determine the initial footprint recognition model when the preset iteration termination condition is reached as the footprint recognition model.

[0099] In this step, the device for identifying the identity information of a newborn based on footprints can construct an initial footprint recognition model, and traverse each training sample in the target training sample set into the initial footprint recognition model for iterative training until a preset iteration termination condition is reached, and the initial footprint recognition model when the preset iteration termination condition is reached is determined as the footprint recognition model.

[0100] It should be noted that the image size of each training sample in the target training sample set when inputting the initial footprint recognition model is 224*112. Tests have shown that the input of 224*112 has a significant improvement in accuracy. Therefore, the image size of the input model used in the present invention is 224*112.

[0101] 105. Determine the identity information of the target newborn according to the footprint depth feature vector corresponding to each footprint image.

[0102] In this embodiment, the device for identifying the identity information of a newborn based on footprints can calculate the similarity between the footprint depth feature vector obtained by the model and each footprint feature in the newborn footprint feature database to obtain a similarity set; and determine whether there is a target footprint feature in the similarity set whose similarity is greater than a similarity threshold. If so, it means that the footprint depth feature and identity information of the target newborn have been stored in the newborn footprint feature database. Therefore, the target identity information corresponding to the target footprint feature in the newborn footprint feature database is determined as the identity information of the target newborn.

[0103] If it does not exist, it means that the newborn's footprint feature database does not store the target newborn's footprint depth features and identity information, and an entry prompt message can be issued. The entry prompt message is used to prompt the user to obtain the target newborn's identity information, such as asking the target newborn's parents or obtaining it through a questionnaire, and entering the target newborn's identity information. At the same time, the target footprint features of the target newborn obtained through the model are associated with the target newborn's identity information and stored in the newborn's footprint feature database to facilitate subsequent identification and comparison.

[0104] In summary, it can be seen that in the embodiment provided by the present invention, the image of the newborn's foot print is quickly captured by high-definition photography, and the foot print features in the image are automatically extracted using a deep learning model, and the extracted foot print features are compared with the foot print features in the newborn feature database, thereby determining the identity information of the newborn, thereby achieving rapid identity confirmation and safety management. This non-contact collection method not only avoids the discomfort and hygiene problems that may be caused by traditional rubbing methods, but also greatly improves the collection efficiency and accuracy.

[0105] The above describes an embodiment of the present invention from the perspective of a method for identifying the identity information of a newborn based on footprints. The following describes an embodiment of the present invention from the perspective of an apparatus for identifying the identity information of a newborn based on footprints.

[0106] See also Figure 5 , a virtual structural diagram of a device for identifying the identity information of a newborn based on footprints in an embodiment of the present invention, the device 500 for identifying the identity information of a newborn based on footprints comprises:

[0107] A determination module 501 is used to determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified;

[0108] A detection module 502 is used to detect the foot sole video frame set to determine whether there is a foot print video frame set that meets a preset requirement in the foot sole video frame set;

[0109] A preprocessing module 503 is used to preprocess each footprint video frame in the footprint video frame set to obtain a target footprint image set when there is a footprint video frame set that meets the preset requirement in the sole video frame set;

[0110] A feature recognition module 504 is used to input the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set;

[0111] The identity recognition module 505 is used to determine the identity information of the target newborn according to the footprint depth feature vector corresponding to each footprint image.

[0112] In a possible design, the detection module 502 is specifically used for:

[0113] Eliminate video frames that do not include the target newborn's footprints from the sole video frame set to obtain an initial sole video frame set;

[0114] Determine whether there are N consecutive video frames in the initial sole video frame set, each of which satisfies the requirement that the footprint is not blocked, the footprint meets the recognition requirement, and the toes of the footprint are completely visible, where N is an integer greater than or equal to 5;

[0115] If yes, the continuous N video frames are determined as the footprint video frame set;

[0116] If not, a prompt message is issued, wherein the prompt message is used to instruct the user to re-photograph the footprints of the target newborn.

[0117] In a possible design, the feature recognition module 504 is further used for:

[0118] Acquire a footprint image of each of the multiple newborns, wherein the footprint image includes a positive sample image whose shooting angle meets the preset requirements and a negative sample image whose shooting angle does not meet the preset requirements, and the footprint image includes a left foot footprint image and a right foot footprint image;

[0119] Annotating the foot print images of training newborns to obtain an initial sample set, wherein the training labeled newborn is any one of the multiple newborns;

[0120] Preprocessing each initial sample in the initial sample set to obtain a target training sample set;

[0121] An initial footprint recognition model is constructed, and the initial footprint recognition model is iteratively trained based on the target training sample set until a preset iteration termination condition is reached, and the initial footprint recognition model when the preset iteration termination condition is reached is determined as the footprint recognition model.

[0122] In a possible design, the feature recognition module 504 annotates the training newborn's foot print image to obtain an initial sample set including:

[0123] Annotating the training newborn's footprint image based on the SAM tool to obtain the overall outline of the training newborn's initial footprint;

[0124] Adjusting the overall contour of the initial footprint to obtain the overall contour of the target footprint;

[0125] Determine the minimum circumscribed rotation rectangle of the training newborn's footprint according to the overall contour of the target footprint, wherein the minimum circumscribed rotation rectangle of the footprint includes a center point, a length, a width, and a rotation angle;

[0126] Annotating the key points and visible states of the toes of the training newborn contained in the minimum circumscribed rotation rectangle of the footprint to obtain an initial annotated footprint image;

[0127] The attributes of the training newborn's footprints in the initial labeled footprint image are labeled to obtain the initial sample set, wherein the attributes of the footprints include whether the footprints are blocked, the image shooting quality, the number of the training newborn, and the footprint position.

[0128] In a possible design, the feature recognition module 504 preprocesses each initial sample in the initial sample set to obtain a target training sample set including:

[0129] Adjusting the angle of the training newborn's footprint in the target initial sample according to the rotation angle of the minimum circumscribed rotation matrix of the footprint to obtain a first training sample;

[0130] cropping the footprints of the training newborn in the first training sample to obtain a footprint image of the training newborn;

[0131] Determining a training combination of the foot print images for training the newborn according to the foot print positions;

[0132] Extracting the footprint pattern of each footprint image in the footprint image training set;

[0133] Determining a texture contour area corresponding to the footprint pattern;

[0134] The texture contour area is added to the corresponding foot print image to obtain the target training sample set.

[0135] In a possible design, the identity recognition module 505 is specifically used for:

[0136] Calculate the similarity between the footprint depth feature vector and each footprint feature in the newborn footprint feature database to obtain a similarity set;

[0137] Determine the target footprint feature in the similarity set that is greater than a similarity threshold;

[0138] The target identity information corresponding to the target footprint feature in the newborn footprint feature database is determined as the identity information of the target newborn.

[0139] In a possible design, the identity recognition module 505 is further used for:

[0140] If the target footprint feature does not exist in the similarity set, an input prompt message is issued;

[0141] Acquire the identity information of the target newborn input by the user according to the input prompt information;

[0142] The identity information of the target newborn is associated with the footprint depth feature vector and stored in the newborn footprint feature database.

[0143] above Figure 5 The apparatus for identifying the identity information of a newborn based on footprints in the embodiment of the present invention is described from the perspective of modular functional entities. The apparatus for identifying the identity information of a newborn based on footprints in the embodiment of the present invention is described in detail from the perspective of hardware processing. Please refer to FIG. 600, which is a schematic diagram of an embodiment of an apparatus 600 for identifying the identity information of a newborn based on footprints in the embodiment of the present invention. The apparatus 600 for identifying the identity information of a newborn based on footprints includes:

[0144] An input device 601, an output device 602, a processor 603 and a memory 604 (the number of the processor 603 can be one or more, Figure 6 In some embodiments of the present invention, the input device 601, the output device 602, the processor 603 and the memory 604 may be connected via a communication bus or other means, wherein: Figure 6 The communication bus connection is taken as an example.

[0145] Wherein, by calling the operation instruction stored in the memory 604, the processor 603 is used to perform the following steps:

[0146] Determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified;

[0147] Detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set;

[0148] If yes, preprocessing each footprint video frame in the footprint video frame set to obtain a target footprint image set;

[0149] Inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set;

[0150] The identity information of the target newborn is determined according to the footprint depth feature vector corresponding to each footprint image.

[0151] By calling the operation instructions stored in the memory 604, the processor 603 is also used to execute Figure 1 Any method in the corresponding embodiment.

[0152] See also Figure 7 , Figure 7 A schematic diagram of an electronic device according to an embodiment of the present invention.

[0153] like Figure 7 As shown, an embodiment of the present invention provides an electronic device, including a memory 710, a processor 720, and a computer program 711 stored in the memory 710 and executable on the processor 720. When the processor 720 executes the computer program 711, the following steps are implemented:

[0154] Determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified;

[0155] Detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set;

[0156] If yes, preprocessing each footprint video frame in the footprint video frame set to obtain a target footprint image set;

[0157] Inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set;

[0158] The identity information of the target newborn is determined according to the footprint depth feature vector corresponding to each footprint image.

[0159] In the specific implementation process, when the processor 720 executes the computer program 711, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0160] Since the electronic device introduced in this embodiment is a device used to implement a device for identifying the identity information of a newborn based on footprints in an embodiment of the present invention, based on the method introduced in the embodiment of the present invention, technical personnel in this field can understand the specific implementation method of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present invention will not be described in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present invention is within the scope of protection of the present invention.

[0161] Please refer to Figure 800, which is a schematic diagram of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention.

[0162] As shown in Figure 800, an embodiment of the present invention further provides a computer-readable storage medium 800, on which a computer program 811 is stored. When the computer program 811 is executed by a processor, the following steps are implemented:

[0163] Determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified;

[0164] Detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set;

[0165] If yes, preprocessing each footprint video frame in the footprint video frame set to obtain a target footprint image set;

[0166] Inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set;

[0167] The identity information of the target newborn is determined according to the footprint depth feature vector corresponding to each footprint image.

[0168] In the specific implementation process, the computer program 811 is executed by the processor to achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0169] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0170] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] The embodiment of the present invention also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device executes the following Figure 1 The process in the corresponding embodiment.

[0175] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There are other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0181] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying the identity information of a newborn based on foot prints, characterized in that: include: Determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified; Detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set; If yes, preprocessing each footprint video frame in the footprint video frame set to obtain a target footprint image set; Inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set; The identity information of the target newborn is determined according to the footprint depth feature vector corresponding to each footprint image.

2. The method according to claim 1, characterized in that The detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets the preset requirement in the continuous foot sole video frame set includes: Eliminate video frames that do not include the target newborn's footprints from the sole video frame set to obtain an initial sole video frame set; Determine whether there are N consecutive video frames in the initial sole video frame set, each of which satisfies the requirement that the footprint is not blocked, the footprint meets the recognition requirement, and the toes of the footprint are completely visible, where N is an integer greater than or equal to 5; If yes, the continuous N video frames are determined as the footprint video frame set; If not, a prompt message is issued, wherein the prompt message is used to instruct the user to re-photograph the footprints of the target newborn.

3. The method according to claim 1, characterized in that The method further comprises: Acquire a footprint image of each of the multiple newborns, wherein the footprint image includes a positive sample image whose shooting angle meets the preset requirements and a negative sample image whose shooting angle does not meet the preset requirements, and the footprint image includes a left foot footprint image and a right foot footprint image; Annotating the foot print images of training newborns to obtain an initial sample set, wherein the training labeled newborn is any one of the multiple newborns; Preprocessing each initial sample in the initial sample set to obtain a target training sample set; An initial footprint recognition model is constructed, and the initial footprint recognition model is iteratively trained based on the target training sample set until a preset iteration termination condition is reached, and the initial footprint recognition model when the preset iteration termination condition is reached is determined as the footprint recognition model.

4. The method according to claim 3, characterized in that The step of labeling the training newborn's foot print images to obtain an initial sample set includes: Annotating the training newborn's footprint image based on the SAM tool to obtain the overall outline of the training newborn's initial footprint; Adjusting the overall contour of the initial footprint to obtain the overall contour of the target footprint; Determine the minimum circumscribed rotation rectangle of the training newborn's footprint according to the overall contour of the target footprint, wherein the minimum circumscribed rotation rectangle of the footprint includes a center point, a length, a width, and a rotation angle; Annotating the key points and visible states of the toes of the training newborn contained in the minimum circumscribed rotation rectangle of the footprint to obtain an initial annotated footprint image; The attributes of the training newborn's footprints in the initial labeled footprint image are labeled to obtain the initial sample set, wherein the attributes of the footprints include whether the footprints are blocked, the image shooting quality, the number of the training newborn, and the footprint position.

5. The method according to claim 4, characterized in that The preprocessing of each initial sample in the initial sample set to obtain a target training sample set includes: Adjusting the angle of the training newborn's footprint in the target initial sample according to the rotation angle of the minimum circumscribed rotation matrix of the footprint to obtain a first training sample; cropping the footprints of the training newborn in the first training sample to obtain a footprint image of the training newborn; Determining a training combination of the foot print images for training the newborn according to the foot print positions; Extracting the footprint pattern of each footprint image in the footprint image training set; Determining a texture contour area corresponding to the footprint pattern; The texture contour area is added to the corresponding foot print image to obtain the target training sample set.

6. The method according to any one of claims 1 to 5, characterized in that The step of determining the identity information of the target newborn according to the footprint depth feature vector corresponding to each footprint image comprises: Calculate the similarity between the footprint depth feature vector and each footprint feature in the newborn footprint feature database to obtain a similarity set; Determine the target footprint feature in the similarity set that is greater than a similarity threshold; The target identity information corresponding to the target footprint feature in the newborn footprint feature database is determined as the identity information of the target newborn.

7. The method according to claim 6, characterized in that The method further comprises: If the target footprint feature does not exist in the similarity set, an input prompt message is issued; Acquire the identity information of the target newborn input by the user according to the input prompt information; The identity information of the target newborn is associated with the footprint depth feature vector and stored in the newborn footprint feature database.

8. A device for identifying the identity information of a newborn based on foot prints, characterized in that: include: A determination module, used to determine a set of foot sole video frames corresponding to a target newborn, wherein the target newborn is a newborn whose identity information is to be identified; A detection module, used for detecting the foot sole video frame set to determine whether there is a foot print video frame set that meets preset requirements in the foot sole video frame set; a preprocessing module, configured to preprocess each footprint video frame in the foot print video frame set to obtain a target footprint image set when there is a footprint video frame set that meets the preset requirement in the foot sole video frame set; A feature recognition module, used for inputting the target footprint image set into a footprint recognition model to obtain a footprint depth feature vector corresponding to each footprint image in the target footprint image set; The identity recognition module is used to determine the identity information of the target newborn according to the footprint depth feature vector corresponding to each footprint image.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to implement the method for identifying the identity information of a newborn based on footprints as described in any one of claims 1 to 7 when executing the computer management program stored in the memory.

10. A computer-readable storage medium having a computer management program stored thereon, characterized in that: When the computer management program is executed by the processor, the method for identifying the identity information of a newborn based on footprints as described in any one of claims 1 to 7 is implemented.

Images