Infant and child face occlusion recognition method, device, equipment and storage medium

Abstract

The present invention relates to the field of image processing technology, solving the problem in the prior art of being unable to effectively determine whether an infant's face is obstructed, and providing a method, device, equipment, and storage medium for identifying infant face obstruction. The method comprises: acquiring multiple frames of images related to a real-time infant care scene; inputting the multiple frames of images into an infant head detection model and marking the infant's head in the image; inputting the partial image within the infant head frame into an infant face detection model and marking the infant's face in the image; inputting the partial image within the infant face frame into an infant face feature extraction model to determine the infant's facial state; based on the infant's facial state, identifying whether the infant's face is obstructed and issuing a safety reminder to the user. When the infant's face is obstructed, the parent is promptly reminded to intervene manually to reduce potential safety risks to the infant.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for recognizing infant facial occlusion. Background Art

[0002] With the progress of urbanization, the use of AI devices for infant care is gradually being accepted by young parents. When using AI devices to care for infants and young children, it is important to judge the current physical condition, posture, behavior, etc. of the infant in real time.

[0003] This includes determining whether there is an obstruction on the infant's face. This determination can effectively prevent the infant from having breathing difficulties due to the face being obstructed by clothing or bed covers. When the infant's face is obstructed, it can promptly remind parents to intervene manually, thereby reducing potential safety risks for the infant. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus, device, and storage medium for identifying infant face occlusion, to solve the problem in the prior art of being unable to effectively determine whether there is an obstruction on the infant's face.

[0005] In a first aspect, an embodiment of the present invention provides a method for recognizing infant face occlusion, characterized in that the method comprises:

[0006] S1: Acquire a real-time video stream related to an infant care scene, and decompose the video stream into multiple frames of images;

[0007] S2: inputting the multiple frames of images into an infant head detection model, marking the infant head in the image with a virtual first target rectangular frame, and recording the first target rectangular frame as an infant head frame;

[0008] S3: inputting the partial image within the infant head frame into an infant face detection model, marking the infant face in the image with a virtual second target rectangular frame, and recording the second target rectangular frame as an infant face frame;

[0009] S4: inputting the partial image within the infant face frame into an infant face feature extraction model to determine the infant face state;

[0010] S5: Based on the facial state of the infant, identify the situation where the infant's face is blocked and issue a safety reminder to the user.

[0011] Preferably, the S2 includes:

[0012] S21: Pre-build and train the first neural network detection algorithm model based on the YoloV6s structure;

[0013] S22: Inputting the real-time image into the first neural network detection algorithm model, and outputting the coordinate point and confidence level of the infant's head to be determined;

[0014] S23: Marking the infant's head in the real-time image with a virtual first target rectangular frame according to the coordinate points and confidence level of the infant's head to be determined, and recording the first target rectangular frame as the infant's head frame.

[0015] Preferably, the S23 includes:

[0016] S231: Determining, based on the coordinate points of the infant's head to be determined, whether the length and width thereof are within a preset length range and a preset width range, respectively;

[0017] S232: Determining whether the to-be-determined infant's head meets the infant's head characteristics according to the confidence level of the to-be-determined infant's head;

[0018] S233: If the length is within a preset length range, the width is within a preset width range, and the infant's head features are met, then the infant's head in the real-time image is marked with the infant's head frame.

[0019] Preferably, the S3 includes:

[0020] S31: Pre-constructing and training a neural network classification algorithm model based on Onet and ResNet18 structures, inputting the local image within the infant's head frame into the neural network classification algorithm model, and outputting vector data vecter and confidence value val of the infant's face to be determined;

[0021] S32: Pre-set a first confidence threshold threshold1 and a second confidence threshold threshold2, and in combination with the confidence val, use a virtual second target rectangular frame to mark the infant's face in the real-time image, and record the second target rectangular frame as the infant's face frame, where 0<threshold1<threshold2.

[0022] Preferably, the S4 includes:

[0023] S41: Pre-constructing and training a second neural network detection algorithm model based on the MobileFaceNet structure, inputting the local image within the infant's face frame into the second neural network detection algorithm model, and outputting the infant's facial feature value;

[0024] S42: performing normalization calculation and classification on the infant's facial feature values ​​according to the vector data vecter and the confidence level val, and outputting the infant's facial status.

[0025] Preferably, the S42 includes:

[0026] S421: When the confidence val is greater than the threshold2, the facial feature value of the infant at this time is recorded as the infant frontal face calibration value according to the vector data vecter;

[0027] S422: If the confidence val is between the threshold1 and the threshold2, performing a vector dot product multiplication operation on the infant's facial feature value at this time and the infant's frontal face calibration value, and outputting the infant's facial state at this time;

[0028] S423: If the confidence val is less than the threshold1, it is considered that the infant's face is not recognized.

[0029] Preferably, the S5 includes:

[0030] S51: continuously inputting the plurality of frames of images and repeating steps S1 to S4 to obtain the facial state of the infant in each of the plurality of frames of images;

[0031] S52: If the infant's face state in the current frame image is frontal or profile, pre-set an infant face occlusion model, and input the next frame image into the infant face occlusion model;

[0032] S53: If the infant's face in the next frame image is in a preset occlusion state, a safety reminder is issued to the user;

[0033] S54: If the infant's face in the next frame image does not appear in the preset occlusion state, no safety reminder is issued to the user.

[0034] In a second aspect, an embodiment of the present invention further provides a device for identifying infant face occlusion, the device comprising:

[0035] A real-time image acquisition module is used to acquire a real-time video stream related to an infant care scene and decompose the video stream into multiple frames of images;

[0036] a head recognition module, configured to input the multiple frames of images into an infant head detection model, mark the infant's head in the image with a virtual first target rectangular frame, and record the first target rectangular frame as an infant head frame;

[0037] a face recognition module, configured to input the partial image within the infant head frame into an infant face detection model, mark the infant face in the image with a virtual second target rectangular frame, and record the second target rectangular frame as an infant face frame;

[0038] a feature extraction module, configured to input the partial image within the infant's face frame into an infant's face feature extraction model to determine the infant's facial state;

[0039] The safety reminder module is used to identify the situation where the infant's face is blocked based on the infant's facial state and issue a safety reminder to the user.

[0040] In a third aspect, an embodiment of the present invention further provides an electronic device comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect of the above-mentioned embodiment.

[0041] In a fourth aspect, an embodiment of the present invention provides a storage medium having computer program instructions stored thereon, which implements the method of the first aspect of the above-mentioned embodiment when the computer program instructions are executed by a processor.

[0042] In summary, the beneficial effects of the present invention are as follows:

[0043] The embodiments of the present invention provide a method, apparatus, device, and storage medium for identifying infant face occlusion, which obtains a real-time video stream related to an infant care scene and decomposes the video stream into multiple frames of images; inputs the multiple frames of images into an infant head detection model, uses a virtual first target rectangular frame to mark the infant's head in the image, and records the first target rectangular frame as an infant head frame; inputs a partial image within the infant head frame into an infant face detection model, uses a virtual second target rectangular frame to mark the infant's face in the image, and records the second target rectangular frame as an infant face frame; inputs the partial image within the infant face frame into an infant face feature extraction model to determine the infant's facial status; based on the infant's facial status, identifies situations where the infant's face is occluded, and issues a safety reminder to the user. First, head detection is used to determine whether it is an infant's head. Secondly, face detection is used to determine whether the infant's face is reliable. Then, face features are extracted to further determine whether the infant's face is straight. Finally, by monitoring the changes in the infant's head and facial status over a period of time, it is decided whether to issue a reminder to the user. This realizes intelligent control to determine whether the infant's face is blocked, effectively avoiding breathing difficulties caused by the infant's face being blocked by clothing or bed quilts, and reducing possible safety risks for infants. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work, and these are all within the scope of protection of the present invention.

[0045] Figure 1 1 is a schematic diagram of the overall workflow of the infant face occlusion recognition method according to Example 1 of the present invention;

[0046] Figure 2 1 is a schematic diagram of a process for identifying an infant's head according to Embodiment 1 of the present invention;

[0047] Figure 3 1 is a schematic diagram of the process of marking an infant's head frame according to Example 1 of the present invention;

[0048] Figure 4 1 is a schematic diagram of the process of marking an infant face frame according to Embodiment 1 of the present invention;

[0049] Figure 5 1 is a schematic diagram of a process for identifying the facial state of an infant according to Embodiment 1 of the present invention;

[0050] Figure 6 2 is a schematic diagram of a process for classifying infant facial states according to embodiment 1 of the present invention;

[0051] Figure 7 1 is a schematic diagram of a process for issuing a safety reminder according to embodiment 1 of the present invention;

[0052] Figure 8 2 is a structural block diagram of an infant face occlusion recognition device according to embodiment 2 of the present invention;

[0053] Figure 9 It is a schematic structural diagram of an electronic device according to embodiment 3 of the present invention. DETAILED DESCRIPTION

[0054] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the present invention.

[0055] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0056] Example 1

[0057] See Figure 1 , an embodiment of the present invention provides a method for recognizing infant face occlusion, the method comprising:

[0058] S1: Acquire a real-time video stream related to an infant care scene, and decompose the video stream into multiple frames of images;

[0059] S2: inputting the multiple frames of images into an infant head detection model, marking the infant head in the image with a virtual first target rectangular frame, and recording the first target rectangular frame as an infant head frame;

[0060] In one embodiment, see Figure 2 , said S2 includes:

[0061] S21: Pre-build and train the first neural network detection algorithm model based on the YoloV6s structure;

[0062] S22: Inputting the real-time image into the first neural network detection algorithm model, and outputting the coordinate point and confidence level of the infant's head to be determined;

[0063] Specifically, before the AI ​​device leaves the factory, a large number of publicly available images of infants and young children at various stages of growth are collected for analysis, focusing on images of daily activities of younger infants and young children. These images are used as training data for the deep learning model to construct and train the first neural network detection algorithm model based on the YoloV6s structure. This model fully learns the image features of a large number of infant and young child head targets and runs offline on the AI ​​device. The AI ​​device's camera obtains a 640x360 YUV image, which is input into the first neural network detection algorithm model, which outputs the lower left corner coordinate point (x1, y1) and upper right corner coordinate point (x3, y3) of the circumscribed rectangular box of the head target detected in the YUV image, as well as the confidence parameter val.

[0064] S23: Marking the infant's head in the real-time image with a virtual first target rectangular frame according to the coordinate points and confidence level of the infant's head to be determined, and recording the first target rectangular frame as the infant's head frame.

[0065] In one embodiment, see Figure 3 , the S23 includes:

[0066] S231: Determining, based on the coordinate points of the infant's head to be determined, whether the length and width thereof are within a preset length range and a preset width range, respectively;

[0067] S232: Determining whether the to-be-determined infant's head meets the infant's head characteristics according to the confidence level of the to-be-determined infant's head;

[0068] S233: If the length is within the preset length range, the width is within the preset width range, and the features of the infant's head are met, the infant's head in the real-time image is marked with the infant's head frame. Specifically, the lower left corner coordinate point (x1, y1) and the upper right corner coordinate point (x3, y3) of the circumscribed rectangular frame of the infant's head target are marked, and the confidence parameter val is set. The user pre-sets a first length threshold L1, a second length threshold L2, a first width threshold W1, a second width threshold W2, and a confidence threshold threshold, wherein 0 < L1 < L2, 0 < W1 < W2, and the specific size and difference between L1 and L2, and the specific size and difference between W1 and W2 are determined according to the actual needs of the user. Determine whether the target length meets the requirements based on the absolute value of the difference between x1 and x3 combined with L1 and L2; determine whether the target width meets the requirements based on the absolute value of the difference between y1 and y3 combined with W1 and W2; determine whether the target feature meets the requirements based on val combined with the threshold; and determine the target head detection result based on the above results, that is, determine whether the detection target is an infant's head. If the absolute value of the difference between x1 and x3 is less than L1 or greater than L2, it is determined that the target length does not meet the requirements; if the absolute value of the difference between x1 and x3 is greater than L1 and less than L2, it is determined that the target length meets the requirements; if the absolute value of the difference between y1 and y3 is less than W1 or greater than W2, it is determined that the target width does not meet the requirements; if the absolute value of the difference between y1 and y3 is greater than W1 and less than W2, it is determined that the target width meets the requirements; if val is less than the threshold, it is determined that the target feature does not meet the requirements; if val is greater than the val threshold, it is determined that the target feature meets the requirements; if the above-mentioned target length does not meet the requirements or the target width does not meet the requirements or the target feature does not meet the requirements, it is determined that the target head detection result is unreliable, that is, the infant head is not detected; if the target length meets the requirements, the target width meets the requirements, and the target feature meets the requirements, it is determined that the target head detection result is reliable, that is, the infant head has been detected.

[0069] S3: inputting the partial image within the infant head frame into an infant face detection model, marking the infant face in the image with a virtual second target rectangular frame, and recording the second target rectangular frame as an infant face frame;

[0070] In one embodiment, see Figure 4 , the S3 includes:

[0071] S31: Pre-constructing and training a neural network classification algorithm model based on Onet and ResNet18 structures, inputting the local image within the infant's head frame into the neural network classification algorithm model, and outputting vector data vecter and confidence value val of the infant's face to be determined;

[0072] Specifically, before the AI ​​device leaves the factory, a large number of publicly available facial images are collected and used as training data for the deep learning model. A neural network classification algorithm model based on the Onet and ResNet18 structures is constructed and trained. If an infant's head is detected, a local YUV image within the rectangular frame of the infant's head is obtained, resized to 96x96 size, and input into the neural network classification algorithm model, which outputs 5 sets of vector data vecter representing the orientation of the face and 1 confidence level val.

[0073] S32: Pre-set a first confidence threshold threshold1 and a second confidence threshold threshold2, and in combination with the confidence val, use a virtual second target rectangular frame to mark the infant's face in the real-time image, and record the second target rectangular frame as the infant's face frame, where 0<threshold1<threshold2.

[0074] Specifically, the user pre-sets a first confidence threshold threshold1 and a second confidence threshold threshold2, where 0<threshold1<threshold2. If the confidence val is less than the threshold1, the face detection result is judged to be unreliable, that is, the back of the infant's head is detected, or most of the face is blocked by clothes, and there is no need to further extract and judge the face information; if the confidence val is greater than the threshold2, the face detection result is judged to be reliable, that is, the front face or side face of the infant is detected, and it is not blocked by clothes, and there is no need to further extract and judge the face information; if the confidence val is between threshold1 and threshold2, it is not sufficient to determine whether there is a reliable face result in the current input image, and further extraction and judgment of the face information is required.

[0075] S4: inputting the partial image within the infant face frame into an infant face feature extraction model to determine the infant face state;

[0076] In one embodiment, see Figure 5 , said S4 includes:

[0077] S41: Pre-constructing and training a second neural network detection algorithm model based on the MobileFaceNet structure, inputting the local image within the infant's face frame into the second neural network detection algorithm model, and outputting the infant's facial feature value;

[0078] Specifically, before AI devices ship, a large number of publicly available facial images are collected and used as training data for a deep learning model. A neural network detection algorithm based on the MobileFaceNet architecture is constructed and trained. This model runs offline on the AI ​​device, taking a local YUV image within the target rectangular frame of the head, resizing it to 112x112 as input, and outputting a data feature containing 256-dimensional facial eigenvalues.

[0079] S42: performing normalization calculation and classification on the infant's facial feature values ​​according to the vector data vecter and the confidence level val, and outputting the infant's facial status.

[0080] Specifically, obtain 5 sets of vector data vectors representing the current face orientation and 256-dimensional eigenvalue data feature of the infant's face in the target rectangular frame of the infant's head in the real-time image frame, first perform normalization calculation on the face eigenvalue of the front face, and convert all the face eigenvalues ​​of different face orientations into eigenvalues ​​of the front face orientation: arrange the vectors according to the spatial position relationship, and obtain the rotation matrices Rx, Ry and Rz about the x, y and z axes of the right-handed Cartesian coordinate system respectively. Then, in the spatial rotation matrix R=Rz*Ry*Rx in the right-handed Cartesian coordinate system, the eigenvalue of a group of front face eigenvalues ​​after rotating around the rotation matrix is ​​feature

[256] , and the eigenvalue of the front face is feature

[256] ×R-1, and obtain the reliability of the infant's face. Combined with the front face eigenvalue, the facial state of the infant is further determined. The AI ​​device obtains the reliability of the infant's face. If the face is unreliable, then no face is detected at this time, corresponding to the state where the back of the infant's head or most of the face is blocked, and the facial features cannot be further extracted, and the next frame image is directly re-input into the head detection process. If the face is reliable, then the face is detected at this time, corresponding to the front face or side face of the infant at this time, and the face is not blocked. At this time, the feature value of the front face is obtained by normalizing the front face, which is the calibration value feature_s, and then the next frame image is re-input into the head detection process. If threshold1<val<threshold2, it is impossible to determine whether the infant's face is reliable at this time. If the calibration value feature_s does not appear in the previous frame image after further detection, that is, the front face of the infant has not appeared in the previous frame image, the AI ​​device does not proceed to the next step and re-inputs the next frame image into the head detection process. If the matching value obtained by comparing the frontal face feature value feature

[256] ×R-1 of the current frame with the frontal face calibration value feature_s is not greater than 0.8, it is considered that the two do not match, corresponding to the fact that it is not the frontal face of an infant or child.

[0081] In one embodiment, see Figure 6, the S42 includes:

[0082] S421: When the confidence val is greater than the threshold2, the facial feature value of the infant at this time is recorded as the infant frontal face calibration value according to the vector data vecter;

[0083] S422: If the confidence val is between the threshold1 and the threshold2, performing a vector dot product multiplication operation on the infant's facial feature value at this time and the infant's frontal face calibration value, and outputting the infant's facial state at this time;

[0084] Specifically, if threshold1<val<threshold2, it is impossible to determine whether the infant's face is reliable. If the calibration value feature_s appears in the previous frame image, the frontal face feature value feature

[256] ×R-1 of the current frame is compared with feature_s to determine the matching value. The AI ​​device pre-sets a matching threshold of 0.8 based on the actual application scenario. If the matching value obtained by the vector dot product multiplication operation of the frontal face feature value feature

[256] ×R-1 of the current frame and the frontal face calibration value feature_s is greater than 0.8, it is considered that the two are matched, corresponding to the frontal face of the infant at this time.

[0085] S423: If the confidence val is less than the threshold1, it is considered that the infant's face is not recognized.

[0086] S5: Based on the facial state of the infant, identify the situation where the infant's face is blocked and issue a safety reminder to the user.

[0087] In one embodiment, see Figure 7 , the S5 includes:

[0088] S51: continuously inputting the plurality of frames of images and repeating steps S1 to S4 to obtain the facial state of the infant in each of the plurality of frames of images;

[0089] S52: If the infant's face state in the current frame image is frontal or profile, pre-set an infant face occlusion model, and input the next frame image into the infant face occlusion model;

[0090] S53: If the infant's face in the next frame image is in a preset occlusion state, a safety reminder is issued to the user;

[0091] S54: If the infant's face in the next frame image does not appear in the preset occlusion state, no safety reminder is issued to the user.

[0092] Specifically, if the confidence val in the current frame image is greater than the threshold2, it is considered that the corresponding infant's face state is the front face or the side face and is not blocked. If the infant's head is not identified in the subsequent input image through step S2, the entire head of the infant may be blocked, which may easily cause dangerous situations such as suffocation, and a safety reminder needs to be issued to the user; if the infant's head is identified through step S2 but the confidence val is less than the threshold1 through step S3, that is, the infant's face is not identified, it is considered that the infant's face is completely blocked, and a safety reminder needs to be issued to the user; if the infant's head is identified through step S2, but the confidence val is between the threshold1 and the threshold2 through step S3, that is, it cannot be identified. When determining whether an infant's face is present and the infant's facial feature values ​​identified by S4 do not match the calibrated frontal face feature values, a large number of infant facial images are collected in advance. These images are used as a training basis to mark key areas such as the infant's mouth and nose, and a third neural network detection algorithm model is output that can detect the infant's mouth and nose. The real-time images are input into the third neural network detection algorithm model to further identify the infant's facial state. If the infant's mouth and nose are identified, it is considered that although the infant's face does not conform to a frontal face, the infant's mouth and nose are not obscured, corresponding to the special case where the infant's forehead is obscured due to a fever or illness, and a safety reminder does not need to be issued to the user. If the infant's mouth and nose are not identified, it is considered that the infant's mouth and nose are also obscured, which may cause suffocation, and a safety reminder needs to be issued to the user. By further identifying whether the infant's mouth and nose are obscured when the frontal face is not identified, the situation where the nursing device misjudges the infant's forehead due to fever or illness and issues an erroneous reminder to the user is avoided.

[0093] Example 2

[0094] See Figure 8 The embodiment of the present invention further provides a device for recognizing infant face occlusion, the device comprising:

[0095] A real-time image acquisition module is used to acquire a real-time video stream related to an infant care scene and decompose the video stream into multiple frames of images;

[0096] a head recognition module, configured to input the multiple frames of images into an infant head detection model, mark the infant's head in the image with a virtual first target rectangular frame, and record the first target rectangular frame as an infant head frame;

[0097] a face recognition module, configured to input the partial image within the infant head frame into an infant face detection model, mark the infant face in the image with a virtual second target rectangular frame, and record the second target rectangular frame as an infant face frame;

[0098] a feature extraction module, configured to input the partial image within the infant's face frame into an infant's face feature extraction model to determine the infant's facial state;

[0099] The safety reminder module is used to identify the situation where the infant's face is blocked based on the infant's facial state and issue a safety reminder to the user.

[0100] Specifically, the infant face occlusion recognition device provided by the embodiment of the present invention includes a real-time image acquisition module for acquiring a real-time video stream related to an infant care scene and decomposing the video stream into multiple frames of images; a head recognition module for inputting the multiple frames of images into an infant head detection model, marking the infant's head in the image with a virtual first target rectangular frame, and recording the first target rectangular frame as an infant head frame; a face recognition module for inputting a local image within the infant head frame into an infant face detection model, marking the infant's face in the image with a virtual second target rectangular frame, and recording the second target rectangular frame as an infant face frame; a feature extraction module for inputting a local image within the infant face frame into an infant face feature extraction model to determine the infant's facial status; and a safety reminder module for identifying situations where the infant's face is occluded based on the infant's facial status and issuing a safety reminder to the user. First, head detection is used to determine whether it is an infant's head. Secondly, face detection is used to determine whether the infant's face is reliable. Then, face features are extracted to further determine whether the infant's face is straight. Finally, by monitoring the changes in the infant's head and facial status over a period of time, it is decided whether to issue a reminder to the user. This realizes intelligent control to determine whether the infant's face is blocked, effectively avoiding breathing difficulties caused by the infant's face being blocked by clothing or bed quilts, and reducing possible safety risks for infants.

[0101] Example 3

[0102] In addition, combined Figure 1 The infant face occlusion recognition method described in the embodiment of the present invention can be implemented by an electronic device. Figure 9 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention is shown.

[0103] The electronic device may include a processor and a memory storing computer program instructions.

[0104] Specifically, the processor may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0105] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a specific embodiment, the memory is a non-volatile solid-state memory. In a specific embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0106] The processor implements any one of the infant face occlusion recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0107] In one example, the electronic device may further include a communication interface and a bus. Figure 9 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0108] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0109] Bus comprises hardware, software or both, couples the parts of electronic equipment to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus can comprise one or more buses.Although the embodiment of the present invention describes and shows specific bus, the present invention considers any suitable bus or interconnection.

[0110] Example 4

[0111] In addition, in conjunction with the infant face occlusion recognition method in the above-mentioned embodiments, embodiments of the present invention may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when executed by a processor, the computer program instructions implement any of the infant face occlusion recognition methods in the above-mentioned embodiments.

[0112] In summary, the embodiments of the present invention provide a method, apparatus, device, and storage medium for recognizing infant or toddler facial occlusion.

[0113] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0114] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in unit, a function card or the like. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0115] It should also be noted that the exemplary embodiments described herein describe methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the steps described above. In other words, the steps may be performed in the order described in the embodiments, or in a different order, or several steps may be performed simultaneously.

[0116] The above description is only a specific embodiment of the present invention. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention.

Claims

1. A method for recognizing infant face occlusion, characterized in that: The method comprises: S1: Acquire a real-time video stream related to an infant care scene, and decompose the video stream into multiple frames of images; S2: inputting the multiple frames of images into an infant head detection model, marking the infant head in the image with a virtual first target rectangular frame, and recording the first target rectangular frame as an infant head frame; S3: inputting the partial image within the infant head frame into an infant face detection model, marking the infant face in the image with a virtual second target rectangular frame, and recording the second target rectangular frame as an infant face frame; S4: inputting the partial image within the infant face frame into an infant face feature extraction model to determine the infant face state; S5: Based on the infant's facial state, identifying whether the infant's face is obscured and issuing a safety reminder to the user; The S2 includes: S21: Pre-build and train the first neural network detection algorithm model based on the YoloV6s structure; S22: Inputting each of the images obtained by decomposing the real-time video stream into the first neural network detection algorithm model, and outputting the coordinate point and confidence level of the infant's head to be determined; S23: marking the infant's head in the real-time image with a virtual first target rectangular frame according to the coordinate point and confidence level of the infant's head to be determined, and recording the first target rectangular frame as the infant's head frame; The S23 includes: S231: Determining, based on the coordinate points of the infant's head to be determined, whether the length and width thereof are within a preset length range and a preset width range, respectively; S232: Determining whether the to-be-determined infant's head meets the infant's head characteristics according to the confidence level of the to-be-determined infant's head; S233: If the length is within a preset length range, the width is within a preset width range, and the infant's head features are met, marking the infant's head in the real-time image with the infant's head frame; The S3 includes: S31: Pre-constructing and training a neural network classification algorithm model based on Onet and ResNet18 structures, inputting the local image within the infant's head frame into the neural network classification algorithm model, and outputting vector data vecter and confidence value val of the infant's face to be determined; S32: presetting a first confidence threshold threshold1 and a second confidence threshold threshold2, and combining the confidence val, marking the infant's face in the real-time image with a virtual second target rectangular frame, and recording the second target rectangular frame as the infant's face frame, wherein 0 < threshold1 < threshold2; The S4 includes: S41: Pre-constructing and training a second neural network detection algorithm model based on the MobileFaceNet structure, inputting the local image within the infant's face frame into the second neural network detection algorithm model, and outputting the infant's facial feature value; S42: performing normalization calculation and classification on the infant's facial feature values ​​according to the vector data vecter and the confidence level val, and outputting the infant's facial status; The S42 includes: S421: When the confidence val is greater than the threshold2, the facial feature value of the infant at this time is recorded as the infant frontal face calibration value according to the vector data vecter; S422: If the confidence val is between the threshold1 and the threshold2, performing a vector dot product multiplication operation on the infant's facial feature value at this time and the infant's frontal face calibration value, and outputting the infant's facial state at this time; S423: If the confidence val is less than the threshold1, it is considered that the infant's face is not recognized.

2. The method for recognizing infant face occlusion according to claim 1, wherein: The S5 includes: S51: continuously inputting the plurality of frames of images and repeating steps S1 to S4 to obtain the facial state of the infant in each of the plurality of frames of images; S52: If the infant's face state in the current frame image is frontal or profile, pre-set an infant face occlusion model, and input the next frame image into the infant face occlusion model; S53: If the infant's face in the next frame image is in a preset occlusion state, a safety reminder is issued to the user; S54: If the infant's face in the next frame image does not appear in the preset occlusion state, no safety reminder is issued to the user.

3. A device for recognizing infant face occlusion, characterized in that: include: A real-time image acquisition module is used to acquire a real-time video stream related to an infant care scene and decompose the video stream into multiple frames of images; a head recognition module, configured to input the multiple frames of images into an infant head detection model, mark the infant's head in the image with a virtual first target rectangular frame, and record the first target rectangular frame as an infant head frame; a face recognition module, configured to input the partial image within the infant head frame into an infant face detection model, mark the infant face in the image with a virtual second target rectangular frame, and record the second target rectangular frame as an infant face frame; a feature extraction module, configured to input the partial image within the infant's face frame into an infant's face feature extraction model to determine the infant's facial state; A safety reminder module, configured to identify situations where the infant's face is obscured based on the infant's facial state and issue a safety reminder to the user; Inputting the multiple frames of images into the infant head detection model, marking the infant head in the image with a virtual first target rectangular frame, and recording the first target rectangular frame as an infant head frame includes: Pre-build and train the first neural network detection algorithm model based on YoloV6s structure; Inputting each of the images obtained by decomposing the real-time video stream into the first neural network detection algorithm model, and outputting the coordinate point and confidence level of the infant's head to be determined; Marking the infant's head in the real-time image with a virtual first target rectangular frame according to the coordinate points and confidence level of the infant's head to be determined, and recording the first target rectangular frame as the infant's head frame; The step of marking the infant's head in the real-time image with a virtual first target rectangular frame according to the coordinate point and confidence level of the infant's head to be determined, and recording the first target rectangular frame as the infant's head frame includes: Determining, based on the coordinate points of the infant's head, whether its length and width are within a preset length range and a preset width range, respectively; determining whether the to-be-determined infant's head meets the infant's head characteristics according to the confidence level of the to-be-determined infant's head; If the length is within a preset length range, the width is within a preset width range, and the features are consistent with the infant's head, then using the infant's head frame to mark the infant's head in the real-time image; The step of inputting the partial image within the infant head frame into the infant face detection model, marking the infant face in the image with a virtual second target rectangular frame, and recording the second target rectangular frame as the infant face frame comprises: Pre-constructing and training a neural network classification algorithm model based on Onet and ResNet18 structures, inputting the local image within the infant's head frame into the neural network classification algorithm model, and outputting vector data vecter and confidence value val of the infant's face to be determined; Presetting a first confidence threshold threshold1 and a second confidence threshold threshold2, and combining the confidence val, using a virtual second target rectangular frame to mark the infant face in the real-time image, and recording the second target rectangular frame as the infant face frame, wherein 0 < threshold1 < threshold2; Inputting the partial image within the infant face frame into the infant face feature extraction model to determine the infant face state includes: Pre-constructing and training a second neural network detection algorithm model based on the MobileFaceNet structure, inputting the local image within the infant's face frame into the second neural network detection algorithm model, and outputting the infant's facial feature value; performing normalized calculation and classification on the infant's facial feature values ​​according to the vector data vecter and the confidence level val, and outputting the infant's facial state; The normalized calculation and classification of the infant's facial feature values ​​based on the vector data vecter and the confidence level val, and the output of the infant's facial status include: When the confidence val is greater than the threshold2, the facial feature value of the infant at this time is recorded as the infant front face calibration value according to the vector data vecter; If the confidence val is between the threshold1 and the threshold2, performing a vector dot product multiplication operation on the infant's facial feature value at this time and the infant's frontal face calibration value, and outputting the infant's facial state at this time; If the confidence val is less than the threshold1, it is considered that the infant's face is not recognized.

4. An electronic device, characterized in that: include: At least one processor, at least one memory, and computer program instructions stored in the memory, which implement the method according to any one of claims 1 to 2 when the computer program instructions are executed by the processor.

5. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images