Infant sleep posture detection method, device and equipment and storage medium

Abstract

The invention discloses an infant sleep posture detection method, device and equipment and a storage medium. The method comprises the following steps: acquiring image data of an infant; using a preset feature extraction model to identify contour features and joint point coordinate features in the image data; and determining a current sleeping posture according to a matching result of the contour features, the joint point coordinate features and features of different sleeping postures in a preset feature library. According to the invention, the contour features and the joint point coordinate features are extracted through the image data of the baby sleeping posture for posture recognition, and the precision of baby posture recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of infant monitoring, and in particular to a method, device, equipment and storage medium for detecting an infant's sleeping posture. Background Art

[0002] In today's fast-paced society, infant care is the most concerned and anxious issue for parents and families. The high cost of nannies and the various problems that arise when nannies take care of infants and young children have deepened this anxiety. Infants and young children sleep for a long time every day. Baby monitoring equipment should not only identify the baby's crying in time, but also determine whether there are risks in the baby's sleep process in time. Experts have found that sudden infant death syndrome is related to sleeping posture, especially the prone sleeping posture with the face down is the most dangerous. At present, baby monitoring equipment has large errors in the monitoring results of infant sleeping posture under different viewing angles, different sleeping positions, and different sleeping backgrounds, resulting in low accuracy in the recognition results of infant sleeping posture. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a method, device, equipment and storage medium for detecting a baby's sleeping posture, which can improve the accuracy of identifying the baby's sleeping posture.

[0004] An embodiment of the present invention provides a method for detecting a baby's sleeping posture, the method comprising:

[0005] Collecting image data of the infant;

[0006] Using a preset feature extraction model to identify contour features and joint point coordinate features in the image data;

[0007] The current sleeping posture is determined according to the matching results of the contour features, the joint point coordinate features and the features of different sleeping postures in a preset feature library.

[0008] Preferably, determining the current sleeping posture according to the matching results of the contour feature, the joint point coordinate feature and the features of different sleeping postures in a preset feature library includes:

[0009] Using a preset first similarity model to calculate the contour similarity between the contour feature and the standard contour features of different sleeping postures in the feature library, and determining a first sleeping posture with the highest contour similarity;

[0010] Using a preset second similarity model to calculate the spatial similarity between the joint point coordinate features and the standard joint point coordinate features of different sleeping postures in the feature library, and determining a second sleeping posture with the highest spatial similarity;

[0011] When the first sleeping position is the same as the second sleeping position, the first sleeping position or the second sleeping position is determined as the current sleeping position.

[0012] As a preferred solution, determining the current sleeping posture according to the matching results of the contour feature, the joint point coordinate feature and the features of different sleeping postures in a preset feature library further includes:

[0013] When the first sleeping posture is different from the second sleeping posture, inputting the contour feature and the joint point coordinate feature into a preset fusion network to output a global feature vector;

[0014] The vector distance between the global feature vector and the sleeping posture vectors of different sleeping postures in the feature library is calculated, and the sleeping posture of the sleeping posture vector with the smallest vector distance is determined as the current sleeping posture.

[0015] As a preferred solution, a preset feature extraction model is used to identify contour features and joint point coordinate features in the image data, including:

[0016] Using an hourglass network to collect features of the infrared image in the image data and extract the contour features;

[0017] The hourglass network is used to collect features of the RGB image in the image data to extract the coordinate features of the joint points.

[0018] As a preferred solution, the contour similarity

[0019] Among them, L t represents the contour feature vector extracted at time t, represents the contour feature vectors of different sleeping postures in the feature library, · represents the dot product of the vector, and ||·|| represents the norm of the vector.

[0020] As a preferred solution, the spatial similarity

[0021] Where i represents the i-th joint point in the joint point coordinate feature, d i represents the Euclidean distance between the joint point coordinate feature and the i-th joint point in the contour feature vector under different sleeping postures in the feature library, S represents the area of ​​the frame where the human skeleton is located, σ i represents the normalization factor of the i-th joint point, v i Indicates whether the i-th joint point is visible, δ(*) represents the visible point screening parameter, δ=1 if visible, otherwise 0.

[0022] As a preferred solution, the process of establishing the feature library includes:

[0023] Obtain image data of babies in different sleeping positions;

[0024] The feature extraction model is used to identify contour features and joint point coordinate features in image data of different sleeping postures;

[0025] Using a preset fusion network to fuse the contour features and the joint point coordinate features to determine sleeping posture vectors of different sleeping postures;

[0026] The feature library is constructed according to the contour features, joint point coordinate features and sleeping posture vectors of different sleeping postures.

[0027] The embodiment of the present invention further provides a device for detecting a baby's sleeping posture, the device comprising:

[0028] An image acquisition module, used for acquiring image data of the baby;

[0029] A feature extraction module, used to identify contour features and joint point coordinate features in the image data using a preset feature extraction model;

[0030] The sleeping posture recognition module is used to determine the current sleeping posture according to the matching results of the contour features, the joint point coordinate features and the features of different sleeping postures in a preset feature library.

[0031] Preferably, the sleeping posture recognition module is specifically used for:

[0032] Using a preset first similarity model to calculate the contour similarity between the contour feature and the standard contour features of different sleeping postures in the feature library, and determining a first sleeping posture with the highest contour similarity;

[0033] Using a preset second similarity model to calculate the spatial similarity between the joint point coordinate features and the standard joint point coordinate features of different sleeping postures in the feature library, and determining a second sleeping posture with the highest spatial similarity;

[0034] When the first sleeping position is the same as the second sleeping position, the first sleeping position or the second sleeping position is determined as the current sleeping position.

[0035] Furthermore, the sleeping posture recognition module is also specifically used for:

[0036] When the first sleeping posture is different from the second sleeping posture, inputting the contour feature and the joint point coordinate feature into a preset fusion network to output a global feature vector;

[0037] The vector distance between the global feature vector and the sleeping posture vectors of different sleeping postures in the feature library is calculated, and the sleeping posture of the sleeping posture vector with the smallest vector distance is determined as the current sleeping posture.

[0038] Preferably, the feature extraction module is specifically used for:

[0039] Using an hourglass network to collect features of the infrared image in the image data and extract the contour features;

[0040] The hourglass network is used to collect features of the RGB image in the image data to extract the coordinate features of the joint points.

[0041] Preferably, the contour similarity

[0042] Among them, L t represents the contour feature vector extracted at time t, represents the contour feature vectors of different sleeping postures in the feature library, · represents the dot product of the vector, and ||·|| represents the norm of the vector.

[0043] Preferably, the spatial similarity

[0044] Where i represents the i-th joint point in the joint point coordinate feature, d i represents the Euclidean distance between the joint point coordinate feature and the i-th joint point in the contour feature vector under different sleeping postures in the feature library, S represents the area of ​​the frame where the human skeleton is located, σ i represents the normalization factor of the i-th joint point, v i Indicates whether the i-th joint point is visible, δ(*) represents the visible point screening parameter, δ=1 if visible, otherwise 0.

[0045] Preferably, the process of establishing the feature library includes:

[0046] Obtain image data of babies in different sleeping positions;

[0047] The feature extraction model is used to identify contour features and joint point coordinate features in image data of different sleeping postures;

[0048] Using a preset fusion network to fuse the contour features and the joint point coordinate features to determine sleeping posture vectors of different sleeping postures;

[0049] The feature library is constructed according to the contour features, joint point coordinate features and sleeping posture vectors of different sleeping postures.

[0050] An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the infant sleeping posture detection method as described in any one of the above embodiments.

[0051] An embodiment of the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for detecting the sleeping posture of an infant as described in any one of the above embodiments is implemented.

[0052] The present invention provides a method, device, equipment and storage medium for detecting the sleeping posture of an infant, which collects image data of an infant; uses a preset feature extraction model to identify contour features and joint point coordinate features in the image data; and determines the current sleeping posture according to the matching results of the contour features, the joint point coordinate features and the features of different sleeping postures in a preset feature library. The present application extracts contour features and joint point coordinate features from the image data of the infant's sleeping posture to perform posture recognition, thereby improving the accuracy of infant posture recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a method for detecting a sleeping posture of an infant provided by an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of contour features and joint point coordinate features provided by an embodiment of the present invention;

[0055] Figure 3 is another flow chart of the method for detecting the sleeping posture of an infant provided by an embodiment of the present invention;

[0056] Figure 4 is a flowchart of a feature recognition process provided by an embodiment of the present invention;

[0057] Figure 5 It is a schematic diagram of the process flow of establishing a feature library provided by an embodiment of the present invention;

[0058] Figure 6 is a structural schematic diagram of a baby sleeping posture detection device provided by an embodiment of the present invention;

[0059] Figure 7 It is a structural diagram of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The embodiment of the present invention provides a method for detecting a baby's sleeping posture. Figure 1 , is a flow chart of a method for detecting a sleeping posture of an infant provided by an embodiment of the present invention, wherein steps S1 to S3 of the method are as follows:

[0062] S1, collects image data of infants;

[0063] S2, using a preset feature extraction model to identify contour features and joint point coordinate features in the image data;

[0064] S3, determining the current sleeping posture according to the matching results of the contour feature, the joint point coordinate feature and the features of different sleeping postures in a preset feature library.

[0065] During the specific implementation of this embodiment, a monitoring device is pre-configured at a designated location of the baby crib, and the user obtains image data of the baby while sleeping.

[0066] It should be noted that the configured monitoring equipment includes sensor devices such as visible light cameras, infrared array sensors and ultrasonic devices that can obtain images of infants' sleeping positions.

[0067] The monitoring device is placed about 1 meter away from the baby for signal collection. The visible light camera collects baby video information, including baby posture information. The infrared array can perform baby image recognition in low-light environments, while the ultrasonic device can eliminate obstacles and achieve monitoring without blind spots.

[0068] During specific implementation, the configured monitoring equipment includes at least one image sensor, and multiple sensor devices may also be integrated to acquire image data.

[0069] After acquiring the image data of the baby, feature extraction is performed on the acquired image data.

[0070] It should be noted that the image feature extraction scheme is a conventional prior art scheme. There are a large number of image feature extraction models or feature extraction methods in the prior art. In this case, the existing feature extraction scheme can be used to extract the features of the image data.

[0071] In order to improve the accuracy of sleeping posture recognition in this case, it is necessary to extract the contour features and joint coordinates of the baby from the image data during feature extraction.

[0072] When performing posture recognition in existing solutions, most solutions use skeleton key points extracted from image data for action recognition to determine the sleeping posture of the baby. However, when performing skeleton key point recognition in existing solutions, due to overlap between key points or key point recognition errors, the accuracy of posture analysis is low, which may lead to failure to recognize specific postures or even recognition of incorrect postures.

[0073] Therefore, the solution proposed in this case extracts the contour features and joint features of the baby, that is, the joint point coordinate features, from the image data during feature recognition.

[0074] It should be noted that the contour features are specifically contour features of the baby's sleeping position, and the joint point coordinate features include at least one of the positions of different joints, distribution relationships, and limb-trunk connection information formed by the joint points.

[0075] See also Figure 2 , is a schematic diagram of contour features and joint point coordinate features provided in an embodiment of the present invention. Figure 2 The figure shows the outline of a sleeping baby and some of the baby's joints, including the top of the head H to the cervical vertebra N, the pelvis P, the knees K and the heels F, where the knees and heels include the left knee K L and left heel F L , and right knee K R and right heel F R .

[0076] It should be noted that, in this embodiment, only one embodiment of contour features and joint point coordinate features is disclosed. In other embodiments, other joint points or other contour features may be used for sleep posture recognition.

[0077] Sleeping posture matching is performed in a preset feature library based on the identified contour features and joint point coordinate features, that is, the features of different sleeping postures in the feature library are matched based on the contour features and joint point coordinate features, the optimal matching result is identified, and the baby's current sleeping posture is determined.

[0078] In this case, contour features and joint point coordinate features are extracted from the image data of the baby's sleeping posture to perform posture recognition, thereby improving the accuracy of baby posture recognition.

[0079] It should be noted that, in this case, posture recognition is combined with posture monitoring to monitor the duration of dangerous postures when the current posture identified is a preset dangerous posture;

[0080] When the duration of the monitored dangerous posture exceeds the preset first duration threshold, an alarm is output. This reduces the risk of dangerous sleeping postures of infants and can also avoid occasional recognition anomalies or continuous movements of infants leading to the recognition of occasional sleeping postures, which frequently trigger alarms and disturb users.

[0081] In another embodiment provided by the present invention, when the current sleeping posture recognition is performed in step S3, the following steps are specifically performed:

[0082] Using a preset first similarity model to calculate the contour similarity between the contour feature and the standard contour features of different sleeping postures in the feature library, and determining a first sleeping posture with the highest contour similarity;

[0083] Using a preset second similarity model to calculate the spatial similarity between the joint point coordinate features and the standard joint point coordinate features of different sleeping postures in the feature library, and determining a second sleeping posture with the highest spatial similarity;

[0084] When the first sleeping position is the same as the second sleeping position, the first sleeping position or the second sleeping position is determined as the current sleeping position.

[0085] In the specific implementation of this embodiment, a video sequence in a group of postures is taken as a sample, and a single data is first processed based on the posture feature to extract the human outline data and the human posture data, and the feature correlation calculation is performed with the posture features in the feature library;

[0086] The contour similarity between the contour feature and the standard contour features of different sleeping postures in the feature library is calculated according to the first similarity model, and the sleeping posture with the highest correlation with the contour feature in the feature library is determined as the first sleeping posture recognized based on the contour.

[0087] The spatial similarity between the joint point coordinate features and the standard joint point coordinate features of different sleeping postures in the feature library is calculated according to the first similarity model, and the sleeping posture with the highest correlation with the joint point features in the feature library is determined as the second sleeping posture identified based on the joint points.

[0088] It should be noted that the first similarity model and the second similarity model can be used as models of feature correlation, similarity and distance calculation schemes in existing technical solutions, and can objectively evaluate the similarity between different features and determine the corresponding sleeping position based on the maximum similarity or correlation.

[0089] After the first sleeping position and the second sleeping position are identified, the first sleeping position and the second sleeping position are compared. When the first sleeping position is the same as the second sleeping position, that is, the sleeping position identified by contour recognition and joint point coordinates is the same, the final current sleeping position is determined.

[0090] After identifying the current sleeping position, if it is a prone sleeping position, the guardian will be reminded to correct the baby's posture in time; if it is a supine sleeping position, the guardian will be reminded to observe the baby in time to see if he or she is choking on milk.

[0091] The sleeping posture is recognized by using contour features and joint point coordinate features respectively. Based on the complementarity between contour features and joint point coordinate features, contour features are used to compensate for overlap or recognition anomalies in joint point coordinates, thereby improving recognition accuracy and the scope of application of scenarios.

[0092] It should be noted that the contour recognition and spatial similarity recognition used in this case require a technical solution to be obtained through corresponding analysis of different feature accuracies and existing defects. Compared with the existing solution that uses a single contour detection or joint point and bone detection, it has higher fault tolerance and higher recognition accuracy for the acquired image data.

[0093] In another embodiment provided by the present invention, when the current sleeping posture recognition is performed in step S3, the following steps are specifically included:

[0094] When the first sleeping posture is different from the second sleeping posture, inputting the contour feature and the joint point coordinate feature into a preset fusion network to output a global feature vector;

[0095] The vector distance between the global feature vector and the sleeping posture vectors of different sleeping postures in the feature library is calculated, and the sleeping posture of the sleeping posture vector with the smallest vector distance is determined as the current sleeping posture.

[0096] In a specific implementation, after the first sleeping posture and the second sleeping posture are identified, the first sleeping posture and the second sleeping posture are compared. When the first sleeping posture is different from the second sleeping posture, it indicates that the contour recognition and joint point coordinate recognition results are inconsistent, and a global feature measurement method is adopted.

[0097] Inputting the contour features and the joint point coordinate features into a preset fusion network, and outputting a global feature vector;

[0098] It should be noted that the fusion network is specifically a global feature fusion network, which outputs the feature vector of the contour features and joint point coordinate features through a 3×3 convolution layer, a pooling layer, and a fully connected layer.

[0099] The vector distance between the global feature vector and the sleeping posture vectors of different sleeping postures in the feature library is calculated, and the sleeping posture of the sleeping posture vector with the smallest vector distance is determined as the current sleeping posture.

[0100] It should be noted that the vector distance can be calculated by a distance calculation method. In this case, Euclidean distance calculation can be used. In other embodiments, other calculation schemes can be used.

[0101] When the results of contour recognition and joint point coordinate recognition are inconsistent, sleeping posture recognition is performed using the global feature vector obtained through feature fusion to improve the applicability of sleeping posture recognition.

[0102] In another embodiment provided by the present invention, when feature recognition is performed in the above step S2, the following steps are specifically included:

[0103] Using an hourglass network to collect features of the infrared image in the image data and extract the contour features;

[0104] The hourglass network is used to collect features of the RGB image in the image data to extract the coordinate features of the joint points.

[0105] Specifically, see Figure 3 , is another flow chart of the method for detecting the sleeping posture of an infant provided by an embodiment of the present invention. When the method is executed:

[0106] The configured infrared camera and RGB camera are used as front-end devices for image acquisition, and infrared images and RGB images are obtained decibel.

[0107] The infrared camera and the RGB camera are configured at different positions, that is, the image acquisition angles of the infrared camera and the RGB camera are different, which can avoid the loss of effective images collected due to occlusion, and can also avoid sleeping posture recognition errors due to angle deviation during single-angle image acquisition through images at different angles, and improve recognition accuracy by making up for angles through multi-angle images.

[0108] After the front-end device collects the image data, it transmits the output to the terminal device where the method is actually executed, which is a computing device such as a controller or an image processor during the specific implementation.

[0109] The terminal device performs image processing and posture detection, obtains the skeleton features and contour features of the human body posture, and determines the contour features and joint point coordinate features.

[0110] When performing feature recognition, see Figure 4 , is a flowchart of a feature recognition process provided by an embodiment of the present invention.

[0111] Design a multi-semantic level posture detector: select the hourglass network, use a multi-stage output heat map to detect the human body structure from coarse to fine, extract basic features of the input image, remove background images and image noise, stage 1 performs global contour detection and outputs the human body contour, stage 2 performs local joint detection and outputs human joints, stage 3 performs joint point detection and outputs human key point information as joint point coordinates.

[0112] A video sequence in a group of postures is taken as a sample. First, a single data is processed based on posture features (for example, human outline data and human posture data are extracted). The processed data is then used as the original input to calculate the overall features of the input data in the 3D space. The feature correlation is then calculated with the posture features in the stored sleeping posture feature library to perform sleeping posture recognition, and information warning is issued based on the recognized sleeping posture.

[0113] In this case, the hourglass network is used to perform feature recognition on different images respectively, and the global feature vector is fused based on the different recognized features to improve the accuracy and efficiency of feature recognition.

[0114] In another embodiment provided by the present invention, the contour similarity

[0115] Among them, L t represents the contour feature vector extracted at time t, represents the contour feature vectors of different sleeping postures in the feature library, · represents the dot product of the vector, and ||·|| represents the norm of the vector.

[0116] In the specific implementation of this embodiment, the contour similarity is measured by cosine similarity, and a video sequence with a length of T is selected. First, use the first-stage detection results of the detector to extract the contour map of the tth image to obtain the human body contour feature data Lt, and use the third-stage posture joint point detection to obtain the joint point coordinate feature data Pt.

[0117] Contour similarity matching uses image appearance similarity measurement and directly adopts cosine similarity:

[0118]

[0119] Among them, L t represents the contour feature vector, represents the contour feature vector corresponding to the supine or prone sleeping posture in the stored sleeping posture feature library, · represents the dot product of the vector, and ||·|| represents the norm of the vector (ie, the length of the vector).

[0120] The minimum distance in the contour similarity corresponds to the first sleeping position identified by the contour.

[0121] In another embodiment provided by the present invention, the spatial similarity

[0122]

[0123] Where i represents the i-th joint point in the joint point coordinate feature, d i represents the Euclidean distance between the joint point coordinate feature and the i-th joint point in the contour feature vector under different sleeping postures in the feature library, S represents the area of ​​the frame where the human skeleton is located, σ i represents the normalization factor of the i-th joint point, v i Indicates whether the i-th joint point is visible, δ(*) represents the visible point screening parameter, δ=1 if visible, otherwise 0.

[0124] In the specific implementation of this embodiment, the spatial similarity is based on the Euclidean distance, and the OKS function is selected.

[0125] Using the posture similarity measurement function OKS (Object Keypoint Similarity), the predicted coordinate data Pt, the coordinates of the skeleton points in different lying postures in the feature library Calculate the similarity between the predicted skeleton points and the skeletons of different sleeping postures. The smaller the similarity, the closer it is to the sleeping posture.

[0126]

[0127] Where i represents the i-th joint point in the joint point coordinate feature, d i The Euclidean distance S between the joint point coordinate feature and the i-th joint point in the contour feature vector under different sleeping postures in the feature library is the scale of the human skeleton, That is, the area of ​​the frame where the human skeleton is located, w and h are the width and height of the frame where the human skeleton is located respectively. i represents the normalization factor of the i-th bone point, the attenuation control constant for different joint points, v i Is the i-th joint point visible? δ(*) selects visible points. If it is visible, δ=1.

[0128] In another embodiment of the present invention, the process of establishing the feature library includes:

[0129] Obtain image data of babies in different sleeping positions;

[0130] The feature extraction model is used to identify contour features and joint point coordinate features in image data of different sleeping postures;

[0131] Using a preset fusion network to fuse the contour features and the joint point coordinate features to determine sleeping posture vectors of different sleeping postures;

[0132] The feature library is constructed according to the contour features, joint point coordinate features and sleeping posture vectors of different sleeping postures.

[0133] When this embodiment is implemented, see Figure 5 , is a schematic diagram of the process flow of establishing a feature library provided by an embodiment of the present invention.

[0134] RGB images and infrared images of infants in different sleeping positions are collected respectively, and image preprocessing and standardization are performed. Then, feature extraction and indexing are performed through the feature extraction model. The RGB image is input for skeleton space information indexing to obtain posture skeleton features, and the infrared image is input for contour feature indexing to obtain contour features. The acquired feature data are processed through convolution, pooling and fully connected layers to obtain the global feature vector of the infant's specific sleeping position.

[0135] The contour features, joint point coordinate features and sleeping posture vectors of different sleeping postures are stored in a sleeping posture feature semantic library to construct the feature library.

[0136] The infant sleeping scenes are divided into supine, left side, right side and prone. First, the data of the monitored infant in different sleeping postures are collected, and the contour features and skeleton features of different infant sleeping postures are extracted. Then the features are stored as an infant sleeping posture feature library. When the infant sleeping data is captured in actual monitoring, the infrared contour and posture skeleton are also used as feature information to compare with the vectors in the infant sleeping posture feature library to identify the sleeping posture category.

[0137] Feature extraction is performed on the input features to be identified, and then the features in the feature library are used to perform contour feature similarity matching, spatial similarity matching and global vector matching to identify the current sleeping posture.

[0138] Another embodiment of the present invention provides a device for detecting a baby's sleeping posture. Figure 6 , is a schematic diagram of the structure of a baby sleeping posture detection device provided by an embodiment of the present invention, the device comprising:

[0139] An image acquisition module, used for acquiring image data of the baby;

[0140] A feature extraction module, used to identify contour features and joint point coordinate features in the image data using a preset feature extraction model;

[0141] The sleeping posture recognition module is used to determine the current sleeping posture according to the matching results of the contour features, the joint point coordinate features and the features of different sleeping postures in a preset feature library.

[0142] It should be noted that the infant sleeping posture detection device provided in the embodiment of the present invention is used to execute all the process steps of the infant sleeping posture detection method in the above embodiment, and the working principles and beneficial effects of the two correspond one to one, so they will not be repeated.

[0143] See also Figure 7 , is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a baby sleep posture detection program. When the processor executes the computer program, the steps in each of the above-mentioned baby sleep posture detection method embodiments are implemented, such as Figure 1 Alternatively, the processor implements the functions of each module in the above-mentioned device embodiments when executing the computer program.

[0144] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the terminal device. For example, the computer program may be divided into various modules, and the specific functions of each module are not described in detail again.

[0145] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the diagram, or may combine certain components, or different components. For example, the terminal device may also include an input / output device, a network access device, a bus, etc.

[0146] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0147] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0148] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0149] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0150] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting a baby's sleeping posture, characterized in that: The method comprises: Collecting image data of the infant; Using a preset feature extraction model to identify contour features and joint point coordinate features in the image data; The current sleeping posture is determined according to the matching results of the contour features, the joint point coordinate features and the features of different sleeping postures in a preset feature library.

2. The method for detecting the sleeping posture of an infant according to claim 1, wherein: Determining the current sleeping posture according to the matching results of the contour feature, the joint point coordinate feature and the features of different sleeping postures in a preset feature library includes: Using a preset first similarity model to calculate the contour similarity between the contour feature and the standard contour features of different sleeping postures in the feature library, and determining a first sleeping posture with the highest contour similarity; Using a preset second similarity model to calculate the spatial similarity between the joint point coordinate features and the standard joint point coordinate features of different sleeping postures in the feature library, and determining a second sleeping posture with the highest spatial similarity; When the first sleeping position is the same as the second sleeping position, the first sleeping position or the second sleeping position is determined as the current sleeping position.

3. The method for detecting the sleeping posture of an infant according to claim 2, wherein: Determining the current sleeping posture according to the matching results of the contour feature, the joint point coordinate feature and the features of different sleeping postures in a preset feature library, further comprising: When the first sleeping posture is different from the second sleeping posture, inputting the contour feature and the joint point coordinate feature into a preset fusion network to output a global feature vector; The vector distance between the global feature vector and the sleeping posture vectors of different sleeping postures in the feature library is calculated, and the sleeping posture of the sleeping posture vector with the smallest vector distance is determined as the current sleeping posture.

4. The method for detecting the sleeping posture of an infant according to claim 1, wherein: Using a preset feature extraction model to identify contour features and joint point coordinate features in the image data includes: Using an hourglass network to collect features of the infrared image in the image data and extract the contour features; The hourglass network is used to collect features of the RGB image in the image data to extract the coordinate features of the joint points.

5. The method for detecting the sleeping posture of an infant according to claim 2, wherein: The contour similarity Among them, L t represents the contour feature vector extracted at time t, represents the contour feature vectors of different sleeping postures in the feature library, · represents the dot product of the vector, and ||·|| represents the norm of the vector.

6. The method for detecting the sleeping posture of an infant according to claim 2, wherein: The spatial similarity Where i represents the i-th joint point in the joint point coordinate feature, d i represents the Euclidean distance between the joint point coordinate feature and the i-th joint point in the contour feature vector under different sleeping postures in the feature library, S represents the area of ​​the frame where the human skeleton is located, σ i represents the normalization factor of the i-th joint point, v i Indicates whether the i-th joint point is visible, δ(*) represents the visible point screening parameter, δ=1 if visible, otherwise 0.

7. The method for detecting the sleeping posture of an infant according to claim 1, wherein: The process of establishing the feature library includes: Obtain image data of babies in different sleeping positions; The feature extraction model is used to identify contour features and joint point coordinate features in image data of different sleeping postures; Using a preset fusion network to fuse the contour features and the joint point coordinate features to determine sleeping posture vectors of different sleeping postures; The feature library is constructed according to the contour features, joint point coordinate features and sleeping posture vectors of different sleeping postures.

8. A device for detecting a baby's sleeping posture, characterized in that: The device comprises: An image acquisition module, used for acquiring image data of the baby; A feature extraction module, used to identify contour features and joint point coordinate features in the image data using a preset feature extraction model; The sleeping posture recognition module is used to determine the current sleeping posture according to the matching results of the contour features, the joint point coordinate features and the features of different sleeping postures in a preset feature library.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method for detecting the sleeping posture of an infant as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for detecting the sleeping posture of an infant according to any one of claims 1 to 7.

Images