Infant sleeping posture recognition method based on double models
By combining the improved Yolov5 network model, Openpose and GoogleNet network model, and using Bayesian classifier methods, the problem of difficult balance between accuracy, real-time and false alarm rates in existing infant sleeping posture recognition technology is solved, achieving higher recognition accuracy and robustness.
Patent Information
- Application Number
- CN202311636871.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-01
- Publication Date
- 2025-03-07
AI Technical Summary
The existing infant sleeping position recognition technology is difficult to maintain a balance between accuracy, real-time and false alarm rate, resulting in unsatisfactory detection results.
The baby sleeping posture recognition method based on the dual model was adopted, combined with the improved Yolov5 network model and Openpose to extract the baby skeleton, and the sleeping posture recognition was performed through the GoogleNet network model. Finally, the recognition results of the dual model were input into the Bayesian classifier for final judgment.
Through the dual-model integration method, the robustness and accuracy of classification performance are improved, the false alarm rate is reduced, and the accurate identification of infant sleeping positions is achieved.
Smart Images
Figure CN119580340A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for recognizing a baby's sleeping posture based on a dual model. Background Art
[0002] With the continuous development of society, the field of infant monitoring has also ushered in the innovation of Internet technology. In this digital age, infant monitoring has gradually become the focus of parents. Among them, Sudden Infant Death Syndrome, also known as "Sudden Infant Death Syndrome" in English, refers to the sudden death of infants during sleep. Prone sleeping is very likely to increase the rate of sudden death. Side sleeping also has the same danger. Supine sleeping can reduce the risk of SIDS.
[0003] Current baby sleeping position recognition technology mainly uses technologies such as image processing, computer vision and deep learning. It provides information about the baby's safety and comfort by monitoring the baby's sleeping position. There are three main ways. One is to provide intuitive picture information through a camera monitoring system. It has good adaptability to the recognition of various sleeping positions, but is easily affected by factors such as light and camera angle; the second is sensor technology methods, which can provide real-time sleeping posture data and are sensitive to body movements and posture changes, but the sensors may cause discomfort, and attention should be paid to the layout on the crib. Some sensors may need to be replaced regularly; the last is deep learning technology, which can learn and recognize complex image features and has strong adaptability. It requires a large amount of labeled data for training and may have high demands on hardware resources and computing resources.
[0004] With the popularity of machine learning and deep learning, researchers continue to explore new methods to improve posture recognition technology. The detection effect of posture recognition has been significantly improved, but there are still some problems. The existing models for monitoring often cannot maintain a balance between accuracy, real-time performance and false alarm rate. In view of the above problems, the present invention proposes a method for infant sleeping posture recognition based on dual models. The method combines the advantages of both models in terms of fast target positioning and accurate recognition capabilities, forming a parallel system to reduce the false alarm rate. Summary of the invention
[0005] The present invention provides a method for identifying infant sleeping posture based on a dual model in order to solve the problems existing in the prior art.
[0006] The technical solutions adopted in the present invention are:
[0007] A method for recognizing a baby's sleeping position based on a dual model comprises the following steps:
[0008] S1) Create a dataset with pictures of babies sleeping.
[0009] S2) Recognize the sleeping posture of babies in all pictures in the dataset by improving the Yolov5 network model;
[0010] S3) Extract the baby skeletons on all the images in the dataset through Openpose;
[0011] S4) Recognizing the sleeping posture of the infant skeleton by using the GoogleNet network model;
[0012] S5) inputting the recognition results of the improved Yolov5 network model and the GoogleNet network model into a Bayesian classifier to make a final judgment on the sleeping position of the infant;
[0013] S6) Setting evaluation indicators, and using the evaluation indicators to evaluate the recognition results of the improved Yolov5 network model and the GoogleNet network model.
[0014] Furthermore, in step S1), web crawler technology is used to crawl baby sleeping pictures. After downloading, data sorting and cleaning are performed, duplicate and irrelevant pictures are deleted, and then all baby sleeping pictures are preprocessed. The preprocessing includes picture size adjustment, picture addition and normalization. After the data set is created, the data set is divided into a training set and a test set.
[0015] Furthermore, in step S2), the improvement of the Yolov5 network model is as follows:
[0016] Add a CA attention mechanism function to the original Yolov5 network model;
[0017] Replace the C3 module in the backbone network of the original Yolov5 network model with the ShuffleNetV2 module.
[0018] Furthermore, in step S3), the process of extracting the infant skeleton by Openpose is as follows:
[0019] 31) Image processing: convert the image format into a format that can be recognized by Openpose;
[0020] 32) Feature extraction: OpenPose uses the Hourglass network as a feature extraction network to extract features from the baby’s body posture in the image;
[0021] 33) Human body posture information detection and output: After feature extraction, the heat map branch and vector field branch of OpenPose work in parallel to obtain the comprehensive posture information of the baby's body, and output the result containing the posture information.
[0022] Furthermore, in step S5), the Bayesian classifier adopts a naive Bayesian classification algorithm, and the expression form of the naive Bayesian classification algorithm is:
[0023]
[0024] Where: Set the feature attribute set of the baby sleeping posture dataset to X{x1,x2,x3...x n}, n is the number, x i To take values, the characteristic attributes here are: the accuracy, recall and average precision of the output of the improved Yolov5 network model and the GoogleNet network model;
[0025] Let c be the set of infant sleeping posture categories, and divide the samples into a certain category, c = {y1, y2, y3...y m}; For the samples to be classified, there are three categories of infant sleeping positions, corresponding to: supine, side-lying, and prone;
[0026] According to Bayes' theorem, the posterior probability of each category is calculated:
[0027]
[0028] P(x|y i )=P(x1|y i )P(x2|y i )…P(x n |y i ), that is, each feature attribute is independent of each other. When a feature has a certain value in the training data, the conditional probability will be zero, affecting the classification result. Laplace smoothing is used to solve the zero probability problem. For a random variable z, its value range is {1, 2, 3…, k}. After m training times, the result is {z (1) ,z (2) ,z (3) ,…,z (m)}, the maximum likelihood estimate is:
[0029]
[0030] After using Laplace smoothing, the calculation formula is:
[0031]
[0032] The improved Yolov5 network model and GoogleNet network model independently process the input data and output their respective accuracy, recall rate and average precision mean.
[0033] Furthermore, in step S6), the evaluation indicators are: accuracy, recall and average precision, which are defined as follows:
[0034]
[0035]
[0036]
[0037]
[0038] Among them, TP means that the model predicts a positive example and the final prediction is correct;
[0039] FP means that the model predicts a positive example, but the result is wrong;
[0040] FN means that the model predicts a negative example and the result is predicted incorrectly. mAP refers to the mean AP of all categories in all images. mAP@0.5 refers to the average precision when the threshold of the IOU loss function is 0.5. The higher the mAP, the higher the model prediction accuracy.
[0041] The present invention has the following beneficial effects:
[0042] The present invention adopts a dual-model integration method to improve the robustness and accuracy of classification performance by integrating the output results of two independent models. The two models model the problem from different perspectives or feature spaces, thereby capturing rich feature information. In order to further optimize the decision-making process of the integrated model, a Bayesian classifier is introduced, and the output results of the two models are used as input features. Through the learning and decision-making of the Bayesian classifier, accurate discrimination of the two-classification problem is finally achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a general route flow chart of the present invention.
[0044] Figure 2 Schematic diagram of the dataset of baby sleeping pictures.
[0045] Figure 3 To improve the structure diagram of Yolov5.
[0046] Figure 4 This is a schematic diagram of the Openpose structure.
[0047] Figure 5 It is the Openpose skeleton diagram.
[0048] Figure 6 Graphs showing the experimental results of a single model and a dual model of the present invention. DETAILED DESCRIPTION
[0049] The present invention will be further described below in conjunction with the accompanying drawings.
[0050] like Figures 1 to 6 The present invention provides a method for recognizing a baby's sleeping posture based on a dual model, comprising the following steps:
[0051] S1) Create a dataset with pictures of babies sleeping.
[0052] S2) Recognize the sleeping posture of babies in all pictures in the dataset by improving the Yolov5 network model;
[0053] S3) Extract the baby skeletons on all the images in the dataset through Openpose;
[0054] S4) Recognizing the sleeping posture of the infant skeleton by using the GoogleNet network model;
[0055] S5) inputting the recognition results of the improved Yolov5 network model and the GoogleNet network model into a Bayesian classifier to make a final judgment on the sleeping position of the infant;
[0056] S6) Setting evaluation indicators, and using the evaluation indicators to evaluate the recognition results of the improved Yolov5 network model and the GoogleNet network model.
[0057] In step S1), a web crawler technology is used to crawl baby sleeping pictures. After downloading, data is sorted and cleaned, duplicate images are deleted, and irrelevant pictures are screened to ensure the quality of the data set. All baby sleeping pictures are preprocessed. The wide application of preprocessing operations includes image size adjustment, normalization, and data augmentation. First, by unifying the image size, ensure that all image inputs are trained on the same scale to improve the consistency and performance of the model. Subsequently, a normalization operation is performed to adjust the pixel value to a specific range, which helps to improve the stability and convergence speed of the model. Further, data augmentation technology is used to expand the training set through random rotation, flipping, scaling and other operations, increase data diversity, and improve the generalization ability of the model.
[0058] Use LabelImg to annotate target detection, mark the location information and category of objects, perform data analysis and visualization through tool annotation, perform feature analysis on the distribution of different categories in the data set, statistics of object sizes, etc., use the annotated data as the training set and test set of the algorithm, perform model training, parameter adjustment and evaluation, and provide key training data for the training of machine learning models.
[0059] Yolov5 recognizes baby sleeping positions:
[0060] (1) The overall structure of YOLOv5 can be divided into three main parts: backbone network, neck structure and head structure. The image is processed by the input layer and sent to the backbone for feature extraction. The backbone obtains feature maps of different sizes, and then the feature fusion network (Neck) fuses these features to generate three feature maps and send them to the prediction head (Head) to complete the object detection.
[0061] (2) The backbone network adopts a combination of Conv module, ShuffleNetV2 module and SPPF module to effectively extract feature information at different levels.
[0062] (3) The Conv module consists of a Conv2d, a BatchNorm2d and a SiLu activation function. The processing flow passes through a Conv2d convolution layer with a convolution kernel of 3 and a step size of 1, followed by BatchNorm2d batch normalization processing, and finally processed by a ReLU activation function.
[0063] (4) The ShuffleNetV2 module is composed of multiple basic units in a certain hierarchical structure. The overall structure includes four stages. The first is a conventional convolutional layer for extracting low-level features. Then, a series of ShuffleNetV2 basic units include operations such as point-by-point convolution, channel reordering, and depth-wise separable convolution to keep the network lightweight while providing sufficient representation capabilities. This is followed by a global average pooling layer to reduce the size of the feature map. The last fully connected layer and Sofxmax layer learn the relationship between the input features and the categories and convert the network's output into a category probability distribution.
[0064] (5) The SPPF module is an improved spatial pyramid pooling layer that integrates multi-scale features and concatenates the feature representations of the same feature map at different scales.
[0065] (6) Through the attention mechanism processing module, the CA attention mechanism function is adopted.
[0066] (7) The Neck structure is actually a feature pyramid (FPN), which combines shallow graphic features with shallow semantic features. The left path interpolates the feature map through upsampling to increase the scale of the feature map to facilitate the fusion of feature maps from the backbone. The right path performs downsampling to obtain feature maps of different scales to better fuse shallow graphic features with deep semantic features.
[0067] (8) The head layer is the Detect module. The network structure of the Detect module is very simple and consists of only three 1×1 convolutions, corresponding to the three detection feature layers, which convert the extracted feature maps into the final output of target detection, including the target category, location and confidence.
[0068] (9) Train the YOLOv5 model so that it can accurately identify the sleeping position of infants based on infant images and output a multidimensional array (BBoxes) of object category, category confidence, box coordinates, width, and height information.
[0069] The process of extracting the baby skeleton through Openpose is:
[0070] (1) Image preprocessing: The input image will first go through preprocessing steps, including image resizing, color space conversion, etc. These steps help convert the input image into a format that the model can process and reduce computational complexity.
[0071] (2) Feature extraction: OpenPose uses the Hourglass network or its variants as the feature extraction network. This network can extract image features at multiple scales to help capture the spatial relationship of human poses.
[0072] (3) Key point detection: After feature extraction, the heat map branch and the vector field branch network work in parallel. The heat map branch generates a key point probability heat map, which indicates the probability that a certain area in the image may contain a specific key point. The vector branch outputs a set of vector fields connecting the key points to form a tensor.
[0073] (4) Heatmap fusion: The position fusion of key points is achieved by multiplying the probability heatmap generated by the heatmap branch with the output of the vector field branch point by point.
[0074] (5) Key point connection: By connecting adjacent key points, a complete human body posture is formed. This step usually adopts a heuristic method to connect key points based on prior knowledge to obtain more accurate human body posture information.
[0075] (6) Pose output: Finally, OpenPose generates an output result containing the poses of a human body or a group of human bodies.
[0076] The principle is as follows: OpenPose inputs an image or video, initializes and fine-tunes the first 10 layers of the backbone network VGG-19, and generates a set of feature maps F as the input of the next stage. In this stage, a set of joint point confidence maps PCM (Part Confidence Maps) and part affinity fields PAF (Part Affinity Fields) are generated. PCM represents the Gaussian response of the pixel at the joint point, that is, S1 =ρ 1 (F), PAF represents the direction of the pixel in the skeleton, that is, L 1 =φ 1 (F), where ρ 1 and φ 1 The CNNs for inference in the first stage, each subsequent stage will combine the prediction of the previous stage with the feature F to produce a refined prediction. In the process of forward and backward propagation, OpenPose will output a feature map, each pixel corresponds to a possible joint point, by decoding and thresholding the feature map, and then according to the spatial relationship between the joint points, the joint point coordinates of the human body posture are obtained, and the human body posture skeleton can be formed through the connection between the joint points.
[0077] Sleeping posture recognition of infant skeleton images using GoogleNet network
[0078] (1) Input layer: The infant skeleton image is resized to 224×224 and input into the GoogleNet network input layer.
[0079] Conv layer: includes two convolutional layers, Conv 1×1 and Conv 3×3. First, it is processed by a convolutional layer with a convolution kernel of 1, a stride of 2, and 64 channels. The output is 112×112×64. After convolution, a ReLU operation is performed. After 3×3 maxpooling (step size is 2), the output is ((112-3+1) / 2)+1=56, that is, 56×56×64, and then a ReLU operation is performed.
[0080] (2) Inception module: The Inception module is the core component of GoogLeNet. It uses 1×1, 3×3 and 5×5 convolution kernels to perform channel linear transformation, capture spatial structure and feature extraction respectively, and then performs maximum pooling operation on the feature output to extract the significant features of the image and capture the main information. Finally, the outputs of different branches are connected by splicing in the channel direction. In this way, the output of the module contains multi-scale feature information extracted from different branches.
[0081] (3) Global average pooling layer: Take the average of all elements in each feature map to obtain a feature map of fixed size. This helps reduce the number of parameters and improve computational efficiency.
[0082] (4) Fully connected layer: A 1x1 convolutional layer may be used before the fully connected layer to reduce the dimension and reduce the computational complexity. The fully connected layer is used to map the output of the convolutional layer to the final category score.
[0083] (5) Softmax layer: The last layer is the Softmax layer, which converts the output of the network into category probability distribution.
[0084] Input the dual model results into the Bayesian classifier to make the final sleeping posture judgment
[0085] Based on the ideas of redundant systems and probabilistic statistics, the present invention uses dual models. In order to improve reliability, the output of the improved YOLOv5 model and the output of OpenPose are trained through a Bayesian classifier. The posterior probability is estimated as accurately as possible based on a limited set of training samples, and the Bayesian decision criterion is used to minimize the decision risk.
[0086] The Naive Bayesian classification algorithm is the most basic type of Bayesian classification algorithm. Its principle is that when a thing has some attribute conditions, the probability that the thing belongs to A is greater than the probability that it belongs to B, then the thing is judged to belong to A. The Naive Bayesian classifier is expressed as:
[0087]
[0088] Where: Set the feature attribute set of the baby sleeping posture dataset to X{x1,x2,x3...x n}, n is the number, x i To take values, the characteristic attributes here are: the accuracy, recall and average precision of the output of the improved Yolov5 network model and the GoogleNet network model;
[0089] Let c be the set of infant sleeping posture categories, and divide the samples into a certain category, c = {y1, y2, y3...y m}; For the samples to be classified, there are three categories of infant sleeping positions, corresponding to: supine, side-lying, and prone;
[0090] According to Bayes' theorem, the posterior probability of each category is calculated:
[0091]
[0092] P(x|y i )=P(x1|y i )P(x2|y i )…P(x n |y i ), that is, each feature attribute is independent of each other. When a feature has a certain value in the training data, the conditional probability will be zero, affecting the classification result. Laplace smoothing is used to solve the zero probability problem. For a random variable z, its value range is {1, 2, 3…, k}. After m training times, the result is {z (1) ,z (2) ,z (3) ,…,z (m)}, the maximum likelihood estimate is:
[0093]
[0094] After using Laplace smoothing, the calculation formula is:
[0095]
[0096] The two models process the input data independently and output their own accuracy, recall, and mean average precision evaluation indicators. Subsequently, the outputs of the two models are concatenated to form a new feature vector that contains the output information of both models. By using the labeled training data, a Bayesian classifier is trained that can calculate the posterior probability of each category based on the input features. In the prediction phase, the classifier makes the final binary classification decision for the new input data based on these probabilities.
[0097] This paper uses accuracy, recall, and mean average precision (mAP) as evaluation indicators of the model, and the relevant definitions are as follows:
[0098]
[0099]
[0100]
[0101]
[0102] Among them, TP means that the model predicts a positive example and the final prediction is correct;
[0103] FP means that the model predicts a positive example, but the result is wrong;
[0104] FN means that the model predicts a negative example and the result is predicted incorrectly. mAP refers to the mean AP of all categories in all images. mAP@0.5 refers to the average precision when the threshold of the IOU loss function is 0.5. The higher the mAP, the higher the model prediction accuracy.
[0105] The present invention adopts a dual-model integration method to improve the robustness and accuracy of classification performance by integrating the output results of two independent models. The two models model the problem from different perspectives or feature spaces, thereby capturing rich feature information.
[0106] In order to further optimize the decision-making process of the integrated model, a Bayesian classifier is introduced, and the output results of the two models are used as input features. Through the learning and decision-making of the Bayesian classifier, accurate discrimination of the binary classification problem is finally achieved. The integration strategy of the present invention not only improves the overall performance, but also has better robustness in dealing with possible deviations or errors in different models. According to the experimental results (such as Figure 6 ), through the synergy of dual models and Bayesian classifier, the effect of classification tasks was successfully improved, providing an effective solution to complex problems.
[0107] The above description is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be regarded as within the protection scope of the present invention.
Claims
1. A method for recognizing infant sleeping posture based on dual models, characterized in that: The steps include: S1) Create a dataset with pictures of infant sleeping positions; S2) Recognize the sleeping posture of babies in all pictures in the dataset by improving the Yolov5 network model; S3) Extract the baby skeletons on all the images in the dataset through Openpose; S4) Recognizing the sleeping posture of the infant skeleton by using the GoogleNet network model; S5) inputting the recognition results of the improved Yolov5 network model and the GoogleNet network model into a Bayesian classifier to make a final judgment on the sleeping position of the infant; S6) Setting evaluation indicators, and using the evaluation indicators to evaluate the recognition results of the improved Yolov5 network model and the GoogleNet network model.
2. The method for recognizing infant sleeping posture based on dual models as claimed in claim 1, characterized in that: In step S1), web crawler technology is used to crawl baby sleeping pictures. After downloading, data sorting and cleaning are performed to delete duplicate and irrelevant pictures. Then all baby sleeping pictures are preprocessed. After the data set is created, the data set is divided into a training set and a test set.
3. The method for recognizing infant sleeping posture based on dual models as claimed in claim 1, characterized in that: In step S2), the improvement of the Yolov5 network model is: Add a CA attention mechanism function to the original Yolov5 network model; Replace the C3 module in the backbone network of the original Yolov5 network model with the ShuffleNetV2 module.
4. The method for recognizing infant sleeping posture based on dual models as claimed in claim 1, characterized in that: In step S3), the process of extracting the baby skeleton by Openpose is as follows: 31) Image processing: convert the image format into a format that can be recognized by Openpose; 32) Feature extraction: OpenPose uses the Hourglass network as a feature extraction network to extract features from the baby’s body posture in the image; 33) Human body posture information detection and output: After feature extraction, the heat map branch and vector field branch of OpenPose work in parallel to obtain the comprehensive posture information of the baby's body, and output the result containing the posture information.
5. The method for recognizing infant sleeping posture based on dual models as claimed in claim 1, characterized in that: In step S5), the Bayesian classifier adopts the naive Bayesian classification algorithm, and the expression form of the naive Bayesian classification algorithm is: Where: Set the feature attribute set of the baby sleeping posture dataset to X{x1,x2,x3...x n }, n is the number, x i To take values, the characteristic attributes here are: the accuracy, recall and average precision of the output of the improved Yolov5 network model and the GoogleNet network model; Let c be the set of infant sleeping posture categories, and divide the samples into a certain category, c = {y1, y2, y3...y m }; For the samples to be classified, there are three categories of infant sleeping positions, corresponding to: supine, side-lying, and prone; According to Bayes' theorem, the posterior probability of each category is calculated: P(x|y i )=P(x1|y i )P(x2|y i )…P(x n |y i ), that is, each feature attribute is independent of each other. When a feature has a certain value in the training data, the conditional probability will be zero, affecting the classification result. Laplace smoothing is used to solve the zero probability problem. For a random variable z, its value range is {1, 2, 3…, k}. After m training times, the result is {z (1) ,z (2) ,z (3) ,…,z (m) }, the maximum likelihood estimate is: After using Laplace smoothing, the calculation formula is: The improved Yolov5 network model and GoogleNet network model independently process the input data and output their respective accuracy, recall rate and average precision mean.
6. The method for recognizing infant sleeping posture based on dual models as claimed in claim 1, characterized in that: In step S6), the evaluation indicators are: accuracy, recall and average precision, which are defined as follows: Among them, TP means that the model predicts a positive example and the final prediction is correct; FP means that the model predicts a positive example, but the result is wrong; FN means that the model predicts a negative example and the result is predicted incorrectly. mAP refers to the mean AP of all categories in all images. mAP@0.5 refers to the average precision when the threshold of the IOU loss function is 0.
5. The higher the mAP, the higher the model prediction accuracy.