Human body posture analysis method and system

Abstract

The embodiment of the present invention discloses a method and system for analyzing human posture, which obtains a certain number of human posture images, pre-processes the human posture images, and obtains training images containing only human body frames; constructs a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer, and an output layer connected in sequence; takes the human body frame as input and the human posture prediction result as output, trains the posture analysis model, and obtains a trained posture analysis model. The human posture analysis method solves the problems of inaccurate detection and repeated detection in the analysis of human posture in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system, electronic equipment and storage medium for analyzing human body posture. Background Art

[0002] Human keypoints detection, also known as human posture estimation, is the prediction of human posture and key points in an image. Human posture estimation is a relatively basic task in computer vision and a prerequisite for human motion recognition, behavior analysis, human-computer interaction, etc. In general, human keypoints detection can be subdivided into single / multi-person keypoint detection and 2D / 3D keypoint detection. At the same time, some algorithms will track key points after completing key point detection, which is also called human posture tracking.

[0003] The existing analysis of human posture relies on the suggestions made by the detection algorithm, which may lead to inaccurate detection and repeated detection;

[0004] There is an urgent need for a method that can accurately and efficiently analyze human posture. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method, system, electronic device and storage medium for analyzing human posture, so as to solve the problems of inaccurate detection and repeated detection in the analysis of human posture in the prior art.

[0006] To achieve the above object, an embodiment of the present invention provides a method for analyzing human body posture, which specifically includes:

[0007] Acquire a certain number of human body posture images, and preprocess the human body posture images to obtain training images containing only human body frames;

[0008] Constructing a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence;

[0009] The key points corresponding to the human body frame are obtained through the convolution layer, the spatial feature data of each key point is obtained through the spatial feature extraction module, the temporal feature data of each key point is obtained through the temporal feature extraction module, the spatial feature data and the temporal feature data are fused through the fusion module to obtain fused data, the fused data is classified through the fully connected layer to obtain a classification result, and the human body posture is predicted through the classification result and the fused data to obtain a human body posture prediction result;

[0010] The human body frame is used as input and the human body posture prediction result is used as output to train the posture analysis model to obtain a trained posture analysis model.

[0011] On the basis of the above technical solution, the present invention can also be improved as follows:

[0012] Furthermore, the step of acquiring a certain number of human posture images and preprocessing the human posture images to obtain training images containing only human body frames includes:

[0013] Detecting the human body in the human body posture image by using a target detection algorithm to obtain a human body frame of each human body;

[0014] Determine whether the human body frame is qualified, and when the human body frame is unqualified, pre-process the human body frame until the human body frame is qualified;

[0015] Based on the skeleton point coordinate file, each skeleton point of each human body is labeled to obtain a labeled training image containing only the human body frame.

[0016] Furthermore, the obtaining of the time feature data of each key point by the time feature extraction module includes:

[0017] Calculate the time characteristic data of each key point by formula (1);

[0018]

[0019] In the formula, h is the time feature data of each key point, X (l) is the network input of the current layer, f i (X) is the convolution operation, σ is the sigmoid nonlinear activation function, is the convolution operator.

[0020] Furthermore, the method of predicting a human body posture by using the classification result and the fusion data to obtain a human body posture prediction result includes:

[0021] Calculate the human body posture prediction result through formula (2);

[0022]

[0023] In the formula, is the human posture prediction result, L is the classification result, z 2 is the spatial feature data of the key point, T is the temporal feature data, is the fused data, W is the learnable parameter of the output layer, and b is the bias term of the output layer.

[0024] Furthermore, the fusion data is classified by the fully connected layer to obtain a classification result, including:

[0025] Obtaining the X-axis coordinate and the Y-axis coordinate of each of the key points, and obtaining the spatial feature data of each of the key points based on the X-axis coordinate and the Y-axis coordinate;

[0026] Based on the spatial feature data and the temporal feature data of the key points, a set of human skeleton key points corresponding to each part of the human body in the human body frame is obtained;

[0027] Calculating the probability value that the fused data belongs to the set of human skeleton key points corresponding to each part of the human body;

[0028] A set of human skeleton key points corresponding to each part of the human body is determined based on the probability value.

[0029] Furthermore, the human body posture analysis method further includes:

[0030] Acquire a human body posture image to be analyzed, and preprocess the human body posture image to be analyzed to obtain an image to be analyzed that only contains a human body frame;

[0031] The image to be analyzed is input into the trained posture analysis model, and the human posture prediction result is output.

[0032] Furthermore, the human body posture analysis method further includes:

[0033] Determining whether the human body posture prediction result is qualified, and when the human body posture prediction result is qualified, determining a corresponding execution instruction based on the human body posture prediction result;

[0034] When the human body posture prediction result is unqualified, a corresponding early warning plan is determined based on the human body posture prediction result, and early warning measures are taken based on the early warning plan.

[0035] A human body posture analysis system, comprising:

[0036] An acquisition module, used to acquire a certain number of human posture images;

[0037] A preprocessing module, used for preprocessing the human body posture image to obtain a training image containing only a human body frame;

[0038] A construction module is used to construct a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence; the key points corresponding to the human body frame are obtained through the convolution layer, the spatial feature data of each key point is obtained through the spatial feature extraction module, the temporal feature data of each key point is obtained through the temporal feature extraction module, the spatial feature data and the temporal feature data are fused through the fusion module to obtain fused data, the fused data is classified through the fully connected layer to obtain a classification result, and the human body posture is predicted through the classification result and the fused data to obtain a human body posture prediction result;

[0039] The training module is used to take the human body frame as input and the human body posture prediction result as output, train the posture analysis model, and obtain a trained posture analysis model.

[0040] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0041] A non-transitory computer-readable storage medium stores a computer program, which implements the steps of the method when executed by a processor.

[0042] The embodiments of the present invention have the following advantages:

[0043] The method for analyzing human body posture of the present invention comprises the following steps: obtaining a certain number of human body posture images, preprocessing the human body posture images, and obtaining training images containing only a human body frame; constructing a posture analysis model, wherein the posture analysis model comprises a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer, and an output layer connected in sequence; obtaining key points corresponding to the human body frame through the convolution layer, obtaining spatial feature data of each key point through the spatial feature extraction module, obtaining temporal feature data of each key point through the temporal feature extraction module, fusing the spatial feature data and the temporal feature data through the fusion module, obtaining fused data, classifying the fused data through the fully connected layer, obtaining a classification result, predicting human body posture through the classification result and the fusion data, and obtaining a human body posture prediction result; taking the human body frame as input and the human body posture prediction result as output, training the posture analysis model, and obtaining a trained posture analysis model; and solving the problems of inaccurate detection and repeated detection in the analysis of human body posture in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0045] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0046] Figure 1 is a flow chart of the method for analyzing human body posture of the present invention;

[0047] Figure 2 is a schematic diagram of the human body posture analysis system of the present invention;

[0048] Figure 3 This is a schematic diagram of the physical structure of the electronic device provided by the present invention.

[0049] The accompanying drawings are denoted as follows:

[0050] Acquisition module 10, preprocessing module 20, construction module 30, training module 40, electronic device 50, processor 501, memory 502, bus 503. DETAILED DESCRIPTION

[0051] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example

[0053] Figure 1 Flow chart of the embodiment of the method for analyzing human body posture of the present invention, as shown in FIG. Figure 1 As shown, a method for analyzing human body posture provided by an embodiment of the present invention comprises the following steps:

[0054] S101, obtaining a certain number of human body posture images, preprocessing the human body posture images, and obtaining training images containing only human body frames;

[0055] Specifically, a human body frame existing in the human body posture image is obtained through a target detection algorithm, and each person existing in the human body posture image is detected, and the number of the human body frames corresponds to the number of people existing in the detection image; a training image is determined based on the human body frame, and then human skeleton key point detection is performed based on the human body frame.

[0056] Detecting the human body in the human body posture image by using a target detection algorithm to obtain a human body frame of each human body;

[0057] Determine whether the human body frame is qualified, and when the human body frame is unqualified, pre-process the human body frame until the human body frame is qualified;

[0058] Based on the skeleton point coordinate file, each skeleton point of each human body is labeled to obtain a labeled training image containing only the human body frame.

[0059] A preferred embodiment of detecting each person by a target detection algorithm is:

[0060] The first step is to sample data according to the input image to be detected and extract feature maps; the algorithm's predicted input image size is 640x640, a three-channel RGB image, the backbone network uses CSPNeXt, and CSPNeXtPAFPN layers for multi-scale fusion feature fusion, and the CSPNeXt, CSPNeXtPAFPN layers are downsampled 8 times, 16 times, and 32 times to obtain feature maps of three sizes: 80x80, 40x40, and 20x20. At this time, after calculation, the number of channels of the three layers remains at 256. After the bbox head layer, two calculation branches are obtained, namely: first: object detection category prediction branch, the number of channels 256 is changed to the number of num_class (such as the coco dataset has 80 categories, the specific number is based on the actual number of categories), 80 corresponds to the number of all categories, and second: bounding box regression branch, the number of channels will also be changed from 256 to 4, where 4 refers to the four point coordinates of the predicted rectangular box.

[0061] The second step is to make regression predictions based on the feature map: initialize three grids according to the feature map size, with sizes of 6400 (80x80), 1600 (40x40), and 400 (20x20), respectively. For example, the shape of the first layer is torch.Size([6400,2]), and the last dimension is 2, which is the horizontal and vertical coordinates of the grid points, and 6400 represents the number of grid points in the current feature layer. After the predict_by_feat_single function, the features of a single image extracted from the head are converted into bbox result inputs, and three lists cls_score_list, bbox_pred_list, and mlvl_priors are obtained. Then, the three feature layers are traversed respectively, and the class category prediction branch and the bbox regression branch are processed respectively. Taking the first layer as an example, the dimension of the bbox prediction branch [4, 80, 80] is transformed to [6400, 4], and the dimension of the category prediction branch [80, 80, 80] is changed to [6400, 80], and normalized to ensure that the category confidence is between 0-1. First, use an nms_pre operation to filter most of the prediction results with low confidence (for example, if the score_thr threshold is set to 0.05, the results with current prediction confidence lower than 0.05 are removed), and then the bbox coordinates, the coordinates of the grid, the confidence, and the label information are obtained. After traversing the three feature layers, the four information obtained from the three layers are integrated and put into the result list. Finally, the prediction results of the network are mapped to the entire image, and the coordinate values ​​of the bbox in the entire image are obtained for nms operation.

[0062] Step 3: Data normalization and prediction results: The final predicted return value is the detection result of each image to be detected after post-processing, including classification confidence, box labels, the four coordinates of the box [4, 80, 80] dimension is transformed to [6400, 4], the category prediction [80, 80, 80] is changed to [6400, 80], and it is normalized to ensure that the category confidence is between 0 and 1. Finally, the predicted cls category, the object's bbox, and the actual confidence result are obtained.

[0063] S102, constructing a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence;

[0064] Specifically, the key points corresponding to the human body frame are obtained through the convolution layer, the spatial feature data of each key point is obtained through the spatial feature extraction module, the temporal feature data of each key point is obtained through the temporal feature extraction module, the spatial feature data and the temporal feature data are fused through the fusion module to obtain fused data, the fused data is classified through the fully connected layer to obtain a classification result, and the human body posture is predicted through the classification result and the fused data to obtain a human body posture prediction result;

[0065] The step of obtaining the time feature data of each key point through the time feature extraction module includes:

[0066] Calculate the time characteristic data of each key point by formula (1);

[0067]

[0068] In the formula, h is the time feature data of each key point, X (l) is the network input of the current layer, f i (X) is the convolution operation, σ is the sigmoid nonlinear activation function, is the convolution operator.

[0069] The method of predicting the human body posture by using the classification result and the fusion data to obtain the human body posture prediction result includes:

[0070] Calculate the human body posture prediction result through formula (2);

[0071]

[0072] In the formula, is the human posture prediction result, L is the classification result, z 2 is the spatial feature data of the key point, T is the temporal feature data, is the fused data, W is the learnable parameter of the output layer, and b is the bias term of the output layer.

[0073] The classifying the fused data through the fully connected layer to obtain a classification result includes:

[0074] Obtaining the X-axis coordinate and the Y-axis coordinate of each of the key points, and obtaining the spatial feature data of each of the key points based on the X-axis coordinate and the Y-axis coordinate;

[0075] The invention divides the horizontal and vertical axes into bins with equal width numbers and discretizes the continuous coordinates into integer bin labels. The model is then trained to predict the bin in which the keypoint is located. By using a large number of bins, the quantization error can be reduced to the sub-pixel level; Simple structure: Due to this novel formulation, SimCC has a very simple structure, using a 1×1 convolutional layer to convert the features extracted by the backbone into vectorized keypoint representations, and using two fully connected layers to perform classification separately;

[0076] SimCC proposes a Gaussian label smoothing strategy to replace one-hot labels with Gaussian distributed soft labels centered on the ground-truth bin, which incorporates inductive biases in model training and brings significant performance improvements.

[0077] Based on the spatial feature data and the temporal feature data of the key points, a set of human skeleton key points corresponding to each part of the human body in the human body frame is obtained;

[0078] Calculating the probability value that the fused data belongs to the set of human skeleton key points corresponding to each part of the human body;

[0079] Each type of fused data is represented by a probability graph, and each fused data in the image is given a probability, which indicates the probability that the fused data belongs to the key point of the corresponding category. Naturally, the closer the fused data is to the key point, the closer the probability is to 1, and the farther the fused data is from the key point, the closer the probability is to 0. This can be simulated by corresponding functions, such as Gaussian. If the same fused data has different distances from different key points, that is, the probability of the position relative to different key points is different, then Max or Average can be used.

[0080] All key points in the image are detected, and then all key points are clustered into different individuals through relevant strategies. Representative algorithms for modeling the relationship between key points include PAF, Associative Embedding, Part Segmentation, and Mid-Range offsets. Currently, the best effect on the MSCOCO dataset is 68.7%.

[0081] A set of human skeleton key points corresponding to each part of the human body is determined based on the probability value.

[0082] S103, taking the human body frame as input and the human body posture prediction result as output, training the posture analysis model to obtain a trained posture analysis model.

[0083] Specifically, a certain number of training images are obtained, and the certain number of training images are divided into a training set, a verification set, and a test set;

[0084] Inputting the training set into the posture analysis model for training to obtain a trained posture analysis model;

[0085] Performing a performance evaluation on the trained posture analysis model based on the verification set to obtain a posture analysis model that meets performance requirements;

[0086] An analysis result of the posture analysis model that meets the performance condition is evaluated based on the test set to obtain an evaluation index corresponding to the posture analysis model.

[0087] Based on the validation set, the trained posture analysis model is evaluated for performance to obtain a posture analysis model that meets the performance conditions; based on the test set, the similarity calculation result of the posture analysis model that meets the performance conditions is evaluated to obtain an evaluation index corresponding to the posture analysis model. The posture analysis model is evaluated for performance to obtain a percentage score (i.e., the highest score is 100 points and the lowest score is 0 points), and a posture analysis model with a score greater than a set value is determined based on the percentage score, for example, a posture analysis model with a score greater than 90 points is a posture analysis model that meets the performance conditions;

[0088] The posture analysis model that meets the performance conditions is evaluated by calculating the evaluation index to obtain the evaluation index of the posture analysis model, and the evaluation value corresponding to each evaluation index is calculated to obtain the evaluation value used to represent the ability value of the posture analysis model at the evaluation index.

[0089] For the key point detection algorithm, the first thing to note is that the local information of the key points is weakly distinguishable, that is, the same local area can easily appear in the background to cause confusion, so a larger receptive field area needs to be considered; secondly, the difficulty of detecting different key points of the human body is different. The detection of key points such as waist and legs is obviously more difficult than the detection of key points near the head, so different key points may need to be treated differently; the present invention applies deep learning to human posture analysis, and uses convolutional layers to express texture information and spatial information. The main network structure is divided into multiple stages, the first stage will produce preliminary key point detection results, and the next few stages will use the predicted output of the previous stage and the features extracted from the original image as input to further improve the key point detection effect.

[0090] The frame skipping detection mechanism in BlazePose is used, where human detection is performed every K frames, and in the interval frames, the bounding box is generated based on the last pose estimation result. In order to achieve smooth prediction of frames, the pose NMS based on OKS and OneEuro filter are used in the post-processing stage.

[0091] The method for analyzing human posture comprises the following steps: obtaining a certain number of human posture images, preprocessing the human posture images, and obtaining training images containing only a human frame; constructing a posture analysis model, wherein the posture analysis model comprises a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer, and an output layer connected in sequence; obtaining key points corresponding to the human frame through the convolution layer, obtaining spatial feature data of each key point through the spatial feature extraction module, obtaining temporal feature data of each key point through the temporal feature extraction module, fusing the spatial feature data and the temporal feature data through the fusion module, obtaining fused data, classifying the fused data through the fully connected layer, obtaining a classification result, predicting human posture through the classification result and the fusion data, and obtaining a human posture prediction result; taking the human frame as input and the human posture prediction result as output, training the posture analysis model, and obtaining a trained posture analysis model. The method solves the problems of inaccurate detection and repeated detection in the analysis of human posture in the prior art.

[0092] Figure 2 FIG. 1 is a diagram showing an embodiment of a human body posture analysis system according to the present invention; Figure 2 As shown, a human body posture analysis system provided by an embodiment of the present invention comprises the following steps:

[0093] An acquisition module 10 is used to acquire a certain number of human posture images;

[0094] A preprocessing module 20 is used to preprocess the human body posture image to obtain a training image containing only a human body frame;

[0095] The pre-processing module 20 is also used for:

[0096] Detecting the human body in the human body posture image by using a target detection algorithm to obtain a human body frame of each human body;

[0097] Determine whether the human body frame is qualified, and when the human body frame is unqualified, pre-process the human body frame until the human body frame is qualified;

[0098] Based on the skeleton point coordinate file, each skeleton point of each human body is labeled to obtain a labeled training image containing only the human body frame.

[0099] A construction module 30 is used to construct a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence; the key points corresponding to the human body frame are obtained through the convolution layer, the spatial feature data of each key point is obtained through the spatial feature extraction module, the temporal feature data of each key point is obtained through the temporal feature extraction module, the spatial feature data and the temporal feature data are fused through the fusion module to obtain fused data, the fused data is classified through the fully connected layer to obtain a classification result, and the human body posture is predicted through the classification result and the fused data to obtain a human body posture prediction result;

[0100] Calculate the time characteristic data of each key point by formula (1);

[0101]

[0102] In the formula, h is the time feature data of each key point, X (l) is the network input of the current layer, f i (X) is the convolution operation, σ is the sigmoid nonlinear activation function, is the convolution operator.

[0103] Calculate the human body posture prediction result through formula (2);

[0104]

[0105] In the formula, is the human posture prediction result, L is the classification result, z 2 is the spatial feature data of the key point, T is the temporal feature data, is the fused data, W is the learnable parameter of the output layer, and b is the bias term of the output layer.

[0106] Obtaining the X-axis coordinate and the Y-axis coordinate of each of the key points, and obtaining the spatial feature data of each of the key points based on the X-axis coordinate and the Y-axis coordinate;

[0107] Based on the spatial feature data and the temporal feature data of the key points, a set of human skeleton key points corresponding to each part of the human body in the human body frame is obtained;

[0108] Calculating the probability value that the fused data belongs to the set of human skeleton key points corresponding to each part of the human body;

[0109] A set of human skeleton key points corresponding to each part of the human body is determined based on the probability value.

[0110] The training module 40 is used to take the human body frame as input and the human body posture prediction result as output, and train the posture analysis model to obtain a trained posture analysis model.

[0111] The human body posture analysis system further comprises an analysis module, and the analysis module is further used for:

[0112] Acquire a human body posture image to be analyzed, and preprocess the human body posture image to be analyzed to obtain an image to be analyzed that only contains a human body frame;

[0113] The image to be analyzed is input into the trained posture analysis model, and the human posture prediction result is output.

[0114] An execution module, used to determine whether the human body posture prediction result is qualified, and when the human body posture prediction result is qualified, determine a corresponding execution instruction based on the human body posture prediction result;

[0115] When the human body posture prediction result is unqualified, a corresponding early warning plan is determined based on the human body posture prediction result, and early warning measures are taken based on the early warning plan.

[0116] The present invention discloses a human posture analysis system, which obtains a certain number of human posture images through an acquisition module 10; preprocesses the human posture images through a preprocessing module 20 to obtain training images containing only a human body frame; constructs a posture analysis model through a construction module 30, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence; obtains key points corresponding to the human body frame through the convolution layer, obtains spatial feature data of each key point through the spatial feature extraction module, obtains temporal feature data of each key point through the temporal feature extraction module, fuses the spatial feature data and the temporal feature data through the fusion module to obtain fused data, classifies the fused data through the fully connected layer to obtain a classification result, predicts human posture through the classification result and the fusion data to obtain a human posture prediction result; takes the human body frame as input and the human body posture prediction result as output through a training module 40, trains the posture analysis model, and obtains a trained posture analysis model. The human posture analysis method solves the problems of inaccurate detection and repeated detection in the analysis of human posture in the prior art.

[0117] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 3 As shown, the electronic device 50 includes: a processor 501 (processor), a memory 502 (memory) and a bus 503;

[0118] The processor 501 and the memory 502 communicate with each other via the bus 503;

[0119] The processor 501 is used to call the program instructions in the memory 502 to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a certain number of human posture images, preprocessing the human posture images, and obtaining training images containing only a human frame; constructing a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer, and an output layer connected in sequence; obtaining key points corresponding to the human frame through the convolution layer, obtaining spatial feature data of each key point through the spatial feature extraction module, obtaining temporal feature data of each key point through the temporal feature extraction module, fusing the spatial feature data and the temporal feature data through the fusion module to obtain fused data, classifying the fused data through the fully connected layer to obtain a classification result, and predicting the human posture through the classification result and the fused data to obtain a human posture prediction result; taking the human frame as input and the human posture prediction result as output, training the posture analysis model to obtain a trained posture analysis model.

[0120] The present embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a certain number of human posture images, preprocessing the human posture images, and obtaining training images containing only a human frame; constructing a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer, and an output layer connected in sequence; obtaining key points corresponding to the human frame through the convolution layer, obtaining spatial feature data of each key point through the spatial feature extraction module, obtaining temporal feature data of each key point through the temporal feature extraction module, fusing the spatial feature data and the temporal feature data through the fusion module to obtain fused data, classifying the fused data through the fully connected layer to obtain a classification result, and predicting the human posture through the classification result and the fused data to obtain a human posture prediction result; taking the human frame as input and the human posture prediction result as output, training the posture analysis model to obtain a trained posture analysis model.

[0121] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various storage media that can store program codes.

[0122] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0124] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for analyzing human body posture, characterized in that: The method specifically comprises: Acquire a certain number of human body posture images, and preprocess the human body posture images to obtain training images containing only human body frames; Constructing a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence; The key points corresponding to the human body frame are obtained through the convolution layer, the spatial feature data of each key point is obtained through the spatial feature extraction module, the temporal feature data of each key point is obtained through the temporal feature extraction module, the spatial feature data and the temporal feature data are fused through the fusion module to obtain fused data, the fused data is classified through the fully connected layer to obtain a classification result, and the human body posture is predicted through the classification result and the fused data to obtain a human body posture prediction result; The classifying the fused data through the fully connected layer to obtain a classification result includes: Obtaining the X-axis coordinate and the Y-axis coordinate of each of the key points, and obtaining the spatial feature data of each of the key points based on the X-axis coordinate and the Y-axis coordinate; Based on the spatial feature data and the temporal feature data of the key points, a set of human skeleton key points corresponding to each part of the human body in the human body frame is obtained; Calculating the probability value that the fused data belongs to the set of human skeleton key points corresponding to each part of the human body; Determine a set of human skeleton key points corresponding to each part of the human body based on the probability value; The method of predicting the human body posture by using the classification result and the fusion data to obtain the human body posture prediction result includes: Calculate the human body posture prediction result through formula (2); In the formula, is the human posture prediction result, L is the classification result, z 2 is the spatial feature data of the key point, T is the temporal feature data, is the fused data, W is the learnable parameter of the output layer, and b is the bias term of the output layer; The human body frame is used as input and the human body posture prediction result is used as output to train the posture analysis model to obtain a trained posture analysis model.

2. The method for analyzing human body posture according to claim 1, characterized in that: The step of obtaining a certain number of human posture images and preprocessing the human posture images to obtain training images containing only human body frames includes: Detecting the human body in the human body posture image by using a target detection algorithm to obtain a human body frame of each human body; Determine whether the human body frame is qualified, and when the human body frame is unqualified, pre-process the human body frame until the human body frame is qualified; Based on the skeleton point coordinate file, each skeleton point of each human body is labeled to obtain a labeled training image containing only the human body frame.

3. The method for analyzing human body posture according to claim 1, characterized in that: The step of obtaining the time feature data of each key point through the time feature extraction module includes: Calculate the time characteristic data of each key point by formula (1); Formula (1); In the formula, h is the time feature data of each key point, X (l) is the network input of the current layer, f i (X) is the convolution operation, σ is the sigmoid nonlinear activation function, is the convolution operator.

4. The method for analyzing human body posture according to claim 1, characterized in that: The human body posture analysis method further comprises: Acquire a human body posture image to be analyzed, and preprocess the human body posture image to be analyzed to obtain an image to be analyzed that only contains a human body frame; The image to be analyzed is input into the trained posture analysis model, and the human posture prediction result is output.

5. The method for analyzing human body posture according to claim 4, characterized in that: The human body posture analysis method further comprises: Determining whether the human body posture prediction result is qualified, and when the human body posture prediction result is qualified, determining a corresponding execution instruction based on the human body posture prediction result; When the human body posture prediction result is unqualified, a corresponding early warning plan is determined based on the human body posture prediction result, and early warning measures are taken based on the early warning plan.

6. A human body posture analysis system, characterized in that: include: An acquisition module, used to acquire a certain number of human posture images; A preprocessing module, used for preprocessing the human body posture image to obtain a training image containing only a human body frame; A construction module is used to construct a posture analysis model, wherein the posture analysis model includes a convolution layer, a spatial feature extraction module, a temporal feature extraction module, a fusion module, a fully connected layer and an output layer connected in sequence; the key points corresponding to the human body frame are obtained through the convolution layer, the spatial feature data of each key point is obtained through the spatial feature extraction module, the temporal feature data of each key point is obtained through the temporal feature extraction module, the spatial feature data and the temporal feature data are fused through the fusion module to obtain fused data, the fused data is classified through the fully connected layer to obtain a classification result, and the human body posture is predicted through the classification result and the fused data to obtain a human body posture prediction result; The building blocks are also used to: Obtaining the X-axis coordinate and the Y-axis coordinate of each of the key points, and obtaining the spatial feature data of each of the key points based on the X-axis coordinate and the Y-axis coordinate; Based on the spatial feature data and the temporal feature data of the key points, a set of human skeleton key points corresponding to each part of the human body in the human body frame is obtained; Calculating the probability value that the fused data belongs to the set of human skeleton key points corresponding to each part of the human body; Determine a set of human skeleton key points corresponding to each part of the human body based on the probability value; Calculate the human body posture prediction result through formula (2); In the formula, is the human posture prediction result, L is the classification result, z 2 is the spatial feature data of the key point, T is the temporal feature data, is the fused data, W is the learnable parameter of the output layer, and b is the bias term of the output layer; The training module is used to take the human body frame as input and the human body posture prediction result as output, train the posture analysis model, and obtain a trained posture analysis model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images