Human body posture detection method and device and electronic equipment

Abstract

The invention relates to the technical field of computers, and discloses a human body posture detection method and device and electronic equipment. A two-hand motion prediction result, a left-hand motion prediction result and a right-hand motion prediction result of a target human body are obtained from an image, and the two-hand motion prediction result, the left-hand motion prediction result and the right-hand motion prediction result are permutated and combined to obtain a more accurate posture prediction result. And finally, providing feedback and suggestions for the user by utilizing the prediction result so as to guide the user to improve the posture expression. Wherein left and right hand features are decoupled during gesture recognition, so that the complexity of gestures is dealt with, and the accuracy of gesture recognition is improved; and the posture improvement of the user is promoted through feedback guidance and suggestion information, so that the expression effect of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and electronic device for detecting human posture. Background Art

[0002] Currently, in many scenarios, users use body gestures in addition to verbal expression, such as in the teaching scenes of normal school students, the speaker's speech process, the process of business communication and negotiation, the interview process, the process of acting and performance, etc. Through the coordination of language and body gestures, they can convey their intentions to the audience.

[0003] However, when expressing themselves, users may experience awkwardness, a sluggish posture, wandering eyes, and mechanical speech. These phenomena are primarily due to a lack of conscious body language training. Therefore, if users can be aware of their body language habits and poor posture during daily practice, understand their positive and negative body language, and eliminate negative body language, this will help them better convey their intentions, engage their audience, and engage in other effective communication in formal settings. Accurately capturing a user's body posture is a prerequisite for ensuring they understand their body language habits and poor posture. Summary of the Invention

[0004] The main technical problem solved by the embodiments of this application is how to improve the accuracy of user body posture recognition.

[0005] To address the above-mentioned technical issues, the present application employs a technical solution: providing a human posture detection method, comprising: acquiring an image including a target human; performing posture detection on the target human in the image to obtain prediction results for both hands, left hand, and right hand movements of the target human; and permuting and combining the two hands, left hand, and right hand movement prediction results to obtain a human posture prediction result for the target human. The human posture prediction result is used by a system to provide guidance and suggestions to a user based on the human posture prediction result. The method involves acquiring the target human posture information from the image, permuting and combining the hand movement results to obtain a more accurate posture prediction result, and finally using the prediction result to provide feedback and suggestions to the user to guide the user in improving their posture expression. During posture recognition, left and right hand features are decoupled to account for posture complexity and improve posture recognition accuracy. The feedback of guidance and suggestions promotes posture improvement for the user, thereby enhancing the user's expression.

[0006] In some embodiments, the posture detection of the target human body in the image to obtain the two-handed motion prediction result, the left-handed motion prediction result and the right-handed motion prediction result of the target human body includes: preprocessing the image; extracting features of the preprocessed image based on preset key point information to obtain key point features; decoupling the whole body and left and right limb features of the target human body according to the key point features to obtain whole body feature data, left limb feature data and right limb feature data; inputting the whole body feature data, the left limb feature data and the right limb feature data into pre-trained two-handed motion classifiers, left hand motion classifiers and right hand motion classifiers respectively to obtain the two-handed motion prediction result, left hand motion prediction result and right hand motion prediction result corresponding to the target human body respectively. By preprocessing the image and extracting features based on key point information, the posture of the target human body can be accurately detected. This embodiment can provide reliable basic data for subsequent hand motion prediction and posture analysis.

[0007] In some embodiments, extracting features from the preprocessed image based on preset key point information to obtain key point features includes: obtaining coordinate information for each key point in the preset key point information; and calculating, based on the coordinate information of the key points, the relative distance between each key point and the body center, the relative distance between each key point, the angle of each key point, and the visibility of each key point; wherein the relative distance between each key point and the body center, the relative distance between each key point, the angle of each key point, and the visibility of each key point together constitute the key point features. By obtaining the coordinate information of each key point in the preset key point information and calculating the relative distance between each key point and the body center, the relative distance between each key point, the angle of each key point, and the visibility of each key point, the spatial structure and joint angles of the human body posture can be captured, providing an accurate data foundation for subsequent posture analysis and hand movement prediction. In addition, considering these four factors can also provide comprehensive key point features, which enables the system to comprehensively consider factors such as the spatial relationship between key points, angle information, and visibility, better describe the details and characteristics of human posture, and thereby improve the accuracy of posture analysis.

[0008] In some embodiments, decoupling the target person's full body and left and right limb features based on the keypoint features to obtain full body feature data, left limb feature data, and right limb feature data includes: extracting a first eigenvector describing the full body features, a second eigenvector describing the left limb features, and a third eigenvector describing the right limb features based on the keypoint features; and normalizing the first, second, and third eigenvectors, wherein the first eigenvector represents the full body feature data, the second eigenvector represents the left limb feature data, and the third eigenvector represents the right limb feature data. By decoupling the keypoint features, the full body and left and right limb features of the target person are extracted separately. This allows for independent feature analysis and processing of the full body and limbs, better capturing the motion characteristics of the full body and limbs. Feature normalization eliminates scale and proportion differences between different people, making the features more comparable and stable, and improving the accuracy of subsequent classifier predictions. Converting complex keypoint features into concise feature vector representations reduces data dimensionality, improves computational efficiency, and provides input for subsequent classifiers.

[0009] In some embodiments, the whole-body feature data, the left limb feature data, and the right limb feature data are respectively input into a pre-trained two-handed motion classifier, a left-hand motion classifier, and a right-hand motion classifier to obtain the two-handed motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results corresponding to the target human body, respectively, including: inputting the first feature vector into the two-handed motion classifier to obtain the two-handed motion prediction results corresponding to the target human body; inputting the second feature vector into the left-hand motion classifier to obtain the left-hand motion prediction results corresponding to the target human body; and inputting the third feature vector into the right-hand motion classifier to obtain the right-hand motion prediction results corresponding to the target human body. By using the two-handed motion classifier, the left-hand motion classifier, and the right-hand motion classifier, the target human body's motion can be classified and predicted in a more fine-grained manner. Different hand motions usually have different meanings and uses. By predicting the motions of both hands, the left hand, and the right hand respectively, the human body's hand motion intentions and behaviors can be understood more accurately.

[0010] In some embodiments, the method further includes: obtaining the two-handed motion classifier, the left-handed motion classifier, and the right-handed motion classifier; the obtaining of the two-handed motion classifier, the left-handed motion classifier, and the right-handed motion classifier includes: collecting a training data set containing two-handed motion, left-handed motion, and right-handed motion, the training data set including samples and posture labels corresponding to each sample; determining a kernel function and a penalty parameter, the kernel function is used to map input features, and the penalty parameter is used to balance interval boundaries and training errors; using (x_n, y_n) in the training data set as input, constructing the objective function and constraints of a convex quadratic programming problem, wherein x_n represents the sample corresponding to , y_n represents the posture label corresponding to the sample, and the constraints include the penalty parameter; according to the objective function and the constraints, the convex quadratic programming problem is solved to obtain the optimal solution; according to the optimal solution, the posture label and the kernel function, a classification decision function is calculated; wherein, when the x_n represents the first eigenvector of the whole body feature, the classification decision function obtained is the two-handed action classifier; when the x_n represents the second eigenvector of the left limb feature, the classification decision function obtained is the left-hand action classifier; when the x_n represents the third eigenvector of the right limb feature, the classification decision function obtained is the right-hand action classifier. wherein, by collecting a training data set, constructing a convex quadratic programming problem and solving the optimal solution, a two-handed action classifier, a left-hand action classifier and a right-hand action classifier can be obtained, which have the advantages of customization, high performance and automated construction.

[0011] In some embodiments, the method of permuting and combining the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results to obtain a human posture prediction result of the target human body includes: obtaining a two-hand motion prediction result sequence, a left-hand motion prediction result sequence, and a right-hand motion prediction result sequence of the same length based on the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results; extracting the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results corresponding to the time step from the two-hand motion prediction result sequence, the left-hand motion prediction result sequence, and the right-hand motion prediction result sequence in chronological order; storing the extracted two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results corresponding to the time step in a preset data structure to form a combination result containing three elements, wherein the combination result containing three elements is used to represent the human posture prediction result corresponding to the time step. By permuting and combining the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results, the motion information of the two hands and the hand can be comprehensively considered, thereby obtaining a more comprehensive and accurate human posture prediction result. In addition, the movements of different hands may affect each other, and the posture of the target person can be better understood and explained by combining the prediction results.

[0012] To address the above technical issues, another technical solution adopted in the embodiments of the present application is to provide a human posture detection device, comprising: an image acquisition module for acquiring an image including a target human; a posture detection module for performing posture detection on the target human in the image to obtain two-hand motion prediction results, a left-hand motion prediction result, and a right-hand motion prediction result for the target human; and a human posture prediction module for permuting and combining the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results to obtain a human posture prediction result for the target human. The human posture prediction result is used by the system to provide guidance and suggestions to the user based on the human posture prediction result. The system obtains the target human posture information from the image, then permutes and combines the hand motion results to obtain a more accurate posture prediction result. Finally, the prediction result is used to provide feedback and suggestions to the user to guide the user in improving their posture expression. During posture recognition, the left and right hand features are decoupled to cope with the complexity of the posture and improve the accuracy of posture recognition. The feedback guidance and suggestions promote the user's posture improvement, thereby improving the user's expression.

[0013] To solve the above technical problems, another technical solution adopted in the embodiments of the present application is to provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the human gesture detection method described above. This electronic device can improve the accuracy of human gesture recognition and the user's expression effect.

[0014] To solve the above technical problems, another technical solution adopted by the embodiments of this application is to provide a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the human posture detection method described above. The human posture detection method stored in this non-volatile computer-readable storage medium can improve the accuracy of human posture recognition and the user's expression effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0016] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0017] Figure 2 This is a flow chart of a method for detecting human body posture provided by an embodiment of the present application;

[0018] Figure 3 This is a flowchart of a method provided by an embodiment of the present application for performing posture detection on a target human body in an image to obtain prediction results of two-handed movements, left-hand movements, and right-hand movements of the target human body;

[0019] Figure 4 This is a schematic diagram corresponding to the preset key point information provided in the embodiment of the present application;

[0020] Figure 5 is a flowchart of a method for obtaining a two-hand motion classifier, a left-hand motion classifier, and a right-hand motion classifier provided in an embodiment of the present application;

[0021] Figure 6 This is a flow chart of a method provided in an embodiment of the present application for permuting and combining two-hand motion prediction results, left-hand motion prediction results, and right-hand motion prediction results to obtain a human body posture prediction result of a target human body;

[0022] Figure 7 Schematic diagram of a human body posture detection device provided in an embodiment of the present application;

[0023] Figure 8 Schematic diagram of the hardware structure of an electronic device for executing the human posture detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0025] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device schematic or the order in the flow chart. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those generally understood by those skilled in the art of the present application. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not used to limit the present application.

[0026] In some current scenarios, such as teacher training, speaker presentations, business negotiations, job interviews, and dramas and performances, there is typically a narrator and a listener. The effectiveness of communication in these scenarios largely depends on the narrator's performance within the context. In addition to using voice and language to convey the content, narrators also employ various body movements and gestures to enhance the presentation, helping listeners gain a deeper understanding and memory of the content. The narrator's body language is a visual representation of their thoughts and emotions during the communication process, playing a crucial role in the reception of audiovisual information.

[0027] In some situations, narrators may display negative body language, such as eye aversion, finger tapping, pacing or restless movement, and a sense of uncertainty. This negative body language can negatively impact communication effectiveness, such as reducing listener attention or causing communication barriers. To improve this, technical tools can be used to assist narrators in developing their communication posture and body language, such as communication training and body language coaching. These tools can help narrators strengthen positive posture and body language, improving communication effectiveness, and reduce or eliminate the impact of negative posture and body language.

[0028] However, in traditional communication posture and body language training technology solutions, the recognition of communication posture and body posture usually focuses on the posture of the entire human body, and may not make a specific distinction for hand movements, resulting in inaccurate human posture detection results, which in turn causes the narrator to be unable to accurately obtain his or her own body posture habits and possible bad postures during the communication posture and body language training process. Therefore, the following embodiment of the present application provides a human posture detection method, which performs more fine-grained detection and prediction of the target human body, especially predicts the movements of both hands, left hand and right hand respectively, and obtains human posture prediction results through permutations and combinations. Compared with traditional methods, the human posture detection method provides more refined and comprehensive human posture prediction results by subdividing hand movements and performing combined predictions, and provides more possibilities for system feedback guidance and suggestion information.

[0029] See also Figure 1 , Figure 1 : is a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario includes a user 10, an image acquisition device 20, and a computing device 30. The user 10 performs body language training in the current scenario. Body language training (or gesture language training) is a training method that improves communication and exchange skills by improving and optimizing body posture, movements, and expressions. The image acquisition device 20 is used to obtain an image of the user 10 performing body language training and send the image to the computing device 30. The image acquisition device 20 can specifically be a device such as a camera or a smart phone, which can capture images of the user 10 performing body language training. The computing device 30 is communicatively connected to the image acquisition device 20 and is used to perform posture detection on the user 10 based on the obtained image to obtain a prediction result of the user 10's two-hand movement, a prediction result of the left hand movement, and a prediction result of the right hand movement; then the two-hand movement prediction result, the left hand movement prediction result, and the right hand movement prediction result are arranged and combined to obtain a human body posture prediction result of the user 10; and then guidance and suggestion information is fed back to the user 10 based on the human body posture prediction result. The computing device 30 may be any device with image processing and computing capabilities, including but not limited to a personal computer (PC) or laptop computer, a server, an embedded system, a mobile device (including a smart phone, a tablet computer, and other mobile computing devices), etc.

[0030] In this application scenario, the image acquisition device 20 is used to capture images of the user's gestures, and the computing device 30 uses the images to perform gesture detection and, through analysis and processing, generates corresponding motion and gesture prediction results. These results can be used to provide feedback and guidance to the user 10, helping them improve and optimize their gesture language expression and communication skills.

[0031] Optionally, user 10 is a teacher training student who conducts body language training in the aforementioned application scenario to improve their body language skills. Specifically, the student first learns theoretical knowledge about body language training, including the basic principles and techniques of posture, gestures, facial expressions, and eye contact. They understand and learn how to use body language to improve their communication and exchange skills. Next, the student can master correct posture and body position through imitation and practice, learning how to stand, sit, and walk, etc., to project a confident and professional image. Next, the student learns through training how to use appropriate gestures and movements to support and emphasize verbal expression. They can practice using gestures to emphasize key points, present examples, or convey emotion to enhance the effectiveness of the message. After learning the theoretical knowledge, practicing basic postures, and performing gesture and movement training, the student conducts a practice drill in a simulated or real-life scenario, such as a simulated teaching session, a speech, or an interview. During this process, the student's training process is recorded by the image capture device 20 and the images are transmitted to the computing device 30. The computing device 30 uses the image to perform posture detection and generates prediction results for the teacher trainee's two-hand movement, left-hand movement, and right-hand movement. These results are then arranged and combined to obtain the teacher trainee's human body posture prediction results. The computing device 30 can provide feedback, guidance, and suggestions to the teacher trainee based on these results to help them improve the effectiveness and influence of their gesture language. The teacher trainee observes their own posture and movements, receives feedback and guidance, and continuously practices and improves. They continuously improve their gesture language expression ability to become more confident, clear, and influential in teaching, speaking, and other communication scenarios. Through this process, the teacher trainee can gradually improve their gesture language training ability, laying a solid foundation for future teaching and communication work.

[0032] It should be noted that the above application scenarios can be applied not only to the body language training process of teacher trainees, but also to other scenarios, such as the speaker's speech process, business communication and negotiation process, interview process, drama and performance process, etc., so as to help users accurately understand their body posture, so that users can achieve better expression effects in the corresponding scenarios.

[0033] See also Figure 2 , Figure 2 : is a flowchart of a human posture detection method provided in an embodiment of the present application, which can be executed by the above-mentioned computing device 30. The method includes the following steps:

[0034] 101: Acquire an image including a target human body.

[0035] The target person refers to the body of the user whose posture is to be detected. This user can be a teacher training student, a teacher, a lecturer, etc. Images of the target person can be acquired through a camera, an image database, or other means. These images can be a sequence of continuous frames, obtained from a video stream or a sequence of continuously captured images. The system processes these continuous frames to perform posture detection and ultimately obtain a predicted human posture result. The target person's images can be acquired in real time, analyzed, and provided with feedback to provide accurate and real-time guidance and suggestions.

[0036] 102: Performing posture detection on the target human body in the image to obtain a two-hand motion prediction result, a left-hand motion prediction result, and a right-hand motion prediction result of the target human body.

[0037] In the application scenarios described above, user postures are often diverse and complex. Different postures involve different left and right upper limbs, and the left and right hands occupy different proportions in different postures. Therefore, the embodiments of the present application decouple the features of the left and right upper limbs to predict two-handed postures and one-handed postures respectively, thereby addressing the diversity of postures and improving the accuracy of posture prediction.

[0038] First, computer vision and deep learning techniques can be used to detect key points of the target person in the image. These key points include the locations of body parts such as the head, shoulders, elbows, wrists, and knees. Then, to account for the diversity of postures and differences between the left and right upper limbs, the posture features of both hands and one hand can be decoupled. This means predicting the postures of the left and right hands separately, rather than predicting them as a whole. This decoupling can better capture the differences between the left and right hands in different postures and improve prediction accuracy. Next, based on the analysis and decoupled features of the key points of the human body, a posture prediction model is established using machine learning (such as support vector machines (SVMs)) or deep learning algorithms (such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs)). This model can learn the mapping relationship between key point information and different postures. By inputting the key point information in the image, the model can output predicted results for the target person's two-handed movements, left-hand movements, and right-hand movements.

[0039] To build an accurate posture prediction model, a training dataset containing diverse postures is required. This dataset should cover various postures in the current user's scenario and include corresponding two-handed and one-handed posture information. Using this dataset, we extract the key point features mentioned above and train and optimize the model based on these key point features to accurately predict different postures.

[0040] 103: Arrange and combine the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results to obtain a human body posture prediction result of the target human body, wherein the human body posture prediction result is used to enable the system to feedback guidance and suggestion information to the user based on the human body posture prediction result.

[0041] The system combines the predicted results for both hands, left hand, and right hand movements to generate a predicted human posture for the target person. This result can be used to provide feedback, guidance, and suggestions to the user, potentially including information on movement optimization and posture correction. Specifically, the system is a computing device that implements the human posture detection method.

[0042] The prediction results for both hands, left hands, and right hands are arranged and combined. The specific arrangement and combination can be set according to the definition and requirements of the posture. For example, the human posture prediction results can be represented as a set of coordinates and angles as required, with coordinates representing the position of each key point in the image or space, and angles representing the connection angles between joints; or classification labels corresponding to specific actions or gestures can be used, such as hands on hips, fists, and open palms.

[0043] This embodiment of the application obtains the target person's posture information from an image, then aligns and combines hand movement results to obtain more accurate posture prediction results. Finally, the prediction results are used to provide feedback and suggestions to the user to guide them in improving their posture expression. During posture recognition, the left and right hand features are decoupled to cope with the complexity of posture and improve the accuracy of posture recognition. The feedback guidance and suggestions promote the user's posture improvement, thereby improving the user's expression effect.

[0044] In some embodiments, see Figure 3 , performing posture detection on the target human body in the image to obtain a two-hand motion prediction result, a left-hand motion prediction result, and a right-hand motion prediction result of the target human body, including:

[0045] 201: Preprocess the image.

[0046] Preprocessing includes operations such as image scaling, cropping, normalization, and denoising. Preprocessing can ensure the quality and consistency of the input image and improve the effect of subsequent feature extraction.

[0047] 202: Extract features from the pre-processed image based on preset key point information to obtain key point features.

[0048] A set of key points is used to represent the posture of the human body. The preset key point information includes key points used to represent the posture of the human body. In this embodiment, the preset key point information specifically includes 17 key points, such as Figure 4 As shown, key points 0 to 16 correspond to the nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles.

[0049] The feature extraction of the pre-processed image based on the preset key point information to obtain the key point features includes:

[0050] (1) Obtain the coordinate information of each key point in the preset key point information. The coordinate information represents the position of the key point in the image. The method for obtaining the coordinate information of the key point in the image may be to use third-party tools and software to perform key point annotation and coordinate extraction. It may also be to estimate the coordinate information of the key point by calculation or measurement based on the rules or laws of a specific scene or problem. It may also be to use a trained posture estimation model that can detect and locate the key points of the human body from the image. These models may be based on machine learning methods (such as random forests, support vector machines, etc.) or deep learning methods (such as convolutional neural networks, key point regression models, etc.); the image is input into the posture estimation model, the posture estimation model will analyze the image, and detect and locate the preset key points. For each key point, the posture estimation model can output its coordinate position in the image; by parsing the output results of the posture estimation model, the coordinate information of each key point can be obtained. These coordinate information represents the position of the key point in the image, usually in pixels.

[0051] (2) Calculating the relative distance between each key point and the center point of the body, the relative distance between each key point, the angle of each key point, and the visibility of each key point based on the coordinate information of the key point; wherein the relative distance between each key point and the center point of the body, the relative distance between each key point, the angle of each key point, and the visibility of each key point together constitute the key point features.

[0052] The center point of the body is defined as the center point of the area formed by the left and right shoulders and the left and right hips, such as Figure 4 The coordinates of the center point shown in (x_center, y_center) are represented by (x_center, y_center). To improve the generalization of the features and eliminate differences caused by different heights and body shapes, the relative distances between each key point and the body center are normalized by dividing by the target person's height (h). Specifically, the relative distance between each key point and the body center is calculated as follows:

[0053] Dist((x_i,y_i),(x_center,y_center)) / h;

[0054] Dist is the Euclidean distance, x_i, y_i represent the coordinates of each key point, and h is the height of the target person.

[0055] The relative distances between the 17 key points and the center point of the body can be calculated by the above formula, thereby obtaining the characteristics of the relative distances between the 17 key points and the center point of the body.

[0056] The relative distances between key points are set based on the influence of the lower limbs on the upper limbs, as well as some upper limb movements. Specifically, whether the lower legs are standing in a standard position, or the legs are too far apart, or the legs are crossed, and whether the upper limbs have palms together, hands on hips, elbows hugged, etc., all of which are strongly correlated with the relative distances of some key points. Therefore, the following six relative values are also calculated as feature vectors. They include:

[0057] Dist((x l_wrist ,y l_wrist ), (x r_wrist ,y r_wrist )) / Dist((x l_hip ,y l_hip ), (x r_hip ,y r_hip ))(1);

[0058] Among them, x l_wrist ,y l_wrist Indicates the coordinates of the left wrist, x r_wrist ,y r_wrist Indicates the coordinates of the right wrist, x l_hip ,y l_hip represents the coordinate of the left hip, x r_hip ,y r_hip represents the coordinates of the right hip. The above formula (1) is the relative distance between the left and right wrists divided by the relative distance between the left and right hips. The result of the division calculation is used to determine whether the user's hands are crossed or separated, that is, to determine whether there is a palm-to-palm gesture. For example, if the result of the division calculation is less than a preset threshold, it is determined that the hands are crossed; otherwise, the hands are separated. The smaller the result of the division calculation, the greater the probability of the hands being crossed, and the more likely the palm-to-palm gesture is.

[0059] Dist((x l_wrist ,y l_wrist ), (x r_wrist ,y r_wrist )) / Dist((x l_elbow ,y l_elbow ), (x r_elbow ,y r_elbow )) (2);

[0060] Among them, x l_wrist ,y l_wrist Indicates the coordinates of the left wrist, x r_wrist ,y r_wrist Indicates the coordinates of the right wrist, x l_elbow ,y l_elbow represents the coordinates of the left elbow, x r_elbow ,y r_elbow Represents the coordinates of the right elbow. The above formula (2) is the relative distance between the left and right wrists divided by the relative distance between the left and right elbows. The result of the division calculation is used to determine whether the user has raised his hand, pointed, hugged his elbows, or put his hands on his hips. For example, if the division calculation result is less than the threshold, it can be determined as a hand-raising action. This is because the hand-raising action will increase the relative distance between the wrists, while the relative distance between the elbows is relatively small, resulting in a larger division calculation result. If the division calculation result is greater than the threshold, it can be determined as one of the actions such as pointing, hugging the elbows, or putting hands on the hips.

[0061] Dist((x l_ankle ,y l_ankle ), (x r_ankle ,y r_snkle )) / Dist((x l_hip ,y l_hip ), (x r_hip ,y r_hip ))(3);

[0062] Among them, x l_ankle ,y l_ankle represents the coordinates of the left ankle joint, x r_ankle ,y r_ankle represents the coordinates of the right ankle joint, x l_hip ,y l_hip represents the coordinate of the left hip, x r_hip ,y r_hip Represents the coordinates of the right hip. The above formula (3) is the relative distance between the left and right ankle joints divided by the relative distance between the left and right hips. The result of the division calculation is used to determine whether the user's feet are standing in a standard position or are wide apart. For example, if the division calculation result is less than the threshold, it can be determined as a standard standing posture. This is because when standing in a standard position, the relative distance between the ankle joints is small, while the relative distance between the hips is large, resulting in a smaller division calculation result. If the division calculation result is greater than the threshold, it can be determined as a posture with the feet wide apart. This is because when the feet are wide apart, the relative distance between the ankle joints is large, while the relative distance between the hips is small, resulting in a larger division calculation result.

[0063] Dist((x l_ankle ,y l_ankle ), (x r_ankle ,y r_ankle )) / Dist((xl_shoulder ,y l_shoulder ), (x r_shoulder ,y r_shoulder )) (4);

[0064] Among them, x l_ankle ,y l_ankle represents the coordinates of the left ankle joint, x r_ankle ,y r_ankle represents the coordinates of the right ankle joint, x l_shoulder ,y l_shoulder Indicates the coordinates of the left shoulder, x r_shoulder ,y r_shoulder The coordinates of the right shoulder are represented by the formula (4). The relative distance between the left and right ankle joints is divided by the relative distance between the left and right shoulders. The result of this division is used to determine the user's foot width and shoulder width, that is, to detect the user's standing posture. For example, if the division result is less than a threshold, it can be determined that the posture is one in which the foot width is less than the shoulder width. If the division result is greater than the threshold, it can be determined that the posture is one in which the foot width is greater than the shoulder width.

[0065] Dist((x l_wrist ,y l_wrist ), (x l_hip ,y l_hip )) / h (5);

[0066] Among them, x l_wrist ,y l_wrist Indicates the coordinates of the left wrist, x l_hip ,y l_hip represents the coordinates of the left hip, and h represents the user's height. Formula (5) above is the relative distance between the left wrist and the left hip divided by the user's height. The result of the division is used to determine the separation of the user's left hand from the body, and the division by the height is for normalization. For example, if the result of the division is less than the threshold, it can be determined that the left hand is relatively close to the body and not significantly separated. If the result of the division is greater than the threshold, it can be determined that the left hand is relatively separated from the body.

[0067] Dist((x r_wrist ,y r_wrist ), (x r_hip ,y r_hip )) / h (6);

[0068] Among them, x r_wrist ,y r_wrist Indicates the coordinates of the right wrist, x r_hip ,y r_hiprepresents the coordinates of the right hip, and h represents the user's height. Formula (6) above is the relative distance between the right wrist and the right hip divided by the user's height. The result of the division is used to determine the separation of the user's right hand from the body, and the division by the height is for normalization. For example, if the result of the division is less than the threshold, it can be determined that the right hand is close to the body and not clearly separated. If the result of the division is greater than the threshold, it can be determined that the right hand is relatively separated from the body.

[0069] The angle of each key point is the bending angle of the part corresponding to each key point, and the bending angle is also an important attribute for distinguishing postures. In this embodiment, a total of 16 key angles are considered, including 6 angles corresponding to the left and right shoulders, 2 angles corresponding to the left and right elbows, 6 angles corresponding to the left and right hips, and 2 angles corresponding to the left and right knees. Figure 4 . These angles can be obtained by using a depth camera, an RGB-D camera, or a configuration of multiple cameras to capture the user's posture information. By using computer vision technology, such as a human posture estimation algorithm, the coordinates of key points can be extracted from images or videos to calculate the angles. Alternatively, a motion capture system can be used to track the movement of the body using multiple sensors or infrared cameras, and generate three-dimensional posture data. The key angles can be calculated by accurately locating the position and orientation of the joints. Other methods can also be used to obtain the above 16 key angles.

[0070] The visibility of each key point is also an important feature for judging posture, and has a significant impact on judging postures such as front view, back view, and hands behind the back. Therefore, the visibility p_i of 17 key points is selected as the feature vector. Each p_i represents the visibility of the corresponding key point, which can be a binary value (0 or 1) indicating whether the key point is visible, or a continuous value indicating the degree of visibility (for example, 0 indicates completely invisible and 1 indicates completely visible).

[0071] Therefore, a total of 56 features were selected (relative distance between each key point and the body center: 17, relative distance between each key point: 6, angle of each key point: 16, visibility of each key point: 17). For each user whose body posture is detected, a 1×56 feature vector can be obtained.

[0072] In this embodiment, by obtaining the coordinate information of each key point in the preset key point information and calculating the relative distance between each key point and the body center point, the relative distance between key points, the angle of the key points, and the visibility of the key points, the spatial structure and joint angles of the human body posture are captured, providing an accurate data foundation for subsequent posture analysis and hand movement prediction. Furthermore, considering these four factors also provides comprehensive key point features. This enables the system to comprehensively consider factors such as the spatial relationship between key points, angle information, and visibility, better describing the details and characteristics of human posture, thereby improving the accuracy of posture analysis.

[0073] 203: Decoupling the whole body and left and right limb features of the target human body according to the key point features to obtain whole body feature data, left limb feature data, and right limb feature data.

[0074] Specifically, based on the key point features, a first eigenvector for describing the whole body features, a second eigenvector for describing the left limb features, and a third eigenvector for describing the right limb features are extracted; the first eigenvector, the second eigenvector and the third eigenvector are normalized; wherein the first eigenvector is used to represent the whole body feature data, the second eigenvector is used to represent the left limb feature data, and the third eigenvector is used to represent the right limb feature data.

[0075] Key point features refer to a series of features obtained by measuring and calculating key points of the human body. Specifically, they can be the 56 features obtained in the above steps, including: the relative distance between each key point and the center point of the body: 17, the relative distance between each key point: 6, the angle of each key point: 16, and the visibility of each key point: 17.

[0076] The first feature vector for describing the whole body feature is extracted based on the key point feature. The first feature vector of 1×56 is extracted based on the key point feature, specifically including:

[0077] The relative distances between each key point and the body center are 17, the relative distances between each key point are 6, the angles of each key point are 16, and the visibility of each key point is 17. This 1×56 feature vector is the first feature vector, which is a representation of key point features and is formed by combining all key point features in a certain order.

[0078] The second feature vector for describing the left limb feature is extracted based on the key point feature. The left limb feature is: based on the key point feature, a 1×46 feature vector is extracted, specifically including:

[0079] The relative distance between each key point and the center point of the body: 14 (removing the distance between the right shoulder, right elbow and right wrist and the center point of the body, but still retaining the characteristics of the right lower limb, because complete left and right lower limb characteristics are crucial).

[0080] The relative distance between key points is 5 (excluding the distance between the right wrist and right hip, i.e. Dist((x_(r_wrist),y_(r_wrist)),(x_(r_hip),y_(r_hip))) / h).

[0081] Key point angles: 13, Figure 4 The angles marked in the figure (excluding the right elbow angle, right shoulder angle, and right upper limb trunk angle).

[0082] Visibility of key points: 14 (excluding the visibility of the right shoulder, right elbow and right wrist).

[0083] The above-mentioned 1×46 feature vector is the second feature vector. The second feature vector is a coding form extracted based on key point features. It only contains some key point features that describe the features of the left limb. The decoupling of the left limb features is achieved by selecting specific key points and features.

[0084] The third feature vector for describing the right limb feature is extracted based on the key point feature. The third feature vector is extracted based on the key point feature. The third feature vector is 1×46, including:

[0085] The relative distances between each key point and the body center point are 14 (excluding the distances between the left shoulder, left elbow, and left wrist and the body center point, while retaining the complete lower limb features).

[0086] The relative distance between key points is 5 (excluding the distance between the left wrist and left hip Dist((x_(l_wrist),y_(l_wrist)),(x_(l_hip),y_(l_hip))) / h.

[0087] Key point angles: 13, Figure 4 The angles marked in the figure (excluding the left elbow angle, left shoulder angle and left upper limb torso angle).

[0088] Visibility of key points: 14 (excluding the visibility of the left shoulder, left elbow and left wrist).

[0089] The above-mentioned 1×46 feature vector is the third feature vector. The third feature vector is also a coding form extracted based on key point features. It only contains some key point features that describe the features of the right limb. The decoupling of the right limb features is achieved by selecting specific key points and features.

[0090] Next, normalization is performed on the first eigenvector, the second eigenvector, and the third eigenvector.

[0091] Since the distance of the camera shooting and the distortion of the camera will cause large deviations in the key points of the human body collected under different devices, in order to improve the generalization and classification accuracy, the feature vectors corresponding to the key point features are first normalized. Among them, the normalization process includes two normalization operations, namely sample dimension normalization and feature dimension normalization. Sample dimension normalization is to perform L2 norm normalization on each sample, as shown in the following formula, each feature is divided by the L2 norm of the sample feature. The formula is:

[0092]

[0093] Among them, each sample x n The eigenvector of

[0094] Feature dimension normalization is to normalize each dimension of the sample according to the following formula, that is, to perform standardization on each dimension. The formula is:

[0095]

[0096]

[0097]

[0098] The above three formulas represent the mean, standard deviation, and normalization, respectively. Using the above formula, the mean and standard deviation of each feature dimension can be calculated, so that the distribution and degree of variation of the data can be obtained; then, by subtracting the mean from each eigenvalue and dividing it by the standard deviation, normalization is performed to obtain the position of the mean value and variation range relative to the entire eigenvector. Among them, the eigenvector is a vector composed of multiple eigenvalues, each eigenvalue represents the value of the eigenvector in the corresponding feature dimension, so the eigenvalue refers to each element or component in the eigenvector. By performing normalization, the value of each feature dimension can be mapped to a value with a similar scale and range, so as to better compare and integrate the feature data.

[0099] In this embodiment, by extracting key point features and mapping them into different feature vectors, the decoupling of the whole body and left and right limb features can be achieved, so that the whole body and limbs can be independently analyzed and processed, and the motion characteristics of the whole body and limbs can be better captured. Among them, the normalization of the feature vectors can eliminate the scale and proportion differences between different human bodies, making the features more comparable and stable, and contributing to the accuracy of subsequent classifier predictions. Among them, the complex key point features are converted into a concise feature vector representation, which can reduce the dimension of the data, improve computational efficiency, and provide input for subsequent classifiers.

[0100] 204: Input the whole-body feature data, the left limb feature data, and the right limb feature data into pre-trained two-hand motion classifiers, left-hand motion classifiers, and right-hand motion classifiers, respectively, to obtain two-hand motion prediction results, left-hand motion prediction results, and right-hand motion prediction results corresponding to the target human body, respectively.

[0101] Specifically, the first feature vector is input into a two-handed motion classifier to obtain a prediction result of the two-handed motion corresponding to the target human body; the second feature vector is input into a left-handed motion classifier to obtain a prediction result of the left-handed motion corresponding to the target human body; the third feature vector is input into a right-handed motion classifier to obtain a prediction result of the right-handed motion corresponding to the target human body. By using a two-handed motion classifier, a left-handed motion classifier, and a right-handed motion classifier, the target human body's motion can be classified and predicted in a more fine-grained manner. Different hand motions usually have different meanings and uses. By predicting the motions of both hands, the left hand, and the right hand separately, the human body's hand motion intentions and behaviors can be understood more accurately.

[0102] The above-mentioned two-handed action classifier is used to classify the two-handed actions of the target human body. It accepts a first eigenvector as input and, based on the hand-related information contained in the eigenvector, such as the position of the hand, key point angles, visibility, etc., determines the two-handed action performed by the target human body. Two-handed actions can be some actions that require the simultaneous use of both hands, such as making a fist or putting the hands together. The left-hand action classifier is used to classify the left-hand actions of the target human body. It accepts a second eigenvector as input and, based on the left-hand-related information contained in the eigenvector, such as the position of the left hand, key point angles, visibility, etc., predicts the left-hand actions performed by the target human body, such as waving or clapping. The right-hand action classifier is used to classify the right-hand actions of the target human body. It accepts a third eigenvector as input and, based on the right-hand-related information contained in the eigenvector, such as the position of the right hand, key point angles, visibility, etc., predicts the right-hand actions performed by the target human body, such as shaking hands or raising hands.

[0103] In this embodiment, the two-handed motion classifier, the left-handed motion classifier, and the right-handed motion classifier are obtained in advance. Specifically, a support vector machine (SVM) is used as a classifier to classify the two-handed motion, the left-handed motion, and the right-handed motion. Figure 5 , obtaining the two-hand action classifier, the left-hand action classifier, and the right-hand action classifier includes:

[0104] 301: Collect a training data set including two-handed movements, left-handed movements, and right-handed movements, wherein the training data set includes samples and posture labels corresponding to each sample; wherein the training data set includes feature vectors (such as the first, second, and third feature vectors mentioned above) and corresponding hand movement labels (two-handed movements, left-hand movements, and right-hand movements).

[0105] 302: Determine a kernel function and a penalty parameter. The kernel function is used to map the input features, and the penalty parameter is used to balance the margin and training error. The kernel function maps the input features into a high-dimensional space for linear classification in that space. The penalty parameter is used to balance the margin and training error, controlling the complexity and generalization ability of the model. Common kernel functions include linear kernels, polynomial kernels, and Gaussian kernels.

[0106] 303: Using (x_n, y_n) in the training dataset as input, construct an objective function and constraints for a convex quadratic programming problem, where x_n represents the feature vector corresponding to the sample, y_n represents the pose label corresponding to the sample, and the constraints include the penalty parameter. The objective function aims to minimize the classification error and maximize the margin. The constraints include the penalty parameter, which controls the degree of penalty for misclassified samples.

[0107] 304: Solve the convex quadratic programming problem according to the objective function and the constraints to obtain an optimal solution; this can be achieved by an optimization algorithm (such as a sequential minimum optimization algorithm).

[0108] 305: Calculate a classification decision function based on the optimal solution, the posture label, and the kernel function. The classification decision function can predict the hand motion category of the target person based on the input feature vector. In SVM, the decision function is typically represented using support vectors, which are sample points located on interval boundaries in the training dataset. Based on the input feature vector and the corresponding kernel function, the decision function can calculate the category prediction result of the sample point.

[0109] During the training process, cross-validation, parameter adjustment, and model evaluation are also required to obtain better classification performance and thus obtain the final classification decision function.

[0110] Among them, when the x_n represents the first eigenvector of the whole body feature, the classification decision function obtained is the two-hand action classifier; when the x_n represents the second eigenvector of the left limb feature, the classification decision function obtained is the left-hand action classifier; when the x_n represents the third eigenvector of the right limb feature, the classification decision function obtained is the right-hand action classifier.

[0111] In this embodiment, by collecting a training data set, constructing a convex quadratic programming problem, and solving the optimal solution, a two-handed action classifier, a left-handed action classifier, and a right-handed action classifier can be obtained, which has the advantages of customization, high performance, and automated construction. It should be noted that in addition to the SVM used above, other machine learning models can also be used.

[0112] In some embodiments, see Figure 6 The permutation and combination of the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results to obtain the human body posture prediction result of the target human body includes:

[0113] 401: Obtaining a two-hand motion prediction result sequence, a left-hand motion prediction result sequence, and a right-hand motion prediction result sequence having the same length according to the two-hand motion prediction result, the left-hand motion prediction result, and the right-hand motion prediction result;

[0114] 402: extracting the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results of corresponding time steps one by one from the two-hand motion prediction result sequence, the left-hand motion prediction result sequence, and the right-hand motion prediction result sequence in chronological order;

[0115] 403: The extracted two-hand motion prediction results, left-hand motion prediction results, and right-hand motion prediction results corresponding to the time step are stored in a preset data structure to form a combination result including three elements, and the combination result including three elements is used to represent the human body posture prediction result corresponding to the time step.

[0116] By extracting and combining the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results by time step, a complete sequence of human posture prediction results is obtained. The prediction result for each time step contains information about the two-hand motion, the left-hand motion, and the right-hand motion. If a time step corresponds to only a single-hand motion, the two-hand motion value can be null. The data structure can be a tuple or a list, etc. The time step can be used to represent a specific time point or time index in a time series. For each prediction result sequence, each time step corresponds to a prediction result. By extracting the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results for the corresponding time step from different prediction result sequences in chronological order, the human posture prediction result for the corresponding time step can be obtained. For example, if the length of the two-hand motion prediction result sequence, the left-hand motion prediction result sequence, and the right-hand motion prediction result sequence are all 10, then the time step can range from 1 to 10, indicating the sequential position of each prediction result in the sequence. The unit of the time step can be seconds, frames, etc.

[0117] In this embodiment, by permuting and combining the prediction results for both hands, left hand motion, and right hand motion, we can comprehensively consider the motion information of both hands and the hand, thereby obtaining a more comprehensive and accurate human posture prediction result. Different hand motions may affect each other, and by combining the prediction results, we can better understand and interpret the target person's posture.

[0118] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a human posture detection device provided in an embodiment of the present application. Figure 7 As shown, the human posture detection device 40 includes:

[0119] An image acquisition module 41 is used to acquire an image including a target human body;

[0120] a posture detection module 42 for performing posture detection on a target human body in the image to obtain a two-hand motion prediction result, a left-hand motion prediction result, and a right-hand motion prediction result of the target human body;

[0121] The human body posture prediction module 43 is used to arrange and combine the two-hand movement prediction results, the left-hand movement prediction results and the right-hand movement prediction results to obtain the human body posture prediction results of the target human body, wherein the human body posture prediction results are used to enable the system to feedback guidance and suggestion information to the user based on the human body posture prediction results.

[0122] It should be noted that the above-mentioned human posture detection device can execute the human posture detection method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the embodiments of the human posture detection device, please refer to the human posture detection method provided in the embodiments of this application.

[0123] See also Figure 8 , Figure 8 Schematic diagram of the hardware structure of the electronic device 50 for performing the human body posture detection method provided in the embodiment of the present application. Figure 8 As shown, the electronic device 50 includes:

[0124] One or more processors 51 and memory 52, Figure 8 A processor 51 is taken as an example.

[0125] The processor 51 and the memory 52 may be connected via a bus or other means. Figure 8 The bus connection is taken as an example.

[0126] The memory 52 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the program instructions / modules corresponding to the human posture detection method in the embodiment of the present application (for example, the attached Figure 7 The processor 51 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions and modules stored in the memory 52, that is, implements the human body posture detection method of the above method embodiment.

[0127] The memory 52 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the human posture detection device, etc. In addition, the memory 52 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 52 may optionally include a memory remotely located relative to the processor 51, and these remote memories may be connected to the human posture detection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] The one or more modules are stored in the memory 52 and, when executed by the one or more processors 51 , perform the human body posture detection method in any of the above method embodiments.

[0129] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0130] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: mobile communication devices, such as smartphones, feature phones, etc.; ultra-mobile personal computer devices, which have computing and processing functions and generally also have mobile Internet access features; servers, devices that provide computing services, and other electronic devices with data interaction functions.

[0131] An embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, for example Figure 8 A processor 51 in the embodiment may enable the one or more processors to execute the human body posture detection method in any of the above method embodiments.

[0132] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by the electronic device, the electronic device is enabled to execute the human posture detection method in any of the above method embodiments.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0134] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that each embodiment can be implemented by means of software plus a general hardware platform, or of course by hardware. It can be understood by those skilled in the art that all or part of the processes in the above embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Based on the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the present application as described above. For the sake of simplicity, they are not provided in detail. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting human posture, characterized in that: include: Acquiring an image including a target human body; Performing posture detection on the target human body in the image to obtain a two-hand motion prediction result, a left-hand motion prediction result, and a right-hand motion prediction result of the target human body; The two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results are arranged and combined to obtain a human posture prediction result of the target human body, wherein the human posture prediction result is used to enable the system to feedback guidance and suggestion information to the user based on the human posture prediction result.

2. The method according to claim 1, characterized in that The performing posture detection on the target human body in the image to obtain a two-hand motion prediction result, a left-hand motion prediction result, and a right-hand motion prediction result of the target human body includes: Preprocessing the image; Perform feature extraction on the pre-processed image based on preset key point information to obtain key point features; Decoupling the whole body and left and right limb features of the target human body according to the key point features to obtain whole body feature data, left limb feature data, and right limb feature data; The whole-body feature data, the left limb feature data and the right limb feature data are respectively input into the pre-trained two-handed motion classifier, the left-hand motion classifier and the right-hand motion classifier to obtain the two-handed motion prediction results, the left-hand motion prediction results and the right-hand motion prediction results corresponding to the target human body.

3. The method according to claim 2, characterized in that The step of extracting features from the pre-processed image based on preset key point information to obtain key point features includes: Obtaining coordinate information of each key point in the preset key point information; Calculating the relative distance between each key point and the center point of the body, the relative distance between each key point, the angle of each key point, and the visibility of each key point based on the coordinate information of the key points; The relative distances between the key points and the center point of the body, the relative distances between the key points, the angles of the key points, and the visibility of the key points together constitute the key point features.

4. The method according to claim 2 or 3, characterized in that Decoupling the whole body and left and right limb features of the target human body according to the key point features to obtain whole body feature data, left limb feature data, and right limb feature data includes: Extracting, based on the key point features, a first feature vector for describing whole body features, a second feature vector for describing left limb features, and a third feature vector for describing right limb features; performing normalization processing on the first eigenvector, the second eigenvector, and the third eigenvector; The first feature vector is used to represent the whole body feature data, the second feature vector is used to represent the left limb feature data, and the third feature vector is used to represent the right limb feature data.

5. The method according to claim 4, characterized in that The whole-body feature data, the left limb feature data, and the right limb feature data are respectively input into pre-trained two-hand motion classifiers, left-hand motion classifiers, and right-hand motion classifiers to obtain two-hand motion prediction results, left-hand motion prediction results, and right-hand motion prediction results corresponding to the target human body, including: Inputting the first feature vector into a two-handed action classifier to obtain a two-handed action prediction result corresponding to the target human body; Inputting the second feature vector into a left-hand action classifier to obtain a left-hand action prediction result corresponding to the target human body; The third feature vector is input into a right-hand action classifier to obtain a right-hand action prediction result corresponding to the target human body.

6. The method according to claim 2, characterized in that The method further includes: obtaining the two-hand motion classifier, the left-hand motion classifier, and the right-hand motion classifier; The obtaining of the two-hand motion classifier, the left-hand motion classifier, and the right-hand motion classifier includes: Collecting a training data set including two-handed movements, left-handed movements, and right-handed movements, wherein the training data set includes samples and a posture label corresponding to each sample; Determining a kernel function and a penalty parameter, wherein the kernel function is used to map the input features and the penalty parameter is used to balance the interval boundary and the training error; Using (x_n, y_n) in the training data set as input, constructing the objective function and constraints of the convex quadratic programming problem, wherein x_n represents the feature vector corresponding to the sample, y_n represents the posture label corresponding to the sample, and the constraints include the penalty parameter; Solving the convex quadratic programming problem according to the objective function and the constraints to obtain an optimal solution; Calculating a classification decision function based on the optimal solution, the posture label, and the kernel function; Wherein, when the x_n represents the first feature vector of the whole body feature, the obtained classification decision function is the two-handed action classifier; When x_n represents the second feature vector of the left limb feature, the obtained classification decision function is the left hand action classifier; When x_n represents the third eigenvector of the right limb feature, the obtained classification decision function is the right hand action classifier.

7. The method according to claim 1, characterized in that The permuting and combining the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results to obtain a human body posture prediction result of the target human body includes: According to the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results, a two-hand motion prediction result sequence, a left-hand motion prediction result sequence, and a right-hand motion prediction result sequence having the same length are obtained; Extracting the two-hand motion prediction results, the left-hand motion prediction results, and the right-hand motion prediction results of corresponding time steps one by one from the two-hand motion prediction result sequence, the left-hand motion prediction result sequence, and the right-hand motion prediction result sequence in chronological order; The extracted two-hand action prediction results, left-hand action prediction results, and right-hand action prediction results corresponding to the time step are stored in a preset data structure to form a combination result containing three elements, and the combination result containing three elements is used to represent the human body posture prediction result corresponding to the time step.

8. A human posture detection device, characterized in that: include: An image acquisition module, configured to acquire an image including a target human body; a posture detection module, configured to perform posture detection on a target human body in the image to obtain a two-hand motion prediction result, a left-hand motion prediction result, and a right-hand motion prediction result of the target human body; A human posture prediction module is used to arrange and combine the two-hand movement prediction results, the left-hand movement prediction results, and the right-hand movement prediction results to obtain a human posture prediction result of the target human body, wherein the human posture prediction result is used to enable the system to feedback guidance and suggestion information to the user based on the human posture prediction result.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images