A yoga action evaluation method, system, storage medium and electronic device

Abstract

This invention relates to a method, system, storage medium, and electronic device for evaluating yoga postures. The method includes: fusing and optimizing yoga posture data from different perspectives using a 3D skeleton information fusion optimization algorithm to obtain occluded and completed 3D human skeleton data; using a trained hierarchical cascaded graph convolutional neural network model to identify the occluded and completed 3D human skeleton data to obtain the yoga posture category and yoga posture completion score corresponding to the yoga posture data; obtaining the difficulty coefficient corresponding to the yoga posture category; and obtaining the evaluation result of the yoga posture data based on the difficulty coefficient and yoga posture completion score. This invention, by constructing a cascaded graph convolutional neural network model to identify and evaluate yoga postures, achieves accurate classification and evaluation of the yoga postures to be evaluated, providing strong support for users to subsequently improve their yoga postures based on the evaluation results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a yoga action evaluation method and system, a storage medium and an electronic device. BACKGROUND

[0002] Currently, visual-based human action analysis aims to recognize and evaluate human actions in images, mainly including human action detection, human action classification and action quality evaluation tasks. Deep learning based on human skeleton information is the mainstream method of human action analysis at present, among which the most widely used model is the recurrent neural network (RNN) and the convolutional neural network (CNN). The method based on RNN usually models the skeleton data as a sequence of coordinate vectors along the spatial and temporal dimensions, where each vector represents a human joint. The method based on CNN usually models the skeleton data as an image according to the hand-designed conversion rule.

[0003] In the prior art, neither the RNN-based method nor the CNN-based method can fully represent the structure of the skeleton data, because the skeleton data is embedded in the form of a graph, rather than a vector sequence or a two-dimensional grid. In recent years, graph convolution networks have generalized convolution to graphs, and the dynamic skeleton model ST-GCN can automatically learn spatial and temporal patterns from images, and has been successfully applied to the field of action analysis, but deep learning methods rely on a large amount of labeled data for model training. Compared with traditional human action analysis tasks, the difficulty of visual-based yoga action analysis lies in the fact that there are many yoga action categories, the poses are not common, and the yoga image data obtained from a single perspective has a serious occlusion problem, making it difficult to effectively recognize and evaluate. SUMMARY

[0004] To solve the above technical problems, the present application provides a yoga action evaluation method, system, storage medium and electronic device.

[0005] The technical scheme of a yoga action evaluation method of the present application is as follows:

[0006] A three-dimensional skeleton information fusion optimization algorithm is used to fuse and optimize the to-be-evaluated yoga action data of different perspectives, to obtain occlusion-completed three-dimensional human skeleton data;

[0007] A trained hierarchical cascaded graph convolution network model is used to identify the occlusion-completed three-dimensional human skeleton data, to obtain the yoga action category and the yoga action completion score corresponding to the to-be-evaluated yoga action data;

[0008] The difficulty coefficient corresponding to the yoga action category is obtained, and the evaluation result of the to-be-evaluated yoga action data is obtained according to the difficulty coefficient and the yoga action completion score.

[0009] The yoga action evaluation method has the following beneficial effects:

[0010] The method of the present application realizes yoga posture classification and accurate evaluation of the yoga action to be evaluated by constructing a cascade graph convolutional neural network model, and provides strong support for subsequent targeted improvement of the yoga action according to the evaluation result.

[0011] On the basis of the above-mentioned scheme, the yoga action evaluation method of the present application can be further improved as follows.

[0012] Further, the three-dimensional skeleton information fusion optimization algorithm is used to fuse and optimize the different-view yoga action data to be evaluated to obtain occlusion-completed three-dimensional human skeleton data, which comprises:

[0013] The iterative closest point algorithm is used to register the different-view yoga action data to be evaluated to obtain a transformation matrix between different views;

[0014] The different-view yoga action data to be evaluated is fused according to the transformation matrix to obtain original three-dimensional human skeleton data;

[0015] The original three-dimensional human skeleton data is optimized according to the parametric human model to obtain the occlusion-completed three-dimensional human skeleton data.

[0016] Further, the different-view yoga action data to be evaluated comprises main-view yoga action data to be evaluated and side-view yoga action data to be evaluated; and the different-view yoga action data to be evaluated is fused according to the transformation matrix to obtain original three-dimensional human skeleton data, which comprises:

[0017] The main-view yoga action data to be evaluated and the side-view yoga action data to be evaluated are fused according to a preset fusion transformation formula and the transformation matrix to obtain the original three-dimensional human skeleton data;

[0018] The preset fusion transformation formula is: f represents the main-view yoga action data to be evaluated, s represents the side-view yoga action data to be evaluated, is a joint coordinate of an i th joint of the original three-dimensional human skeleton data, T sf is the transformation matrix, S fi is a joint coordinate of an i th joint of the main-view yoga action data to be evaluated, S si is a joint coordinate of an i th joint of the side-view yoga action data to be evaluated, W fi is S fi a corresponding weight, Wsi S si corresponding weight, θ i ∈(0, 90), z i is a depth value corresponding to the ith joint, θ i is a joint angle corresponding to the ith joint, c i is a confidence coefficient corresponding to the ith joint, W i is a weight corresponding to the ith joint.

[0019] Further, the optimization of the original three-dimensional human skeleton data according to the parameterized human body model to obtain the occlusion-completed three-dimensional human skeleton data comprises:

[0020] According to the optimization of the original three-dimensional human skeleton data according to the parameterized human body model, when the objective function reaches the minimum, the occlusion-completed three-dimensional human skeleton data is obtained;

[0021] Wherein, the objective function is: E fused (θ, β) = ω pro E pro + ω shape E shape , E pro indicates a function of aligning the two-dimensional projection of the original three-dimensional human skeleton data to the three-dimensional joint, E shape indicates a human body shape prior constraint function, ω pro is a preset balance weight of E pro , ω shape is a preset balance weight of E shape , θ indicates the length of the control skeleton, and β indicates the posture of each joint.

[0022] Further, it further comprises: collecting original yoga action data of different angles by using a calibration camera, and performing key frame screening on the original yoga action data of each angle to obtain the different angle to be evaluated yoga action data.

[0023] Further, it further comprises:

[0024] Collecting a plurality of yoga action training data of different angles, and using the three-dimensional skeleton information fusion optimization algorithm to obtain the three-dimensional human skeleton training data corresponding to each different angle of the yoga action training data;

[0025] Each three-dimensional human skeleton training data is marked with hierarchical action categories and scored with action quality, and all the marked three-dimensional human skeleton training data are used to train the original hierarchical cascaded graph convolutional network model to obtain the trained hierarchical cascaded graph convolutional network model.

[0026] Further, the trained hierarchical cascaded graph convolutional network model comprises: a first-level cascaded network, a second-level cascaded network and a third-level cascaded network; the occlusion-completed three-dimensional human skeleton data is recognized by using the trained hierarchical cascaded graph convolutional network model, so as to obtain the yoga action category corresponding to the to-be-evaluated yoga action data and the yoga action completion score.

[0027] The first-level cascaded network and the second-level cascaded network are used for coarse-grained and fine-grained action recognition of the occlusion-completed three-dimensional human skeleton data, so as to obtain the yoga action category.

[0028] The third-level cascaded network is used for action evaluation of the occlusion-completed three-dimensional human skeleton data, so as to obtain the yoga action completion score.

[0029] The technical scheme of the yoga action evaluation system of the present application is as follows:

[0030] The system comprises: a processing module, an identification module and an evaluation module.

[0031] The processing module is used for: using a three-dimensional skeleton information fusion optimization algorithm to fuse and optimize the to-be-evaluated yoga action data of different angles, so as to obtain occlusion-completed three-dimensional human skeleton data.

[0032] The identification module is used for: using a trained hierarchical cascaded graph convolutional network model to recognize the occlusion-completed three-dimensional human skeleton data, so as to obtain the yoga action category corresponding to the to-be-evaluated yoga action data and the yoga action completion score.

[0033] The evaluation module is used for: obtaining the difficulty coefficient corresponding to the yoga action category, and obtaining the evaluation result of the to-be-evaluated yoga action data according to the difficulty coefficient and the yoga action completion score.

[0034] The beneficial effects of the yoga action evaluation system of the present application are as follows:

[0035] The system of the present application realizes yoga posture classification and accurate evaluation of the to-be-evaluated yoga action by constructing a cascaded graph convolutional neural network model to recognize and evaluate the yoga action, which provides strong support for users to improve the yoga action according to the evaluation result.

[0036] The technical scheme of the storage medium of the present application is as follows:

[0037] The storage medium stores instructions, and when a computer reads the instructions, the computer executes the steps of the yoga action evaluation method of the present application.

[0038] The technical scheme of the electronic device of the present application is as follows:

[0039] A computer program product comprising a memory, a processor, and a computer program stored on the memory and loadable on the processor, characterized in that the processor, when executing the computer program, causes the computer to perform the steps of a yoga action evaluation method according to the present application. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A flowchart of a yoga action evaluation method according to an embodiment of the present application;

[0041] Figure 2 A yoga action category diagram in a yoga action evaluation method according to an embodiment of the present application;

[0042] Figure 3 A recognition accuracy diagram of a yoga action in a yoga action evaluation method according to an embodiment of the present application;

[0043] Figure 4 An evaluation result diagram of a yoga action in a yoga action evaluation method according to an embodiment of the present application;

[0044] Figure 5 A structure diagram of a yoga action evaluation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] As shown in the drawings, a yoga action evaluation method according to an embodiment of the present application includes the following steps: Figure 1

[0046] S1, using a three-dimensional skeleton information fusion optimization algorithm to fuse and optimize different view angle yoga action data to be evaluated, to obtain occlusion-completed three-dimensional human skeleton data.

[0047] The three-dimensional skeleton information fusion optimization algorithm refers to obtaining different length images and skeleton sequences of a certain yoga action from different angles, selecting multiple key frame data after spatio-temporal alignment to describe the yoga action, using an iterative closest point algorithm (ICP algorithm) to align and fuse the three-dimensional human skeleton posture for the key frame skeleton data under each angle, and optimizing the human skeleton data through embedding a parameterized human model (SMPL), to obtain the occlusion-completed three-dimensional human skeleton data corresponding to the yoga action.

[0048] The yoga action data to be evaluated is image data of a yoga action after preprocessing.

[0049] The occlusion-completed three-dimensional human skeleton data is three-dimensional human skeleton data obtained after optimization by the three-dimensional skeleton information fusion optimization algorithm.

[0050] ​S2, recognizing the occlusion-completed three-dimensional human skeleton data by using the trained hierarchical cascaded graph convolutional network model to obtain a yoga action category and a yoga action completion score corresponding to the to-be-evaluated yoga action data.

[0051] The trained hierarchical cascaded graph convolutional network model is obtained by using a large amount of training yoga action data to perform hierarchical category label and score label annotation and training.

[0052] The yoga action category includes two-level classification, specifically including a first-level and a second-level. The name of each level is named according to a professional yoga action classification standard. The first level is a coarse-grained classification, and the second level is a fine-grained classification. For example, as shown in the table, the first level includes but is not limited to: Inverted III and Balance VIII, and the second level includes but is not limited to: Single-Leg Downward-Facing Dog-17 and Crossed Balance I-15. Figure 2

[0053] The yoga action completion score is divided into four levels (0-3). The higher the score, the closer the to-be-evaluated yoga action data is to the standard action data.

[0054] S3, obtaining a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the to-be-evaluated yoga action data according to the difficulty coefficient and the yoga action completion score.

[0055] Each yoga action category corresponds to a difficulty coefficient. The difficulty coefficient is obtained by referring to the body position standard of fitness yoga published on the official website of the National Health Yoga Guidance Committee. The evaluation result is composed of the product of the difficulty coefficient and the yoga action completion score.

[0056] Preferably, the S1 includes:

[0057] S11, registering the to-be-evaluated yoga action data in different perspectives by using an iterative closest point algorithm to obtain a transformation matrix between different perspectives.

[0058] The iterative closest point algorithm (ICP algorithm) is prior art. Specifically, the ICP algorithm is used to register the to-be-evaluated yoga action data in different perspectives (main position and side position in this embodiment), and the three-dimensional coordinates of the registration result are used as initial values. Then, the three-dimensional points are calculated through the corresponding depth image and camera intrinsic parameters. After ICP algorithm registration, the transformation matrix T between the two views (from the side position to the main position) is obtained. sf .

[0059] S12, fusing the to-be-evaluated yoga action data in different perspectives according to the transformation matrix to obtain original three-dimensional human skeleton data. ​

[0060] Specifically, the joint depth values, angle values and recognition confidence of the yoga action data to be evaluated at different view angles are fused by a transformation matrix.

[0061] S13, optimizing the original three-dimensional human skeleton data according to the parameterized human body model, to obtain the occlusion-completed three-dimensional human skeleton data.

[0062] Wherein, the parameterized human body model (SMPL) is prior art. Specifically, the original three-dimensional human skeleton data is adjusted by using the parameterized human body model (SMPL) to adjust the posture of each human joint and the position of the whole human body, to obtain the occlusion-completed three-dimensional human skeleton data.

[0063] Preferably, the yoga action data to be evaluated at different view angles includes main-view yoga action data to be evaluated and side-view yoga action data to be evaluated; the S12 includes:

[0064] According to a preset fusion transformation formula and the transformation matrix, the main-view yoga action data to be evaluated and the side-view yoga action data to be evaluated are fused to obtain the original three-dimensional human skeleton data;

[0065] Wherein, the preset fusion transformation formula is: f represents the main-view yoga action data to be evaluated, s represents the side-view yoga action data to be evaluated, is the joint coordinate of the i-th joint of the original three-dimensional human skeleton data, T sf is the transformation matrix, S fi is the joint coordinate of the i-th joint of the main-view yoga action data to be evaluated, S si is the joint coordinate of the i-th joint of the side-view yoga action data to be evaluated, W fi is S fi is the corresponding weight, W si is S si is the corresponding weight, is θ i ∈(0, 90), z i is the depth value corresponding to the i-th joint, θ i is the joint angle corresponding to the i-th joint, c i is the confidence coefficient corresponding to the i-th joint, W i is the weight corresponding to the i-th joint.

[0066] Preferably, the S13 includes:

[0067] The original three-dimensional human skeleton data is optimized based on the parameterized human body model. When the objective function reaches its minimum, the occlusion-completed three-dimensional human skeleton data is obtained.

[0068] The objective function is: E fused (θ, β) = ω pro E pro +ω shape E shape E pro E represents a function that aligns the two-dimensional projection of the original three-dimensional human skeleton data to the three-dimensional joints. shape Let ω represent the prior constraint function for human body shape. pro For E pro The preset balance weight, ω shape For E shape The preset balance weights are θ, which controls the length of the skeleton, and β, which controls the posture of each joint.

[0069] Preferably, it also includes: using a calibration camera to collect raw yoga movement data from different perspectives, and performing keyframe filtering on the raw yoga movement data from each perspective to obtain yoga movement data to be evaluated from the different perspectives.

[0070] The camera calibration process involves obtaining the internal parameters of each camera in the main and side view positions using the Kinect Azure SDK, and calculating the geometric relationship between the main and side views of the two cameras using the Matlab stereo camera calibration tool. The keyframe selection process involves mapping human skeleton data onto RGB images and visualizing it. Qualified keyframe data requires a complete number of key points with generally accurate positions. Each keyframe stores three different types of data: a color image, a depth image, and 3D human skeleton data.

[0071] More preferably, it also includes:

[0072] Yoga movement training data from multiple different perspectives were collected, and the three-dimensional skeleton information fusion optimization algorithm was used to obtain the three-dimensional human skeleton training data corresponding to each yoga movement training data from different perspectives.

[0073] The optimization method for yoga movement training data using a 3D skeleton information fusion optimization algorithm is consistent with the aforementioned algorithm.

[0074] Each 3D human skeleton training data is labeled with hierarchical action categories and assigned action quality scores. The original hierarchical convolutional graph network model is then trained using all labeled 3D human skeleton training data to obtain the trained hierarchical convolutional graph network model.

[0075] The original hierarchical cascaded graph convolutional network is obtained by cascading improvement of the two-stream graph convolutional model (2S-AGCN). The three-fold cascaded structure of the improved two-stream graph convolutional model is used to realize coarse-grained to fine-grained classification of yoga movements and quality assessment of specific yoga postures.

[0076] Specifically, two cameras, a main camera and a side camera, were used to film multiple yoga poses. Camera calibration was performed before each pose was filmed. During each calibration, approximately 100 checkerboard-style images were captured. The checkerboard grid was 9×12, with each grid cell having an actual side length of 10cm. The intrinsic parameter matrix for each camera was obtained using the Kinect Azure SDK. The geometric relationship between the main and side cameras was calculated using a Matlab stereo camera calibration tool. The average reprojection error of the stereo camera calibration was 2.81 pixels.

[0077] Secondly, keyframe selection: For each action segment, frame extraction is performed, leaving 5 keyframe RGB images for each action. The Kinect SDK is used to map the skeleton data onto the RGB images and visualize them. Finally, through manual selection, 2 keyframes are retained for each static action and 10-20 frames for dynamic actions. The selected keyframe data must have a complete number of key points and be in basically accurate positions, accurately identifying the left and right sides of the upper and lower limbs of the human body, without any left-right reversal. Each keyframe stores three different types of data: a color image, a depth image, and 3D skeleton data. The color image is used to describe texture information, the depth image is used to describe distance information, and the 3D skeleton data is used to describe the human skeleton pose.

[0078] Finally, the 3D human skeleton training data collected from different scenes were labeled and stored hierarchically. The folders for the 7 male subjects were designated M01, M02, ..., M07, and the folders for the 15 female subjects were designated F01, F02, ..., F15. Then, the data was categorized according to the first-level action classification, numbered A01, A02, ..., A10. Further detailed second-level classifications were performed under the first-level classification, with pose IDs (a01, a03, ..., a117) representing the folder names of the second-level classifications. Each action stored its main camera and side camera pose data separately, and each sub-pose folder corresponding to each action segment contained color images, depth maps, and human skeleton data files. The two-stream graph convolutional model (2S-AGCN) was improved to obtain a hierarchical cascaded graph convolutional network action recognition model. A yoga pose recognition task was constructed using the first and second levels of the cascaded network, yielding results at both the first and second levels. After completing the first-level classification using 2S-AGCN, erroneous samples are filtered out. Then, each type of action is input into the corresponding second-level network to identify the specific action name. The advantage of this approach is that, since the main network and branch networks are in a unified framework, the feature maps extracted at the beginning can be shared throughout the entire network, rather than collecting features from the raw data at each layer of the network.

[0079] For example, Figure 3 The analysis of motion recognition accuracy on the constructed 3D human skeleton training dataset is shown, with an accuracy of 84.03% when FC (fully convolutional layer in a deep neural network) = 256.

[0080] Preferably, the trained hierarchical cascaded graph convolutional network model includes: a first-level cascaded network, a second-level cascaded network, and a third-level cascaded network.

[0081] The first-level cascaded network is used to identify the broad categories of yoga movements (coarse-grained). The second-level cascaded network is used to identify the posture names of yoga movements (fine-grained), and the third-level cascaded network is used to obtain the movement completion score of the yoga movement.

[0082] S2 includes:

[0083] S21. Using the first-level cascaded network and the second-level cascaded network, coarse-grained and fine-grained motion recognition is performed on the occluded and completed three-dimensional human skeleton data to obtain the yoga motion category.

[0084] The first-level cascaded network of the 2S-AGCN model is used to obtain the yoga movement categories. Then, each category of yoga movement is input into the corresponding second-level cascaded network to identify the specific posture name.

[0085] S22. The third-level cascaded network is used to evaluate the occlusion-completed three-dimensional human skeleton data to obtain the yoga movement completion score.

[0086] The third-level cascaded network of the 2S-AGCN model is used to predict the completion score for each yoga pose.

[0087] Specifically, such as Figure 4 The effect of the three-dimensional human skeleton training dataset on the constructed yoga fitness dataset for motion quality assessment is shown. Taking motion numbers 15 and 83 as examples, it demonstrates the automatic scoring of the target hierarchical cascaded graph convolutional network model based on the motions of four different subjects.

[0088] The technical solution of this embodiment identifies and evaluates yoga movements by constructing a cascaded graph convolutional neural network model, thereby achieving yoga posture classification and accurate evaluation of the yoga movements to be evaluated, providing strong support for users to make targeted improvements to their yoga movements based on the evaluation results.

[0089] In another embodiment of a yoga movement assessment method of the present invention, it includes:

[0090] S01. A three-dimensional skeleton information fusion optimization algorithm is used to fuse and optimize the yoga movement data to be evaluated from different perspectives to obtain occlusion-complete three-dimensional human skeleton data.

[0091] S02. The trained hierarchical convolutional graph network model is used to identify the occluded and completed 3D human skeleton data to obtain the yoga movement category and yoga movement completion score corresponding to the yoga movement data to be evaluated.

[0092] S03. Obtain the difficulty coefficient corresponding to the yoga movement category, and obtain the evaluation result of the yoga movement data to be evaluated based on the difficulty coefficient and the yoga movement completion score.

[0093] S04. Based on the difference between the evaluation result and the corresponding standard result, obtain the target difference data, and determine whether the target difference data is greater than the preset difference data. If the determination result is yes, optimize the yoga movement data to be evaluated based on the target difference data, and return to step S1 until the target difference data is less than the preset difference data, and obtain the evaluation result of the yoga movement data to be evaluated.

[0094] In this embodiment, a preset difference data point can be set. For example, the preset difference data point for a certain yoga pose is 4, and the standard result is 15. When the subject's evaluation result is 10, the target difference data point is 5 (greater than the preset difference data point 4). At this point, the subject can understand the differences between their yoga pose to be evaluated and the corresponding standard yoga pose. The subject can continuously adjust their pose based on these differences, thereby re-evaluating and ultimately obtaining an evaluation result that meets the standard. By using the above steps, the differences between the poses performed by different subjects and the standard pose can be evaluated. While accurately identifying these differences, it also encourages subjects to continuously improve and optimize their poses, achieving the goals of accurate prediction and effective training.

[0095] Figure 5 As shown, a yoga movement assessment system 200 according to an embodiment of the present invention includes: a processing module 210, a recognition module 220 and an assessment module 230;

[0096] The processing module 210 is used to: use a three-dimensional skeleton information fusion optimization algorithm to fuse and optimize yoga movement data to be evaluated from different perspectives to obtain occlusion-complete three-dimensional human skeleton data;

[0097] The recognition module 220 is used to: use the trained hierarchical convolutional graph network model to recognize the occluded and completed three-dimensional human skeleton data, and obtain the yoga movement category and yoga movement completion score corresponding to the yoga movement data to be evaluated;

[0098] The evaluation module 230 is used to: obtain the difficulty coefficient corresponding to the yoga movement category, and obtain the evaluation result of the yoga movement data to be evaluated based on the difficulty coefficient and the yoga movement completion score.

[0099] Preferably, the processing module 210 includes: a first processing module, a second processing module, and a third processing module;

[0100] The first processing module is used to: register the yoga movement data to be evaluated from different perspectives using an iterative nearest-point algorithm to obtain the transformation matrix between different perspectives.

[0101] The second processing module is used to: fuse the yoga movement data to be evaluated from different perspectives according to the transformation matrix to obtain the original three-dimensional human skeleton data.

[0102] The third processing module is used to: optimize the original three-dimensional human skeleton data according to the parametric human body model to obtain the occlusion-completed three-dimensional human skeleton data.

[0103] Preferably, the yoga movement data to be evaluated from different perspectives includes: yoga movement data to be evaluated from a main viewpoint and yoga movement data to be evaluated from a side viewpoint; the second processing module is specifically used for:

[0104] According to the preset fusion transformation formula and the transformation matrix, the yoga movement data to be evaluated from the main viewpoint and the yoga movement data to be evaluated from the side viewpoint are fused to obtain the original three-dimensional human skeleton data.

[0105] The preset fusion transformation formula is as follows: f represents the yoga movement data to be evaluated from the main viewpoint, and s represents the yoga movement data to be evaluated from the side viewpoint. Let T be the joint coordinate of the i-th joint in the original three-dimensional human skeleton data. sf Let S be the transformation matrix. fi S represents the joint coordinates of the i-th joint in the yoga movement data to be evaluated from the main perspective. si W represents the joint coordinates of the i-th joint in the yoga movement data to be evaluated from the side view. fi For S fi The corresponding weight, W si For S si The corresponding weights θ i ∈(0, 90), z i Let θ be the depth value corresponding to the i-th joint. i Let c be the joint angle corresponding to the i-th joint. i W represents the confidence coefficient corresponding to the i-th joint. i The weight is the weight corresponding to the i-th joint.

[0106] Preferably, the third processing module is specifically used for:

[0107] The original three-dimensional human skeleton data is optimized based on the parameterized human body model. When the objective function reaches its minimum, the occlusion-completed three-dimensional human skeleton data is obtained.

[0108] The objective function is: E fused (θ, β) = ω pro E pro +ω shape E shape E pro E represents a function that aligns the two-dimensional projection of the original three-dimensional human skeleton data to the three-dimensional joints. shape Let ω represent the prior constraint function for human body shape. pro For E pro The preset balance weight, ω shape For E shapeThe preset balance weights are θ, which controls the length of the skeleton, and β, which controls the posture of each joint.

[0109] Preferably, it also includes: a preprocessing module;

[0110] The preprocessing module is used to: acquire raw yoga movement data from different perspectives using a calibration camera, and perform keyframe filtering on the raw yoga movement data from each perspective to obtain yoga movement data to be evaluated from the different perspectives.

[0111] Preferably, it further includes: a training module; the training module is used for:

[0112] Multiple yoga movement training data from different perspectives were collected, and the three-dimensional skeleton information fusion optimization algorithm was used to obtain the three-dimensional human skeleton training data corresponding to each yoga movement training data from different perspectives.

[0113] Each 3D human skeleton training data is labeled with hierarchical action categories and assigned action quality scores. The original hierarchical convolutional graph network model is then trained using all labeled 3D human skeleton training data to obtain the trained hierarchical convolutional graph network model.

[0114] Preferably, the target hierarchical concatenated graph convolutional network model includes: a first-level concatenated network, a second-level concatenated network, and a third-level concatenated network;

[0115] The identification module 220 is specifically used for:

[0116] The first-level cascaded network and the second-level cascaded network are used to perform coarse-grained and fine-grained motion recognition on the occluded and completed 3D human skeleton data to obtain the yoga motion category;

[0117] The third-level cascaded network is used to evaluate the occlusion-completed 3D human skeleton data to obtain the yoga pose completion score.

[0118] The technical solution of this embodiment identifies and evaluates yoga movements by constructing a cascaded graph convolutional neural network model, thereby achieving yoga posture classification and accurate evaluation of the yoga movements to be evaluated, providing strong support for users to make targeted improvements to their yoga movements based on the evaluation results.

[0119] The steps for implementing the corresponding functions of each parameter and module in the yoga movement assessment system 200 of the present invention described above can be referred to the parameters and steps in the embodiments of the yoga movement assessment method described above, and will not be repeated here.

[0120] An embodiment of the present invention provides a storage medium, comprising: the storage medium storing instructions, which, when a computer reads the instructions, cause the computer to execute the steps of a yoga movement assessment method as described above. For details, please refer to the parameters and steps in the embodiment of the yoga movement assessment method described above, which will not be repeated here.

[0121] Computer storage media, such as USB flash drives and external hard drives.

[0122] An electronic device provided by an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The device is characterized in that, when the processor executes the computer program, it causes the computer to perform the steps of a yoga movement assessment method as described above. Specific details regarding the parameters and steps in the embodiments of the yoga movement assessment method described above are available for reference and will not be repeated here.

[0123] Those skilled in the art will know that the present invention can be implemented as a method, system, storage medium, and electronic device.

[0124] Therefore, the present invention can be implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention can also be implemented as a computer program product contained in one or more computer-readable media, which contains computer-readable program code. Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Although embodiments of the invention have been shown and described above, it is to be understood that these embodiments are exemplary and should not be construed as limiting the invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the invention.

Claims

1. A method for assessing yoga postures, characterized in that, include: A 3D skeleton information fusion optimization algorithm is used to fuse and optimize yoga movement data to be evaluated from different perspectives to obtain occlusion-complete 3D human skeleton data. The trained hierarchical convolutional graph network model is used to identify the occluded and completed 3D human skeleton data to obtain the yoga movement category and yoga movement completion score corresponding to the yoga movement data to be evaluated. Obtain the difficulty coefficient corresponding to the yoga movement category, and obtain the evaluation result of the yoga movement data to be evaluated based on the difficulty coefficient and the yoga movement completion score; The algorithm employs a 3D skeleton information fusion and optimization method to fuse and optimize yoga movement data from different perspectives, resulting in occlusion-completed 3D human skeleton data, including: The iterative nearest point algorithm is used to register the yoga movement data to be evaluated from different perspectives to obtain the transformation matrix between different perspectives; The yoga movement data to be evaluated from different perspectives are fused according to the transformation matrix to obtain the original three-dimensional human skeleton data. The original 3D human skeleton data is optimized based on the parametric human body model to obtain the occlusion-completed 3D human skeleton data. The yoga movement data to be evaluated from different perspectives includes: yoga movement data to be evaluated from a frontal view and yoga movement data to be evaluated from a side view; the process of fusing the yoga movement data to be evaluated from different perspectives according to the transformation matrix to obtain the original three-dimensional human skeleton data includes: According to the preset fusion transformation formula and the transformation matrix, the yoga movement data to be evaluated from the main viewpoint and the yoga movement data to be evaluated from the side viewpoint are fused to obtain the original three-dimensional human skeleton data. The preset fusion transformation formula is as follows: , This indicates the yoga movement data to be evaluated from the perspective of the user. This indicates the side-view data of the yoga movements to be evaluated. Let be the joint coordinates of the i-th joint in the original three-dimensional human skeleton data. Let be the transformation matrix. Let be the joint coordinates of the i-th joint in the yoga movement data to be evaluated from the main perspective. Let be the joint coordinates of the i-th joint in the yoga movement data to be evaluated from the side view. for The corresponding weights for The corresponding weights , Let be the depth value corresponding to the i-th joint. Let be the joint angle corresponding to the i-th joint. Let be the confidence coefficient corresponding to the i-th joint. The weight is the weight corresponding to the i-th joint.

2. The yoga posture assessment method according to claim 1, characterized in that, The optimization of the original 3D human skeleton data based on the parametric human body model to obtain the occlusion-completed 3D human skeleton data includes: The original three-dimensional human skeleton data is optimized based on the parameterized human body model. When the objective function reaches its minimum, the occlusion-completed three-dimensional human skeleton data is obtained. The objective function is: , This function represents aligning the two-dimensional projection of the original three-dimensional human skeleton data onto the three-dimensional joints. This represents the prior constraint function for human body morphology. for The preset balance weights, for The preset balance weights, This indicates control over bone length. This indicates the posture of each joint.

3. The yoga posture assessment method according to claim 1, characterized in that, Also includes: A calibrated camera was used to collect raw yoga movement data from different perspectives, and keyframes were filtered from the raw yoga movement data from each perspective to obtain yoga movement data to be evaluated from the different perspectives.

4. The yoga posture assessment method according to claim 1, characterized in that, Also includes: Multiple yoga movement training data from different perspectives were collected, and the three-dimensional skeleton information fusion optimization algorithm was used to obtain the three-dimensional human skeleton training data corresponding to each yoga movement training data from different perspectives. Each 3D human skeleton training data is labeled with hierarchical action categories and assigned action quality scores. The original hierarchical convolutional graph network model is then trained using all labeled 3D human skeleton training data to obtain the trained hierarchical convolutional graph network model.

5. The yoga movement assessment method according to claim 4, characterized in that, The trained hierarchical cascaded graph convolutional network model includes: a first-level cascaded network, a second-level cascaded network, and a third-level cascaded network; the process of using the trained hierarchical cascaded graph convolutional network model to identify the occluded and completed 3D human skeleton data to obtain the yoga movement category and yoga movement completion score corresponding to the yoga movement data to be evaluated includes: The first-level cascaded network and the second-level cascaded network are used to perform coarse-grained and fine-grained motion recognition on the occluded and completed 3D human skeleton data to obtain the yoga motion category; The third-level cascaded network is used to evaluate the occlusion-completed 3D human skeleton data to obtain the yoga pose completion score.

6. A yoga movement assessment system, characterized in that, include: Processing module, identification module, and evaluation module; The processing module is used to: use a three-dimensional skeleton information fusion optimization algorithm to fuse and optimize yoga movement data to be evaluated from different perspectives to obtain occlusion-complete three-dimensional human skeleton data; The recognition module is used to: use the trained hierarchical convolutional graph network model to recognize the occluded and completed three-dimensional human skeleton data, and obtain the yoga movement category and yoga movement completion score corresponding to the yoga movement data to be evaluated; The evaluation module is used to: obtain the difficulty coefficient corresponding to the yoga movement category, and obtain the evaluation result of the yoga movement data to be evaluated based on the difficulty coefficient and the yoga movement completion score; The processing module includes: a first processing module, a second processing module, and a third processing module; The first processing module is used to: register the yoga movement data to be evaluated from different perspectives using an iterative nearest point algorithm to obtain the transformation matrix between different perspectives; The second processing module is used to: fuse the yoga movement data to be evaluated from different perspectives according to the transformation matrix to obtain the original three-dimensional human skeleton data; The third processing module is used to: optimize the original three-dimensional human skeleton data according to the parametric human body model to obtain the occlusion-completed three-dimensional human skeleton data; The yoga movement data to be evaluated from different perspectives includes: yoga movement data to be evaluated from the main viewpoint and yoga movement data to be evaluated from the side viewpoint; the second processing module is specifically used for: According to the preset fusion transformation formula and the transformation matrix, the yoga movement data to be evaluated from the main viewpoint and the yoga movement data to be evaluated from the side viewpoint are fused to obtain the original three-dimensional human skeleton data. The preset fusion transformation formula is as follows: , This indicates the yoga movement data to be evaluated from the perspective of the user. This indicates the side-view data of the yoga movements to be evaluated. Let be the joint coordinates of the i-th joint in the original three-dimensional human skeleton data. Let be the transformation matrix. Let be the joint coordinates of the i-th joint in the yoga movement data to be evaluated from the main perspective. Let be the joint coordinates of the i-th joint in the yoga movement data to be evaluated from the side view. for The corresponding weights for The corresponding weights , Let be the depth value corresponding to the i-th joint. Let be the joint angle corresponding to the i-th joint. Let be the confidence coefficient corresponding to the i-th joint. The weight is the weight corresponding to the i-th joint.

7. A storage medium, characterized in that, The storage medium stores instructions that, when read by a computer, cause the computer to execute a yoga movement assessment method as described in any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the computer to perform a yoga movement assessment method as described in any one of claims 1 to 5.

Images