Aerobics training action analysis method, system and storage medium

Abstract

The present invention relates to the field of image recognition technology, and more specifically to a method, system and storage medium for analyzing aerobics training movements. The method comprises: step S1: obtaining a user's first movement video in real time and monitoring the user's heart rate; step S2: obtaining a second movement video, periodically obtaining sub-movement frames from the second movement video, and extracting a first joint trajectory, comparing it with a reference joint trajectory, obtaining a first comparison result, and obtaining a sub-movement trajectory according to the first comparison result, the second joint trajectory and the third joint trajectory; step S3: comparing the line changes of key points with the reference lines of key points, and obtaining a second comparison result; step S4: estimating the first heart rate change of each target sub-section in the sub-movement trajectory and comparing it with the second heart rate change, and obtaining a third comparison result. The present invention solves the problem of inaccurate analysis results of aerobics training movements and improves the accuracy of the analysis results of aerobics training movements.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and more specifically to a method, system and storage medium for analyzing aerobics training movements. Background Art

[0002] With the continuous development of science and technology, the application of motion capture, computer vision, deep learning and other technologies in the field of motion analysis has become increasingly mature. These technologies not only improve the accuracy and real-time performance of motion analysis, but also expand its application field. For example, in aerobics or other training, motion analysis can help coaches more accurately evaluate the user's technical level and formulate targeted training plans; similar prior art includes Chinese patent CN113691867A, which discloses a motion analysis method, device, electronic device and storage medium. The method includes: obtaining multiple original video files of the target human body; each of the original video files is obtained by synchronously shooting the target human body by a camera device set at different positions; synthesizing the multiple original video files to obtain a stereoscopic video file; extracting the first motion feature data of the target human body from the multiple original video files and the stereoscopic video file; comparing the first motion feature data with the second motion feature data of the demonstration human body; generating exercise suggestions based on the comparison results and the body data received from at least one wearable device worn by the target human body; prompting the exercise suggestions, and displaying at least one of the multiple original video files and the stereoscopic video files, which can improve the accuracy of motion analysis and provide more suitable exercise suggestions. Similar prior art includes Korean patent KR102201050B1, which discloses a dance movement analysis and evaluation device installed on a mobile device, including a shooting unit for shooting a test dance image, and a test object composed of joints and joints. A link to the test object in the test dance image is used. The object generation unit generates a pattern, and the evaluation unit compares the test object pattern with the sample object pattern, and evaluates the joint angles of the test dance image object pattern and the sample object pattern by comparing the tested joint angles. Both of the above documents solve the problem of motion capture, but it is not possible to accurately analyze whether the above movements are standard based on the appearance of the movements alone, and the analysis results are not accurate enough. Summary of the invention

[0003] In order to better solve the above problems, the present invention provides a method for analyzing aerobics training movements. When a user performs aerobics training, the user wears professional aerobics clothing and monitoring equipment, and performs movement guidance through a reference video. The method comprises the following steps:

[0004] Step S1: acquiring a multi-angle first action video of the user in real time through multiple cameras of the camera unit, and monitoring the user's heart rate in real time through the monitoring device;

[0005] Step S2: adjusting the playback speed of the first action video to obtain a second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video, periodically obtaining sub-action frames from the second action video, and extracting the first joint trajectory of the sub-action frame by an extraction unit, and comparing the first joint trajectory with the corresponding reference joint trajectory from the reference video to obtain a first comparison result, and obtaining the sub-action trajectory according to the first comparison result and the second joint trajectory of the previous cycle and the third joint trajectory of the next cycle;

[0006] Step S3: obtaining the action rule corresponding to each target sub-segment in the sub-action trajectory, amplifying the key point part corresponding to the action rule in the sub-action video, and comparing the line change of the amplified key point part with the reference line of the key point part, and obtaining a second comparison result, wherein the sub-action video is the video corresponding to the sub-action trajectory in the second action video;

[0007] Step S4: estimating a first heart rate change of each target sub-segment in the sub-motion trajectory and comparing the first heart rate change with a second heart rate change detected by the monitoring device corresponding to the target sub-segment to obtain a third comparison result;

[0008] Step S5: obtaining the analysis result of the sub-action according to the second comparison result and the third comparison result corresponding to each target sub-segment in the sub-action trajectory.

[0009] As a preferred technical solution of the present invention, step S2 includes:

[0010] Step S21: reading all the reference videos from the storage unit, adjusting the playback speed of the first action video, and using the adjusted first action video as the second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video;

[0011] Step S22: placing the second action video and all the reference videos into a coordinate system with time as the x-axis, periodically acquiring the sub-action frames corresponding to a preset time period from the second action video, and extracting the first joint trajectory from the sub-action frames through an extraction unit, and extracting the reference joint trajectory from the reference video, wherein the preset time period is a time period corresponding to each of the reference videos;

[0012] Step S23: comparing the first joint trajectory with the reference joint trajectory, and when the first comparison result shows that the similarity between the two is greater than or equal to a preset similarity, using the first joint trajectory as the sub-action trajectory.

[0013] As a preferred technical solution of the present invention, the step S23 further includes: when the first comparison result is that the similarity between the two is less than the preset similarity, the first joint trajectory is divided into sub-segments and a target sub-segment whose similarity to the reference joint trajectory is greater than or equal to the preset similarity is obtained, and at least one reference sub-segment of the reference joint trajectory is obtained based on the target sub-segment and the reference joint trajectory, excluding a plurality of the target sub-segments;

[0014] The second joint trajectory of the sub-action frame of the previous cycle and the third joint trajectory of the sub-action frame of the next cycle are respectively obtained, and the second joint trajectory and the third joint trajectory are respectively compared with each of the reference sub-segments, and other target sub-segments whose similarity with the reference sub-segment is greater than or equal to the preset similarity are obtained, and the sub-action trajectories of all the target sub-segments are obtained in chronological order.

[0015] As a preferred technical solution of the present invention, step S3 includes the following steps:

[0016] Step S31: reading the action rule corresponding to each target sub-segment in the sub-action trajectory from a storage unit, and obtaining the action key point of the target sub-segment based on the action rule, wherein the action key point is the force application part of the action corresponding to the target sub-segment;

[0017] Step S32: cutting out the sub-action video corresponding to the target sub-segment from the second action video, amplifying the key points of the sub-action video corresponding to the key points of the action, and obtaining line changes of the key points in the sub-action video based on the amplified key points;

[0018] Step S33: comparing the line change with the reference line of the key point, and obtaining a second comparison result.

[0019] As a preferred technical solution of the present invention, step S4 includes the following steps:

[0020] Step S41: acquiring each of the target sub-segments in the sub-motion trajectory, calculating the first exercise amount of the user corresponding to each of the segment trajectories through the segment trajectory of each of the target sub-segments and the user information, and calculating the first heart rate change of the user in the target sub-segment based on the heart rate of the user at the initial moment of the segment trajectory and the first exercise amount, wherein the user information includes the weight of the user;

[0021] Step S42: Obtain the user's heart rate at the start time and the end time of each target sub-segment through the monitoring device, and take the difference between the heart rate at the start time and the heart rate at the end time as the second heart rate change, and compare the first heart rate change with the second heart rate change to obtain the third comparison result.

[0022] As a preferred technical solution of the present invention, step S5 includes: when the second comparison result corresponding to each of the target sub-segments is less than or equal to the first threshold and the third comparison result corresponding to each of the target sub-segments is less than or equal to the second threshold, if the time of the target sub-segment in the sub-action trajectory is continuous, then the sub-action is a standard action and suitable for the training intensity of the user; if the time of the target sub-segment in the sub-action trajectory is discontinuous, then the sub-action is a qualified action; when at least one corresponding second comparison result in all the target sub-segments is greater than the first threshold, the sub-action is unqualified; when the second comparison result corresponding to each of the target sub-segments is less than or equal to the first threshold and at least one corresponding third comparison result in all the target sub-segments is greater than the second threshold, the training intensity of the sub-action is not suitable for the user.

[0023] As a preferred technical solution of the present invention, the step S5 also includes: when the training intensity of the sub-action is not suitable for the user, obtaining at least one third comparison result greater than the second threshold value corresponding to the target sub-segment, and reducing the action in the reference video corresponding to the target sub-segment from the first difficulty level to the second difficulty level, and restoring to the first difficulty level after the set training time.

[0024] As a preferred technical solution of the present invention, the step S5 further includes a step S6: repeating the steps S1 to S5 to obtain the analysis results of all the sub-actions.

[0025] The present invention also provides a system for analyzing aerobics training movements, the system being used to implement the above method, the system comprising:

[0026] A camera unit, used for acquiring a multi-angle first action video of the user in real time through multiple cameras;

[0027] Monitoring equipment for monitoring the user's heart rate;

[0028] The processing unit is configured to: adjust the playback speed of the first action video to obtain a second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video, periodically obtain sub-action frames from the second action video, and extract the first joint trajectory of the sub-action frame through the extraction unit, and compare the first joint trajectory with the corresponding reference joint trajectory from the reference video to obtain a first comparison result, and obtain the sub-action trajectory according to the first comparison result and the second joint trajectory of the previous cycle and the third joint trajectory of the next cycle;

[0029] The comparison unit is configured to: obtain the action rule corresponding to each target sub-segment in the sub-action trajectory, amplify the key point part corresponding to the action rule in the sub-action video, compare the line change of the amplified key point part with the reference line of the key point part, and obtain a second comparison result; estimate the first heart rate change of each target sub-segment in the sub-action trajectory and compare the first heart rate change with the second heart rate change detected corresponding to the target sub-segment to obtain a third comparison result;

[0030] An analysis unit is used to obtain an analysis result of the sub-action according to the second comparison result and the third comparison result corresponding to each of the target sub-segments in the sub-action trajectory.

[0031] The present invention also provides a computer storage medium, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the above method.

[0032] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0033] The present invention intercepts the above-mentioned sub-action frames corresponding to each of the above-mentioned reference videos from the above-mentioned second action video, extracts the above-mentioned first joint trajectory from the above-mentioned sub-action frames, and extracts the reference joint trajectory from the above-mentioned reference video, compares the above-mentioned first joint trajectory with the above-mentioned reference joint trajectory, and obtains the above-mentioned first comparison result. When the above-mentioned first comparison result is that the similarity between the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory is greater than or equal to the above-mentioned preset similarity, the above-mentioned first joint trajectory is used as the above-mentioned sub-action trajectory, and when the similarity between the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory is less than the above-mentioned preset similarity, the above-mentioned The first joint trajectory is divided into sub-segments, and the above sub-segments are compared with the above reference joint trajectory to obtain a target sub-segment whose similarity with the above reference joint trajectory is greater than or equal to the above preset similarity, and a reference sub-segment in the above reference joint trajectory that does not match the above target sub-segment is also obtained, and the sub-segments in the above second joint trajectory corresponding to the sub-action frame of the previous cycle of this cycle and the above third joint trajectory corresponding to the action frame of the next cycle are respectively compared with the above reference sub-segments, and other target sub-segments whose similarity with the above reference sub-segment is greater than or equal to the above preset similarity are obtained, and for the above target sub-segments that cannot be matched The reference sub-segment of the sub-segment generates a prompt message to prompt the user that the training action corresponding to the reference sub-segment is missing, and all the target sub-segments are spliced ​​in chronological order to obtain the sub-action trajectory. Since each aerobics action has a training target, which corresponds to a different training part, i.e., the force part of the muscle, the sub-action video corresponding to the target sub-segment is cut out from the second action video, and the force part corresponding to the sub-action video is enlarged to obtain the line change of the force part, and the line change is compared with the reference line change to obtain the second comparison result, so as to obtain the second comparison result through the first The second comparison result determines whether the force exerted at the force-exerting part of the above-mentioned sub-action is correct. By comparing the first heart rate change and the second heart rate change of the user in each of the above-mentioned target sub-segments, it is possible to accurately analyze whether the action corresponding to the above-mentioned segment trajectory is suitable for the user, thereby improving the accuracy of the action analysis. According to the second comparison result and the third comparison result corresponding to each target sub-segment in the sub-action trajectory, the analysis result of the sub-action is obtained and the difficulty level of the action corresponding to each of the above-mentioned target sub-segments is accurately adjusted. Through the mutual cooperation of the above-mentioned technical solutions, not only can the aerobics training actions be accurately analyzed, but also the training actions can be made more suitable for the training intensity of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The present invention is a flow chart of the aerobics training action analysis method;

[0035] Figure 2 It is a structural diagram of the aerobics training action analysis system of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] The present invention provides a method for analyzing aerobics training movements. When a user performs aerobics training, the user wears professional aerobics clothing and monitoring equipment, and receives movement guidance through reference videos, such as Figure 1 As shown, the method comprises the following steps:

[0038] Step S1: acquiring a multi-angle first action video of the user in real time through multiple cameras of the camera unit, and monitoring the user's heart rate in real time through the monitoring device;

[0039] Specifically, the first action videos of the user at multiple angles are acquired in real time through the multiple cameras, and more comprehensive training action video information of the user during aerobics training can be acquired through the uploaded first action video. The heart rate of the user during aerobics training is also detected in real time through the monitoring device. Through the technical solution, not only the first action video of the user in all directions during aerobics training can be acquired, but also the heart rate changes of the user during aerobics training can be acquired, thereby facilitating accurate analysis of the aerobics training actions.

[0040] Step S2: adjusting the playback speed of the first action video to obtain a second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video, periodically obtaining sub-action frames from the second action video, and extracting the first joint trajectory of the sub-action frame by an extraction unit, and comparing the first joint trajectory with the corresponding reference joint trajectory from the reference video to obtain a first comparison result, and obtaining the sub-action trajectory according to the first comparison result and the second joint trajectory of the previous cycle and the third joint trajectory of the next cycle;

[0041] Specifically, since the second action video is the entire action video of the user performing aerobics training, in order to facilitate the capture of the sub-action frames corresponding to each sub-action from the second action video, and since the playback speed of the second action video is consistent with that of the reference video, the second action video and all the reference videos are placed in the same coordinate system with time as the x-axis in chronological order, and the sub-action frames that are aligned in time with the reference video of each sub-action are the corresponding videos of the user's sub-actions. However, since the user's sub-actions may not be performed in a standard manner or there may be omissions in the middle, some wrong actions may be added, or the order of actions may be reversed, It is found that the above-mentioned sub-action frame cannot completely contain the sub-action trajectory corresponding to the above-mentioned reference video. Therefore, the above-mentioned sub-action frame corresponding to each of the above-mentioned reference video is intercepted from the above-mentioned second action video, and the above-mentioned first joint trajectory is extracted from the above-mentioned sub-action frame, and the reference joint trajectory is extracted from the above-mentioned reference video. By comparing the above-mentioned first joint trajectory with the above-mentioned reference joint trajectory and obtaining the above-mentioned first comparison result, when the above-mentioned first comparison result is that the similarity between the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory is greater than or equal to the above-mentioned preset similarity, that is, the above-mentioned first joint trajectory contains all the trajectories in the above-mentioned reference joint trajectory and the similarity between the two is relatively high, Here, the first joint trajectory is used as the sub-action trajectory. When the similarity between the first joint trajectory and the reference joint trajectory is less than the preset similarity, the first joint trajectory is divided into sub-segments, and the sub-segments are compared with the reference joint trajectory to obtain a target sub-segment with a similarity greater than or equal to the preset similarity to the reference joint trajectory, and a reference sub-segment in the reference joint trajectory that does not match the target sub-segment is also obtained. The reference sub-segment may be the action that the user has missed, the action sequence is incorrect, or other additional actions have been added. Therefore, through the sub-action of the previous cycle of this cycle, The sub-segments in the above-mentioned second joint trajectory corresponding to the action frame and the above-mentioned third joint trajectory corresponding to the action frame of the next cycle are respectively compared with the above-mentioned reference sub-segments, and the other above-mentioned target sub-segments whose similarity with the above-mentioned reference sub-segments is greater than or equal to the above-mentioned preset similarity are obtained. For the above-mentioned reference sub-segments that cannot match the above-mentioned target sub-segments, prompt information is generated to prompt that the training action corresponding to the above-mentioned reference sub-segments is missing, and all the above-mentioned target sub-segments are also spliced ​​in chronological order to obtain the above-mentioned sub-action trajectory. Through the above-mentioned technical solution, a relatively complete sub-action trajectory for each of the above-mentioned reference videos can be obtained, providing a basis for accurate action analysis.

[0042] Step S3: obtaining the action rule corresponding to each target sub-segment in the sub-action trajectory, amplifying the key point part corresponding to the action rule in the sub-action video, and comparing the line change of the amplified key point part with the reference line of the key point part, and obtaining a second comparison result, wherein the sub-action video is the video corresponding to the sub-action trajectory in the second action video;

[0043] Specifically, the above-mentioned target sub-segment only reflects the movement trajectory of the above-mentioned user sub-movement close to the above-mentioned reference movement. However, evaluating aerobics training movements is not only to judge whether the movements are qualified based on the appearance of the movements. Since each aerobics movement has a training target, which corresponds to different training parts, i.e., the force parts of the muscles, the above-mentioned action rules of each above-mentioned target sub-segment, i.e., the force parts of the movements, are also used, and the above-mentioned sub-movement video corresponding to the above-mentioned target sub-segment is cut out from the above-mentioned second action video, and the corresponding force parts in the above-mentioned sub-movement video are enlarged to obtain the line changes of the above-mentioned force parts, and the above-mentioned line changes are compared with the above-mentioned reference line changes to obtain the above-mentioned second comparison result, so as to judge whether the force parts of the above-mentioned sub-movements are correct through the second comparison result, and then more accurately judge whether the above-mentioned sub-movements are qualified.

[0044] Step S4: estimating a first heart rate change of each target sub-segment in the sub-motion trajectory and comparing the first heart rate change with a second heart rate change detected by the monitoring device corresponding to the target sub-segment to obtain a third comparison result;

[0045] Specifically, since each user has different physical conditions, such as flexibility and muscle strength, the first exercise amount of the user corresponding to each of the above-mentioned segment trajectories is calculated by the movement amplitude corresponding to the segment trajectory of each of the above-mentioned target sub-segments in the above-mentioned sub-action trajectory and the weight of the above-mentioned user, and the first heart rate change of the above-mentioned user after the above-mentioned first exercise amount is calculated according to the heart rate of the above-mentioned user at the initial moment of the above-mentioned segment trajectory. The heart rate of the above-mentioned user at the starting moment and the ending moment of the above-mentioned target sub-segment is obtained according to the above-mentioned monitoring equipment, and the difference between the two is taken as the above-mentioned second heart rate change. When the user normally performs the action corresponding to the above-mentioned segment trajectory, the above-mentioned first heart rate change and the above-mentioned second heart rate change are relatively close. When the difficulty of the action corresponding to the above-mentioned segment trajectory is close to the user's tolerance limit, the difference between the above-mentioned first heart rate change and the above-mentioned second heart rate change is large, indicating that although the user can complete the above-mentioned action, the intensity of the above-mentioned action may be too large for the user. Therefore, through the above-mentioned technical solution, it is possible to accurately analyze whether the action corresponding to the above-mentioned segment trajectory is suitable for the user, thereby improving the accuracy of the action analysis.

[0046] Step S5: obtaining the analysis result of the sub-action according to the second comparison result and the third comparison result corresponding to each target sub-segment in the sub-action trajectory.

[0047] Specifically, when the second comparison result of each of the target sub-segments in the sub-action trajectory is less than or equal to the first threshold value and the third comparison result of each of the target sub-segments is less than or equal to the second threshold value, that is, the user's force point is accurate in each of the target sub-segments, and the first heart rate change and the second heart rate change of the user in each of the target sub-segments are relatively close, it means that when the user performs the action corresponding to each of the target sub-segments, the action rule is met and the intensity is suitable for the user. By checking whether each of the target sub-segments is time-continuous, the sub-action is a standard action. If there is at least one time discontinuity between the target sub-segments, the sub-action is qualified. When at least one of the corresponding second comparison results in all the above-mentioned target sub-segments is greater than the above-mentioned first threshold, that is, the action in at least one target sub-segment does not comply with the corresponding action rule, and therefore the above-mentioned sub-action is considered unqualified; when the above-mentioned second comparison results corresponding to each of the above-mentioned target sub-segments are less than or equal to the first threshold and when at least one of the corresponding third comparison results in all the above-mentioned target sub-segments is greater than the above-mentioned second threshold, that is, the actions corresponding to the above-mentioned target sub-segments comply with the action rules, but the actual heart rate change in at least one target sub-segment is significantly different from the estimated heart rate change, indicating that the intensity of the above-mentioned sub-action is not suitable for the above-mentioned user. Through the above-mentioned technical solution, the accuracy of training action analysis is improved.

[0048] Furthermore, the step S2 comprises:

[0049] Step S21: reading all the reference videos from the storage unit, adjusting the playback speed of the first action video, and using the adjusted first action video as the second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video;

[0050] Specifically, by reading the reference video from the storage unit, each sub-action corresponds to one reference video, and by adjusting the playback speed of the first action video so that the playback speed of the second action video is consistent with that of the reference video, it is convenient to obtain the sub-action trajectory corresponding to the first action video.

[0051] Step S22: placing the second action video and all the reference videos into a coordinate system with time as the x-axis, periodically acquiring the sub-action frames corresponding to a preset time period from the second action video, and extracting the first joint trajectory from the sub-action frames through an extraction unit, and extracting the reference joint trajectory from the reference video, wherein the preset time period is a time period corresponding to each of the reference videos;

[0052] Specifically, since the second action video is the entire action video of the user performing aerobics training, in order to facilitate the extraction of sub-action frames corresponding to each sub-action from the second action video, and since the playback speed of the second action video is consistent with that of the reference video, the second action video and all the reference videos are placed in the same coordinate system with time as the x-axis in chronological order, and the sub-action frames that are aligned in time with the reference video of each sub-action are the corresponding videos of the user's sub-actions. However, since the user's sub-actions may not be performed in a standard manner or there may be omissions, some wrong actions added, or the order of actions reversed, the sub-action frames cannot completely contain the sub-action trajectories corresponding to the reference video. Therefore, the sub-action frames corresponding to each reference video are extracted from the second action video, and the first joint trajectory is extracted from the sub-action frames. The extraction process is a prior art and will not be repeated here. The reference joint trajectory is extracted from the reference video. Through the above technical solution, a foundation is laid for further comparing the first joint trajectory with the reference joint trajectory to determine whether the sub-action frame includes the sub-action trajectory in the reference video.

[0053] Step S23: comparing the first joint trajectory with the reference joint trajectory, and when the first comparison result shows that the similarity between the two is greater than or equal to a preset similarity, using the first joint trajectory as the sub-action trajectory.

[0054] Specifically, by comparing the above-mentioned first joint trajectory with the above-mentioned reference joint trajectory and obtaining the above-mentioned first comparison result, when the above-mentioned first comparison result is that the similarity between the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory is greater than or equal to the above-mentioned preset similarity, that is, the above-mentioned first joint trajectory contains all the trajectories in the above-mentioned reference joint trajectory and the similarity between the two is relatively high, therefore, the above-mentioned first joint trajectory is used as the above-mentioned sub-action trajectory, and the above-mentioned first joint trajectory is also the only target sub-trajectory of the above-mentioned sub-action trajectory. Through the above-mentioned technical solution, the sub-action trajectory can be obtained when the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory have a high similarity.

[0055] Further, the step S23 further comprises: when the first comparison result is that the similarity between the two is less than the preset similarity, dividing the first joint trajectory into sub-segments and obtaining a target sub-segment having a similarity with the reference joint trajectory greater than or equal to the preset similarity, and obtaining at least one reference sub-segment of the reference joint trajectory excluding a plurality of the target sub-segments based on the target sub-segment and the reference joint trajectory;

[0056] The second joint trajectory of the sub-action frame of the previous cycle and the third joint trajectory of the sub-action frame of the next cycle are respectively obtained, and the second joint trajectory and the third joint trajectory are respectively compared with each of the reference sub-segments, and other target sub-segments whose similarity with the reference sub-segment is greater than or equal to the preset similarity are obtained, and the sub-action trajectories of all the target sub-segments are obtained in chronological order.

[0057] Specifically, when the similarity between the first joint trajectory and the reference joint trajectory is less than the preset similarity, the first joint trajectory is divided into sub-segments, and the sub-segments are compared with the reference joint trajectory to obtain a target sub-segment whose similarity to the reference joint trajectory is greater than or equal to the preset similarity, and a reference sub-segment in the reference joint trajectory that does not match the target sub-segment is also obtained. The reference sub-segment may be the action that the user has missed, the action sequence is incorrect, or other additional actions have been added. Therefore, the second level corresponding to the sub-action frame of the previous cycle of this cycle is obtained. The sub-segments in the above-mentioned third joint trajectory corresponding to the joint trajectory and the sub-action frames of the next cycle are compared with the above-mentioned reference sub-segments respectively, and the other above-mentioned target sub-segments whose similarity with the above-mentioned reference sub-segments is greater than or equal to the above-mentioned preset similarity are obtained, and for the above-mentioned reference sub-segments that cannot match the above-mentioned target sub-segments, a prompt message is generated to prompt the user that the training action corresponding to the above-mentioned reference sub-segments is missing, and all the above-mentioned target sub-segments are also spliced ​​in chronological order to obtain the above-mentioned sub-action trajectory. Through the above-mentioned technical scheme, a relatively complete sub-action trajectory for each of the above-mentioned reference videos can be obtained, providing a basis for accurate action analysis.

[0058] Furthermore, step S3 includes the following steps:

[0059] Step S31: reading the action rule corresponding to each target sub-segment in the sub-action trajectory from a storage unit, and obtaining the action key point of the target sub-segment based on the action rule, wherein the action key point is the force application part of the action corresponding to the target sub-segment;

[0060] Step S32: cutting out the sub-action video corresponding to the target sub-segment from the second action video, amplifying the key points of the sub-action video corresponding to the key points of the action, and obtaining line changes of the key points in the sub-action video based on the amplified key points;

[0061] Step S33: comparing the line change with the reference line of the key point, and obtaining a second comparison result.

[0062] Specifically, the above-mentioned target sub-segment only reflects the movement trajectory of the above-mentioned user sub-movement close to the above-mentioned reference movement. However, evaluating aerobics training movements is not only to judge whether the movements are qualified based on the appearance of the movements. Since each aerobics movement has a training target, which corresponds to different training parts, i.e., the force parts of the muscles, the above-mentioned action rules of each above-mentioned target sub-segment, i.e., the force parts of the movements, are also used, and the above-mentioned sub-movement video corresponding to the above-mentioned target sub-segment is cut out from the above-mentioned second action video, and the corresponding force parts in the above-mentioned sub-movement video are enlarged to obtain the line changes of the above-mentioned force parts, and the above-mentioned line changes are compared with the above-mentioned reference line changes to obtain the above-mentioned second comparison result, so as to judge whether the force parts of the above-mentioned sub-movements are correct through the second comparison result, and then more accurately judge whether the above-mentioned sub-movements are qualified.

[0063] Furthermore, the step S4 comprises the following steps:

[0064] Step S41: acquiring each of the target sub-segments in the sub-motion trajectory, calculating the first exercise amount of the user corresponding to each of the segment trajectories through the segment trajectory of each of the target sub-segments and the user information, and calculating the first heart rate change of the user in the target sub-segment based on the heart rate of the user at the initial moment of the segment trajectory and the first exercise amount, wherein the user information includes the weight of the user;

[0065] Step S42: Obtain the user's heart rate at the start time and the end time of each target sub-segment through the monitoring device, and take the difference between the heart rate at the start time and the heart rate at the end time as the second heart rate change, and compare the first heart rate change with the second heart rate change to obtain the third comparison result.

[0066] Specifically, since each user has different physical conditions, such as flexibility and muscle strength, the first exercise amount of the user corresponding to each of the above-mentioned segment trajectories is calculated through the movement amplitude corresponding to the segment trajectory of each of the above-mentioned target sub-segments in the above-mentioned sub-motion trajectory and the weight of the above-mentioned user, and the first heart rate change of the above-mentioned user after the above-mentioned first exercise amount is calculated based on the heart rate of the above-mentioned user at the initial moment of the above-mentioned segment trajectory, and the heart rate of the above-mentioned user at the starting moment and the ending moment of the above-mentioned target sub-segment is obtained according to the above-mentioned monitoring equipment, and the difference between the two is used as the above-mentioned second heart rate change, wherein the above-mentioned second heart rate change is the actual heart rate change of the above-mentioned user, and the above-mentioned first heart rate change is the above-mentioned second heart rate change. The estimated amount of user heart rate change. When the user normally performs the action corresponding to the above-mentioned segment trajectory, the above-mentioned first heart rate change and the above-mentioned second heart rate change are relatively close. When the difficulty of the action corresponding to the above-mentioned segment trajectory is close to the user's tolerance limit, the difference between the above-mentioned first heart rate change and the above-mentioned second heart rate change is large, which means that although the user can complete the above-mentioned action, the intensity of the above-mentioned action may be too large for the user. Therefore, through the above-mentioned technical solution, the third comparison result of the above-mentioned first heart rate change and the above-mentioned second heart rate change can be obtained, and then the above-mentioned third comparison result can be used to accurately analyze whether the action corresponding to the above-mentioned segment trajectory is suitable for the user, thereby improving the accuracy of the action analysis.

[0067] Further, step S5 includes: when the second comparison result corresponding to each of the target sub-segments is less than or equal to the first threshold and the third comparison result corresponding to each of the target sub-segments is less than or equal to the second threshold, if the time of the target sub-segment in the sub-action trajectory is continuous, then the sub-action is a standard action and suitable for the training intensity of the user; if the time of the target sub-segment in the sub-action trajectory is discontinuous, then the sub-action is a qualified action; when at least one corresponding second comparison result in all the target sub-segments is greater than the first threshold, the sub-action is unqualified; when the second comparison result corresponding to each of the target sub-segments is less than or equal to the first threshold and at least one corresponding third comparison result in all the target sub-segments is greater than the second threshold, the training intensity of the sub-action is not suitable for the user.

[0068] Specifically, when the second comparison result of each of the target sub-segments in the sub-action trajectory is less than or equal to the first threshold value and the third comparison result of each of the target sub-segments is less than or equal to the second threshold value, that is, the user's force point is accurate in each of the target sub-segments, and the first heart rate change and the second heart rate change of the user in each of the target sub-segments are relatively close, it means that when the user performs the action corresponding to each of the target sub-segments, the action rules are met and the intensity is suitable for the user. If the time between each of the target sub-segments is continuous, it means that there are no redundant other actions in the middle of the sub-action, so the sub-action is a standard action. If there is at least one time discontinuity between the target sub-segments, it contains redundant other actions. Although the action is not up to standard, it is qualified; when at least one corresponding second comparison result in all the above target sub-segments is greater than the first threshold, that is, the action in at least one target sub-segment does not conform to the corresponding action rule, and therefore the sub-action is considered unqualified; when the second comparison result corresponding to each of the above target sub-segments is less than or equal to the first threshold and at least one corresponding third comparison result in all the above target sub-segments is greater than the second threshold, that is, the actions corresponding to the above target sub-segments conform to the action rules, but the actual heart rate change in at least one target sub-segment is significantly different from the estimated heart rate change, indicating that the intensity of the sub-action is not suitable for the above user. Through the above technical solution, the accuracy of training action analysis is improved.

[0069] Furthermore, the step S5 also includes: when the training intensity of the sub-action is not suitable for the user, obtaining at least one third comparison result greater than the second threshold value corresponding to the target sub-segment, and reducing the action in the reference video corresponding to the target sub-segment from the first difficulty level to the second difficulty level, and restoring to the first difficulty level after the set training time.

[0070] Specifically, when the training intensity of the above-mentioned sub-action is not suitable for the above-mentioned user, the above-mentioned target sub-segment corresponding to at least one of the above-mentioned third comparison results greater than the above-mentioned second threshold value is obtained, and the above-mentioned target sub-segment corresponding to the action in the above-mentioned reference video is reduced from the first difficulty level to the above-mentioned second difficulty level. Since the user is training with reference to the above-mentioned reference video, therefore, by reducing the difficulty level of the above-mentioned reference video, the user is guided to reduce the difficulty level of the training action. After training for the above-mentioned set time, due to the improvement of the user's physical fitness, muscle strength, etc. after the above-mentioned training for the set time, the above-mentioned second difficulty level may not be suitable for the above-mentioned user. Therefore, after training for the above-mentioned set time, the difficulty level of the above-mentioned target sub-segment is restored to the above-mentioned first difficulty level. Through the above-mentioned technical solution, the difficulty level of the action corresponding to each of the above-mentioned target sub-segments can be accurately adjusted, so as to be more suitable for the user's training intensity.

[0071] Furthermore, the step S5 further includes a step S6: repeating the steps S1 to S5 to obtain the analysis results of all the sub-actions.

[0072] Specifically, through the above technical solution, all analysis results of all the above sub-actions can be accurately obtained.

[0073] The present invention also provides a system for analyzing aerobics training movements, the system being used to implement the above method, such as Figure 2 As shown, the system comprises:

[0074] A camera unit, used for acquiring a multi-angle first action video of the user in real time through multiple cameras;

[0075] Monitoring equipment for monitoring the user's heart rate;

[0076] The processing unit is configured to: adjust the playback speed of the first action video to obtain a second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video, periodically obtain sub-action frames from the second action video, and extract the first joint trajectory of the sub-action frame through the extraction unit, and compare the first joint trajectory with the corresponding reference joint trajectory from the reference video to obtain a first comparison result, and obtain the sub-action trajectory according to the first comparison result and the second joint trajectory of the previous cycle and the third joint trajectory of the next cycle;

[0077] The comparison unit is configured to: obtain the action rule corresponding to each target sub-segment in the sub-action trajectory, amplify the key point part corresponding to the action rule in the sub-action video, compare the line change of the amplified key point part with the reference line of the key point part, and obtain a second comparison result; estimate the first heart rate change of each target sub-segment in the sub-action trajectory and compare the first heart rate change with the second heart rate change detected corresponding to the target sub-segment to obtain a third comparison result;

[0078] An analysis unit is used to obtain an analysis result of the sub-action according to the second comparison result and the third comparison result corresponding to each of the target sub-segments in the sub-action trajectory.

[0079] The present invention also provides a computer storage medium, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the above method.

[0080] In summary, the present invention captures the above-mentioned sub-action frames corresponding to each of the above-mentioned reference videos from the above-mentioned second action video, extracts the above-mentioned first joint trajectory from the above-mentioned sub-action frames, and extracts the reference joint trajectory from the above-mentioned reference video, compares the above-mentioned first joint trajectory with the above-mentioned reference joint trajectory, and obtains the above-mentioned first comparison result. When the above-mentioned first comparison result is that the similarity between the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory is greater than or equal to the above-mentioned preset similarity, the above-mentioned first joint trajectory is used as the above-mentioned sub-action trajectory, and when the similarity between the above-mentioned first joint trajectory and the above-mentioned reference joint trajectory is less than the above-mentioned preset similarity, The first joint trajectory is divided into sub-segments, and the sub-segments are compared with the reference joint trajectory to obtain a target sub-segment whose similarity with the reference joint trajectory is greater than or equal to the preset similarity. A reference sub-segment in the reference joint trajectory that does not match the target sub-segment is also obtained. The sub-segments in the second joint trajectory corresponding to the sub-action frame of the previous cycle of this cycle and the sub-segments in the third joint trajectory corresponding to the action frame of the next cycle are respectively compared with the reference sub-segments, and other target sub-segments whose similarity with the reference sub-segment is greater than or equal to the preset similarity are obtained. For the sub-segments that do not match the target sub-segment, The reference sub-segment of the target sub-segment generates a prompt message to prompt the user that the training action corresponding to the reference sub-segment is missing, and all the target sub-segments are spliced ​​in chronological order to obtain the sub-action trajectory. Since each aerobics action has a training target, which corresponds to a different training part, i.e., the muscle force part, the sub-action video corresponding to the target sub-segment is cut out from the second action video, and the force part corresponding to the sub-action video is enlarged to obtain the line change of the force part, and the line change is compared with the reference line change to obtain the second comparison result, thereby The second comparison result determines whether the force exerted at the force-exerting part of the above-mentioned sub-action is correct. By comparing the first heart rate change and the second heart rate change of the user in each of the above-mentioned target sub-segments, it is possible to accurately analyze whether the action corresponding to the above-mentioned segment trajectory is suitable for the user, thereby improving the accuracy of the action analysis. According to the second comparison result and the third comparison result corresponding to each target sub-segment in the sub-action trajectory, the analysis result of the sub-action is obtained and the difficulty level of the action corresponding to each of the above-mentioned target sub-segments is accurately adjusted. Through the mutual cooperation of the above-mentioned technical solutions, not only can the aerobics training actions be accurately analyzed, but also the training actions can be made more suitable for the training intensity of the user.

[0081] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0083] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for analyzing aerobics training movements, wherein a user wears professional aerobics clothing and monitoring equipment during aerobics training and receives movement guidance through a reference video, wherein: The method comprises the following steps: Step S1: acquiring a multi-angle first action video of the user in real time through multiple cameras of the camera unit, and monitoring the user's heart rate in real time through the monitoring device; Step S2: adjusting the playback speed of the first action video to obtain a second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video, periodically obtaining sub-action frames from the second action video, and extracting the first joint trajectory of the sub-action frame by an extraction unit, and comparing the first joint trajectory with the corresponding reference joint trajectory from the reference video to obtain a first comparison result, and obtaining the sub-action trajectory according to the first comparison result and the second joint trajectory of the previous cycle and the third joint trajectory of the next cycle; Step S3: obtaining the action rule corresponding to each target sub-segment in the sub-action trajectory, amplifying the key point part corresponding to the action rule in the sub-action video, and comparing the line change of the amplified key point part with the reference line of the key point part, and obtaining a second comparison result, wherein the sub-action video is the video corresponding to the sub-action trajectory in the second action video; Step S4: estimating a first heart rate change of each target sub-segment in the sub-motion trajectory and comparing the first heart rate change with a second heart rate change detected by the monitoring device corresponding to the target sub-segment to obtain a third comparison result; Step S5: obtaining the analysis result of the sub-action according to the second comparison result and the third comparison result corresponding to each target sub-segment in the sub-action trajectory; Wherein, the step S2 comprises: Step S21: reading all the reference videos from the storage unit, adjusting the playback speed of the first action video, and using the adjusted first action video as the second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video; Step S22: placing the second action video and all the reference videos into a coordinate system with time as the x-axis, periodically acquiring the sub-action frames corresponding to a preset time period from the second action video, and extracting the first joint trajectory from the sub-action frames through an extraction unit, and extracting the reference joint trajectory from the reference video, wherein the preset time period is a time period corresponding to each of the reference videos; Step S23: comparing the first joint trajectory with the reference joint trajectory, and when the first comparison result shows that the similarity between the two is greater than or equal to a preset similarity, taking the first joint trajectory as the sub-action trajectory; The step S23 further comprises: when the first comparison result is that the similarity between the two is less than the preset similarity, dividing the first joint trajectory into sub-segments and obtaining a target sub-segment having a similarity with the reference joint trajectory greater than or equal to the preset similarity, and obtaining at least one reference sub-segment of the reference joint trajectory excluding a plurality of the target sub-segments based on the target sub-segment and the reference joint trajectory; The second joint trajectory of the sub-action frame of the previous cycle and the third joint trajectory of the sub-action frame of the next cycle are respectively obtained, and the second joint trajectory and the third joint trajectory are respectively compared with each of the reference sub-segments, and other target sub-segments whose similarity with the reference sub-segment is greater than or equal to the preset similarity are obtained, and the sub-action trajectories of all the target sub-segments are obtained in chronological order.

2. The method according to claim 1, characterized in that: The step S3 comprises the following steps: Step S31: reading the action rule corresponding to each target sub-segment in the sub-action trajectory from a storage unit, and obtaining the action key point of the target sub-segment based on the action rule, wherein the action key point is the force application part of the action corresponding to the target sub-segment; Step S32: cutting out the sub-action video corresponding to the target sub-segment from the second action video, amplifying the key points of the sub-action video corresponding to the key points of the action, and obtaining line changes of the key points in the sub-action video based on the amplified key points; Step S33: comparing the line change with the reference line of the key point, and obtaining a second comparison result.

3. The method according to claim 1, characterized in that The step S4 comprises the following steps: Step S41: acquiring each of the target sub-segments in the sub-motion trajectory, calculating the first exercise amount of the user corresponding to each of the segment trajectories through the segment trajectory of each of the target sub-segments and the user information, and calculating the first heart rate change of the user in the target sub-segment based on the heart rate of the user at the initial moment of the segment trajectory and the first exercise amount, wherein the user information includes the weight of the user; Step S42: Obtain the user's heart rate at the start time and the end time of each target sub-segment through the monitoring device, and take the difference between the heart rate at the start time and the heart rate at the end time as the second heart rate change, and compare the first heart rate change with the second heart rate change to obtain the third comparison result.

4. The method according to claim 1, characterized in that: The step S5 includes: when the second comparison result corresponding to each of the target sub-segments is less than or equal to the first threshold and the third comparison result corresponding to each of the target sub-segments is less than or equal to the second threshold, if the time of the target sub-segments in the sub-action trajectory is continuous, the sub-action is a standard action and suitable for the training intensity of the user; if the time of the target sub-segments in the sub-action trajectory is discontinuous, the sub-action is a qualified action; when at least one corresponding second comparison result in all of the target sub-segments is greater than the first threshold, the sub-action is unqualified; when the second comparison result corresponding to each of the target sub-segments is less than or equal to the first threshold and at least one corresponding third comparison result in all of the target sub-segments is greater than the second threshold, the training intensity of the sub-action is not suitable for the user.

5. The method according to claim 4, characterized in that The step S5 also includes: when the training intensity of the sub-action is not suitable for the user, obtaining at least one third comparison result greater than the second threshold value corresponding to the target sub-segment, and reducing the action in the reference video corresponding to the target sub-segment from the first difficulty level to the second difficulty level, and restoring to the first difficulty level after the set training time.

6. The method according to claim 1, characterized in that The step S5 further includes a step S6: repeating the steps S1 to S5 to obtain the analysis results of all the sub-actions.

7. A system for analyzing aerobics training movements, the system being used to implement the method according to any one of claims 1 to 6, characterized in that: The system comprises: A camera unit, used for acquiring a multi-angle first action video of the user in real time through multiple cameras; Monitoring equipment for monitoring the user's heart rate; The processing unit is configured to: adjust the playback speed of the first action video to obtain a second action video, wherein the playback speed of the second action video is consistent with the playback speed of the reference video, periodically obtain sub-action frames from the second action video, and extract the first joint trajectory of the sub-action frame through the extraction unit, and compare the first joint trajectory with the corresponding reference joint trajectory from the reference video to obtain a first comparison result, and obtain the sub-action trajectory according to the first comparison result and the second joint trajectory of the previous cycle and the third joint trajectory of the next cycle; The comparison unit is configured to: obtain the action rule corresponding to each target sub-segment in the sub-action trajectory, amplify the key point part corresponding to the action rule in the sub-action video, compare the line change of the amplified key point part with the reference line of the key point part, and obtain a second comparison result; estimate the first heart rate change of each target sub-segment in the sub-action trajectory and compare the first heart rate change with the second heart rate change detected corresponding to the target sub-segment to obtain a third comparison result; An analysis unit is used to obtain an analysis result of the sub-action according to the second comparison result and the third comparison result corresponding to each of the target sub-segments in the sub-action trajectory.

8. A computer storage medium, characterized in that: The storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

Images