A real-time motion determination system based on a motion data and method thereof
Patent Information
- Authority / Receiving Office
- TW · TW
- Patent Type
- Applications
- Current Assignee / Owner
- MINAE LTD
- Filing Date
- 2025-01-17
- Publication Date
- 2026-08-01
AI Technical Summary
Existing motion detection technologies lack real-time feedback capability and accuracy in judging user actions, failing to effectively compare user motions with benchmark actions.
A real-time motion judgment system utilizing a camera module, motion capture module, and comparison module to capture and analyze user key points, compare them with reference data, and generate judgment data based on deep learning and tolerance tables.
Enables accurate and instantaneous judgment of user motions conforming to benchmark motions, providing real-time feedback and improved interactivity.
Smart Images
Figure TWG2TA001069578_001 
Figure TWG2TA001069578_002 
Figure TWG2TA001069578_003
Abstract
Description
[Technical Field]
[0001] This invention relates to the technical field of user action judgment, and more particularly to an instant action judgment system and method for comparing and judging user actions with reference actions. [Previous Technology]
[0002] In terms of motion judgment technology, the user image is usually determined by capturing images in each frame, and the user's current position and posture are determined based on the user image. Then, the user's action is determined based on the posture of each frame (such as posture comparison or judgment of falling, etc.). Alternatively, simple sensor technology (such as gyroscope or accelerometer) is used as a motion sensor to perform motion sensing for the user's actions.
[0003] However, these technologies cannot meet the requirements for accurate action judgment and analysis, and have problems such as lack of real-time feedback capability, inaccurate action capture and insufficient interactivity. They are technologies that cannot achieve real-time capture of user actions and comparison with benchmark actions, and thus accurately and instantly judge whether the user actions conform to the benchmark actions.
[0004] Therefore, there is an urgent need in the present technology for a real-time motion judgment system and method based on motion data to improve the problems existing in the prior art. [Summary of the Invention]
[0005] The purpose of this invention is to provide a real-time action judgment system based on action data, so as to realize the real-time capture of user actions and compare them with the benchmark actions, thereby accurately and instantly judging whether the user actions conform to the benchmark actions.
[0006] To achieve the above-mentioned objective, the present invention provides a real-time motion judgment system based on motion data, comprising: a camera module configured to capture motion data of a user; a motion capture module connected to the camera module to receive the motion data, the motion capture module performing a feature analysis program based on the motion data to obtain a plurality of user key points, and setting the motion data according to each user key point to generate a motion feature; and a comparison module connected to the motion capture module to receive the motion feature, the comparison module comparing the motion feature with a reference motion feature in the same time sequence as the motion feature in a reference data, and generating judgment data based on a comparison result.
[0007] Preferably, the motion data includes a preset motion data and a comparison motion data. When the motion capture module executes the feature analysis program, the motion capture module detects the skeletal points of the basic user image in the preset motion data based on deep learning to generate a key point model. Then, based on the multiple key point positions in the key point model, it selects a user's body position, multiple individual body endpoint positions, and multiple individual body positions connected to the body position as multiple first key points. The motion capture module then obtains multiple dynamic user images from the preset motion data. Based on the motion capture module comparing the image change position of the basic user image and the multiple dynamic user images, and the movement relationship between the multiple first key points in the image change position, it confirms the other multiple key point positions in the key point model as multiple second key points. The motion capture module generates multiple user key points based on the multiple first key points and the multiple second key points. The motion capture module sets the comparison motion data based on each user key point and generates the motion feature.
[0008] Preferably, the motion data includes preset motion data and comparison motion data. When the motion capture module executes the feature analysis program, the motion capture module obtains a basic user image from the preset motion data. The motion capture module detects the skeletal points of the basic user image in the preset motion data based on deep learning to generate a keypoint model, and sets the basic user image as a plurality of first keypoints according to the keypoint model. The motion capture module then obtains a plurality of dynamic user images from the preset motion data and sets the skeletal points of the basic user image as a plurality of first keypoints according to the keypoint model. A preset motion model is compared with a plurality of dynamic user images to generate correction data. The motion capture module corrects at least one of the first key points corresponding to the image change position according to the correction data to serve as a second key point. The motion capture module generates a plurality of user key points according to a plurality of the first key points and a plurality of the second key points. The motion capture module sets the comparison motion data according to each user key point and generates the motion feature. The preset motion model corresponds to the motion in the plurality of dynamic user images.
[0009] Preferably, when the comparison module compares the action feature with a reference action feature in the same time sequence as the action feature in a reference data, the comparison module determines a plurality of reference key points in the reference action feature that correspond to each user key point, and then compares the movement amount of each user key point in the comparison action data with the movement amount of each reference key point in the same time sequence, and takes an error value of each user key point as the comparison result, and generates the judgment data based on the comparison result.
[0010] Preferably, the real-time motion judgment system includes: a fitting module connected to the motion capture module and the comparison module. The fitting module receives the motion feature from the motion capture module and modifies a user image size in the motion feature based on the reference target size in the reference data. The fitting module generates the modified motion feature based on the modified user image size and transmits it to the comparison module, so that the comparison module compares the modified motion feature with the reference motion feature in the reference data that is in the same time sequence as the modified motion feature.
[0011] Preferably, when the fitting module modifies the user size in the motion data, the fitting module modifies the user image size in the motion data proportionally to the height of the reference target size in the reference data, so that the height of the reference target size is equal to the height of the modified user image size.
[0012] Preferably, when the comparison module compares the action feature with a reference action feature in the same time sequence as the action feature in a reference data, the comparison module determines a plurality of reference key points in the reference action feature that correspond to each user key point, and then compares them according to a difference value between each user key point in the comparison action data and the corresponding reference key points in the same time sequence, and determines each difference value based on an allowance table as the comparison result. The comparison module generates the judgment data according to the comparison result.
[0013] Preferably, the comparison module determines each difference value based on a tolerance table as the comparison result. When the comparison module generates the judgment data based on the comparison result, the comparison module determines whether the difference value of each user key point falls within a tolerance range according to the tolerance table. If the difference value falls within the tolerance range, the comparison module outputs a matching comparison result. If the difference value is not within the tolerance range, the comparison module outputs a non-matching comparison result. The tolerance range is set based on the distance difference between the user key point and the corresponding reference key point and a human body part classification.
[0014] Preferably, the real-time action judgment system includes: a scoring module connected to the comparison module to receive the judgment data, the scoring module constructing a scoring table based on a plurality of current states and a plurality of parts, and calculating the judgment data based on the scoring table and generating a scoring data.
[0015] To achieve the above-mentioned objectives, the present invention also provides a real-time action judgment method applied to the real-time action judgment system described above, comprising: capturing a user's action data using a camera module; performing a feature analysis program based on the action data using a motion capture module to obtain a plurality of user key points, and generating an action feature based on each of the user key points; and comparing the action feature with a reference action feature in a reference data at the same time sequence as the action feature using a comparison module, and generating judgment data based on a comparison result.
[0016] In order to make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments listed below with reference to the figures are described in detail below.
Implementation Method
[0017] The advantages, features and technical methods of the present invention will be more readily understood by referring to the exemplary embodiments and accompanying drawings. The present invention may be implemented in different forms and should not be construed as being limited to the embodiments set forth herein. Rather, the embodiments provided will make this disclosure more thorough, complete and fully convey the scope of the invention to those skilled in the art. The present invention will be defined only as provided in the appended claims.
[0018] In addition, the terms "comprising" and / or "including" refer to the presence of the said features, areas, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, areas, wholes, steps, operations, elements, components and / or combinations thereof.
[0019] To facilitate your understanding of the content of this invention and the effects it can achieve, the following detailed description is provided in conjunction with the accompanying drawings of various specific embodiments:
[0020] Please refer to Figures 1 to 5, which are schematic diagrams of the configuration relationship of each module of the real-time motion judgment system of the present invention, a schematic diagram of the first key point of the basic user image in one embodiment, a schematic diagram of other multiple key point positions as second key points in one embodiment, a schematic diagram of the key point model setting of the basic user image as the first key point in another embodiment, and a schematic diagram of the image change position correction and the first key point corresponding to the image change position as the second key point in another embodiment. As shown in the figures, in order to realize the technology of real-time capture of user actions and comparison with reference actions, and then accurately and instantly determine whether the user actions conform to the reference actions, the real-time motion judgment system based on motion data of the present invention includes a camera module 100, a motion capture module 200, and a comparison module 300.
[0021] The camera module 100 is a camera device used to capture images of a user. In another embodiment, the camera module 100 may also be a camera device supporting depth sensing technology (such as a Time of Flight (TOF) camera or a binocular camera). The camera module 100 is configured to capture motion data 10 of the user, which refers to data that includes at least an image of the user. Thus, when the real-time motion judgment system is activated, the user can stand within the capture range of the camera module 100, allowing the camera module 100 to capture the motion data 10.
[0022] The motion capture module 200 is a processor used to analyze user key points and generate corresponding motion features. The motion capture module 200 is connected to the camera module 100 to receive the motion data 10. The motion capture module 200 performs a feature analysis program based on the motion data 10 to obtain a plurality of user key points 20, and sets the motion data 10 according to each user key point 20 to generate a motion feature 21.
[0023] The motion data 10 includes a preset motion data 101 and a comparison motion data 102. The preset motion data 101 indicates that the user presents a corresponding motion posture (e.g., a preset standing posture) according to a preset posture template, so that the camera module 100 can acquire the preset motion data 101 of the user's motion posture corresponding to the preset posture template. The comparison motion data 102 indicates that the user presents a corresponding motion posture (e.g., a dance motion in each frame) according to a posture presented in a reference data 40, so that the camera module 100 can acquire the comparison motion data 102 of the user's motion posture corresponding to the posture presented in the reference data 40.
[0024] When the motion capture module 200 performs a feature analysis program based on the motion data 10, please refer to Figures 2 and 3. In one embodiment, the motion capture module 200 obtains a basic user image 1011 from the preset motion data 101. The basic user image 1011 is a full-body image of the user presenting a corresponding action posture according to a preset posture template. The motion capture module 200 can detect the basic user image 1011 in the preset motion data 101 based on deep learning (e.g., PoseNet, OpenPose, or MediaPipe). The user's skeletal points in image 1011 are used to generate a keypoint model 22. Then, based on the keypoint model 22, a user's body position, a plurality of individual body endpoint positions, and a plurality of individual body positions connected to the body position are selected as a plurality of first keypoints 221. It should be noted that the individual body endpoint positions represent the position of the head, palm, foot, finger joints, and / or toe joints, etc., and the individual body positions connected to the body position represent the positions where the head, hand, and / or foot connect to the body, etc.
[0025] Subsequently, the motion capture module 200 acquires a plurality of dynamic user images 1012 from the preset motion data 101, and determines the positions of other plurality of key points in the key point model 22 as a plurality of second key points 222 based on an image change position 10121 of the plurality of dynamic user images 1012 and the movement relationship between the plurality of first key points 221 in the image change position 10121. The dynamic user image 1012 represents a different action from the basic user image 1011, such as raising both hands, squatting, or turning, so that the motion capture module 200 can compare the image change position 10121 between the basic user image 1011 and the dynamic user image 1012. When the image change position 10121 represents the action of raising both hands, the movement relationship can be represented as the movement relationship between the first key point 221 at the palm position and the first key point 221 at the position where the hand connects to the body. Since the dynamic user image 1012 is a preset posture template with an action different from that of the basic user image 1011, when the first key point 221 at the palm position and the first key point 221 at the position where the hand connects to the body (e.g., the shoulder position) are different, the movement relationship between the first key point 221 at the palm position and the first key point 221 at the position where the hand connects to the body (e.g., the shoulder position) can be different. When point 221 moves, the preset movement relationship of its hand movement can be confirmed (for example, the movement relationship of the three points of shoulder, elbow and palm forming a triangle RT). Thus, based on the movement relationship of the first key point 221 of the palm position in the image change position 10121 and the first key point 221 of the position where the hand connects to the body, the elbow key point position in the key point model 22 can be confirmed as the second key point 222. When confirming the elbow key point position in the key point model 22, it means determining or modifying (when there is a deviation between the preset elbow key point position in the key point model 22 and the confirmed elbow key point position in the key point model, the preset elbow key point position can be modified) the elbow key point position in the key point model 22 as the second key point 222.
[0026] Thus, the motion capture module 200 can generate a plurality of user key points 20 based on a plurality of the first key points 221 and a plurality of the second key points 222, and set the comparison motion data 102 based on each user key point 20, and generate the motion feature 21. That is to say, after setting the comparison motion data 102 based on the plurality of user key points 20, the subsequent input comparison motion data 102 can determine the position of each key point of the user based on the plurality of user key points 20, and generate the motion feature 21 accordingly.
[0027] When the motion capture module 200 performs a feature analysis program based on the motion data 10, please refer to Figures 4 and 5. In another embodiment, after obtaining the key point model 22 as described above, the motion capture module 200 can set the basic user image 1011 as a plurality of first key points 221 according to the position of each key point in the key point model 22. Then, it can obtain a plurality of dynamic user images 1012 from the preset motion data 101, and compare the image change position 10121 of the plurality of dynamic user images 1012 with a preset motion model of the key point model 22 and generate a correction data 23. The motion capture module 200 corrects at least one of the first key points 221 corresponding to the image change position 10121 according to the correction data 23 as a second key point 222. The preset motion model corresponds to the motions in the plurality of dynamic user images 1012. In other words, the preset motion model is a preset motion pattern. For example, when a user raises both hands, the preset motion model has preset hand, arm, and shoulder positions. When acquiring the corresponding dynamic user image 1012 with the user's hands raised, the motion capture module 200 compares the preset motion model with the image change position 10121 of the dynamic user image 1012 (i.e., the hand, arm, and shoulder positions in the dynamic user image 1012) and generates correction data 23. If the correction data 23 indicates a deviation between any key point in the preset motion model and the first key point 221 (e.g., the first key point of the elbow position) at the corresponding image change position 10121, the motion capture module 200 can correct at least one of the first key points 221 corresponding to the image change position 10121 based on the correction data 23. (That is, the first key point 221 of the elbow position is corrected) to serve as the second key point 222.
[0028] Subsequently, the motion capture module 200 can generate the plurality of user key points 20 based on the plurality of the first key points 221 and the plurality of the second key points 222. The motion capture module 200 sets the comparison motion data 102 based on each user key point 20 and generates the motion feature 21.
[0029] The comparison module 300 is a processor used to compare data and generate corresponding judgment results. The comparison module 300 is connected to the motion capture module 200 to receive the motion feature 21, so that the comparison module 300 can compare the motion feature with a reference motion feature in a reference data 40 that is in the same time sequence as the motion feature 21, and generate judgment data 30 according to the comparison result. The comparison result mainly determines whether the movement amount or position of each user key point 20 in the motion feature 21 matches or is similar to that of the corresponding reference key point in the reference motion feature 401, and then generates the judgment data 30 according to the comparison result of each user key point 20, and then outputs the judgment data 30 through a display device (e.g., a networked TV).
[0030] The reference data 40 mentioned above can also be output by the display device to be displayed to the user and provide the user with the same or similar actions as the reference data 40, so that the camera device 100 can capture the user's actions and generate the action data 10, and perform subsequent comparison and judgment based on the comparison action data 102 in the action data 10.
[0031] In addition, the acquisition of the reference data 40 can be achieved by connecting a selection module 400 and the comparison module 300. When the selection module 400 selects the reference data 40 from a database (for example, the user selects the corresponding dance video (i.e., the reference data 40) using the selection module 400), the reference data 40 is transmitted to the comparison module 300 so that the comparison module 300 can obtain the corresponding reference motion features.
[0032] The motion capture module 200 and the comparison module 300 may be application-specific integrated circuits (ASICs) or graphics processing units (GPUs) to achieve low latency and high performance real-time feedback.
[0033] Please refer to Figure 6 again, which is a schematic diagram of the present invention to determine the error value of the user key point by comparing the movement amount of the user key point with the movement amount of the reference key point in the same time sequence. As shown in the figure, when the comparison module 300 compares the action feature 21 with a reference action feature 401 in the reference data 40 in the same time sequence as the action feature 21, the comparison module 300 can determine a plurality of reference key points 4011 in the reference action feature 401 that correspond to each user key point 20, and then compare the movement amount of each user key point 20 in the comparison action data 102 with the movement amount of each reference key point 4011 in the same time sequence, and take the error value E of each user key point 20 as the comparison result, and generate the judgment data 30 based on the comparison result.
[0034] For example, when the reference data 40 is a dance video, the reference motion feature 401 represents the dancer's movements in the video, and each reference keypoint 4011 represents the position of each skeletal keypoint of the dancer when the dancer performs the movements. However, the dance video above is only an example, and the reference data 40 may also be other related videos containing reference movements.
[0035] Thus, the comparison module 300 can use Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and / or Dynamic Time Warping (DTW) to compare the temporal and image features of each user keypoint 20 and each reference keypoint 4011 to determine the movement amount of each user keypoint 20 and the movement amount of each reference keypoint 4011 in the same temporal sequence in the comparison action data 102. When the movement amount of any user keypoint 20 has an error with the movement amount of its corresponding reference keypoint 4011, the error value E can be generated to determine the similarity between the user keypoint 20 and the reference keypoint 4011 in the same temporal sequence and to serve as the comparison result. For example, when the error value E between the user's elbow key point 20 and the reference key point 4011's elbow key point is 0.9 cm, the similarity is 92%; when the error value E is 1.8 cm, the similarity is 83%, etc.
[0036] Please refer to Figures 7 and 8, which are schematic diagrams of the configuration relationship of the fitting module of the present invention and schematic diagrams of the allowable range of difference values based on the allowable table. As shown in the figures. The real-time motion judgment system disclosed in this invention may include a fitting module 500, which is an image processor that modifies a reference image to fit a user image. The fitting module 500 is connected to the motion capture module 200 and the comparison module 300. After receiving the motion feature from the motion capture module 200, the fitting module 500 can modify the size of a user image 211 in the motion feature 21 based on the size of the reference target 402 in the reference data 40 (e.g., the height of a person in a dance video), so that the size of the reference target 402 in the reference data 40 is the same as the size of the user image 211. More specifically, when the fitting module 500 modifies the height based on the size of the reference target 402 in the reference data 40, the height of the user image 211 will be modified to be equal to the height of the reference target 402 in the reference data 40, but other size values of the user image 211 (e.g., width) will be modified proportionally to conform to the original body shape of the user image 211. Subsequently, the fitting module 500 generates the modified motion feature 21 based on the modified user image 211 size and transmits it to the comparison module 300. The comparison module 300 then compares the modified motion feature 21 with the reference motion feature 401 in the reference data 40, which is in the same time sequence as the modified motion feature 21. The reference target 402 can also be other targets that can be used as a reference, such as hands, feet, or the head, which are capable of movement.
[0037] However, since each person's skeletal key points usually have slight differences, for example, some people have longer hands or longer feet, and the joint points of their elbows or knees will be different from those of other people of the same height. Therefore, even if the height of the user image 211 is modified by the fitting module 500 to be equal to the height of the reference target 402 in the reference data 40, in order to more accurately determine whether the user's movements conform to the movements in the reference data 40, when the comparison module 300 compares the reference movement feature 401 with the movement feature 21 in the reference data 40 based on the reference movement feature 21 in the same time sequence, the comparison module 300 determines the reference movement feature 401 that corresponds to each of the user's key points 20. Each of the reference key points 4011 is compared with the user key point 20 and the corresponding reference key point 4011 in the same time sequence according to the difference value D. Here, the difference value D represents the distance difference between the user key point 20 and the corresponding reference key point 4011 in the same time sequence. At this time, the comparison module 300 can judge each difference value D based on an allowance table as the comparison result, and generate the judgment data 30 based on the comparison result.
[0038] The allowable range AR is set based on the distance difference between the user key point 20 and the corresponding reference key point 4011 when they are in a preset position (e.g., the elbow key point position when the hand is naturally vertical). For example, if the distance difference is -0.3 cm on the X-axis and -1 cm on the Y-axis of the user key point 20, the allowable range AR will be set to a circle or an ellipse according to the distance difference. Furthermore, the allowable range AR will be set according to a body part classification. For example, if the user key point 20 is the elbow key point, since the swing amplitude of the elbow is large, a larger allowable range AR can be given when setting the allowable range AR. Or if the user key point 20 is the shoulder key point, since the swing amplitude of the shoulder is small, a smaller allowable range AR can be given when setting the allowable range AR, etc.
[0039] The acquisition of the reference data 40 can be achieved by connecting the selection module 400 and the fitting module 500, so that the selection module 400 transmits the reference data 40 to the fitting module 500.
[0040] Please refer to Figure 9 again, which is a schematic diagram of the configuration relationship of the scoring module of the present invention. As shown in the figure, the real-time action judgment system disclosed in the present invention may include a scoring module 600, which is connected to the comparison module 300 to receive the judgment data 30. The scoring module 600 can construct a scoring table based on a plurality of current states and a plurality of parts, and the scoring table can be represented as follows:
[0041] Scoring Sheet: Current status → Location ↓ State A State B State C Part A 1 2 5 Part B 0.3 0.8 2 Part C 0.6 1.5 3
[0042] When determining the error value E, the current state and location can be represented as follows:
[0043] State A indicates an error greater than 1 cm or a similarity of less than 90%;
[0044] State B indicates an error greater than 2 cm or a similarity of less than 80%;
[0045] State C indicates an error greater than 5 cm or a similarity less than 65%;
[0046] Part A indicates a key point on the shoulder;
[0047] Location B indicates the key point of the elbow;
[0048] Location C indicates the key point of the knee.
[0049] When determining the difference value D, the current state and location can be represented as follows:
[0050] State A indicates that the allowable range is exceeded by 1 cm;
[0051] State B indicates that the allowable range is exceeded by 2 cm;
[0052] State C indicates that the allowable range is exceeded by 5 cm;
[0053] Part A indicates a key point on the shoulder;
[0054] Location B indicates a key point on the elbow;
[0055] Location C indicates a key point on the knee.
[0056] Accordingly, the scoring module 600 can obtain a corresponding score value based on the current state of different body parts, and calculate the remaining score based on a plurality of such score values (for example, the remaining score after deducting each of the score values from 100 points) to generate a score data 60. Furthermore, the score data 60 may also include a plurality of such current states and a plurality of such body parts, so that when the display device receives the score data 60 and displays it to the user, it can accurately provide the user with a score for their actions and suggestions for improvement for the corresponding body parts, thereby improving the user's learning efficiency and entertainment experience.
[0057] Furthermore, the motion capture module 200 can also be connected to a cloud device to output the user key point model of each user key point 20 to the cloud device, so as to store and update the user key point model through the cloud device, and process it in conjunction with the edge computing device to realize the functions of real-time motion processing feedback and long-term data analysis.
[0058] Please refer to Figure 10 again, which is a flowchart of the real-time action judgment method of the present invention. As shown in the figure, in order to realize the technology of real-time capture of user actions and comparison with reference actions, and thus accurately and instantly determine whether the user actions conform to the reference actions, the present invention also provides a real-time action judgment method applied to the real-time action judgment system described above, which includes:
[0059] S101: Capture user motion data using a camera module;
[0060] S102: The motion capture module performs a feature analysis program based on the motion data to obtain a plurality of user key points, and generates motion features based on each user key point;
[0061] S103: Use the comparison module to compare the action feature with the benchmark action feature in the benchmark data that is in the same time sequence as the action feature, and generate judgment data based on the comparison results.
[0062] The present invention discloses a preferred embodiment. Any partial changes or modifications that are derived from the technical concept of the present invention and can be easily deduced by those skilled in the art are not outside the scope of the patent rights of the present invention.
[0063] In summary, the present invention, in terms of purpose, means and effects, demonstrates technical features that are distinct from those of the conventional, and its invention is practical and meets all the requirements for a patent. We respectfully request that your review committee examine the invention and grant a patent as soon as possible so that it may benefit society. We would be truly grateful for your assistance. [Simplified Explanation of the Diagram]
[0064] Figure 1 is a schematic diagram of the configuration relationship of each module of the real-time action judgment system of the present invention; Figure 2 is a schematic diagram of the first key point of the basic user image in one embodiment of the present invention; Figure 3 is a schematic diagram of the positions of other multiple key points as second key points in one embodiment of the present invention; Figure 4 is a schematic diagram of the key point model setting of the basic user image as the first key point in another embodiment of the present invention; Figure 5 is a schematic diagram of the image change position correction and the first key point corresponding to the image change position as the second key point in another embodiment of the present invention; Figure 6 is a schematic diagram of the present invention for judging the error value of the user key point by comparing the movement amount of the user key point with the movement amount of the reference key point in the same time sequence; Figure 7 is a schematic diagram of the configuration relationship of the fitting module of the present invention; Figure 8 is a schematic diagram of the allowable range of the difference value based on the allowable table of the present invention; Figure 9 is a schematic diagram of the configuration relationship of the scoring module of the present invention; Figure 10 is a flowchart of the steps of the real-time action judgment method of the present invention.
Claims
1. A real-time motion judgment system based on motion data, comprising: a camera module configured to capture motion data of a user; a motion capture module connected to the camera module to receive the motion data, the motion capture module performing a feature analysis program based on the motion data to obtain a plurality of user key points, and setting the motion data according to each of the user key points to generate a motion feature; and a comparison module connected to the motion capture module to receive the motion feature, the comparison module comparing the motion feature with a reference motion feature in the same time sequence as the motion feature in the reference data, and generating judgment data based on a comparison result; wherein, The motion data includes preset motion data and comparison motion data. When the motion capture module executes the feature analysis program, it acquires a basic user image from the preset motion data. Based on deep learning, the motion capture module detects the skeletal points of the basic user image in the preset motion data to generate a keypoint model. Based on the keypoint model, it sets the basic user image as a plurality of first keypoints. The motion capture module then acquires a plurality of dynamic user images from the preset motion data. Based on a preset motion model of the keypoint model, it compares an image change position of the plurality of dynamic user images and generates correction data. Based on the correction data, the motion capture module corrects at least one of the first keypoints in the image change position as a second keypoint. Based on the plurality of first keypoints and the plurality of second keypoints, the motion capture module generates a plurality of user keypoints. Based on each user keypoint, the motion capture module sets the comparison motion data and generates the motion feature. The preset motion model corresponds to the motion in the plurality of dynamic user images.
2. The real-time action determination system as described in claim 1, wherein, When comparing the action feature with a reference action feature in the same time sequence as the action feature in a reference data, the comparison module determines a plurality of reference key points in the reference action feature that correspond to each user key point. Then, it compares the movement amount of each user key point in the comparison action data with the movement amount of each reference key point in the same time sequence, and uses an error value of each user key point as the comparison result. The judgment data is then generated based on the comparison result.
3. The real-time action determination system as described in claim 1, comprising: A fitting module is connected to the motion capture module and the comparison module. The fitting module receives the motion feature from the motion capture module and modifies a user image size in the motion feature based on the reference target size in the reference data. The fitting module generates the modified motion feature based on the modified user image size and transmits it to the comparison module, so that the comparison module compares the modified motion feature with the reference motion feature in the reference data at the same time sequence as the modified motion feature.
4. The real-time action determination system as described in claim 3, wherein, When the fitting module modifies the user size in the motion data, it modifies the user image size in the motion data proportionally to the height of the reference target size in the reference data, so that the height of the reference target size is equal to the height of the modified user image size.
5. The real-time action determination system as described in claim 3, wherein, When comparing the action feature with a reference action feature in the same time sequence as the action feature in a reference data, the comparison module determines a plurality of reference key points in the reference action feature that correspond to each user key point. Then, it compares the user key points in the comparison action data with the corresponding reference key points in the same time sequence, and determines each difference value based on an allowance table as the comparison result. The comparison module generates the judgment data based on the comparison result.
6. The real-time action determination system as described in claim 5, wherein, The comparison module uses a tolerance table to determine the difference values as the comparison result. When generating the judgment data based on the comparison result, the comparison module determines whether the difference value of each user key point falls within a tolerance range according to the tolerance table. If the difference value falls within the tolerance range, the comparison module outputs a matching comparison result. If the difference value does not fall within the tolerance range, the comparison module outputs a non-matching comparison result. The tolerance range is set based on the distance difference between the user key point and the corresponding reference key point, as well as a human body part classification.
7. The real-time action determination system as described in claim 1, comprising: A scoring module is connected to the comparison module to receive the judgment data. The scoring module constructs a scoring table based on a plurality of current states and a plurality of parts, and calculates the judgment data based on the scoring table and generates a scoring data.
8. A real-time action judgment method applied to a real-time action judgment system as described in any one of claims 1 to 7, comprising: capturing a user's action data using a camera module; performing a feature analysis program based on the action data using a motion capture module to obtain a plurality of user key points, and generating an action feature based on each of the user key points; and comparing the action feature with a reference action feature in a reference data at the same time sequence as the action feature using a comparison module, and generating judgment data based on a comparison result.