A motion recognition method based on human skeletal joint points

Abstract

The present invention belongs to the technical field of human motion recognition. A motion recognition method based on human skeletal joint points is as follows: When a human body is in motion, an image acquisition device is used to collect images of the actions of human skeletal joint points, and action images of skeletal joint points can be obtained; The action images of skeletal joint points are frame-divided, and the data in each frame image is extracted to find the positions of human skeletal joint points in each frame image, analyze the changes in the positions of human skeletal joint points in consecutive images in different frames, and analyze the actual movement distances of human skeletal joint points according to the changes in the positions of human skeletal joint points; The present invention collects and stores action feature data in various motion states, and then uses these data to construct a database, and further can accurately judge and analyze the motion states of different users according to their action feature data.

Description

Technical Field

[0001] The present invention relates to the technical field of human motion recognition, and specifically to a motion recognition method based on human bone joint positions. Background Art

[0002] Currently, more and more sports scenarios start to use smart devices to collect motion data. Then, based on the motion data, the motion state of the user is recognized, and thus the real-time motion state of the user is judged. When the existing smart devices collect motion data, they calculate the number of steps according to the offset of the center of gravity of the device caused by the human body, and then use the distance difference in the positioning system to judge the motion speed.

[0003] However, when the smart devices collect the number of motion steps and the motion speed, they are easily interfered by external factors. For example, if the smart device shakes, it will cause a large deviation in the number of motion steps. When the user's motion trajectory is arc-shaped, it will cause a large deviation in the judgment of the motion speed. Moreover, these are only basic motion data, and it is difficult to comprehensively understand the real-time motion state of the user and predict the real-time motion state of the user.

[0004] When the human body is in motion, it will inevitably cause changes in the positions of human bone joints. By judging the changes in the positions of human bone joints, the motion state of the user can be accurately judged. Therefore, a method for analyzing the changes in the positions of human bone joints is needed to accurately judge and recognize the motion. Summary of the Invention

[0005] The purpose of the present invention is to provide a motion recognition method based on human bone joint positions to solve the problem of large deviations easily occurring in the existing methods when collecting motion data as mentioned in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A motion recognition method based on human bone joint positions, and the steps of the motion recognition method based on human bone joint positions are as follows:

[0007] Step 1: According to the image acquisition device, collect the bone joint position actions during human motion to obtain bone joint position action images;

[0008] Step 2: Perform frame-by-frame processing on the image, extract the data in each frame of the image, find the positions of the human bone joints in each frame of the image, compare the changes in the positions of the human bone joints in multiple adjacent frames of the image, and obtain the actual motion distance of the human bone joints according to the changes in the positions of the human bone joints;

[0009] Step 3: Obtain the movement speed and movement direction based on the actual movement distance and corresponding movement time period of the human bone joint points, obtain action feature data based on the movement speed and the movement direction, and perform importance analysis on the action feature data to obtain important action feature data;

[0010] Step 4: Analyze the user's movement state by comparing the important action feature data with the standard action feature data stored in the database under different movement states;

[0011] Step 5: Classify and train the important action feature data through a classification algorithm to obtain different movement states of the user, and store and record the different movement states of the user and the important action feature data in the database.

[0012] Preferably, in Step 1, the action images of the bone joint points include images of all bone joint points performing actions.

[0013] Preferably, in Step 2, when analyzing the actual movement distance of the human bone joint points, it includes analyzing and calculating in three directions of length, width, and height.

[0014] Preferably, in Step 3, the movement speed includes instantaneous speed and cross-point speed; the instantaneous speed refers to the movement speed of the human bone joint points between two adjacent frames of images; the cross-point speed refers to the movement speed of the human bone joint points between non-adjacent frames of images.

[0015] Preferably, after the instantaneous speed is collected, a time-speed curve is drawn using the instantaneous speed and time.

[0016] Preferably, the analysis of the user's movement state includes: using the cross-point speed to obtain action feature data for judging the user's movement state; when making a detailed judgment on the movement state under the same movement state, the instantaneous speed is used to obtain action feature data for comparison and analysis.

[0017] Preferably, in Step 4, when the collected important action feature data cannot be matched with the standard action feature data stored in the database, the important action feature data is stored and Steps 1 to 4 are repeated until the database is updated and supplemented after matching.

[0018] Preferably, in Step 3, when performing importance analysis on the action feature data, it includes: under the same movement state, if the action feature data appears irregularly, it is determined that the action feature data is unimportant action feature data, and the action feature data of this bone joint point is not stored; if the action feature data appears regularly, it is determined that the action feature data is important action feature data, and the action feature data of this bone joint point is stored.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] 1) The present invention uses an image acquisition device to obtain action images of human bone joint points, and then analyzes the position changes and speed changes of the bone joint points according to the image data. Subsequently, these data are associated with the user's motion state. When it is necessary to judge the user's motion state, only the action feature data of the bone joint points need to be collected for comparison and analysis, and then the user's motion state can be accurately judged;

[0021] 2) The present invention collects and stores the action feature data in various motion states, and then uses these data to construct a large database, and then can accurately judge and analyze the motion state according to the action feature data of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the working process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0025] Embodiment:

[0026] Please refer to Figure 1 , the present invention provides a technical solution: a motion recognition method based on human bone joint points, and the motion recognition method based on human bone joint points is as follows:

[0027] Step 1: During the movement of the human body, the human bone joint points will also change accordingly. The action of the human bone joint points is collected by an image acquisition device. In order to obtain more data, the action images of all human bone joint points can be collected, and these action images together constitute the action images of the bone joint points;

[0028] Step 2: Frame the action images of the skeletal joint points. Since the movement of the skeletal joint points will cause the positions of the skeletal joint points in the images of different frames to change, the present invention extracts the data in each frame of the image, finds the positions of the human skeletal joint points in each frame of the image, analyzes the changes in the positions of the human skeletal joint points in the consecutive images of different frames. Through the changes in these positions, the change distance of the skeletal joint points on the image can be calculated. Then, according to the geometric conversion, the change distance on the image can be converted into the real world, and thus the actual movement distance of the skeletal joint points can be obtained;

[0029] Step 3: The time interval between each frame of the image is fixed. The time difference between two frames of the image can be obtained through the interval difference between the two frames of the image. Using the actual movement distance of the skeletal joint points between the two frames of the image and the time difference between the two frames of the image, the movement speed of the skeletal joint points can be calculated. At the same time, by connecting the skeletal joint points in each frame of the image, the moving direction of the skeletal joint points can be judged. According to the movement speed and the movement direction, the action feature data in different time periods are statistically analyzed. According to the statistically analyzed action feature data of different skeletal joint points, importance analysis is carried out. When the action feature data appears repeatedly in the same motion state, it is determined that the action feature data is important data; when the action feature data appears irregularly in the same motion state, it is determined that the action feature data is non-important data. When collecting and storing the action feature data later, when the action feature data of the skeletal joint point is non-important data, the action feature data of the skeletal joint point is not stored. When analyzing the motion state according to the action feature data later, the action feature data of the skeletal joint point is not collected either, thus reducing the amount of data collected;

[0030] Step 4: When analyzing the motion state of the user, compare the collected important action feature data with the stored action feature data. When the important action feature data collected is consistent with the action feature data of a certain motion state in the stored action feature data, the movement of the human skeletal joint point corresponds to the motion state;

[0031] Step 5: Obtain the action feature data in different motion states and conduct big data statistics. Collect and store the action feature data of users of different heights. These action feature data are carried out in different motion states, such as running and high jumping. Use the neural network to learn and judge the action feature data, associate the action feature data with the motion state, and at the same time store and record the training results to establish a database.

[0032] When the human body is in motion, the skeletal joint points will move in three planes. In step 2, when analyzing the actual movement distance of the human skeletal joint points, the analysis and calculation are carried out in the three directions of length, width, and height respectively. In this way, the combined motion can be converted into component motions in three planes. By analyzing the component motions in the three planes, that is, analyzing the actual movement distances in the three planes, more accurate data can be obtained.

[0033] In step 3, when analyzing the movement speed, there are two states. One is that the analyzed data is used as standard data, and the other is that the analyzed data is used for comparison. The standard data needs to be supported by sufficient data. When analyzing the movement speed, the instantaneous speed is constructed by using the movement speed between the human skeletal joint positions in any two adjacent frames of images, and the cross-point speed is constructed by using the movement speed between the human skeletal joint positions in non-adjacent frames of images. When comparing the data, in order to reduce the calculation amount, the cross-point speed can be used for comparison. In order to understand the movement state more accurately, the movement state can be subdivided. When only a general judgment of the movement state is needed, the cross-point speed is used to obtain the action feature data for comparison and analysis. When a detailed judgment of the movement state is made, the instantaneous speed is used to obtain the action feature data for comparison and analysis.

[0034] After the instantaneous speed is collected, the time-speed curve is drawn by using the instantaneous speed and time. By using the time-speed curve, it is convenient for users to intuitively understand the change of the instantaneous speed.

[0035] Since the data collection cannot cover comprehensively, it may occur that the important action feature data collected cannot be matched with the action feature data stored in the database. At this time, the collected important action feature data is saved and steps 1 to 4 are repeated until the important action feature data collected is matched with the saved important action feature data. Then, the saved important action feature data is updated and supplemented, and thus the database is improved.

[0036] The above shows and describes the basic principles, main features and advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention, and any reference signs in the claims should not be regarded as limiting the claimed rights.

[0037] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will appreciate that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A motion recognition method based on human skeletal joint points, characterized in that: The motion recognition method based on human skeletal joint points is as follows: Step 1: According to the skeletal joint point actions during human motion collected by an image acquisition device, obtain skeletal joint point action images; Step 2: Perform frame-by-frame processing on the images, extract the data in each frame of the image, find the positions of the human skeletal joint points in each frame of the image, compare the changes in the positions of the human skeletal joint points in multiple adjacent frames of images, and obtain the actual motion distances of the human skeletal joint points according to the changes in the positions of the human skeletal joint points; Step 3: Obtain the motion speed and motion direction according to the actual motion distances of the human skeletal joint points and the corresponding motion time periods, obtain action feature data according to the motion speed and the motion direction, perform importance analysis on the action feature data, and obtain important action feature data; The motion speed includes instantaneous speed and cross-point speed; the instantaneous speed refers to the motion speed of the human skeletal joint points between two adjacent frames of images; the cross-point speed refers to the motion speed of the human skeletal joint points between non-adjacent frames of images; Step 4: Analyze the user's motion state by comparing the important action feature data with the standard action feature data in the database stored under different motion states; Step 5: Perform classification training on the important action feature data through a classification algorithm to obtain different motion states of the user, and store and record the different motion states of the user and the important action feature data in the database.

2. The motion recognition method based on human skeletal joint points according to claim 1, characterized in that: In step 1, the skeletal joint point action images include images of all skeletal joint points performing actions.

3. The motion recognition method based on human skeletal joint points according to claim 1, characterized in that: In step 2, when analyzing the actual motion distances of the human skeletal joint points, it includes analyzing and calculating in three directions of length, width, and height.

4. The motion recognition method based on human skeletal joint points according to claim 1, characterized in that: After collecting the instantaneous speed, use the instantaneous speed and time to draw a time-speed curve.

5. The motion recognition method based on human skeletal joint points according to claim 1, characterized in that: Analyzing the user's motion state includes: using the cross-point speed to obtain action feature data for judging the user's motion state; when making a detailed judgment on the motion state under the same motion state, use the instantaneous speed to obtain action feature data for comparison and analysis.

6. The motion recognition method based on human skeletal joint points according to claim 1, characterized in that: In step 4, when the collected important action feature data cannot be matched with the standard action feature data stored in the database, store the important action feature data and repeat steps 1 to 4 until after matching, update and supplement the database.

7. The motion recognition method based on human skeletal joint points according to claim 1, It is characterized in that: In step three, when performing importance analysis on the action feature data, it includes that under the same motion state, if the action feature data appears irregularly, it is determined that the action feature data is unimportant action feature data, and the action feature data of this bone joint point position is not stored; if the action feature data appears regularly, it is determined that the action feature data is important action feature data, and the action feature data of this bone joint point position is stored.

Images