A method for recognizing and predicting human motion

Abstract

The application relates to a human action recognition and prediction method, which comprises the following steps: collecting foot acceleration data in real time; dividing a step phase based on the foot movement data, and constructing a step phase feature vector according to the foot movement data of each step phase; putting the step phase feature vector into a step phase classification model for classification and recognition to obtain a step phase classification result; and predicting whether the next action is a target action based on the step phase classification result. The application can accurately predict the target action and accelerate the starting speed of a response device.

Description

Technical Field

[0001] This invention relates to the field of human motion recognition, and in particular to a method for recognizing and predicting human motion. Background Technology

[0002] Currently, common motion prediction or recognition methods are mainly based on image information. For example, document CN115393964 A discloses a fitness motion recognition method and device based on BlazePose. This invention uses a lightweight convolutional neural network based on BlazePose to estimate human pose in images, obtaining the joint position information of the user during exercise. The joint position information is converted into feature vectors, and the current motion category is extracted based on KNN classification. Angle analysis and distance thresholds are used to determine whether the user's pose in the current motion category is accurate, and suggestions for motion improvement are provided based on the results. Document CN115273244A discloses a human motion recognition method and system based on graph neural networks. It processes input videos through a pre-trained graph neural network-based human motion recognition network and outputs motion classification. This human motion recognition network includes a 2D joint recognition network, a 3D joint recognition network, and a fully connected classification layer. It uses downsampling and corresponding upsampling layers with short connections to achieve 2D feature extraction and joint recognition of the data. The output is used as the skeleton graph input of the graph neural network, thereby outputting accurate 3D motion and recognizing the corresponding motion category from the motion sequence. Reference 108284444A discloses a multi-modal human motion prediction method based on the Tc-ProMps algorithm under human-computer collaboration. The prediction model is mainly divided into two modules: offline training and online prediction. The offline module trains human motion samples to obtain the feature weight vector w and its probability distribution representing a certain type of motion skill. The online module observes human motion through visual information and performs online recognition and rolling prediction of human motion.

[0003] Machine learning-based action recognition algorithms first classify the data and then transmit the classification results to the terminal device for response. However, human movements sometimes occur very rapidly. If the data for an action is fully collected and analyzed, the action has already been completed, missing the opportunity to respond. If recognition and analysis are performed at the beginning of the movement, the terminal device needs to be driven to initiate or complete the response action before the action is finished, which remains a significant challenge. Image-based action prediction algorithms require external devices such as cameras to acquire and analyze image data, which is not feasible in many scenarios. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for recognizing and predicting human movements, which can accurately predict the next human movement.

[0005] The technical solution adopted by this invention to solve its technical problem is: to provide a method for recognizing and predicting human movements, comprising the following steps:

[0006] Real-time collection of foot movement data;

[0007] The foot motion data is used to segment gait phases, and a gait phase feature vector is constructed based on the foot motion data of each gait phase.

[0008] The step phase feature vector is put into the step phase classification model for classification and recognition to obtain the step phase classification result;

[0009] Based on the gait classification results, predict whether the next action is the target action.

[0010] Furthermore, the foot motion data includes foot acceleration data, which includes horizontal axis acceleration, vertical axis acceleration, and vertical axis acceleration. The horizontal axis acceleration is the acceleration in the lateral displacement direction of the human body, the vertical axis acceleration is the acceleration in the forward movement direction of the human body, and the vertical axis acceleration is the acceleration in the vertical direction of the human body.

[0011] Furthermore, the step phase feature vector includes the mean horizontal axis acceleration, the mean vertical axis acceleration, and the mean vertical axis acceleration.

[0012] Furthermore, the step of segmenting gait phases based on the foot movement data and constructing a gait phase feature vector based on the foot movement data for each gait phase includes:

[0013] Set a gait reference cycle, which is the estimated time required for the human body to complete one gait phase;

[0014] Within the gait reference cycle, the periodic change pattern of the foot motion data is used to determine whether the current gait phase is completed. If it is completed, the foot motion data obtained within the current gait reference cycle is divided into a gait phase, the gait phase feature vector is constructed, and the next gait reference cycle begins. Otherwise, the foot motion data is collected again.

[0015] If it is determined that the current step phase has not been completed within the reference cycle of the step phase, the foot motion data obtained within the entire reference cycle of the step phase is divided into a step phase, and the step phase feature vector is constructed.

[0016] Furthermore, when the human body is moving, determining whether the current gait phase is completed based on the periodic changes in the foot movement data includes:

[0017] Set the positive and negative reference peak values ​​for the longitudinal axis acceleration;

[0018] When the longitudinal axis acceleration reaches one positive reference peak and one negative reference peak and then returns to the initial acceleration, it is considered that the human body has completed a gait phase.

[0019] Furthermore, the foot movement data also includes foot pressure data.

[0020] Furthermore, when the human body is moving, determining whether the current gait phase is completed based on the periodic changes in the foot movement data includes:

[0021] Set the positive and negative reference peak values ​​for the longitudinal axis acceleration;

[0022] When the longitudinal axis acceleration reaches one positive reference peak and one negative reference peak and then returns to the initial acceleration, and the foot pressure data increases and then decreases, it is considered that the human body has completed a gait phase.

[0023] Furthermore, the step phase classification result includes target action, preparatory action, and non-target action, wherein the preparatory action is the previous step phase or a combination of the previous multiple step phases of the target action.

[0024] Furthermore, the step-by-step prediction of whether the next action is the target action based on the step-by-step classification result includes:

[0025] If the current step phase is the non-preparatory action, return to the step of real-time acquisition of foot movement data;

[0026] If the current step phase is the preparatory action, and the preparatory action includes a step phase, the next action is predicted to be the target action;

[0027] If the current step phase is the preparatory action, and the preparatory action includes multiple step phases, determine whether the current step phase is the last step phase in the preparatory action. If so, predict the next action as the target action; otherwise, record the current step phase and return to the step of real-time acquisition of foot movement data.

[0028] Furthermore, the step classification model is constructed based on support vector machines.

[0029] Furthermore, it also includes the step of initiating a response action based on the phase classification result.

[0030] Beneficial effects

[0031] By employing the above-mentioned technical solution, this invention has the following advantages and positive effects compared with the prior art: Based on the characteristics of human movement, this invention divides human movements into different phases for analysis, and takes the preceding phase of the target movement as a preparatory movement. It uses a classification model to identify the preparatory movement and then predict the target movement, rather than identifying the target movement that is in progress or has already been completed. When the preparatory movement is completed, the identification process of the preparatory movement is also completed simultaneously. When the target movement begins, the response has already started, enabling the response device to predict the target movement earlier and providing more time to prepare subsequent corresponding actions. Attached Figure Description

[0032] Figure 1 This is a schematic diagram showing the breakdown of human movement stages in an embodiment of the present invention;

[0033] Figure 2 This is a flowchart of an embodiment of the present invention;

[0034] Figure 3 This is a flowchart of action prediction and response in an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of foot data for a human body's emergency stop action in an embodiment of the present invention. Detailed Implementation

[0036] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0037] Embodiments of the present invention relate to a method for recognizing and predicting human movements, such as... Figure 2 As shown, sensors are used to collect foot acceleration and plantar pressure data. The controller reads and processes the collected data. First, the data is smoothed and filtered to obtain preprocessed data. Since each step of human movement can be roughly considered a periodic motion, the gait can be divided according to the periodic characteristics of the data. The feature vector of each gait phase is then fed into a trained classification model for classification and recognition. For example... Figure 1 As shown, each step is divided into target action, non-target action, and preparation action according to the requirements. Preparation action can be a combination of one or several steps. The target action is completed after the preparation action is completed.

[0038] Predicting the response process as follows Figure 3As shown, if the classification result is a non-preparatory action (i.e., a target action or a non-target action), the action prediction process ends, no response is initiated, and gait data is re-inputted for classification and recognition. If the classification result is a preparatory action or part of a preparatory action, gait recording begins. If a single gait or a combination of several gait phases is identified as a preparatory action, the next gait phase is predicted to be the target action, and a response initiation signal is output to drive the response. The response action can be any behavior targeting the target action, such as recording or feedback from a wearable device, or enabling or disabling functions of an external device. The target action can also be any action in any movement; by constructing a model based on the acquired data, this invention can be applied to various movements.

[0039] Taking the sudden stop action in basketball as an example, if the stopping action is taken as the target action, then the step before stopping is the preparatory action, and the running action is the non-target action.

[0040] First, foot pressure and acceleration data for the corresponding movements need to be collected for data processing and modeling. During the data processing stage, a moving average filtering method is used to preprocess the sensor data. After processing, gait phases are divided based on the periodicity of human movement.

[0041] Preprocessed data such as Figure 4 As shown, in the upper graph, the horizontal axis represents sampling points, and the vertical axis represents acceleration. In the lower graph, the horizontal axis represents sampling points, and the vertical axis represents pressure values. As the subject's foot lands, supports, pushes off, and swings, besides the foot pressure on the inner forefoot initially increasing and then decreasing, the acceleration in the x, y, and z directions also changes. The data has better periodicity but a larger data volume. The y-direction represents the forward direction of running. When the runner pushes off the ground, they exert force against the ground to receive feedback from the ground's reaction force, providing energy for initiation. Therefore, the plantar pressure and acceleration in the y-direction begin to increase. After the foot initiates the movement and leaves the ground, the plantar pressure remains relatively stable, mainly due to the pressure from the foot and shoe sole adjusted by the shoe's support. After leaving the ground, the acceleration in the y-direction begins to decrease. When the subject's foot lands, the plantar pressure increases, and the acceleration in the y-direction increases, entering the next cycle. During running, the acceleration in the y-direction fluctuates slightly between the foot landing and leaving the ground again. Acceleration in other directions also exhibits a certain periodicity. Acceleration in the z-direction decreases when the feet lift off the ground and increases when the feet land, fluctuating within a certain range during landing and takeoff. Based on these characteristics, after defining the gait phases, the mean values ​​of acceleration in the x, y, and z directions (x.mean(), y.mean(), z.mean()) are used to construct a feature vector, which is then fed into a gait phase classification model based on Support Vector Machine (SVM) for classification.

[0042] The data acquired in real time during movement is first divided into gait phases based on periodicity. Taking the movement of a human body moving forward as an example, a baseline gait phase period and a baseline peak value of acceleration in the y-direction are preset based on experience. Within the current period, if the acceleration reaches the baseline peak value once in the positive and once in the negative directions and then returns to near the initial value, the human body is considered to have completed a gait phase. To obtain more accurate results, plantar pressure data can be used for auxiliary judgment. If, within the current period, the acceleration reaches the baseline peak value once in the positive and once in the negative directions and then returns to near the initial value, and the plantar pressure data increases and then decreases, the human body is considered to have completed a gait phase.

[0043] If the system consistently determines that a step phase has not been completed within a given cycle, then the data from the entire step phase baseline cycle is divided into a single step phase. Feature vectors are constructed based on these divided step phases and fed into the model for recognition. If the recognition indicates a preparatory action, i.e., the step before an emergency stop, then the next step is considered an emergency stop action, triggering a response activity. Simultaneously, the human body enters the emergency stop action, allowing for targeted activities to begin at the start of the emergency stop without missing it.

[0044] The following are the specific prediction results. Using 30% of all existing samples as the samples to be identified, and the remaining 70% of the existing sample data, the algorithm was designed and modeled. The overall recognition accuracy of the algorithm is 88.9%, and the recognition accuracy of the pre-emergency stop action is 73.1%.

[0045] For example, in the running and shooting motion of football, if the shooting motion is taken as the target motion, then the step before the shooting motion is taken as the preparatory motion, and the running motion in the first part of the motion is taken as the non-target motion. According to the method of the present invention, the preparatory motion is identified after the step before the shooting motion is completed, the next step is predicted to be the shooting motion, and a response signal is issued, which can control other devices to react to the shooting motion.

[0046] In some sports with stronger regularity, such as the triple jump, the regular stepping motion before takeoff can be used as a preparatory action. That is, the preparatory action is a combination of several step phases. After recognizing it, a response is made to the takeoff action.

[0047] In certain scenarios, such as daily walking, the normal walking gait can be used as a non-target action. When encountering some sudden situations, such as falling, the step before the fall is taken as a preparatory action. Once the fall is detected, it can be predicted that the next step will be a fall. When the fall begins, external devices can be activated to prevent and respond to it.

Claims

1. A method for recognizing and predicting human movements, characterized in that, Includes the following steps: The S100 collects foot motion data in real time. The foot motion data includes horizontal axis acceleration, vertical axis acceleration and vertical axis acceleration. The horizontal axis acceleration is the acceleration in the lateral displacement direction of the human body, the vertical axis acceleration is the acceleration in the forward movement direction of the human body, and the vertical axis acceleration is the acceleration in the vertical direction of the human body. S200 performs gait phase segmentation based on the foot motion data and constructs a gait phase feature vector based on the foot motion data of each gait phase. The gait phase feature vector includes the mean horizontal axis acceleration, the mean vertical axis acceleration, and the mean vertical axis acceleration. S300 inputs the step phase feature vector into the step phase classification model for classification and recognition, and obtains the step phase classification result. The step phase classification result includes target action, preparatory action, and non-target action. The preparatory action is the step phase preceding the target action or a combination of the previous step phases or multiple previous step phases. S400 predicts whether the next action is the target action based on the gait classification results; Step S200 includes: S201 sets the gait reference cycle, which is the estimated time required for the human body to complete one gait phase. S202 Within the gait reference cycle, based on the periodic variation pattern of the foot motion data, it is determined whether the current gait phase is completed. If it is completed, the foot motion data obtained within the current gait reference cycle is divided into a gait phase, the gait phase feature vector is constructed, and the next gait reference cycle begins; otherwise, the foot motion data is collected again. When the human body is walking, determining whether the current gait phase is completed based on the periodic variation pattern of the foot motion data includes: setting a positive reference peak value and a negative reference peak value for the longitudinal axis acceleration; when the longitudinal axis acceleration reaches one positive reference peak value and one negative reference peak value and then returns to the initial acceleration, it is considered that the human body has completed a gait phase. S203 If it is determined that the current step phase has not been completed within the step phase reference cycle, the foot motion data obtained within the entire step phase reference cycle is divided into a step phase, and the step phase feature vector is constructed. Step S400 includes: If the current step phase is not a preparatory action, return to the step of real-time acquisition of foot movement data; If the current step phase is the preparatory action, and the preparatory action includes a step phase, the next action is predicted to be the target action; If the current step phase is the preparatory action, and the preparatory action includes multiple step phases, determine whether the current step phase is the last step phase in the preparatory action. If so, predict the next action as the target action; otherwise, record the current step phase and return to the step of real-time acquisition of foot movement data.

2. The method according to claim 1, characterized in that, The foot movement data also includes foot pressure data.

3. The method according to claim 2, characterized in that, When a human body is walking, the determination of whether the current gait phase is completed based on the periodic changes in the foot movement data includes: Set the positive and negative reference peak values ​​for the longitudinal axis acceleration; When the longitudinal axis acceleration reaches one positive reference peak and one negative reference peak and then returns to the initial acceleration, and the foot pressure data increases and then decreases, it is considered that the human body has completed a gait phase.

4. The method according to claim 1, characterized in that, The gait classification model is built on support vector machines.

5. The method according to claim 1, characterized in that, It also includes the step of initiating a response action based on the phase classification result.

Images