A method for real-time collection and processing of motion data

Abstract

The present invention relates to the field of intelligent information processing, and in particular to a method for real-time collection and processing of motion data. The content includes: collecting multi-dimensional motion data and correcting it, performing spatiotemporal enhancement processing on the corrected motion data to obtain enhanced spatiotemporal data; mapping the enhanced spatiotemporal data into a graph convolutional network to extract spatiotemporal features; predicting the motion state based on the spatiotemporal features; introducing a reward mechanism and a feedback mechanism to evaluate the effect of motion behavior and dynamically adjust the motion behavior strategy. It solves the problem that the motion data collected in the traditional method cannot be aligned in time and space, affecting the accuracy of data fusion and the accuracy of motion state analysis; it cannot effectively capture the spatiotemporal dependencies in complex motion, and the accuracy of motion pattern recognition and motion state reasoning is low; it lacks dynamic optimization capabilities and is difficult to adjust the prediction strategy in real time; it cannot dynamically adjust the analysis strategy and optimization scheme, and the breadth and practicality of the application scenario are limited.

Description

Technical Field

[0001] The present invention relates to the field of intelligent information processing, and in particular to a method for real-time collection and processing of motion data. Background Art

[0002] With the rapid development of wearable devices, intelligent health monitoring systems, and sports training auxiliary technologies, the real-time collection and processing of sports data has become an important research direction in the current information processing field. Modern people have an increasing demand for health management and sports performance optimization. In scenes such as fitness, rehabilitation, and competitive sports, real-time collection of sports data through multi-sensor devices and intelligent processing of them can help users fully understand their sports status, evaluate their sports performance, and obtain scientific sports advice. For example, devices worn on the wrist, ankle, or chest can monitor the user's motion trajectory, speed, acceleration, angular velocity, spatial orientation, and other parameters in real time through built-in sensors such as accelerometers, gyroscopes, and magnetometers; heart rate sensors, electromyography sensors, etc. can record physiological characteristics during exercise. By comprehensively analyzing these data, the complex temporal and spatial changes of users during exercise can be revealed.

[0003] In practical applications, the processing of motion data usually faces many challenges. For example, the data collected by sensors has various dimensions and different sampling frequencies, which means that the motion data needs to be aligned in time series; in addition, multiple sensors are distributed in different locations, and their measurement results will be affected by the differences in spatial positions. Sensor data will also be disturbed by the complexity of the external environment and the noise problems of the sensor itself, which may lead to a decrease in data quality. To address this problem, modern data analysis technology is gradually developing in the direction of combining spatiotemporal features, deep learning, and feedback optimization. At the same time, the demand for motion data analysis with increasing dynamic and real-time requirements has also put forward higher requirements on the existing processing architecture and algorithm design.

[0004] Traditional motion data processing methods have the following technical problems: the collected motion data cannot be aligned in time and space, affecting the accuracy of data fusion and the precision of motion state analysis; it is impossible to effectively capture the spatiotemporal dependencies in complex motions, resulting in low accuracy in motion pattern recognition and motion state reasoning; it lacks dynamic optimization capabilities, making it difficult to adjust prediction strategies in real time to adapt to complex motion scenarios, and is unable to provide efficient feedback and guidance during motion; it is impossible to dynamically adjust analysis strategies and optimization plans for different motion modes such as walking, running, and cycling, which severely limits the breadth and practicality of application scenarios. Summary of the invention

[0005] The present invention provides a method for real-time collection and processing of motion data to solve the problems that in traditional motion data processing methods, the collected motion data cannot be aligned in time and space, affecting the accuracy of data fusion and the precision of motion state analysis; the spatiotemporal dependencies in complex motions cannot be effectively captured, resulting in low precision of motion pattern recognition and motion state reasoning; there is a lack of dynamic optimization capability, making it difficult to adjust prediction strategies in real time to adapt to complex motion scenes, and it is impossible to provide efficient feedback and guidance during the motion process; and it is impossible to dynamically adjust analysis strategies and optimization schemes for different motion modes such as walking, running, and cycling, which seriously limits the breadth and practicality of application scenarios.

[0006] A method for real-time collection and processing of motion data of the present invention specifically includes the following technical solutions:

[0007] A method for real-time collection and processing of motion data comprises the following steps:

[0008] S1. Collect multi-dimensional motion data and correct it to obtain corrected motion data; perform spatiotemporal enhancement processing on the corrected motion data to obtain enhanced spatiotemporal data; map the enhanced spatiotemporal data to a graph convolutional network to extract spatiotemporal features;

[0009] S2. Input spatiotemporal features into the deep reinforcement learning model to predict the motion state; introduce reward mechanism and feedback mechanism to evaluate the effect of motion behavior and dynamically adjust the motion behavior strategy.

[0010] Preferably, the S1 specifically includes:

[0011] The multi-dimensional motion data is uniformly time-series and spatially aligned, and the data from different sensors are fused through spatiotemporal correction to obtain the corrected motion data.

[0012] Preferably, the S1 specifically includes:

[0013] In the process of space-time correction, weighting and acceleration correction are introduced. The specific formula is as follows:

[0014] ,

[0015] in, is the corrected motion data; is the number of sensors; is the index variable of the sensor; It is Sensors at time The weight coefficient of It is Sensors at time The collected raw data; is the acceleration correction term; is the acceleration correction factor.

[0016] Preferably, the S1 specifically includes:

[0017] In the process of spatiotemporal enhancement, spatial correlation is introduced, and spatial data fusion is performed in a weighted manner to obtain enhanced spatiotemporal data; the specific formula is as follows:

[0018] ,

[0019] in, is the enhanced spatiotemporal data, indicating that and spatial location The motion data value after spatiotemporal filtering and enhancement; is the initial time; is the number of spatiotemporal enhanced features, indicating the number of spatial features involved in data enhancement; is the spatial feature gain coefficient; It is A spatial characteristic function used to describe the spatial position Impact on data augmentation; is the time decay coefficient; is the noise attenuation function; It is the acceleration correction coefficient, which is used to adjust the contribution of the acceleration correction part in the data enhancement process to the enhanced spatiotemporal data.

[0020] Preferably, the S1 specifically includes:

[0021] The enhanced spatiotemporal data is mapped to the nodes of each sensor and input into the graph convolutional network. The weight matrix is ​​introduced to aggregate the node features of the previous layer and the neighboring node features to generate a new node representation, and nonlinear transformation is introduced through the activation function to obtain the spatiotemporal features.

[0022] Preferably, the S2 specifically includes:

[0023] The spatiotemporal features are input into the deep reinforcement learning model to predict the current motion state; a reward mechanism is introduced to calculate the reward value based on the current motion state, combined with the historical motion state and spatiotemporal features.

[0024] Preferably, the S2 specifically includes:

[0025] In the deep reinforcement learning model, a feedback mechanism is introduced to calculate the feedback amount based on the current reward value and the average of historical rewards, combined with the spatiotemporal characteristics of each dimension.

[0026] Preferably, the S2 specifically includes:

[0027] Based on the feedback amount, the movement behavior strategy is dynamically adjusted, and the training process is guided according to the real-time reward value.

[0028] The beneficial effects of the technical solution of the present invention are:

[0029] 1. The present invention introduces weighting and acceleration correction and fuses the measurement data of multiple sensors, so that the generated corrected motion data has higher accuracy and reliability. The corrected motion data can effectively reflect the actual motion state, laying a solid foundation for subsequent data enhancement and processing.

[0030] 2. The present invention introduces the joint modeling of spatiotemporal features in the data enhancement link, comprehensively considering the temporal changes and spatial distribution laws of the data; the spatial correlation of the sensor is modeled through the spatial feature function, and the historical data is weighted in combination with the time attenuation coefficient, thereby enhancing the robustness and dynamic adaptability of the data.

[0031] 3. The present invention uses a graph convolutional network to extract features from the enhanced spatiotemporal data, fully mining the dependencies and spatiotemporal features between sensor nodes; through the multi-layer structure of the graph convolutional network, combined with the adjacency relationship between sensors and the characteristics of nonlinear activation functions, it successfully captures complex motion patterns and dynamic change laws.

[0032] 4. The present invention combines the deep reinforcement learning model with the reward mechanism and feedback mechanism to achieve dynamic optimization of the motion state; the reward mechanism not only takes into account the current motion behavior effect, but also combines the changing characteristics of the historical motion state and action, so that the deep reinforcement learning model can be continuously adjusted and optimized in a dynamic environment; by training the deep reinforcement learning model, a more intelligent motion behavior strategy is generated, which effectively improves the accuracy and real-time performance of motion state prediction.

[0033] 5. Through the feedback mechanism, the present invention can optimize the exercise behavior strategy in real time according to the deviation between the current reward value and the historical reward, combined with the contribution of spatiotemporal characteristics; the calculation of the feedback amount not only reflects the current exercise state deviation, but also provides users with personalized adjustment suggestions by comprehensively analyzing the influence of multiple feature dimensions, such as changing the step frequency, stride or exercise intensity; the feedback mechanism can respond quickly to environmental changes, helping users to optimize their exercise performance in real time during exercise, thereby achieving higher exercise efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The present invention is a flow chart of a method for real-time collection and processing of motion data. DETAILED DESCRIPTION

[0035] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0037] The specific scheme of the method for real-time collection and processing of motion data provided by the present invention is described in detail below with reference to the accompanying drawings.

[0038] See attached Figure 1 , which shows a flow chart of a method for real-time collection and processing of motion data provided by an embodiment of the present invention, the method comprising the following steps:

[0039] S1. Collect multi-dimensional motion data and correct it to obtain corrected motion data; perform spatiotemporal enhancement processing on the corrected motion data to obtain enhanced spatiotemporal data; map the enhanced spatiotemporal data to a graph convolutional network to extract spatiotemporal features;

[0040] Multi-dimensional motion data is collected from multiple sensors, including accelerometers, gyroscopes, magnetometers, etc. The sensors can provide motion data of different dimensions, such as acceleration, angular velocity, orientation, etc. In practical applications, sensors are often distributed in different spatial locations and may have different measurement perspectives for the same motion phenomenon, resulting in spatial and temporal inconsistency of the data. In addition, the error of the sensor itself will also affect the quality of the motion data. For example, factors such as temperature changes and equipment aging will cause deviations in the sensor's measurement results.

[0041] Therefore, it is necessary to uniformly time-series and spatially align the multi-dimensional motion data to ensure that the timestamps of the sensor data are consistent and to eliminate the spatial differences caused by different sensor positions. Specifically, in the time-series process, the sampling frequencies and timestamps of all sensors need to be synchronized, and time interpolation technology is used to match the motion data at different sampling time points; while spatial alignment relies on the accurate calibration of the sensor position, and the spatial differences between different sensors are corrected by establishing a suitable mathematical model to ensure a unified spatial coordinate system when data is fused.

[0042] Since the clocks of multiple sensors are not synchronized, the time sensor will be offset, resulting in data misalignment, which will affect the accuracy of motion analysis. In addition, the sensors may be in different spatial positions, and the distance and angle differences between them will also cause spatial inconsistency. Therefore, time-space correction is used to eliminate the timing error and spatial offset between the data of different sensors. The time-space correction adopts weighted and acceleration correction to fuse the motion data from different sensors to obtain the corrected motion data. The specific formula of time-space correction is as follows:

[0043] ,

[0044] in, is the corrected motion data, which represents the comprehensive result after considering multiple sensor data and performing weighting and acceleration correction; is the number of sensors; is the index variable of the sensor; It is Sensors at time The weight coefficient is dynamically adjusted according to the accuracy and measurement error of the sensor, and the inverse of the measurement error can be used; It is Sensors at time The collected raw data; is the acceleration correction term, which indicates the rate of change of the original data; is the acceleration correction coefficient, which is used to control the strength of the acceleration correction and is obtained through experiments. The corrected motion data can effectively eliminate the inconsistency of time and space and ensure that the outputs of each sensor are aligned in time and space.

[0045] In order to expand the diversity of motion data and ensure its consistency in time and space, the modified motion data is subjected to spatiotemporal enhancement processing to obtain enhanced spatiotemporal data; spatiotemporal filtering is used to eliminate noise and inconsistency, and enhancement is performed to capture motion characteristics and environmental changes; spatiotemporal enhancement processing not only considers the time series characteristics of the modified motion data, but also introduces spatial correlation to improve the robustness of the data, and uses a weighted method to perform spatial data fusion. At the same time, the historical data is weighted by the time attenuation coefficient, and a noise attenuation function is introduced to reduce the impact of noise. The specific formula is as follows:

[0046] ,

[0047] in, is the enhanced spatiotemporal data, indicating that and spatial location The motion data value after spatiotemporal filtering and enhancement; is the initial time; is the number of spatiotemporal enhanced features, indicating the number of spatial features involved in data enhancement; is the spatial feature gain coefficient, which is used to control the contribution of each spatial feature to data enhancement and is obtained through experiments; It is A spatial characteristic function used to describe the spatial position The impact on data augmentation is set based on the geometric model or physical characteristics of the sensor; is the time decay coefficient, which is used to control the impact of historical data on data enhancement and is obtained through experiments; is the noise attenuation function, which is used to reduce the influence of external interference or sensor noise; It is the acceleration correction coefficient, which is used to adjust the contribution of the acceleration correction part in the data enhancement process to the enhanced spatiotemporal data. Through enhancement and spatiotemporal filtering, the enhanced spatiotemporal data is made more spatiotemporally consistent and robust, thus providing high-quality data support for the input of the graph convolutional network (GCN).

[0048] The dimension of the enhanced spatiotemporal data includes sensor measurements at multiple moments and enhanced spatiotemporal features. The enhanced spatiotemporal data is mapped to the nodes of each sensor and input into the graph convolutional network. As a powerful graph data processing method, the graph convolutional network can effectively capture the spatiotemporal dependencies between sensor nodes, including the relationship and motion patterns between sensors, during the learning process of spatiotemporal data. The output of each layer of the graph convolutional network is a new node representation generated by aggregating the node features of the previous layer and the features of neighboring nodes based on the weight matrix, and nonlinear transformations are introduced through activation functions to further improve the expression of spatiotemporal features. Specifically, the feature value of the current node not only depends on its own features, but also is affected by the weighted influence of its neighboring node information, thereby realizing the aggregation and propagation of spatiotemporal features. The formula is as follows:

[0049] ,

[0050] ,

[0051] in, It is the graph convolutional network The feature matrix output by the layer represents the spatiotemporal features after graph convolution; It is an activation function, usually using nonlinear functions such as ReLU, to control the nonlinear expression ability of graph convolutional networks; Is a node The neighbor set of,reflects the spatiotemporal dependency between sensors; It is The weight matrix of the layer is used for graph convolution operations to learn the characteristics of graph structure data, obtained through experiments; It is The sensor in Feature representation in the layer; It is The bias term of the layer is a parameter obtained by back-propagation optimization, which is used to control the propagation and integration of spatiotemporal features; Is Mapped, It is the first The data corresponding to each sensor; is a The weight matrix of is obtained through experiments. The goal of graph convolutional networks is to transfer and aggregate information through multi-layer networks, learn the spatiotemporal dependencies of data, and provide high-dimensional feature representation for the reasoning of motion states. Graph convolutional networks are used to effectively extract complex features in spatiotemporal data and convert them into high-dimensional feature matrices that can be used in deep reinforcement learning (DRL) models.

[0052] S2. Input spatiotemporal features into the deep reinforcement learning model to predict the motion state; introduce reward mechanism and feedback mechanism to evaluate the effect of motion behavior and dynamically adjust the motion behavior strategy.

[0053] The spatiotemporal features extracted by the graph convolutional network are used as the input of the deep reinforcement learning model. After multiple training and adjustments of the deep reinforcement learning model, the current motion state is predicted based on the spatiotemporal features, historical reward values, and historical motion states, and a reward mechanism is established to calculate the reward value; the reward value not only reflects the effect of the behavior at the current moment, but also takes into account the changes in historical motion states and actions. The specific formula is:

[0054] ,

[0055] in, It's in time The reward value indicates the effect of the action selected at the current moment; It's in time The reward value of is a discount factor used to balance current rewards with future rewards, , obtained through experiments; It is the adjustment coefficient of the motion state change, which determines the impact of the current motion state change on the reward value and is obtained through experiments; and Respectively, in time and The motion state indicator variable at the time reflects the motion state characteristics of the current and previous moments; is the reward adjustment coefficient, which is used to control the contribution of spatiotemporal features extracted after graph convolution to the reward value. It is obtained through experiments. The reward value reflects the effectiveness of the behavior taken at the current moment, and is also closely related to the historical motion state changes and action selection. It can adjust its motion behavior strategy based on past behavior experience and current feedback signals, thereby continuously optimizing the performance and prediction accuracy of the deep reinforcement learning model.

[0056] In the deep reinforcement learning model, a feedback mechanism is introduced to calculate the feedback amount based on the current reward value and the average of historical rewards, and then adjust the movement behavior strategy to guide the deep reinforcement learning model to choose the next action; guide the training process based on the real-time reward value to avoid overfitting or underfitting and ensure continuous improvement of training. The calculation of the feedback amount is not only based on the difference between the current reward value and the average of historical rewards, but also takes into account the contribution of each feature dimension. The calculation formula of the feedback amount is:

[0057] ,

[0058] in, is The amount of feedback in time represents the amount of adjustment of the current predicted movement state in the movement behavior strategy; It is the average of historical rewards, indicating the average level of rewards over a period of time; is the number of spatiotemporal feature dimensions; It is The weight coefficient of the dimensional spatiotemporal feature is used to control the contribution of each dimensional spatiotemporal feature to the feedback amount and is obtained through experiments; It is the graph convolutional network The output of the layer The feedback can be used to adjust the current motion parameters in real time. For example, by analyzing the reward value and spatiotemporal characteristics, it is recommended that users increase their step frequency, reduce exercise intensity, or adjust their stride length, etc., to directly improve their current exercise performance. The core role of feedback is to guide adjustments through the deviation between the current reward value and the historical reward, improve decision-making efficiency, optimize the prediction of motion status in real time, and make subsequent decisions more accurately.

[0059] As a specific example, assume that the application scenario is a smart sports bracelet that is used by runners to monitor and optimize their exercise status in real time. The current goal is to adjust the user's step frequency through feedback to achieve the optimal state of energy consumption. At this moment, the system calculates the current reward value based on the motion data collected by multiple sensors (such as accelerometers and heart rate sensors). For example, the deviation between the current cadence and the target cadence leads to higher energy consumption, and the reward value is 0.6; the historical average reward is 0.8, which means that there have been better states in history. Calculate the feedback amount to reflect the direction and strength of the current state. If , indicating that the current state has a certain negative deviation from the historical optimal state. Assume that the weight of the cadence feature is 0.7 and the weight of the heart rate feature is 0.3. Calculate the feedback amount:

[0060] ,

[0061] Assumptions , ,but:

[0062] ,

[0063] The feedback amount is -0.27, indicating that the current cadence is too high and needs to be reduced to reduce energy consumption. The system adjusts the cadence strategy based on the feedback amount. For example: the current cadence is 180 steps / minute, and the system calculates the cadence adjustment range as: ,in is the proportional coefficient of the step frequency adjustment, assuming ; The adjusted step frequency is: The system detects the impact of the adjusted step frequency on the reward value in real time. If the new reward value increases, for example, 0.75, it means that the adjustment is effective. The system will further incorporate the new reward value into the calculation of the historical reward average value to dynamically optimize the subsequent exercise behavior strategy.

[0064] In summary, a method for real-time collection and processing of motion data is completed.

[0065] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0066] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for real-time collection and processing of motion data, characterized in that: The following steps are involved: S1. Collect multi-dimensional motion data and perform spatiotemporal correction to obtain corrected motion data. The specific formula is as follows: ， in, is the corrected motion data; is the number of sensors; is the index variable of the sensor; It is Sensors at time The weight coefficient of It is The raw data collected by each sensor at time; is the acceleration correction term; is the acceleration correction factor; The corrected motion data is subjected to spatiotemporal enhancement processing, spatial correlation is introduced, and spatial data fusion is performed in a weighted manner to obtain enhanced spatiotemporal data; the specific formula for spatiotemporal enhancement processing is as follows: ， in, is the enhanced spatiotemporal data, indicating that and spatial location The motion data value after spatiotemporal filtering and enhancement; is the initial time; is the number of spatiotemporal enhanced features, indicating the number of spatial features involved in data enhancement; is the spatial feature gain coefficient; It is A spatial characteristic function is used to describe the spatial position Impact on data augmentation; is the time decay coefficient; is the noise attenuation function; is the acceleration correction coefficient, which is used to adjust the contribution of the acceleration correction part in the data enhancement process to the enhanced spatiotemporal data; Map the enhanced spatiotemporal data into the graph convolutional network to extract spatiotemporal features; S2. Input spatiotemporal features into the deep reinforcement learning model to predict the motion state; introduce reward mechanism and feedback mechanism to evaluate the effect of motion behavior and dynamically adjust the motion behavior strategy.

2. The method for real-time collection and processing of motion data according to claim 1, characterized in that: The S1 specifically includes: The multi-dimensional motion data is uniformly time-series and spatially aligned, and the data from different sensors are fused through spatiotemporal correction to obtain the corrected motion data.

3. The method for real-time collection and processing of motion data according to claim 2, characterized in that: The S1 specifically includes: The enhanced spatiotemporal data is mapped to the nodes of each sensor and input into the graph convolutional network. The weight matrix is ​​introduced to aggregate the node features of the previous layer and the neighboring node features to generate a new node representation, and nonlinear transformation is introduced through the activation function to obtain the spatiotemporal features.

4. The method for real-time collection and processing of motion data according to claim 1, characterized in that: The S2 specifically includes: The spatiotemporal features are input into the deep reinforcement learning model to predict the current motion state; a reward mechanism is introduced to calculate the reward value based on the current motion state, combined with the historical motion state and spatiotemporal features.

5. The method for real-time collection and processing of motion data according to claim 4, characterized in that: The S2 specifically includes: In the deep reinforcement learning model, a feedback mechanism is introduced to calculate the feedback amount based on the current reward value and the average of historical rewards, combined with the spatiotemporal characteristics of each dimension.

6. A method for real-time collection and processing of motion data according to claim 5, characterized in that: The S2 specifically includes: Based on the feedback amount, the movement behavior strategy is dynamically adjusted, and the training process is guided according to the real-time reward value.

Images