Motion Prediction Method Based on Frequency Domain Mask and Human Motion Risk Feedback Device

Abstract

The present invention discloses a motion prediction method based on frequency domain masking and a human body motion risk feedback device, which relates to the technical field of signal processing. By obtaining multiple frequency domain channel data through a preposed frequency domain mask, restoring them and then combining with a multi-layer perceptron for feature extraction, filtering and restoring through a postposed mask, and finally fusing residuals for effective prediction, thereby improving the prediction accuracy and model robustness. At the same time, it adapts to complex environments with long time series and dynamic changes, and provides an efficient, accurate and versatile human body motion risk feedback device with functions of biological information collection, data processing and interactive output.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a motion prediction method based on frequency-domain masking and a human motion risk feedback device. Background Technique

[0002] Human motion prediction is an important application in the fields of computer vision and deep learning, and is widely used in many fields such as security monitoring, smart home, virtual reality and augmented reality, and autonomous driving. Traditional human motion prediction methods are mostly based on time series analysis techniques, such as convolutional neural networks, long short-term memory networks, and recently popular models based on self-attention mechanisms. These methods usually rely on historical data sequences to predict future actions. The human motion risk feedback device is a motion monitoring device designed for people with physical disabilities, aiming to avoid joint or muscle injuries in the lives of such people.

[0003] Traditional time-domain based feature extraction methods are often limited by the information dimension when dealing with complex actions, and cannot fully capture the details of actions. Especially when facing long time series and dynamic changes, the prediction accuracy is low. Existing methods generally lack the full utilization of the frequency interval features in the action sequence, resulting in poor robustness of the prediction model in dynamic changes, information loss, and high-noise environments; in addition, traditional interaction systems rely on simple pattern recognition of the current motion, and may not be able to correctly identify action features under the changes of complex environments, and cannot prevent damaging motions in advance, and thus cannot effectively play the role of the device. Summary of the Invention

[0004] The purpose of the present invention is to provide a motion prediction method based on frequency-domain masking and a human motion risk feedback device, which can more comprehensively extract the detailed information of the action sequence, improve the prediction accuracy and model robustness, and at the same time adapt to the complex environment of long time series and dynamic changes, and provide an efficient, accurate, and versatile human motion risk feedback device with the functions of biological information collection, data processing, and interactive output.

[0005] To achieve the above object, the present invention provides a motion prediction method based on frequency-domain masking, including the following steps:

[0006] Step 1: The pre-time-frequency conversion component converts the motion time-series data into motion frequency-domain data by using Fourier transform;

[0007] Step 2: The pre-low-frequency masking component, the pre-medium-frequency masking component, and the pre-high-frequency masking component respectively multiply the motion frequency-domain data by the low-frequency masking vector, the medium-frequency masking vector, and the high-frequency masking vector to obtain motion low-frequency data, motion medium-frequency data, and motion high-frequency data;

[0008] Step 3: The pre-time-frequency conversion component uses the inverse Fourier transform to convert the motion low-frequency data, motion intermediate-frequency data, and motion high-frequency data into motion low-frequency time-series data, motion intermediate-frequency time-series data, and motion high-frequency time-series data;

[0009] Step 4: The feature extraction component extracts the data features of the motion low-frequency time-series data, motion intermediate-frequency time-series data, and motion high-frequency time-series data through a multi-layer perceptron network, and sequentially converts them into motion low-frequency time-series feature data, motion intermediate-frequency time-series feature data, and motion high-frequency time-series feature data;

[0010] Step 5: The post-time-frequency conversion component uses the Fourier transform to sequentially convert the motion low-frequency time-series feature data, motion intermediate-frequency time-series feature data, and motion high-frequency time-series feature data into first motion low-frequency feature data, first motion intermediate-frequency feature data, and first motion high-frequency feature data;

[0011] Step 6: The post low-frequency masking component multiplies the first motion low-frequency feature data by a low-frequency masking vector to obtain second motion low-frequency feature data, the post intermediate-frequency masking component multiplies the first motion intermediate-frequency feature data by an intermediate-frequency masking vector to obtain second motion intermediate-frequency feature data, and the post high-frequency masking component multiplies the first motion high-frequency feature data by a high-frequency masking vector to obtain second motion high-frequency feature data;

[0012] Step 7: The post frequency-domain synthesis component uses addition fusion of the second motion low-frequency feature data, second motion intermediate-frequency feature data, and second motion high-frequency feature data to obtain a second motion frequency-domain feature;

[0013] Step 8: The post-time-frequency conversion component uses the inverse Fourier transform to convert the second motion frequency-domain feature data into motion frequency-combined time-series feature data;

[0014] Step 9: The residual time-series feature extraction component converts the motion time-series data into motion residual time-series feature data through a multi-layer perceptron network;

[0015] Step 10: The prediction fusion component uses addition fusion to convert the motion frequency-combined time-series feature data and the motion residual time-series feature data into motion prediction data.

[0016] Preferably, the multi-layer perceptron network in Step 4 and Step 9 includes a cosine time-frequency conversion component, a pre-feature mapping component, a multi-layer perceptron feature extraction component, a post-feature mapping component, and a cosine frequency-time conversion component.

[0017] Preferably, the multi-layer perceptron network is a single-input, single-output network.

[0018] Preferably, the data transmission process of the multi-layer perceptron network includes the following steps:

[0019] S1. The pre - cosine time - frequency conversion component uses the discrete cosine transform to convert the input time - series data into input cosine frequency - domain data;

[0020] S2. The pre - feature mapping component uses a fully - connected neural network to convert the input cosine frequency - domain data into pre - extracted feature data;

[0021] S3. The multi - layer perceptron feature extraction component uses a multi - layer perceptron to extract features, converting the pre - extracted features into post - extracted feature data;

[0022] S4. The post - feature mapping component uses a fully - connected neural network to convert the post - extracted feature data into output frequency - domain data;

[0023] S5. The cosine frequency - time conversion component uses the inverse discrete cosine transform to convert the output frequency - domain data into output time - series data.

[0024] The human motion risk feedback device includes a biological information collection module, a motion data processing module, and a feedback interaction module. The feedback interaction module is connected to the motion data processing module, and the motion data processing module is connected to the biological information collection module. The biological information collection module collects motion time - series data, the motion data processing module processes the motion time - series data to obtain motion prediction data, and the feedback interaction module processes the motion prediction data.

[0025] Preferably, the motion data processing module includes a pre - time - frequency conversion component, a pre - masking component, a pre - frequency - time conversion component, a feature extraction component, a post - time - frequency conversion component, a post - masking component, a post - frequency - domain synthesis component, a post - frequency - time conversion component, a residual time - series feature extraction component, and a prediction fusion component;

[0026] The pre - masking component includes a pre - low - frequency masking component, a pre - medium - frequency masking component, and a pre - high - frequency masking component;

[0027] The post - masking component includes a post - low - frequency masking component, a post - medium - frequency masking component, and a post - high - frequency masking component.

[0028] Preferably, the feedback interaction component includes a data receiving component, a pattern matching component, and a feedback component;

[0029] The data receiving component receives the motion prediction data wirelessly; the pattern matching component uses a risk data conversion model to convert the motion prediction data into joint motion risk data; the feedback component converts the part of the joint risk data that is greater than the specified threshold into a vibration level and acts on the vibratable hardware unit.

[0030] Preferably, the risk data conversion model uses a multi - layer perceptron model and is trained based on a dataset of motion sequence data and motion risk data to obtain its mapping relationship as the conversion logic of the model.

[0031] Preferably, the motion data processing module runs on a computer device and includes a processor, a memory, a communication interface, and a bus.

[0032] Therefore, the present invention adopts the above-mentioned motion prediction method based on frequency domain masking and the human motion risk feedback device, and has the following beneficial effects:

[0033] (1) Through the frequency domain masking technology, the motion data can be more finely decomposed into multiple frequency channels, thereby improving the prediction accuracy and robustness of complex actions. Especially in an environment with dynamic changes, information loss, or high noise, the prediction accuracy can be significantly improved.

[0034] (2) Adopting a multi-layer perception model with a relatively small amount of computation ensures the real-time performance of the model while guaranteeing the accuracy.

[0035] (3) The feedback interaction module can real-time judge potential dangers during the motion according to the prediction results, and timely remind the user to adjust the actions through tactile feedback, reducing the risk of injury during the motion and improving the safety during the motion process.

[0036] (4) The interactive feedback device provided by the present invention has strong versatility. It can not only be applied in the field of motion monitoring, but also can be widely applied to multiple fields such as virtual reality and augmented reality after logical modification, with high market adaptability and broad application prospects.

[0037] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Description of the Drawings

[0038] Figure 1 is the flowchart of the motion prediction method based on frequency domain masking of the present invention;

[0039] Figure 2 is the flowchart of the feature extraction component of the present invention;

[0040] Figure 3 is the flowchart of the human motion risk feedback device of the present invention;

[0041] Figure 4 is the schematic diagram of the feedback convergence module of the present invention;

[0042] Figure 5 is the operation diagram of the motion data processing module of the present invention. Detailed Embodiments

[0043] The following further illustrates the technical solutions of the present invention through the drawings and embodiments.

[0044] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings as understood by those of ordinary skill in the art to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. Words such as "comprising" or "including" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. Words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Upper", "lower", "left", "right", etc. are only used to indicate relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0045] Embodiment

[0046] Please refer to Figures 1 - 5 , the present invention provides a human motion risk feedback device, which is composed of a biological information acquisition module, a motion data processing module and a feedback interaction module. The human motion risk feedback device acquires motion timing data through the biological information acquisition module, the motion data processing module processes the motion timing data to obtain motion prediction data, and the feedback interaction module processes the motion prediction data to feedback the motion risk on the human joints by vibration.

[0047] The biological information acquisition module is composed of an action capture device worn on the human body, which is used to acquire the motion timing data of the human body in real time and transmit the motion data to the motion data processing module wirelessly; the wireless method is selected from various different network transmission methods including wireless network, Bluetooth, etc.

[0048] The motion data processing module is composed of a pre-time-frequency conversion component, a pre-low-frequency masking component, a pre-medium-frequency masking component, a pre-high-frequency masking component, a pre-frequency-time conversion component, a feature extraction component, a post-time-frequency conversion component, a post-low-frequency masking component, a post-medium-frequency masking component, a post-high-frequency masking component, a post-frequency domain synthesis component, a post-frequency-time conversion component, a residual time-series feature extraction component and a prediction fusion component.

[0049] The motion data processing module runs on a computer device, including a processor, a memory, a communication interface and a bus.

[0050] The feature extraction module consists of a cosine time-frequency conversion component, a pre-feature mapping component, a multi-layer perceptron feature extraction module, a post-feature mapping component, and a cosine frequency-time conversion component. Define the multi-layer perceptron network as a single-input and single-output network. Define the input and output of the multi-layer perceptron network as input time-series data and output time-series data respectively, and different changes should be made to different data names during the call process; the pre-cosine time-frequency conversion component uses the discrete cosine transform to convert the input time-series data into input cosine frequency-domain data; the pre-feature mapping component uses a fully-connected neural network to convert the input cosine frequency-domain data into pre-extracted feature data; the multi-layer perceptron feature extraction component uses a multi-layer perceptron to extract features and convert the pre-extracted features into post-extracted feature data; the post-feature mapping component uses a fully-connected neural network to convert the post-extracted feature data into output frequency-domain data; the cosine frequency-time conversion component uses the inverse discrete cosine transform to convert the output frequency-domain data into output time-series data.

[0051] The feedback interaction module consists of a data receiving component, a pattern matching component, and a feedback component. The data receiving component receives motion prediction data from the motion data processing module wirelessly; the pattern matching component uses a risk data conversion model to convert the motion prediction data into action joint risk data; the feedback component converts the part of the joint risk data that is greater than the specified threshold into a vibration level and acts on the vibratable hardware unit; the risk data conversion model uses a multi-layer perceptron model and is trained based on a dataset of motion sequence data and motion risk data to obtain its mapping relationship as the conversion logic of the model.

[0052] A motion prediction method based on a frequency-domain mask includes the following steps:

[0053] Step 1: The pre-time-frequency conversion component uses the Fourier transform to convert the motion time-series data into motion frequency-domain data.

[0054] Step 2: The pre-low-frequency mask component multiplies the motion frequency-domain data by a low-frequency mask vector to obtain motion low-frequency data. The pre-middle-frequency mask component multiplies the motion frequency-domain data by a middle-frequency mask vector to obtain motion middle-frequency data. The pre-high-frequency mask component multiplies the motion frequency-domain data by a high-frequency mask vector to obtain motion high-frequency data.

[0055] Step 3: The pre-frequency-time conversion component uses the inverse Fourier transform to convert the motion low-frequency data, motion middle-frequency data, and motion high-frequency data into motion low-frequency time-series data, motion middle-frequency time-series data, and motion high-frequency time-series data.

[0056] Step 4: The feature extraction component extracts the data features of the motion low-frequency time-series data, motion middle-frequency time-series data, and motion high-frequency time-series data through a multi-layer perceptron network, and converts them into motion low-frequency time-series feature data, motion middle-frequency time-series feature data, and motion high-frequency time-series feature data in sequence.

[0057] Step Five: The post-positioned time-frequency conversion component uses Fourier transform to sequentially convert the motion low-frequency time-series feature data, motion medium-frequency time-series feature data, and motion high-frequency time-series feature data into first motion low-frequency feature data, first motion medium-frequency feature data, and first motion high-frequency feature data.

[0058] Step Six: The post-positioned low-frequency masking component multiplies the first motion low-frequency feature data by the low-frequency masking vector to obtain second motion low-frequency feature data. The post-positioned medium-frequency masking component multiplies the first motion medium-frequency feature data by the medium-frequency masking vector to obtain second motion medium-frequency feature data. The post-positioned high-frequency masking component multiplies the first motion high-frequency feature data by the high-frequency masking vector to obtain second motion high-frequency feature data.

[0059] Step Seven: The post-positioned frequency-domain synthesis component uses addition fusion to combine the second motion low-frequency feature data, second motion medium-frequency feature data, and second motion high-frequency feature data to obtain second motion frequency-domain features.

[0060] Step Eight: The post-positioned frequency-time conversion component uses inverse Fourier transform to convert the second motion frequency-domain feature data into motion frequency-combined time-series feature data.

[0061] Step Nine: The residual time-series feature extraction component uses a multi-layer perceptron network to convert the motion time-series data into motion residual time-series feature data.

[0062] Step Ten: The prediction fusion component uses addition fusion to combine the motion frequency-combined time-series feature data and the motion residual time-series feature data into motion prediction data.

[0063] For the present invention, replacing the Fourier transform with the wavelet transform can also achieve the purpose of the present invention.

[0064] Therefore, the present invention adopts the above-mentioned motion prediction method based on frequency-domain masking and the human motion risk feedback device. Through the frequency-domain masking technology, the motion data can be more finely decomposed into multiple frequency channels, thereby improving the prediction accuracy and robustness of complex actions. Especially in an environment with dynamic changes, information loss, or high noise, the prediction accuracy can be significantly improved. Using a multi-layer perceptron model with a relatively small amount of computation, while ensuring the accuracy, the real-time performance of the model is guaranteed. The feedback interaction module can real-time judge the potential dangers during the motion according to the prediction results, and timely remind the user to adjust the actions through tactile feedback, reducing the risk of injury during the motion and improving the safety during the motion process. The interactive feedback device provided by the present invention has strong versatility and can not only be applied in the field of motion monitoring, but also can be widely applied in multiple fields such as virtual reality and augmented reality after logical modification, having high market adaptability and broad application prospects.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions of the present invention or make equivalent replacements, and these modifications or equivalent replacements do not enable the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A motion prediction method based on frequency domain masking, characterized in that, It includes the following steps: Step 1: The pre-time-frequency conversion component uses Fourier transform to convert the motion time-series data into motion frequency-domain data; Step 2: The pre-low-frequency masking component, the pre-medium-frequency masking component, and the pre-high-frequency masking component respectively multiply the motion frequency-domain data by the low-frequency masking vector, the medium-frequency masking vector, and the high-frequency masking vector to obtain the motion low-frequency data, the motion medium-frequency data, and the motion high-frequency data; Step 3: The pre-frequency-time conversion component uses inverse Fourier transform to convert the motion low-frequency data, the motion medium-frequency data, and the motion high-frequency data into the motion low-frequency time-series data, the motion medium-frequency time-series data, and the motion high-frequency time-series data; Step 4: The feature extraction component extracts the data features of the motion low-frequency time-series data, the motion medium-frequency time-series data, and the motion high-frequency time-series data through a multi-layer perceptron network, and sequentially converts them into the motion low-frequency time-series feature data, the motion medium-frequency time-series feature data, and the motion high-frequency time-series feature data; Step 5: The post-time-frequency conversion component uses Fourier transform to sequentially convert the motion low-frequency time-series feature data, the motion medium-frequency time-series feature data, and the motion high-frequency time-series feature data into the first motion low-frequency feature data, the first motion medium-frequency feature data, and the first motion high-frequency feature data; Step 6: The post-low-frequency masking component multiplies the first motion low-frequency feature data by the low-frequency masking vector to obtain the second motion low-frequency feature data, the post-medium-frequency masking component multiplies the first motion medium-frequency feature data by the medium-frequency masking vector to obtain the second motion medium-frequency feature data, and the post-high-frequency masking component multiplies the first motion high-frequency feature data by the high-frequency masking vector to obtain the second motion high-frequency feature data; Step 7: The post-frequency-domain synthesis component uses addition fusion to combine the second motion low-frequency feature data, the second motion medium-frequency feature data, and the second motion high-frequency feature data to obtain the second motion frequency-domain feature; Step 8: The post-frequency-time conversion component uses inverse Fourier transform to convert the second motion frequency-domain feature data into the motion frequency-combined time-series feature data; Step 9: The residual time-series feature extraction component converts the motion time-series data into the motion residual time-series feature data through a multi-layer perceptron network; Step 10: The prediction fusion component uses addition fusion to convert the motion frequency-combined time-series feature data and the motion residual time-series feature data into the motion prediction data.

2. The motion prediction method based on frequency domain masking according to claim 1, wherein: In Step 4 and Step 9, the multi-layer perceptron network includes a cosine time-frequency conversion component, a pre-feature mapping component, a multi-layer perceptron feature extraction component, a post-feature mapping component, and a cosine frequency-time conversion component.

3. The motion prediction method based on frequency domain masking according to claim 2, wherein: The multi-layer perceptron network is a single-input and single-output network.

4. The motion prediction method based on frequency domain masking according to claim 3, wherein The data transmission process of the multi-layer perceptron network includes the following steps: S1: The pre-cosine time-frequency conversion component uses discrete cosine transform to convert the input time-series data into the input cosine frequency-domain data; S2: The pre-feature mapping component uses a fully connected neural network to convert the input cosine frequency-domain data into the feature pre-extraction data; S3: The multi-layer perceptron feature extraction component uses a multi-layer perceptron to extract features and converts the feature pre-extraction into the feature post-extraction data; S4: The post-feature mapping component uses a fully connected neural network to convert the feature post-extraction data into the output frequency-domain data; S5. The cosine frequency-time conversion component uses the inverse discrete cosine transform to convert the output frequency-domain data into output timing data.

5. Human motion risk feedback device, applying the motion prediction method based on frequency domain masking according to any one of the above claims 1-4, characterized in that: It includes a biological information acquisition module, a motion data processing module, and a feedback interaction module. The feedback interaction module is connected to the motion data processing module, and the motion data processing module is connected to the biological information acquisition module. The biological information acquisition module acquires motion timing data, the motion data processing module processes the motion timing data to obtain motion prediction data, and the feedback interaction module processes the motion prediction data.

6. The human body movement risk feedback device according to claim 5, wherein: The motion data processing module includes a pre-time-frequency conversion component, a pre-masking component, a pre-frequency-time conversion component, a feature extraction component, a post-time-frequency conversion component, a post-masking component, a post-frequency-domain synthesis component, a post-frequency-time conversion component, a residual timing feature extraction component, and a prediction fusion component; The pre-masking component includes a pre-low-frequency masking component, a pre-medium-frequency masking component, and a pre-high-frequency masking component; The post-masking component includes a post-low-frequency masking component, a post-medium-frequency masking component, and a post-high-frequency masking component.

7. The human motion risk feedback device according to claim 6, characterized in that: The feedback interaction component includes a data receiving component, a pattern matching component, and a feedback component; The data receiving component receives the motion prediction data wirelessly; the pattern matching component uses a risk data conversion model to convert the motion prediction data into action joint risk data; The feedback component converts the part of the joint risk data that is greater than the specified threshold into a vibration level and acts on the vibratable hardware unit.

8. The human body movement risk feedback device according to claim 7, characterized in that: The risk data conversion model uses a multi-layer perceptron model and is trained based on a data set of motion sequence data and motion risk data to obtain its mapping relationship as the conversion logic of the model.

9. The human body movement risk feedback device according to claim 8, wherein: The motion data processing module runs on a computer device and includes a processor, a memory, a communication interface, and a bus.

Images