Motion prediction method based on frequency domain mask and human body motion risk feedback device

Abstract

The invention discloses a motion prediction method based on a frequency domain mask and a human motion risk feedback device, and relates to the technical field of signal processing, multiple pieces of frequency domain channel data are obtained through a front frequency domain mask, feature extraction is performed in combination with a multi-layer perceptron after restoration, filtering restoration is performed in combination with a rear mask, and finally effective prediction is performed by fusing residual errors. Therefore, prediction precision and model robustness are improved, meanwhile, the human motion risk feedback device adapts to complex environments with long-time sequences and dynamic changes, and the human motion risk feedback device which is efficient, accurate and widely used and has the functions of biological information collection, data processing and interactive output is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a motion prediction method based on frequency domain mask and a human motion risk feedback device. Background Art

[0002] Human motion prediction is an important application in the field of computer vision and deep learning, and is widely used in security monitoring, smart home, virtual reality and augmented reality, autonomous driving and other fields. Traditional human motion prediction methods are mostly based on time series analysis techniques, such as convolutional neural networks, long short-term memory networks, and the recently popular models based on self-attention mechanisms. These methods usually rely on historical data sequences to predict future movements. The human motion risk feedback device is a motion monitoring device designed for people with physical defects, aiming to prevent such people from suffering joint or muscle injuries in their lives.

[0003] Traditional time-domain-based feature extraction methods are often limited by information dimensions when dealing with complex movements, and are unable to fully capture the details of the movement, especially when facing long time series and dynamic changes, and the prediction accuracy is low. Existing methods generally lack the ability to fully utilize the frequency interval characteristics in the action sequence, resulting in poor robustness of the prediction model in dynamic changes, information loss and high noise environments; in addition, traditional interactive systems rely on simple pattern recognition of the current movement, and may not be able to correctly identify the movement characteristics under complex environmental changes, and cannot prevent damaging movements in advance, and thus cannot effectively exert the effect of the device. Summary of the invention

[0004] The purpose of the present invention is to provide a motion prediction method and a human motion risk feedback device based on frequency domain mask, which can more comprehensively extract the detailed information of the action sequence, improve the prediction accuracy and model robustness, and adapt to long time series and dynamically changing complex environments, and provide an efficient, accurate and versatile human motion risk feedback device with the functions of biological information collection, data processing and interactive output.

[0005] To achieve the above object, the present invention provides a motion prediction method based on frequency domain mask, comprising the following steps: Step 1: The pre-time-frequency conversion component converts the motion time series data into motion frequency domain data using Fourier transform; Step 2: The front low-frequency mask component, the front intermediate-frequency mask component and the front high-frequency mask component respectively multiply the motion frequency domain data with the low-frequency mask vector, the intermediate-frequency mask vector and the high-frequency mask vector to obtain motion low-frequency data, motion intermediate-frequency data and motion high-frequency data; Step 3: The pre-frequency-time conversion component converts the motion low-frequency data, the motion medium-frequency data and the motion high-frequency data into motion low-frequency time series data, motion medium-frequency time series data and motion high-frequency time series data by using inverse Fourier transform; Step 4: The feature extraction component extracts data features of the motion low-frequency time series data, the motion medium-frequency time series data, and the motion high-frequency time series data through a multi-layer perception network, and converts them into motion low-frequency time series feature data, motion medium-frequency time series feature data, and motion high-frequency time series feature data in sequence; Step 5: The post-time-frequency conversion component uses Fourier transform to convert the motion low-frequency time series feature data, the motion medium-frequency time series feature data and the motion high-frequency time series feature data into first motion low-frequency feature data, first motion medium-frequency feature data and first motion high-frequency feature data in sequence; Step 6: The rear low-frequency mask component obtains second motion low-frequency feature data by multiplying the first motion low-frequency feature data with the low-frequency mask vector, the rear intermediate frequency mask component obtains second motion intermediate frequency feature data by multiplying the first motion intermediate frequency feature data with the intermediate frequency mask vector, and the rear high-frequency mask component obtains second motion high-frequency feature data by multiplying the first motion high-frequency feature data with the high-frequency mask vector; Step 7: The post-frequency domain synthesis component uses addition to fuse the second motion low-frequency feature data, the second motion medium-frequency feature data, and the second motion high-frequency feature data to obtain the second motion frequency domain feature; Step 8: The post-frequency-time conversion component converts the second motion frequency domain feature data into motion frequency-time series feature data by using inverse Fourier transform; Step 9: The residual time series feature extraction component converts the motion time series data into motion residual time series feature data through a multi-layer perception network; Step 10: The prediction fusion component uses additive fusion to convert the motion frequency temporal feature data and the motion residual temporal feature data into motion prediction data.

[0006] Preferably, the multi-layer perception network in step 4 and step 9 includes a cosine time-frequency conversion component, a pre-feature mapping component, a multi-layer perception feature extraction component, a post-feature mapping component and a cosine frequency-time conversion component.

[0007] Preferably, the multi-layer perception network is a single-input, single-output network.

[0008] Preferably, the multi-layer perception network data transmission process includes the following steps: S1, the pre-cosine time-frequency conversion component converts the input time series data into input cosine frequency domain data using discrete cosine transform; S2, the pre-feature mapping component uses a fully connected neural network to convert the input cosine frequency domain data into feature pre-extraction data; S3, the multi-layer perceptron feature extraction component uses a multi-layer perceptron to extract features and converts feature pre-extraction into post-feature extraction data; S4, the post-feature mapping component uses a fully connected neural network to convert the post-feature extraction data into output frequency domain data; S5. The cosine frequency-time conversion component converts the output frequency domain data into output time series data using an inverse discrete cosine transform.

[0009] The human motion risk feedback device includes a biological information acquisition module, a motion data processing module and a feedback interaction module. The feedback interaction module is connected to the motion data processing module, and the motion data processing module is connected to the biological information acquisition module. The biological information acquisition module acquires motion time series data, the motion data processing module processes the motion time series data to obtain motion prediction data, and the feedback interaction module processes the motion prediction data.

[0010] Preferably, the motion data processing module includes a pre-time-frequency conversion component, a pre-mask component, a pre-frequency-time conversion component, a feature extraction component, a post-time-frequency conversion component, a post-mask component, a post-frequency domain synthesis component, a post-frequency-time conversion component, a residual time series feature extraction component and a prediction fusion component; The front mask assembly includes a front low-frequency mask assembly, a front medium-frequency mask assembly and a front high-frequency mask assembly; The rear mask assembly includes a rear low-frequency mask assembly, a rear intermediate-frequency mask assembly and a rear high-frequency mask assembly.

[0011] Preferably, the feedback interaction component includes a data receiving component, a pattern matching component and a feedback component; The data receiving component receives the motion prediction data wirelessly; the pattern matching component converts the motion prediction data into action joint risk data using a risk data conversion model; the feedback component converts the part of the joint risk data that is greater than a specified threshold into a vibration degree, which acts on the vibrating hardware unit.

[0012] Preferably, the risk data conversion model utilizes a multi-layer perceptron model, based on a data set of motion sequence data and motion risk data, and trains to obtain a mapping relationship as the conversion logic of the model.

[0013] Preferably, the motion data processing module runs on a computer device, including a processor, a memory, a communication interface and a bus.

[0014] Therefore, the present invention adopts the above-mentioned motion prediction method based on frequency domain mask and human motion risk feedback device, which has the following beneficial effects: (1) Through frequency domain masking technology, motion data can be more finely decomposed into multiple frequency channels, thereby improving the prediction accuracy and robustness of complex actions, especially in environments with dynamic changes, information loss or high noise, which can significantly improve the accuracy of prediction.

[0015] (2) A multi-layer perception model with low computational complexity is adopted to ensure the real-time performance of the model while ensuring accuracy.

[0016] (3) The feedback interaction module can judge the potential dangers during exercise in real time based on the prediction results, and promptly remind the user to adjust the movement through tactile feedback, thereby reducing the risk of injury during exercise and improving safety during exercise.

[0017] (4) The interactive feedback device provided by the present invention has strong versatility and can be used not only in the field of motion monitoring, but also can be widely used in multiple fields such as virtual reality and augmented reality after logical modification. It has high market adaptability and broad application prospects.

[0018] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the motion prediction method based on frequency domain mask of the present invention; Figure 2 This is a flow chart of the feature extraction component of the present invention; Figure 3 This is a flow chart of the human body motion risk feedback device of the present invention; Figure 4 It is a schematic diagram of the feedback intersection module of the present invention; Figure 5 This is the operation diagram of the dynamic data processing module of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0021] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0022] Example See also Figure 1-Figure 5 The present invention provides a human motion risk feedback device, which is composed of a biological information acquisition module, a motion data processing module and a feedback interaction module. The human motion risk feedback device obtains motion time series data through the biological information acquisition module, the motion data processing module processes the motion time series data to obtain motion prediction data, and the feedback interaction module processes the motion prediction data to feedback motion risks on human joints through vibration.

[0023] The bio-information acquisition module is composed of a motion capture device worn on the human body, which is used to collect the human body's motion timing data in real time and transmit the motion data to the motion data processing module wirelessly; the wireless method is selected from a variety of different network transmission methods including wireless network, Bluetooth, etc.

[0024] The motion data processing module is composed of a front time-frequency conversion component, a front low-frequency mask component, a front intermediate frequency mask component, a front high-frequency mask component, a front frequency-time conversion component, a feature extraction component, a rear time-frequency conversion component, a rear low-frequency mask component, a rear intermediate frequency mask component, a rear high-frequency mask component, a rear frequency domain synthesis component, a rear frequency-time conversion component, a residual time series feature extraction component and a prediction fusion component.

[0025] The motion data processing module runs on a computer device, including a processor, a memory, a communication interface and a bus.

[0026] The feature extraction module consists of a cosine time-frequency conversion component, a pre-feature mapping component, a multi-layer perception feature extraction module, a post-feature mapping component, and a cosine frequency-time conversion component. The multi-layer perception network is defined as a single-input, single-output network, and the input and output of the multi-layer perception network are defined as input time series data and output time series data, respectively. Different data names should be changed differently during the call process; the pre-cosine time-frequency conversion component uses discrete cosine transform to convert input time series data into input cosine frequency domain data; the pre-feature mapping component uses a fully connected neural network to convert input cosine frequency domain data into feature pre-extraction data; the multi-layer perception feature extraction component uses a multi-layer perceptron to extract features and converts feature pre-extraction into feature post-extraction data; the post-feature mapping component uses a fully connected neural network to convert feature post-extraction data into output frequency domain data; the cosine frequency-time conversion component uses inverse discrete cosine transform to convert output frequency domain data into output time series data.

[0027] The feedback interaction module consists of a data receiving component, a pattern matching component and a feedback component. The data receiving component receives the motion prediction data from the motion data processing module wirelessly; the pattern matching component uses the risk data conversion model to convert the motion prediction data into action joint risk data; the feedback component converts the part of the joint risk data greater than the specified threshold into a vibration degree, which acts on the vibrating hardware unit; the risk data conversion model uses a multi-layer perceptron model, based on the data set of motion sequence data and motion risk data, to train the mapping relationship as the conversion logic of the model.

[0028] The motion prediction method based on frequency domain mask includes the following steps: Step 1: The pre-time-frequency conversion component uses Fourier transform to convert the motion time series data into motion frequency domain data.

[0029] Step 2: The front low-frequency mask component multiplies the motion frequency domain data with the low-frequency mask vector to obtain motion low-frequency data, the front intermediate frequency mask component multiplies the motion frequency domain data with the intermediate frequency mask vector to obtain motion intermediate frequency data, and the front high-frequency mask component multiplies the motion frequency domain data with the high-frequency mask vector to obtain motion high-frequency data.

[0030] Step 3: The pre-frequency-to-time conversion component uses inverse Fourier transform to convert the motion low-frequency data, motion medium-frequency data, and motion high-frequency data into motion low-frequency time series data, motion medium-frequency time series data, and motion high-frequency time series data.

[0031] Step 4: The feature extraction component extracts data features of motion low-frequency time series data, motion medium-frequency time series data, and motion high-frequency time series data through a multi-layer perception network, and converts them into motion low-frequency time series feature data, motion medium-frequency time series feature data, and motion high-frequency time series feature data in turn.

[0032] Step 5: The post-time-frequency conversion component uses Fourier transform to convert the motion low-frequency time series feature data, the motion medium-frequency time series feature data, and the motion high-frequency time series feature data into first motion low-frequency feature data, first motion medium-frequency feature data, and first motion high-frequency feature data in sequence.

[0033] Step six, the post-low-frequency mask component obtains the second motion low-frequency feature data by multiplying the first motion low-frequency feature data with the low-frequency mask vector, the post-intermediate frequency mask component obtains the second motion intermediate frequency feature data by multiplying the first motion intermediate frequency feature data with the intermediate frequency mask vector, and the post-high-frequency mask component obtains the second motion high-frequency feature data by multiplying the first motion high-frequency feature data with the high-frequency mask vector.

[0034] Step 7: The post-frequency domain synthesis component uses addition to fuse the second motion low-frequency feature data, the second motion medium-frequency feature data, and the second motion high-frequency feature data to obtain the second motion frequency domain feature.

[0035] Step 8: The post-frequency-time conversion component converts the second motion frequency domain feature data into motion frequency-combined time series feature data using inverse Fourier transform.

[0036] Step 9: The residual time series feature extraction component converts the motion time series data into motion residual time series feature data through a multi-layer perception network.

[0037] Step 10: The prediction fusion component uses additive fusion to convert the motion frequency time series feature data and the motion residual time series feature data into motion prediction data.

[0038] For the present invention, replacing Fourier transform with wavelet transform can also achieve the purpose of the present invention.

[0039] Therefore, the present invention adopts the above-mentioned motion prediction method based on frequency domain mask and human motion risk feedback device. Through the frequency domain mask technology, the motion data can be more finely decomposed into multiple frequency channels, thereby improving the prediction accuracy and robustness of complex movements, especially in dynamic changes, information loss or high noise environments, which can significantly improve the accuracy of prediction. A multi-layer perception model with a small amount of calculation is adopted to ensure the real-time performance of the model while ensuring accuracy. The feedback interaction module can judge the potential dangers in the movement in real time according to the prediction results, and promptly remind the user to adjust the movement through tactile feedback, reduce the risk of injury in the movement, and improve the safety during the movement. The interactive feedback device provided by the present invention has strong versatility, which can not only be used in the field of motion monitoring, but also can be widely used in multiple fields such as virtual reality and augmented reality after logical modification, and has high market adaptability and broad application prospects.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A motion prediction method based on frequency domain mask, characterized in that: The following steps are involved: Step 1: The pre-time-frequency conversion component converts the motion time series data into motion frequency domain data using Fourier transform; Step 2: The front low-frequency mask component, the front intermediate-frequency mask component and the front high-frequency mask component respectively multiply the motion frequency domain data with the low-frequency mask vector, the intermediate-frequency mask vector and the high-frequency mask vector to obtain motion low-frequency data, motion intermediate-frequency data and motion high-frequency data; Step 3: The pre-frequency-time conversion component converts the motion low-frequency data, the motion medium-frequency data and the motion high-frequency data into motion low-frequency time series data, motion medium-frequency time series data and motion high-frequency time series data by using inverse Fourier transform; Step 4: The feature extraction component extracts data features of the motion low-frequency time series data, the motion medium-frequency time series data, and the motion high-frequency time series data through a multi-layer perception network, and converts them into motion low-frequency time series feature data, motion medium-frequency time series feature data, and motion high-frequency time series feature data in sequence; Step 5: The post-time-frequency conversion component uses Fourier transform to convert the motion low-frequency time series feature data, the motion medium-frequency time series feature data and the motion high-frequency time series feature data into first motion low-frequency feature data, first motion medium-frequency feature data and first motion high-frequency feature data in sequence; Step 6: The rear low-frequency mask component obtains second motion low-frequency feature data by multiplying the first motion low-frequency feature data with the low-frequency mask vector, the rear intermediate frequency mask component obtains second motion intermediate frequency feature data by multiplying the first motion intermediate frequency feature data with the intermediate frequency mask vector, and the rear high-frequency mask component obtains second motion high-frequency feature data by multiplying the first motion high-frequency feature data with the high-frequency mask vector; Step 7: The post-frequency domain synthesis component uses addition to fuse the second motion low-frequency feature data, the second motion medium-frequency feature data, and the second motion high-frequency feature data to obtain the second motion frequency domain feature; Step 8: The post-frequency-time conversion component converts the second motion frequency domain feature data into motion frequency-time series feature data by using inverse Fourier transform; Step 9: The residual time series feature extraction component converts the motion time series data into motion residual time series feature data through a multi-layer perception network; Step 10: The prediction fusion component uses additive fusion to convert the motion frequency temporal feature data and the motion residual temporal feature data into motion prediction data.

2. The motion prediction method based on frequency domain mask according to claim 1, characterized in that: The multi-layer perception network in step 4 and step 9 includes a cosine time-frequency conversion component, a pre-feature mapping component, a multi-layer perception feature extraction component, a post-feature mapping component and a cosine frequency-time conversion component.

3. The motion prediction method based on frequency domain mask according to claim 2, characterized in that: The multi-layer perception network is a single-input, single-output network.

4. The motion prediction method based on frequency domain mask according to claim 3, characterized in that: The multi-layer perception network data transmission process includes the following steps: S1, the pre-cosine time-frequency conversion component converts the input time series data into input cosine frequency domain data using discrete cosine transform; S2, the pre-feature mapping component uses a fully connected neural network to convert the input cosine frequency domain data into feature pre-extraction data; S3, the multi-layer perceptron feature extraction component uses a multi-layer perceptron to extract features and converts feature pre-extraction into post-feature extraction data; S4, the post-feature mapping component uses a fully connected neural network to convert the post-feature extraction data into output frequency domain data; S5. The cosine frequency-time conversion component converts the output frequency domain data into output time series data using an inverse discrete cosine transform.

5. A human motion risk feedback device, using the motion prediction method based on frequency domain mask as described in any one of claims 1 to 4, characterized in that: It includes a biological information acquisition module, a motion data processing module and a feedback interaction module. The feedback interaction module is connected to the motion data processing module, and the motion data processing module is connected to the biological information acquisition module. The biological information acquisition module acquires motion timing data, the motion data processing module processes the motion timing data to obtain motion prediction data, and the feedback interaction module processes the motion prediction data.

6. The human body movement risk feedback device according to claim 5, characterized in that: The motion data processing module includes a pre-time-frequency conversion component, a pre-mask component, a pre-frequency-time conversion component, a feature extraction component, a post-time-frequency conversion component, a post-mask component, a post-frequency domain synthesis component, a post-frequency-time conversion component, a residual time series feature extraction component and a prediction fusion component; The front mask assembly includes a front low-frequency mask assembly, a front medium-frequency mask assembly and a front high-frequency mask assembly; The rear mask assembly includes a rear low-frequency mask assembly, a rear intermediate-frequency mask assembly and a rear high-frequency mask assembly.

7. The human body movement risk feedback device according to claim 6, characterized in that: The feedback interaction component includes a data receiving component, a pattern matching component and a feedback component; The data receiving component receives the motion prediction data wirelessly; the pattern matching component converts the motion prediction data into action joint risk data using the risk data conversion model; The feedback component converts the part of the joint risk data that is greater than a specified threshold into a vibration degree, which acts on the vibrating hardware unit.

8. The human body movement risk feedback device according to claim 7, characterized in that: The risk data conversion model uses a multi-layer perceptron model and is based on a dataset of motion sequence data and motion risk data. The mapping relationship is trained as the conversion logic of the model.

9. The human body movement risk feedback device according to claim 8, characterized in that: The motion data processing module runs on a computer device, including a processor, a memory, a communication interface and a bus.

Images