A human motion information recognition system and method

Abstract

The present invention relates to a human motion information recognition system and method. First, a motion information recognition neural network is obtained through a method. The method steps mainly include: synthesizing a neural network, selecting n common human motions, and dividing the n common human motions into motion y1, motion y2,..., motion y n , forming a set of digital images corresponding to motion y i . Data preprocessing is performed on the elements in the set of digital images corresponding to motion y i to obtain an input set, and the neural network is trained and tested to finally obtain the required neural network. The system is based on the content in the method and is established: Through the present invention, the motion form can be recognized relatively accurately.

Description

Technical Field

[0001] The present invention relates to the field of sports biomechanics, and more particularly, to a human motion information recognition system and method. Background Art

[0002] With the rapid development of sensor technology and pervasive computing technology, sensor-based human activity recognition technology has become increasingly popular, and this technology is widely used in industries such as animal activity information collection, medical health, sports management, and physical education.

[0003] In the collection of human motion information, information from different types of behaviors is usually collected from a set of dedicated wearable motion sensors, such as accelerometers, gyroscopes, and magnetometers. Since the acceleration and angular velocity data of human motions such as the human body and animal body change according to human motions, these data can be used to infer human activities. The miniaturization and flexibility of wearable motion sensors allow humans to wear or carry mobile devices embedded with various sensing units, which is different from fixed sensors. Moreover, this type of sensor has characteristics such as low cost, low power consumption, high capacity, miniaturization, and low environmental dependence.

[0004] However, when using wearable motion sensors for human activity recognition, in the prior art, inertial sensors are often only used for information collection, and information such as speed and acceleration during human activities is obtained through inertial sensors, and then human activities are recognized based on this information. However, such sensors cannot obtain the bending characteristics of some joints or links, which has a certain impact on the accuracy of activity recognition. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a human motion information recognition system and method that can obtain motion information and recognize motion forms.

[0006] A method for obtaining a human motion information recognition neural network provided by the present invention has the following technical solutions:

[0007] A method for obtaining a human motion information recognition neural network includes the following steps:

[0008] Step 1: Set an input module, a learnable noise reduction module for denoising input data, a convolutional module for obtaining a feature image, a fully connected layer module for recognizing the feature image, and an output module in sequence according to the running direction, and synthesize a neural network accordingly;

[0009] Step 2: Select n common human motions, and divide the n common human motions into motion y1, motion y2,..., motion y n ;

[0010] Step 3: Through a sensor composed of an inertial sensor and a bending sensor, obtain the stretching amount, acceleration, and angular acceleration information of the target athlete during the performance of the n common human movements for movement y i When performing movement y, and use the vector synthesized by the stretching amount, acceleration, and angular acceleration data each time when performing movement y i as an element to form a digital image, and further obtain the digital image set corresponding to movement y with the digital image as an element, where 1 ≤ i ≤ n; successively obtain the digital image set corresponding to movement y1, the digital image set corresponding to movement y2,..., the digital image set corresponding to movement y i in accordance with the above method in this step; n

[0011] Step 4: Perform data preprocessing on each element in the digital image set corresponding to movement y i to form the input set corresponding to movement y i ; successively obtain the input set corresponding to movement y1, the input set corresponding to movement y2,..., the input set corresponding to movement y n in accordance with the above method in this step;

[0012] Step 5: Use each element in the input set corresponding to movement y i as input data, and movement y i as the ideal output to train the neural network. After the training is passed, a test neural network is obtained;

[0013] Step 6: Construct a test set matching the test neural network, and test the test neural network through the test set. After passing the test, a motion information recognition neural network is obtained.

[0014] Adopting the above technical solution, compared with the prior art, the technical solution provided by the present invention application can at least bring the following beneficial effects: This method is based on a sensor synthesized by an inertial sensor and a bending sensor for data collection, obtaining the acceleration (in three directions), angular acceleration (in three directions), and stretching amount characteristics of the target object. In this way, while obtaining the overall motion characteristics, the bending characteristics of some joints or links are also obtained, thereby improving the accuracy of human motion recognition. At the same time, when constructing the neural network model, a learnable noise reduction module is added on the basis of the existing convolutional network for processing images, so that the noise in the data can be removed when processing the data.

[0015] ​Preferably, the learnable noise reduction module set in the first step includes a transformation unit, a filtering unit, and an inverse transformation unit that are sequentially arranged according to the running order. The transformation unit is used to perform discrete Fourier transform processing on digital signals, the filtering unit uses a filtering algorithm to filter the digital signals after the discrete Fourier transform processing, and the inverse transformation unit performs inverse discrete Fourier transform processing on the digital signals after the filtering processing. Such a learnable filter helps to adaptively remove noise in the data while reducing noise.

[0016] Preferably, the filtering algorithm includes a high-pass filter and a low-pass filter arranged according to the running order. This filter combination has characteristics such as high precision and convergence model for denoising processing, which helps to improve the filtering and denoising efficiency.

[0017] Preferably, the sensors used in the third step include a textile wrist sensor and a textile knee sensor. In this way, the acceleration, angular acceleration, and displacement corresponding to the knee and wrist joints of the target object can be obtained simultaneously, thereby improving the accurate acquisition of the motion characteristic data of the target object.

[0018] Preferably, in the fourth step, each element in the digital image set corresponding to the motion y i After data preprocessing, the input set corresponding to the motion y i The process of forming includes the following steps:

[0019] Step 4.1, clear the abnormal data of each element in the digital image set corresponding to the motion y i To form the denoising processing set corresponding to the motion y i ;

[0020] Step 4.2, perform denoising processing on each element in the denoising processing set corresponding to the motion y i Through a Kalman filter to form the digital segmentation set corresponding to the motion y i ;

[0021] Step 4.3, adopt a sliding window method to segment each element in the digital segmentation set corresponding to the motion y i To form the digital normalization set corresponding to the motion y i ;

[0022] Step 4.4, perform normalization processing on each element in the digital normalization set corresponding to the motion y i Finally, form the input set corresponding to the motion y i ;

[0023] After the above processing, the data obtained by the sensor can be successfully transmitted to the input module of the neural network, thereby improving the training and testing efficiency of the neural network.

[0024] A human motion information recognition system provided by the present invention has the following technical solutions:

[0025] A human motion information recognition system includes a digital signal acquisition module, a digital signal preprocessing module, a motion information recognition neural network established based on the human motion information recognition neural network acquisition method of the present technical solution, and a terminal display module, which are sequentially arranged along the digital signal propagation direction; the digital signal acquisition module is used to acquire the stretching amount, acceleration, and angular acceleration information of the target object, and form a digital image with the stretching amount, acceleration, and angular acceleration information as elements; the digital signal preprocessing module is used to preprocess the digital image and form an input digital image; the motion information recognition neural network is used to recognize the input digital image and obtain the motion form represented by the input digital image; the terminal display module is used to display the motion form on the terminal device.

[0026] Adopting the above technical solution, compared with the prior art, the technical solution provided by the present invention application can at least bring the following beneficial effects: by setting a digital signal acquisition module, a digital signal preprocessing module, a motion information recognition neural network, and a terminal display module, the digital images corresponding to the stretching amount, acceleration, and angular acceleration information of the target object during motion are processed in an orderly manner, and the digital image is processed and recognized by the neural network, so as to judge the specific motion form of the target object. Since the neural network adopted is the neural network trained and tested by the method in the present technical solution, and this neural network has the ability to reduce noise and can accurately judge the motion form.

[0027] Preferably, the digital signal acquisition module includes a textile wristband sensor and a textile knee sensor provided with a signal output unit, and a summary unit that is information-conducted with the signal output unit. The signal transmission unit is used to send the stretching amount, acceleration, and angular acceleration information obtained by the textile wristband sensor and the textile knee sensor to the summary unit. The summary unit forms the digital image through signal summarization and transmits it to the digital signal preprocessing module; in this way, by wearing a textile knee sensor on the knee of the target object and a textile wristband sensor provided with a signal output unit on the wrist, it is beneficial to collect the motion acceleration, angular acceleration, and stretching amount of the target object with almost no external physical burden, and transmit it to the digital signal preprocessing module through the summary unit, thereby facilitating the accurate, efficient, and orderly collection of motion characteristics and improving the efficiency for subsequent processing.

[0028] Preferably, the digital signal preprocessing module includes an abnormal data elimination unit, a noise reduction unit, a segmentation unit, and a normalization unit, which are sequentially arranged along the propagation direction of the digital image. The abnormal data elimination unit is used to eliminate abnormal data in the digital image and form a noise-reduced processed digital image. The noise reduction unit is used to perform noise reduction processing on the noise-reduced processed digital image and form a segmentation-processed digital image. The segmentation unit is used to perform segmentation processing on the segmentation-processed digital image and form a normalized processed digital image. The normalization unit is used to perform normalization processing on the corresponding data of the normalized processed digital image and form an input digital image. This is beneficial for preprocessing digital signals and improving the recognition efficiency of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flowchart of a method for obtaining a human motion information recognition neural network in the present invention;

[0030] Figure 2 It is a schematic structural diagram of a human motion information recognition system in the present invention;

[0031] Figure 3 It is a schematic structural diagram of a motion information recognition neural network in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. It should be noted that if "1≤i≤n" is mentioned at the end of a step in this technical solution, the operations listed in this step must be performed on these n items from 1 to n. This is the spirit of the Einstein protocol.

[0033] A method for obtaining a human motion information recognition neural network provided by an embodiment of the present invention includes the following steps:

[0034] Step 1: An input module, a learnable noise reduction module for performing noise reduction processing on input data, a convolution module for obtaining a feature image, a fully connected layer module for recognizing the feature image, and an output module are sequentially arranged according to the running direction, and a neural network is synthesized accordingly;

[0035] Step 2: Select n common human motions, and divide the n common human motions into motion y1, motion y2,..., motion y n ;

[0036] Step 3: Through a sensor composed of an inertial sensor and a bending sensor, obtain the stretching amount, acceleration, and angular acceleration information of the target athlete when performing these n common human motions for motion y i and, for each time of performing motion y iA digital image is composed of vectors synthesized from the stretching amount, acceleration, and angular acceleration data at a certain time, and further a digital image is obtained as the motion y of the elements. i The corresponding set of digital images, where 1 ≤ i ≤ n;

[0037] Step Four: For each element in the set of digital images corresponding to the motion y i After performing data preprocessing on each element, an input set corresponding to the motion y i is formed, where 1 ≤ i ≤ n;

[0038] Step Five: Using each element in the input set corresponding to the motion y i as input data and the motion y i as the ideal output, the neural network is trained. After successful training, a test neural network is obtained, where 1 ≤ i ≤ n;

[0039] Step Six: Construct a test set that matches the test neural network and use the test set to test the test neural network. After passing this test, a motion information recognition neural network is obtained.

[0040] The learnable noise reduction module set in Step One of this embodiment includes a transformation unit for performing discrete Fourier transform processing on digital signals, a filtering unit for filtering the digital signals after the discrete Fourier transform processing using a filtering algorithm, and an inverse transformation unit for performing inverse discrete Fourier transform processing on the filtered digital signals, which are arranged in sequence according to the running order. The filtering algorithm in this embodiment includes a high-pass filter and a low-pass filter arranged in sequence according to the running order.

[0041] The sensors used in Step Three of this embodiment include a textile wrist sensor and a textile knee sensor, which are respectively worn on the wrist and knee of the target object.

[0042] Step Four of this embodiment includes the following steps:

[0043] Step 4.1: Remove the abnormal data of each element in the set of digital images corresponding to the motion y i to form a denoising processing set corresponding to the motion y i where 1 ≤ i ≤ n;

[0044] Step 4.2: Perform noise reduction processing on each element in the denoising processing set corresponding to the motion y i using a Kalman filter to form a digital segmentation set corresponding to the motion y i where 1 ≤ i ≤ n;

[0045] Step 4.3: Adopt a sliding window method for the motion y iPerform segmentation processing on each element in the corresponding digital segmentation set to form motion y i The corresponding digital normalization set, where 1 ≤ i ≤ n;

[0046] Step 4.4. Normalize each element in the digital normalization set corresponding to motion y i to finally form the input set corresponding to motion y i where 1 ≤ i ≤ n.

[0047] A human motion information recognition system provided in this embodiment includes a digital signal acquisition module, a digital signal preprocessing module, a motion information recognition neural network established by using a method for obtaining a human motion information recognition neural network according to this technical solution, and a terminal display module, which are sequentially arranged along the digital signal propagation direction; wherein, the digital signal acquisition module is used to acquire the stretching amount, acceleration, and angular acceleration information of the target object, and form a digital image according to the stretching amount, acceleration, and angular acceleration information; the digital signal preprocessing module is used to preprocess the digital image and form an input digital image; the motion information recognition neural network is used to recognize the input digital image to obtain the motion form represented by the input digital image; the terminal display module is used to display the motion form on the terminal device.

[0048] In this embodiment, the digital signal acquisition module includes a textile wrist guard sensor and a textile knee guard sensor provided with a signal output unit, and a summary unit that is information-conducted with the signal output unit. Among them, the signal transmission unit is used to send the stretching amount, acceleration, and angular acceleration information obtained by the textile wrist guard sensor and the textile knee guard sensor to the summary unit, and the summary unit forms a digital image through signal summarization and transmits it to the digital signal preprocessing module.

[0049] In this embodiment, the digital signal preprocessing module includes an abnormal data elimination unit that is sequentially arranged along the digital image propagation direction and is used to eliminate abnormal data in the digital image and form a noise reduction processed digital image, a noise reduction unit that is used to perform noise reduction processing on the noise reduction processed digital image and form a segmentation processed digital image, a segmentation unit that is used to perform segmentation processing on the segmentation processed digital image and form a normalization processed digital image, and a normalization unit that is used to perform normalization processing on the data corresponding to the normalization processed digital image and form an input digital image.

[0050] Specifically, the selected exercise forms in this embodiment are eight forms: walking, running, jumping, going upstairs, going downstairs, standing, lying, and sitting, which can be represented by the eight numbers 1-8 respectively. The wrist guard and knee guard sensors each include a six-axis inertial measurement unit (IMU) and a stretching and angle strain sensor. The signal transmission unit can be a Bluetooth wireless communication module, and the aggregation unit can be a Bluetooth receiver, which is usually set at the upper computer and can receive real-time data through Bluetooth transmission.

[0051] The four units of the digital signal preprocessing module are essentially algorithmic program units that correspond one-to-one to the four sub-steps included in step four of this embodiment. The discrete Fourier transform and inverse transform can be carried out in the following manner;

[0052] Given a sequence {x n}, n ∈ [0, N - 1], there is a formula:

[0053]

[0054] For the given X after transformation processing k , the original sequence {x n} is restored through the inverse transform, and the formula is:

[0055]

[0056] In summary, although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to this embodiment without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for obtaining a neural network for identifying human motion information, characterized in that: It includes the following steps: Step 1: Sequentially set an input module, a learnable noise reduction module for performing noise reduction processing on input data, a convolutional module for obtaining a feature image, a fully connected layer module for recognizing the feature image, and an output module in the running direction, and synthesize a neural network accordingly; Step 2: Select n common human motions and divide the n common human motions into motion y1, motion y2, ..., motion y n ; Step 3: Obtain, by means of a sensor composed of an inertial sensor and a bending sensor, the stretching amount, acceleration, and angular acceleration information of the target athlete during the performance of the n common human motions, where the motion is y i When performing the motion y, form a digital image with vectors synthesized from the data of the stretching amount, acceleration, and angular acceleration each time the motion y is performed, and further obtain a digital image set corresponding to the motion y with the digital images as elements, where 1 ≤ i ≤ n; successively obtain the digital image set corresponding to the motion y1, the digital image set corresponding to the motion y2,..., the digital image set corresponding to the motion y i in accordance with the above method in this step; i The digital image set corresponding to the motion y is obtained; n ​ Step 4: Perform data preprocessing on each element in the digital image set corresponding to the motion y i to form the input set corresponding to the motion y i ; successively obtain the input sets corresponding to the motion y1, the input sets corresponding to the motion y2, …, the input sets corresponding to the motion y n according to the above method in this step; Step Five: Using each element in the input set corresponding to the motion y i as input data, and using the motion y i as the ideal output, train the neural network. After the training is passed, a test neural network is obtained; Step 6: Construct a test set that matches the test neural network, and test the test neural network through the test set. After passing the test, a motion information recognition neural network is obtained.

2. The method for obtaining a neural network for identifying human motion information according to claim 1, characterized in that: The learnable noise reduction module set in Step 1 includes a transformation unit for performing discrete Fourier transform processing on digital signals, a filtering unit for filtering the digital signals after the discrete Fourier transform processing using a filtering algorithm, and an inverse transformation unit for performing inverse discrete Fourier transform processing on the digital signals after the filtering processing, which are sequentially set in the running order.

3. The method for obtaining a human motion information recognition neural network according to claim 2, characterized in that: The filtering algorithm includes a high-pass filter and a low-pass filter set in the running order.

4. The method for obtaining a neural network for identifying human motion information according to claim 3, characterized in that: The sensors used in Step 3 include a textile wristband sensor and a textile knee pad sensor.

5. The method for obtaining a neural network for identifying human motion information according to claim 4, wherein: In the fourth step, the motion y i After preprocessing the data of each element in the corresponding digital image set, the motion y i The process of forming the corresponding input set includes the following steps: Step 4.1, clear the abnormal data of each element in the digital image set corresponding to the movement y i to form a denoising processing set corresponding to the movement y i ; Step 4.2, perform noise reduction processing on each element in the noise reduction processing set corresponding to the motion y i to form the digital segmentation set corresponding to the motion y i after the noise reduction processing; Step 4.3: Using a sliding window method, segment each element in the digital segmentation set corresponding to the motion y i to form the digital normalization set corresponding to the motion y i ; Step 4.4, normalize each element in the digital normalization concentration corresponding to the motion y i to finally form the input set corresponding to the motion y i corresponding thereto.

6. A human motion information recognition system, characterized in that: It includes a digital signal acquisition module, a digital signal preprocessing module, a motion information recognition neural network established based on the human motion information recognition neural network acquisition method described in any one of claims 1-5, and a terminal display module, which are sequentially set along the digital signal propagation direction; the digital signal acquisition module is used to collect the stretching amount, acceleration, and angular acceleration information of the target object, and form a digital image with the stretching amount, acceleration, and angular acceleration information as elements; the digital signal preprocessing module is used to preprocess the digital image and form an input digital image; the motion information recognition neural network is used to recognize the input digital image and obtain the motion form represented by the input digital image; The terminal display module is used to display the motion form on the terminal device.

7. The human motion information recognition system according to claim 6, characterized in that: The digital signal acquisition module includes a textile wristband sensor and a textile knee pad sensor provided with a signal output unit, and a summary unit that is information-conducted with the signal output unit. The signal output unit is used to send the stretching amount, acceleration, and angular acceleration information obtained by the textile wristband sensor and the textile knee pad sensor to the summary unit. The summary unit forms the digital image through signal summarization and transmits it to the digital signal preprocessing module.

8. The human motion information recognition system according to claim 7, characterized in that: The digital signal preprocessing module includes an abnormal data elimination unit for eliminating abnormal data in the digital image and forming a noise reduction processed digital image, a noise reduction unit for performing noise reduction processing on the noise reduction processed digital image and forming a segmentation processed digital image, a segmentation unit for performing segmentation processing on the segmentation processed digital image and forming a normalized processed digital image, and a normalization unit for performing normalization processing on the corresponding data of the normalized processed digital image and forming an input digital image, which are sequentially set along the digital image propagation direction.

Images