An imu data sample collection, labeling and training method and system

Abstract

This invention discloses an IMU data sample acquisition, annotation, and training method, comprising: acquiring IMU data generated when a user performs a preset fitness movement as first IMU data; annotating the first IMU data and generating IMU sample data corresponding to different speeds; correcting the IMU sample data according to the fitness video corresponding to the preset fitness movement to form corrected IMU samples; training a recognition model using the corrected IMU data samples; and using the recognition model to recognize the preset fitness movement. This invention also discloses an IMU data sample acquisition, annotation, and training system. This invention's IMU data sample acquisition, annotation, and training method and system, by acquiring IMU data of fitness movements at different speeds during the sampling process, enable the trained model to accurately recognize fitness movements at different speeds, effectively improving the accuracy of fitness movement recognition.

Description

Technical Field

[0001] This invention relates to the field of intelligent fitness technology, specifically to an IMU data sample acquisition, annotation, and training method and system. Background Technology

[0002] Pose and motion recognition using IMU data has begun to be widely adopted. Currently, motion state recognition largely relies on trained models to identify IMU data, and the related technology is becoming increasingly mature. However, when recognizing and scoring fitness movements, due to differences in the physical conditions of different users, the speed of movement when performing the same fitness movement often varies. This makes it difficult for existing recognition models to accurately judge the standard of the movement when the user's movement is performed correctly but at a speed different from the training sample speed, and sometimes the recognition model cannot recognize the movement at all. Summary of the Invention

[0003] The technical problem to be solved by this invention is that the existing recognition models have low accuracy in recognizing actions performed by users at different speeds. The purpose is to provide an IMU data sample collection, annotation, and training method and system to solve the above problem.

[0004] This invention is achieved through the following technical solution:

[0005] In one aspect, this embodiment provides an IMU data sample acquisition, annotation, and training method, including:

[0006] The IMU data generated when the user performs a preset fitness movement is collected through a preset data acquisition process and used as the first IMU data; the first IMU data includes the IMU data collected when the user performs the preset fitness movement at different rates;

[0007] The first IMU data is labeled, and IMU sample data corresponding to different rates are generated;

[0008] The IMU sample data is corrected based on the fitness video corresponding to the preset fitness movements to form a corrected IMU sample.

[0009] The modified IMU data samples are used to train a recognition model; the recognition model is used to recognize the preset fitness movements.

[0010] In existing technologies, recognition models are mainly trained using clustering algorithms and deep learning neural networks, and the technical system is already very mature. However, the inventors discovered in practice that for fitness exercises, different people complete some exercises at different speeds. Users often adjust their exercise speed according to their physical condition. For example, when performing jumping jacks, a standard jumping jack in a video typically takes about 1000ms. However, if a user has good physical fitness, they can reduce the time to 800ms, while if a user has poor physical fitness, the time may extend to 1500ms to 2000ms. If the recognition model is trained based on samples taken from the approximately 1000ms speed in the video, the user's completion speed will become the standard for recognition and scoring, which is clearly unreasonable.

[0011] In this embodiment, during the initial IMU data acquisition, IMU data of fitness movements performed at different speeds are also collected. After annotation and correction, the resulting IMU data is used for model training. The resulting model can effectively recognize movements at different speeds. In this embodiment, the initial IMU data can be annotated manually or automatically. Correcting the IMU samples based on the fitness video involves determining whether a particular IMU sample corresponds to a specific movement within the video. By collecting IMU data of fitness movements at different speeds during the sampling process, this embodiment enables the trained model to accurately recognize fitness movements at different speeds, effectively improving the accuracy of fitness movement recognition.

[0012] Furthermore, the preset data acquisition process includes:

[0013] Play the fitness video of the preset fitness movements;

[0014] The IMU data collected from the user during the preparatory movement of the preset fitness exercise for a first preset duration is used as the second IMU data, and the IMU sensor is initialized using the second IMU data.

[0015] After completing the ekf initialization, IMU data of the user completing the preset fitness movement at the first preset rate is collected as the third IMU data, and IMU data of the user completing the preset fitness movement at the second preset rate is collected as the fourth IMU data.

[0016] The first IMU data is formed by repeatedly collecting multiple sets of third IMU data and multiple sets of fourth IMU data.

[0017] Furthermore, the pre-defined data acquisition process also includes:

[0018] Each time a set of third or fourth IMU data is acquired, the IMU sensor is rotated in a preset plane and the next set of third or fourth IMU data is acquired.

[0019] Furthermore, training the recognition model using the modified IMU data samples includes:

[0020] Extract IMU data corresponding to different rates from the modified IMU samples, and train multiple recognition models corresponding to different rates using the IMU data corresponding to different rates;

[0021] Obtain the data length of IMU data corresponding to different rates, and establish a mapping relationship between the data length and the corresponding recognition model.

[0022] Furthermore, when performing the preset fitness movement recognition, the detected IMU data is acquired as the fifth IMU data;

[0023] Obtain the effective length of the fifth IMU data, where the effective length is the length of consecutively valid data in the fifth IMU data;

[0024] The reference length is determined by using at least two data lengths that are closest to the effective length.

[0025] The reference length is fitted to form a correction gradient, and the fifth IMU data is corrected according to the correction gradient to IMU data corresponding to the reference length as corrected MIU data;

[0026] The corrected MIU data is input into the recognition model corresponding to the reference length for recognition.

[0027] In another aspect, this embodiment provides an IMU data sample acquisition, annotation, and training system, comprising:

[0028] The acquisition unit is configured to collect IMU data generated when the user performs a preset fitness movement through a preset data acquisition process, and use it as first IMU data; the first IMU data includes IMU data collected when the user performs the preset fitness movement at different rates;

[0029] The annotation unit is configured to annotate the first IMU data and generate IMU sample data corresponding to different rates;

[0030] The correction unit is configured to correct the IMU sample data according to the fitness video corresponding to the preset fitness movement to form a corrected IMU sample.

[0031] The training unit is configured to train a recognition model using the modified IMU data samples; the recognition model is used to recognize the preset fitness movements.

[0032] Furthermore, the preset data acquisition process includes:

[0033] Play the fitness video of the preset fitness movements;

[0034] The IMU data collected from the user during the preparatory movement of the preset fitness exercise for a first preset duration is used as the second IMU data, and the IMU sensor is initialized using the second IMU data.

[0035] After completing the ekf initialization, IMU data of the user completing the preset fitness movement at the first preset rate is collected as the third IMU data, and IMU data of the user completing the preset fitness movement at the second preset rate is collected as the fourth IMU data.

[0036] The first IMU data is formed by repeatedly collecting multiple sets of third IMU data and multiple sets of fourth IMU data.

[0037] Furthermore, the pre-defined data acquisition process also includes:

[0038] Each time a set of third or fourth IMU data is acquired, the IMU sensor is rotated in a preset plane and the next set of third or fourth IMU data is acquired.

[0039] Furthermore, the training model is configured as follows:

[0040] Extract IMU data corresponding to different rates from the modified IMU samples, and train multiple recognition models corresponding to different rates using the IMU data corresponding to different rates;

[0041] Obtain the data length of IMU data corresponding to different rates, and establish a mapping relationship between the data length and the corresponding recognition model.

[0042] Furthermore, it also includes a recognition unit, configured to acquire the detected IMU data as the fifth IMU data when the preset fitness movement recognition is performed;

[0043] Obtain the effective length of the fifth IMU data, where the effective length is the length of consecutively valid data in the fifth IMU data;

[0044] The reference length is determined by using at least two data lengths that are closest to the effective length.

[0045] The reference length is fitted to form a correction gradient, and the fifth IMU data is corrected according to the correction gradient to IMU data corresponding to the reference length as corrected MIU data;

[0046] The corrected MIU data is input into the recognition model corresponding to the reference length for recognition.

[0047] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0048] This invention discloses an IMU data sample acquisition, annotation, and training method and system. By acquiring IMU data of fitness movements at different speeds during the sampling process, the trained model can accurately identify fitness movements at different speeds, effectively improving the accuracy of fitness movement recognition. Attached Figure Description

[0049] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0050] Figure 1 This is a schematic diagram of the method steps in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the system architecture of an embodiment of the present invention. Detailed Implementation

[0052] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with examples and drawings. The exemplary embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0053] Example

[0054] Based on the above, please refer to the following: Figure 1 This is a flowchart illustrating an IMU data sample acquisition, annotation, and training method provided in an embodiment of the present invention. This IMU data sample acquisition, annotation, and training method can be applied to... Figure 2 The IMU data sample acquisition, annotation, and training system further includes, in detail, the IMU data sample acquisition, annotation, and training method, which may include the contents described in steps S1-S4.

[0055] S1: Collect IMU data generated when the user performs a preset fitness movement through a preset data acquisition process as the first IMU data; the first IMU data includes IMU data collected when the user performs the preset fitness movement at different rates;

[0056] S2: Label the first IMU data and generate IMU sample data corresponding to different rates;

[0057] S3: Correct the IMU sample data according to the fitness video corresponding to the preset fitness movement to form a corrected IMU sample;

[0058] S4: Use the modified IMU data samples to train the recognition model; the recognition model is used to recognize the preset fitness movements.

[0059] In existing technologies, recognition models are mainly trained using clustering algorithms and deep learning neural networks, and the technical system is already very mature. However, the inventors discovered in practice that for fitness exercises, different people complete some exercises at different speeds. Users often adjust their exercise speed according to their physical condition. For example, when performing jumping jacks, a standard jumping jack in a video typically takes about 1000ms. However, if a user has good physical fitness, they can reduce the time to 800ms, while if a user has poor physical fitness, the time may extend to 1500ms to 2000ms. If the recognition model is trained based on samples taken from the approximately 1000ms speed in the video, the user's completion speed will become the standard for recognition and scoring, which is clearly unreasonable.

[0060] In this embodiment, during the initial IMU data acquisition, IMU data of fitness movements performed at different speeds are also collected. After annotation and correction, the resulting IMU data is used for model training. The resulting model can effectively recognize movements at different speeds. In this embodiment, the initial IMU data can be annotated manually or automatically. Correcting the IMU samples based on the fitness video involves determining whether a particular IMU sample corresponds to a specific movement within the video. By collecting IMU data of fitness movements at different speeds during the sampling process, this embodiment enables the trained model to accurately recognize fitness movements at different speeds, effectively improving the accuracy of fitness movement recognition.

[0061] In one embodiment, the preset data acquisition process includes:

[0062] Play the fitness video of the preset fitness movements;

[0063] The IMU data collected from the user during the preparatory movement of the preset fitness exercise for a first preset duration is used as the second IMU data, and the IMU sensor is initialized using the second IMU data.

[0064] After completing the ekf initialization, IMU data of the user completing the preset fitness movement at the first preset rate is collected as the third IMU data, and IMU data of the user completing the preset fitness movement at the second preset rate is collected as the fourth IMU data.

[0065] The first IMU data is formed by repeatedly collecting multiple sets of third IMU data and multiple sets of fourth IMU data.

[0066] In one embodiment, the preset data acquisition process further includes:

[0067] Each time a set of third or fourth IMU data is acquired, the IMU sensor is rotated in a preset plane and the next set of third or fourth IMU data is acquired.

[0068] This embodiment provides a specific scheme for acquiring IMU data under different motion states. In this embodiment, eKF initialization is performed using second IMU data before movement to ensure accuracy, and two preset data acquisition rates are set. After acquiring each set of third or fourth IMU data, to ensure data richness, the IMU sensor is rotated a certain angle within a preset plane before acquisition. For example, for a wrist-worn IMU sensor, it can be rotated a certain range around the forearm axis to acquire data from the same area, thereby reducing recognition errors caused by the way the user wears the IMU sensor.

[0069] In one embodiment, training a recognition model using the modified IMU data samples includes:

[0070] Extract IMU data corresponding to different rates from the modified IMU samples, and train multiple recognition models corresponding to different rates using the IMU data corresponding to different rates;

[0071] Obtain the data length of IMU data corresponding to different rates, and establish a mapping relationship between the data length and the corresponding recognition model.

[0072] In one embodiment, when the preset fitness movement recognition is performed, the detected IMU data is acquired as the fifth IMU data;

[0073] Obtain the effective length of the fifth IMU data, where the effective length is the length of consecutively valid data in the fifth IMU data;

[0074] The reference length is determined by using at least two data lengths that are closest to the effective length.

[0075] The reference length is fitted to form a correction gradient, and the fifth IMU data is corrected according to the correction gradient to IMU data corresponding to the reference length as corrected MIU data;

[0076] The corrected MIU data is input into the recognition model corresponding to the reference length for recognition.

[0077] In implementing this embodiment, the inventors discovered in practice that different users' movement rates can vary greatly. Training too many recognition models for different movement rates would increase the computational burden and make it difficult to encompass all possible movement rates of all users. Therefore, in this embodiment, recognition is performed by correcting the detected IMU data through linear fitting within a limited recognition model.

[0078] Specifically, during the recognition process, it is necessary to obtain the effective length of the fifth IMU data. The effective length refers to the length of consecutive valid data, which corresponds to the duration of a user completing a certain action or a set of actions. In this embodiment, the length can be either the number of consecutive data points or the duration of consecutive data points. After selecting at least two baseline lengths, linear fitting can be performed on these lengths, and the gradient of the fitted data can be obtained.

[0079] After obtaining the fitted gradient, the fifth IMU data can be corrected. The correction method mainly involves correcting the effective length of the fifth IMU data to a certain baseline length, and then using the recognition model corresponding to the baseline length for recognition. This can effectively improve the recognition accuracy.

[0080] Please see Figure 2 Based on the same inventive concept, an IMU data sample acquisition, annotation, and training system is also provided, the system comprising:

[0081] The acquisition unit is configured to collect IMU data generated when the user performs a preset fitness movement through a preset data acquisition process, and use it as first IMU data; the first IMU data includes IMU data collected when the user performs the preset fitness movement at different rates;

[0082] The annotation unit is configured to annotate the first IMU data and generate IMU sample data corresponding to different rates;

[0083] The correction unit is configured to correct the IMU sample data according to the fitness video corresponding to the preset fitness movement to form a corrected IMU sample.

[0084] The training unit is configured to train a recognition model using the modified IMU data samples; the recognition model is used to recognize the preset fitness movements.

[0085] In one embodiment, the preset data acquisition process includes:

[0086] Play the fitness video of the preset fitness movements;

[0087] The IMU data collected from the user during the preparatory movement of the preset fitness exercise for a first preset duration is used as the second IMU data, and the IMU sensor is initialized using the second IMU data.

[0088] After completing the ekf initialization, IMU data of the user completing the preset fitness movement at the first preset rate is collected as the third IMU data, and IMU data of the user completing the preset fitness movement at the second preset rate is collected as the fourth IMU data.

[0089] The first IMU data is formed by repeatedly collecting multiple sets of third IMU data and multiple sets of fourth IMU data.

[0090] In one embodiment, the preset data acquisition process further includes:

[0091] Each time a set of third or fourth IMU data is acquired, the IMU sensor is rotated in a preset plane and the next set of third or fourth IMU data is acquired.

[0092] In one embodiment, the training model is configured as follows:

[0093] Extract IMU data corresponding to different rates from the modified IMU samples, and train multiple recognition models corresponding to different rates using the IMU data corresponding to different rates;

[0094] Obtain the data length of IMU data corresponding to different rates, and establish a mapping relationship between the data length and the corresponding recognition model.

[0095] In one embodiment, the system further includes a recognition unit configured to acquire detected IMU data as fifth IMU data when the preset fitness movement recognition is performed.

[0096] Obtain the effective length of the fifth IMU data, where the effective length is the length of consecutively valid data in the fifth IMU data;

[0097] The reference length is determined by using at least two data lengths that are closest to the effective length.

[0098] The reference length is fitted to form a correction gradient, and the fifth IMU data is corrected according to the correction gradient to IMU data corresponding to the reference length as corrected MIU data;

[0099] The corrected MIU data is input into the recognition model corresponding to the reference length for recognition.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0102] The units described as separate components may or may not be physically separate. As will be apparent to those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0103] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for IMU data sample acquisition, annotation, and training, characterized in that, include: The IMU data generated when the user performs preset fitness movements is collected through a preset data collection process and used as the first IMU data; The first IMU data includes IMU data collected when the user performs the preset fitness movements at different rates; The first IMU data is labeled, and IMU sample data corresponding to different rates are generated; The IMU sample data is corrected based on the fitness video corresponding to the preset fitness movements to form a corrected IMU sample. The modified IMU samples are used to train the recognition model; The recognition model is used to recognize the preset fitness movements; Training the recognition model using the modified IMU samples includes: Extract IMU data corresponding to different rates from the modified IMU samples, and train multiple recognition models corresponding to different rates using the IMU data corresponding to different rates; Obtain the data length of IMU data corresponding to different rates, and establish a mapping relationship between the data length and the corresponding recognition model.

2. The IMU data sample acquisition, annotation, and training method according to claim 1, characterized in that, The preset data acquisition process includes: Play the fitness video of the preset fitness movements; The IMU data collected from the user during the preparatory movement of the preset fitness exercise for a first preset duration is used as the second IMU data, and the IMU sensor is initialized using the second IMU data. After completing the ekf initialization, IMU data of the user completing the preset fitness movement at the first preset rate is collected as the third IMU data, and IMU data of the user completing the preset fitness movement at the second preset rate is collected as the fourth IMU data. The first IMU data is formed by repeatedly collecting multiple sets of third IMU data and multiple sets of fourth IMU data.

3. The IMU data sample acquisition, annotation, and training method according to claim 2, characterized in that, The preset data acquisition process also includes: Each time a set of third or fourth IMU data is acquired, the IMU sensor is rotated in a preset plane and the next set of third or fourth IMU data is acquired.

4. The IMU data sample acquisition, annotation, and training method according to claim 1, characterized in that, When the preset fitness movement recognition is performed, the detected IMU data is acquired as the fifth IMU data; Obtain the effective length of the fifth IMU data, where the effective length is the length of consecutively valid data in the fifth IMU data; The reference length is determined by using at least two data lengths that are closest to the effective length. The reference length is fitted to form a correction gradient, and the fifth IMU data is corrected according to the correction gradient to IMU data corresponding to the reference length as corrected MIU data; The corrected MIU data is input into the recognition model corresponding to the reference length for recognition.

5. An IMU data sample acquisition, annotation, and training system, characterized in that, include: The acquisition unit is configured to collect IMU data generated when the user performs a preset fitness movement through a preset data acquisition process, and use it as the first IMU data; The first IMU data includes IMU data collected when the user performs the preset fitness movements at different rates; The annotation unit is configured to annotate the first IMU data and generate IMU sample data corresponding to different rates; The correction unit is configured to correct the IMU sample data according to the fitness video corresponding to the preset fitness movement to form a corrected IMU sample. The training unit is configured to train a recognition model using the modified IMU samples; the recognition model is used to recognize the preset fitness movements. The training unit is also configured to: Extract IMU data corresponding to different rates from the modified IMU samples, and train multiple recognition models corresponding to different rates using the IMU data corresponding to different rates; Obtain the data length of IMU data corresponding to different rates, and establish a mapping relationship between the data length and the corresponding recognition model.

6. The IMU data sample acquisition, annotation, and training system according to claim 5, characterized in that, The preset data acquisition process includes: Play the fitness video of the preset fitness movements; The IMU data collected from the user during the preparatory movement of the preset fitness exercise for a first preset duration is used as the second IMU data, and the IMU sensor is initialized using the second IMU data. After completing the ekf initialization, IMU data of the user completing the preset fitness movement at the first preset rate is collected as the third IMU data, and IMU data of the user completing the preset fitness movement at the second preset rate is collected as the fourth IMU data. The first IMU data is formed by repeatedly collecting multiple sets of third IMU data and multiple sets of fourth IMU data.

7. The IMU data sample acquisition, annotation, and training system according to claim 6, characterized in that, The preset data acquisition process also includes: Each time a set of third or fourth IMU data is acquired, the IMU sensor is rotated in a preset plane and the next set of third or fourth IMU data is acquired.

8. The IMU data sample acquisition, annotation, and training system according to claim 5, characterized in that, It also includes a recognition unit, configured to acquire the detected IMU data as the fifth IMU data when the preset fitness movement recognition is performed; Obtain the effective length of the fifth IMU data, where the effective length is the length of consecutively valid data in the fifth IMU data; The reference length is determined by using at least two data lengths that are closest to the effective length. The reference length is fitted to form a correction gradient, and the fifth IMU data is corrected according to the correction gradient to IMU data corresponding to the reference length as corrected MIU data; The corrected MIU data is input into the recognition model corresponding to the reference length for recognition.

Images