IMU Data Loss Completion Method and System

Abstract

This invention discloses an IMU data packet loss completion method, comprising: determining whether packet loss occurs between adjacent IMU data in an IMU data stream; if packet loss is determined to have occurred, defining the area between the adjacent IMU data with packet loss as a packet loss region; and completing the IMU data in the packet loss region based on the adjacent IMU data. This invention also discloses an IMU data packet loss completion system. The IMU data packet loss completion method and system of this invention, by detecting and completing the packet loss region, solve the problem of misaligned data caused by IMU data packet loss, providing effective data basis for subsequent action detection.

Description

Technical Field

[0001] This invention relates to information data processing technology, specifically to a method and system for compensating for lost IMU data packets. Background Technology

[0002] IMU stands for Inertial Measurement Unit, a sensor primarily used to detect and measure acceleration and rotational motion. Its principle is based on the law of inertia. These sensors range from ultra-small MEMS sensors to laser gyroscopes with extremely high measurement accuracy; whether it's a MEMS sensor only a few millimeters in size or a fiber optic device with a diameter of nearly half a meter, they all employ this principle.

[0003] Currently, IMU technology has begun to be widely used in human motion sensing. Monitoring human movement through IMU sensors configured on wristbands or other electronic devices is very common. However, due to human body obstruction or other electromagnetic interference, data packet loss is prone to occur during real-time data transmission of these wearable IMU sensors, reducing the accuracy of IMU data. Summary of the Invention

[0004] The technical problem to be solved by this invention is the lack of effective means to handle packet loss in IMU data transmission in the prior art. The purpose is to provide a method and system for IMU data packet loss recovery to solve the above-mentioned problem.

[0005] This invention is achieved through the following technical solution:

[0006] On the one hand, this embodiment provides an IMU data packet loss recovery method, including:

[0007] Determine whether there is packet loss between adjacent IMU data in the IMU data stream;

[0008] If packet loss is determined to have occurred, the area between the adjacent IMU data points where packets are lost is defined as the packet loss region.

[0009] IMU data is supplemented in the packet loss area based on the adjacent IMU data.

[0010] In existing technologies, the current human motion information acquisition is generally based on a single IMU sensor. However, when the inventors used multiple IMU sensors for information acquisition and synchronously acquired video information with IMU information, they discovered that, unlike the signal transmission of a single IMU sensor, accurately determining human motion at a certain moment requires aligning different IMU information or video signals on the time axis. If IMU information is lost during the alignment process, it will cause a time difference in alignment, reducing the accuracy of motion detection. Therefore, the inventors adopted an IMU data packet loss compensation method to complete the lost data and ensure that the IMU data can be continuously aligned.

[0011] In this embodiment, it is determined whether there is packet loss between adjacent IMU data in the IMU data stream. This determination can be made using existing packet loss detection technologies based on different protocols, or by identifying each IMU data to pinpoint the location of the packet loss. If packet loss occurs, this embodiment locates the lost packet region and completes the data in the lost region based on adjacent IMU data. This completion can be achieved using techniques such as interpolation. It should be understood that regardless of the completion method used, it must be within the scope of this embodiment. This embodiment, through the detection and completion of lost packet regions, solves the problem of misaligned data caused by IMU data packet loss, providing effective data for subsequent action detection.

[0012] Furthermore, determining whether packet loss occurs between adjacent IMU data streams includes:

[0013] Obtain the identification information of two adjacent IMU data in the IMU data stream; the identification information is the position of the IMU data relative to the first IMU data in the IMU data stream.

[0014] When the identification information of two adjacent IMU data is not continuous, it is determined that the two IMU data have been lost.

[0015] Furthermore, the IMU data completion for the packet loss area based on the adjacent IMU data includes:

[0016] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0017] IMU data is generated by interpolating the packet loss area based on the number of lost packets, the first IMU data, and the second IMU data; the first IMU data is the previous IMU data of the packet loss area; and the second IMU data is the next IMU data of the packet loss area.

[0018] Furthermore, the following formula is used to complete the IMU data:

[0019]

[0020] In the formula, C i Let i be the i-th IMU data that needs to be supplemented in the packet loss region, n be the number of packet losses, A be the first IMU data, B be the second IMU data, and i ∈ [1, n].

[0021] Furthermore, the IMU data completion for the packet loss area based on the adjacent IMU data includes:

[0022] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0023] The number of packet losses is used as the data interval between the first IMU data and the second IMU data; the first IMU data is the previous IMU data in the packet loss area; the second IMU data is the next IMU data in the packet loss area.

[0024] Based on the first IMU data, the IMU data stream is traversed and retrieved, and IMU data whose difference from the first IMU data meets a preset value is selected as the first candidate data;

[0025] The data that is spaced from the first candidate data by the number of packet losses is used as the second candidate data;

[0026] The second IMU data is compared with the second candidate data, and the second candidate data with the smallest comparison difference is used as the reference data;

[0027] The number of IMU data points preceding the reference data is added to the packet loss area.

[0028] On the other hand, this embodiment provides an IMU data packet loss completion system, including:

[0029] The judgment unit is configured to determine whether there is packet loss between adjacent IMU data in the IMU data stream;

[0030] The determining unit is configured to define the area between adjacent IMU data that have lost packets as a packet loss region if it is determined that packet loss has occurred.

[0031] The completion unit is configured to complete the IMU data in the packet loss area based on the adjacent IMU data.

[0032] Furthermore, the decision unit is also configured as follows:

[0033] Obtain the identification information of two adjacent IMU data in the IMU data stream; the identification information is the position of the IMU data relative to the first IMU data in the IMU data stream.

[0034] When the identification information of two adjacent IMU data is not continuous, it is determined that the two IMU data have been lost.

[0035] Furthermore, the completion unit is also configured as follows:

[0036] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0037] IMU data is generated by interpolating the packet loss area based on the number of lost packets, the first IMU data, and the second IMU data; the first IMU data is the previous IMU data of the packet loss area; and the second IMU data is the next IMU data of the packet loss area.

[0038] Furthermore, the completion unit performs IMU data completion using the following formula:

[0039]

[0040] In the formula, C i Let i be the i-th IMU data that needs to be supplemented in the packet loss region, n be the number of packet losses, A be the first IMU data, B be the second IMU data, and i ∈ [1, n].

[0041] Furthermore, the completion unit is also configured as follows:

[0042] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0043] The number of packet losses is used as the data interval between the first IMU data and the second IMU data; the first IMU data is the previous IMU data in the packet loss area; the second IMU data is the next IMU data in the packet loss area.

[0044] Based on the first IMU data, the IMU data stream is traversed and retrieved, and IMU data whose difference from the first IMU data meets a preset value is selected as the first candidate data;

[0045] The data that is spaced from the first candidate data by the number of packet losses is used as the second candidate data;

[0046] The second IMU data is compared with the second candidate data, and the second candidate data with the smallest comparison difference is used as the reference data;

[0047] The number of IMU data points preceding the reference data is added to the packet loss area.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] The present invention provides an IMU data packet loss completion method and system that solves the problem of misaligned data caused by IMU data packet loss by detecting and completing the packet loss area, thus providing effective data basis for subsequent action detection. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0051] Figure 1 This is a schematic diagram of the inventive steps of an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the system architecture of an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0054] Example

[0055] Please refer to the following: Figure 1 This is a flowchart illustrating the IMU data packet loss completion method provided in this embodiment of the invention. The IMU data packet loss completion method can be applied to... Figure 2 The IMU data packet loss compensation system in the text further includes, in detail, the IMU data packet loss compensation method, which may include the content described in steps S1-S3.

[0056] S1: Determine whether there is packet loss between adjacent IMU data in the IMU data stream;

[0057] S2: If packet loss is determined to have occurred, the area between the adjacent IMU data points where packets are lost is defined as the packet loss region;

[0058] S3: Complete the IMU data for the packet loss area based on the adjacent IMU data.

[0059] In existing technologies, the current human motion information acquisition is generally based on a single IMU sensor. However, when the inventors used multiple IMU sensors for information acquisition and synchronously acquired video information with IMU information, they discovered that, unlike the signal transmission of a single IMU sensor, accurately determining human motion at a certain moment requires aligning different IMU information or video signals on the time axis. If IMU information is lost during the alignment process, it will cause a time difference in alignment, reducing the accuracy of motion detection. Therefore, the inventors adopted an IMU data packet loss compensation method to complete the lost data and ensure that the IMU data can be continuously aligned.

[0060] In this embodiment, it is determined whether there is packet loss between adjacent IMU data in the IMU data stream. This determination can be made using existing packet loss detection technologies based on different protocols, or by identifying each IMU data to pinpoint the location of the packet loss. If packet loss occurs, this embodiment locates the lost packet region and completes the data in the lost region based on adjacent IMU data. This completion can be achieved using techniques such as interpolation. It should be understood that regardless of the completion method used, it must be within the scope of this embodiment. This embodiment, through the detection and completion of lost packet regions, solves the problem of misaligned data caused by IMU data packet loss, providing effective data for subsequent action detection.

[0061] In a specific embodiment, determining whether packet loss occurs between adjacent IMU data in the IMU data stream includes:

[0062] Obtain the identification information of two adjacent IMU data in the IMU data stream; the identification information is the position of the IMU data relative to the first IMU data in the IMU data stream.

[0063] When the identification information of two adjacent IMU data is not continuous, it is determined that the two IMU data have been lost.

[0064] In this embodiment, as a specific implementation scheme, packet loss is determined by using the identification information of two adjacent IMU data sets. For example, as one method of obtaining the identification information in this embodiment, it includes:

[0065] When the IMU sensor is powered on, the identification information is initialized to a preset value;

[0066] When the IMU sensor generates data, it assigns the data with the current identifier information incremented by a fixed value as the first identifier information.

[0067] For example, each data entry has a `count` field, which is initialized to 0 upon power-on. Afterward, for each new data entry, `count` is incremented and assigned to that new data entry. When storing data, the `count` of each new data entry is subtracted from the `count` of the first new data entry to obtain the `count` relative to the first new data entry. In the event of packet loss during IMU sensor data transmission, data alignment is performed using `count`.

[0068] In this embodiment, if no packet loss occurs, the identification information of two adjacent IMU data will be continuous. However, if packet loss occurs, the identification information of two adjacent IMU data will be discontinuous and interrupted. For example, if the identification information of five consecutive IMU data is 10, 11, 12, 14, 15, then the IMU data corresponding to the identification information 12 and 14 is missing the IMU data with identification information 13, indicating that packet loss has occurred. At this time, the area between the IMU data corresponding to the identification information 12 and 14 is the packet loss area.

[0069] In one specific embodiment, IMU data completion for the packet loss area based on the adjacent IMU data includes:

[0070] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0071] IMU data is generated by interpolating the packet loss area based on the number of lost packets, the first IMU data, and the second IMU data; the first IMU data is the previous IMU data of the packet loss area; and the second IMU data is the next IMU data of the packet loss area.

[0072] In implementing this embodiment, the inventors discovered that when collecting IMU data of human motion, since muscle exertion is a continuous process, the IMU data of human motion is also a continuous process. In this embodiment, the IMU data is acquired by the IMU sensor and can be an unprocessed acceleration signal, an integrated velocity signal, a displacement signal with double integration, or a processed coordinate signal. Regardless of the type of signal, it should be continuous in data acquisition by IMU sensors with sampling rates generally exceeding 100Hz. Therefore, in this embodiment, the packet loss region is interpolated using the first and second IMU data to generate IMU completion data in the packet loss region. The interpolation process can be linear interpolation or interpolation based on certain fixed functions. Since the number of packet losses between two adjacent IMU data points may be greater than one, the number of packet losses is also used as a parameter for interpolation in this embodiment.

[0073] In one specific embodiment, IMU data completion is performed using the following formula:

[0074]

[0075] In the formula, C i Let i be the i-th IMU data that needs to be supplemented in the packet loss region, n be the number of packet losses, A be the first IMU data, B be the second IMU data, and i ∈ [1, n].

[0076] In this embodiment, a purely linear fitting method is adopted, that is, using the first IMU data and the second IMU data as endpoints, the packet loss area is linearly fitted and interpolated. The interpolation value depends on the position of each lost packet in the fitted line segment. This method is easy to calculate and the accuracy is acceptable when the number of lost packets is small.

[0077] In one specific embodiment, IMU data completion for the packet loss area based on the adjacent IMU data includes:

[0078] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0079] The number of packet losses is used as the data interval between the first IMU data and the second IMU data; the first IMU data is the previous IMU data in the packet loss area; the second IMU data is the next IMU data in the packet loss area.

[0080] Based on the first IMU data, the IMU data stream is traversed and retrieved, and IMU data whose difference from the first IMU data meets a preset value is selected as the first candidate data;

[0081] The data that is spaced from the first candidate data by the number of packet losses is used as the second candidate data;

[0082] The second IMU data is compared with the second candidate data, and the second candidate data with the smallest comparison difference is used as the reference data;

[0083] The number of IMU data points preceding the reference data is added to the packet loss area.

[0084] In this embodiment, unlike the linear interpolation scheme in the previous embodiments, a historical data filling method is adopted. The inventors discovered that if a segment of IMU data is similar, it indicates that the user performed essentially the same action on that part of the body. It should be understood that this essentially the same action may only be a displacement in a certain direction, rather than the overall action being identical. Based on the above discovery, the inventors used historical data to fill in the packet loss area. A preliminary search was performed using the first IMU data to retrieve IMU data that met the preset difference requirements as the first candidate data. Then, the data at the corresponding intervals of the first candidate data were used as the second candidate data for secondary filtering. This can filter out data in the previous IMU data that matches the current packet loss area, and use these data to fill in the gaps. Although this requires certain computing resources, it can effectively improve the accuracy of the data completion.

[0085] Please see Figure 2 Based on the same inventive concept, an IMU data loss completion system is also provided, the system comprising:

[0086] The judgment unit is configured to determine whether there is packet loss between adjacent IMU data in the IMU data stream;

[0087] The determining unit is configured to define the area between adjacent IMU data that have lost packets as a packet loss region if it is determined that packet loss has occurred.

[0088] The completion unit is configured to complete the IMU data in the packet loss area based on the adjacent IMU data.

[0089] In one specific embodiment, the determination unit is further configured to:

[0090] Obtain the identification information of two adjacent IMU data in the IMU data stream; the identification information is the position of the IMU data relative to the first IMU data in the IMU data stream.

[0091] When the identification information of two adjacent IMU data is not continuous, it is determined that the two IMU data have been lost.

[0092] In one specific embodiment, the completion unit is further configured as follows:

[0093] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0094] IMU data is generated by interpolating the packet loss area based on the number of lost packets, the first IMU data, and the second IMU data; the first IMU data is the previous IMU data of the packet loss area; and the second IMU data is the next IMU data of the packet loss area.

[0095] In one specific embodiment, the completion unit performs IMU data completion using the following formula:

[0096]

[0097] In the formula, C i Let i be the i-th IMU data that needs to be supplemented in the packet loss region, n be the number of packet losses, A be the first IMU data, B be the second IMU data, and i ∈ [1, n].

[0098] In one specific embodiment, the completion unit is further configured as follows:

[0099] The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data.

[0100] The number of packet losses is used as the data interval between the first IMU data and the second IMU data; the first IMU data is the previous IMU data in the packet loss area; the second IMU data is the next IMU data in the packet loss area.

[0101] Based on the first IMU data, the IMU data stream is traversed and retrieved, and IMU data whose difference from the first IMU data meets a preset value is selected as the first candidate data;

[0102] The data that is spaced from the first candidate data by the number of packet losses is used as the second candidate data;

[0103] The second IMU data is compared with the second candidate data, and the second candidate data with the smallest comparison difference is used as the reference data;

[0104] The number of IMU data points preceding the reference data is added to the packet loss area.

[0105] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0107] The units described as separate components may or may not be physically separate. As will be apparent to those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0108] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An IMU data packet loss recovery method, characterized in that, include: Determine whether there is packet loss between adjacent IMU data in the IMU data stream; If packet loss is determined to have occurred, the area between the adjacent IMU data points where packets are lost is defined as the packet loss region. IMU data is supplemented in the packet loss area based on the adjacent IMU data; Determining whether packet loss occurs between adjacent IMU data streams includes: Obtain the identification information of two adjacent IMU data in the IMU data stream; the identification information is the position of the IMU data relative to the first IMU data in the IMU data stream. When the identification information of two adjacent IMU data is not continuous, it is determined that the two IMU data have been lost; IMU data completion for the packet loss area based on the adjacent IMU data includes: The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data. The number of packet losses is used as the data interval between the first IMU data and the second IMU data; the first IMU data is the previous IMU data in the packet loss area; the second IMU data is the next IMU data in the packet loss area. Based on the first IMU data, the IMU data stream is traversed and retrieved, and IMU data whose difference from the first IMU data meets a preset value is selected as the first candidate data; The data that is spaced from the first candidate data by the number of packet losses is used as the second candidate data; The second IMU data is compared with the second candidate data, and the second candidate data with the smallest comparison difference is used as the reference data; The number of IMU data points preceding the reference data is added to the packet loss area.

2. The IMU data packet loss compensation method according to claim 1, characterized in that, IMU data completion for the packet loss area based on the adjacent IMU data includes: The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data. IMU data is generated by interpolating the packet loss area based on the number of lost packets, the first IMU data, and the second IMU data; the first IMU data is the previous IMU data of the packet loss area; and the second IMU data is the next IMU data of the packet loss area.

3. The IMU data packet loss compensation method according to claim 2, characterized in that, IMU data completion is performed using the following formula: In the formula, Ci is the i-th IMU data that needs to be completed in the packet loss area, n is the number of lost packets, A is the first IMU data, B is the second IMU data, and i∈[1,n].

4. An IMU data packet loss recovery system, characterized in that, include: The judgment unit is configured to determine whether there is packet loss between adjacent IMU data in the IMU data stream; The determining unit is configured to define the area between adjacent IMU data points that have lost packets as a packet loss region if it is determined that packet loss has occurred. The completion unit is configured to complete the IMU data in the packet loss area based on the adjacent IMU data. The decision unit is also configured as follows: Obtain the identification information of two adjacent IMU data in the IMU data stream; the identification information is the position of the IMU data relative to the first IMU data in the IMU data stream; When the identification information of two adjacent IMU data is not continuous, it is determined that the two IMU data have been lost; The completion unit is also configured to: The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data. The number of packet losses is used as the data interval between the first IMU data and the second IMU data; the first IMU data is the previous IMU data in the packet loss area; the second IMU data is the next IMU data in the packet loss area. Based on the first IMU data, the IMU data stream is traversed and retrieved, and IMU data whose difference from the first IMU data meets a preset value is selected as the first candidate data; The data that is spaced from the first candidate data by the number of packet losses is used as the second candidate data; The second IMU data is compared with the second candidate data, and the second candidate data with the smallest comparison difference is used as the reference data; The number of IMU data points preceding the reference data is added to the packet loss area.

5. The IMU data packet loss compensation system according to claim 4, characterized in that, The completion unit is also configured to: The number of lost packets in the packet loss area is determined based on the difference in the identification information of two adjacent IMU data. IMU data is generated by interpolating the packet loss area based on the number of lost packets, the first IMU data, and the second IMU data; the first IMU data is the previous IMU data of the packet loss area; and the second IMU data is the next IMU data of the packet loss area.

6. The IMU data packet loss compensation system according to claim 5, characterized in that, The completion unit performs IMU data completion using the following formula: In the formula, Ci is the i-th IMU data that needs to be completed in the packet loss area, n is the number of lost packets, A is the first IMU data, B is the second IMU data, and i∈[1,n].

Images