Motion capture method and system based on inertial sensors

Abstract

This application relates to the field of motion capture technology, and specifically relates to a motion capture method and system based on inertial sensors. The method includes: collecting the roll angle, pitch angle, and yaw angle of each inertial sensor at each acquisition moment in each cycle, as well as each acceleration sequence of each inertial sensor in each cycle; obtaining the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; obtaining the average mutation index of each inertial sensor at each fluctuation acquisition moment in each cycle; obtaining the behavior intensity index of each inertial sensor at each fluctuation acquisition moment in each cycle; obtaining the acceleration response coefficient and attitude angle response coefficient of each inertial sensor in each cycle; obtaining the improved value of the noise covariance matrix of each inertial sensor in each cycle to correct the error of the inertial sensor. This application improves the accuracy and stability of motion capture.

Description

Technical Field

[0001] This application relates to the field of motion capture technology, and specifically to a motion capture method and system based on inertial sensors. Background Art

[0002] As an important means of obtaining digital attitude information, motion capture technology is widely used in fields such as rehabilitation medicine, virtual reality, intelligent robots, and sports training. Among them, the motion capture technology based on inertial sensors (IMUs) has become the mainstream technology of existing motion capture due to its advantages such as small size, low light requirements, and no need for external auxiliary equipment. However, since inertial sensors calculate position and attitude by measuring acceleration and integrating angular velocity, which is an integration process based on time, over time, large cumulative errors will occur in inertial sensors, affecting the capture accuracy of motion postures.

[0003] In existing technologies for improving the motion capture accuracy of inertial sensors, the patent "CN114533039B" corrects the errors caused by the relative displacement between inertial sensors and bones during motion, but does not consider correcting the cumulative errors of inertial sensors over time. The patent "CN112957033B" obtains attitude and coordinate information through inertial sensors, obtains three-dimensional coordinate information through ultra-wideband positioning technology, and uses a filtering algorithm for fusion to calculate the full-body attitude; however, this method still does not solve the error problem of inertial sensors, and in an indoor environment, if there is no error in the inertial sensor but an error occurs in the ultra-wideband positioning technology, it will further increase the error and affect the capture of motion postures.

[0004] Therefore, the existing technologies have not truly realized the correction of the errors of inertial sensors, and when improving the motion capture accuracy of inertial sensors, the existing Kalman filter is usually used for attitude calculation. However, the noise covariance matrix for error correction by the Kalman filter is a fixed value set according to prior knowledge, and it is not adaptively adjusted according to the actual scene characteristics. It does not consider that the actions of the observed person during the motion capture process are random, and the random variation degree of the actions will have a greater impact on the process noise, resulting in errors in the final attitude angle calculation. Summary of the Invention

[0005] To solve the above technical problems, the purpose of this application is to provide a motion capture method and system based on inertial sensors, and the specific technical solutions adopted are as follows:

[0006] In a first aspect, an embodiment of this application provides a motion capture method based on inertial sensors, and the method includes the following steps:

[0007] Collect the roll angle, pitch angle, and yaw angle of each inertial sensor at each acquisition moment in each cycle, as well as each acceleration sequence of each inertial sensor in each cycle;

[0008] Based on the differences between the fluctuation points in the acceleration sequence, the difference of the acceleration sequence, and the differences between the means of each local acceleration sequence, obtain the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle;

[0009] Based on the average situation of the mutation index, obtain the average mutation index of each inertial sensor at each fluctuating acquisition moment in each cycle;

[0010] Based on the distance between the local acceleration data sequences and the average mutation index, obtain the behavior intensity index of each inertial sensor at each fluctuating acquisition moment in each cycle;

[0011] Based on the behavior intensity index, obtain the acceleration response coefficient of each inertial sensor in each cycle;

[0012] Based on the roll angle, pitch angle, and yaw angle, obtain the attitude angle response coefficient of each inertial sensor in each cycle;

[0013] Based on the acceleration response coefficient and the attitude angle response coefficient, obtain the improved value of the noise covariance matrix of each inertial sensor in each cycle, and use the improved value of the noise covariance matrix of each inertial sensor in each cycle as the value of the noise covariance matrix when using the Kalman filter for attitude solution in the next adjacent cycle of each inertial sensor in each cycle to correct the error of the inertial sensor.

[0014] Furthermore, the roll angle, pitch angle, and yaw angle include:

[0015] For each inertial sensor, obtain the pitch angle and roll angle at each acquisition moment through the triaxial accelerometer as the first pitch angle and the first roll angle at each acquisition moment, and obtain the yaw angle at each acquisition moment through the triaxial magnetometer as the first yaw angle at each acquisition moment; obtain the pitch angle, roll angle, and yaw angle at each acquisition moment through the triaxial gyroscope as the second pitch angle, the second roll angle, and the second yaw angle at each acquisition moment.

[0016] Furthermore, the method for obtaining the mutation index is:

[0017] For each acceleration sequence of each inertial sensor in each cycle, use the peak-valley detection algorithm to obtain all the peaks and valleys in each acceleration sequence of each cycle as each fluctuation point; for each fluctuation point, calculate the absolute value of the difference between the acceleration data of the fluctuation point and the acceleration data at the previous adjacent acquisition moment as the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle;

[0018] Obtain the gregariousness index of each fluctuation point based on the difference situation of the acceleration data sequence;

[0019] For each fluctuation point in each acceleration sequence of each inertial sensor in each period, a window with a preset size is constructed centered on the fluctuation point as the local window of each fluctuation point, and the absolute value of the difference between the mean value of all acceleration data before the fluctuation point in the local window and the mean value of all acceleration data after the fluctuation point in the local window is used as the front-back mean difference of each fluctuation point;

[0020] The calculation formula of the mutation index is: ; where, is the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each period; is the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each period, is the gregariousness index of each fluctuation point in each acceleration sequence of each inertial sensor in each period, is the front-back mean difference of each fluctuation point in each acceleration sequence of each inertial sensor in each period; is the preset tuning parameter coefficient.

[0021] Furthermore, the method for obtaining the gregariousness index is:

[0022] For each acceleration sequence of each inertial sensor in each period, calculate the absolute value of the first-order difference value of all elements in the acceleration sequence, and take the maximum value of the absolute value of the first-order difference value of all elements in the acceleration sequence as the maximum difference value of the acceleration sequence;

[0023] For each fluctuation point in each acceleration sequence of each inertial sensor in each period, calculate the absolute value of the difference between the acceleration data of the fluctuation point and the maximum difference value as the gregariousness index of each fluctuation point.

[0024] Furthermore, the method for obtaining the average mutation index is:

[0025] For each fluctuation point in each acceleration sequence of each inertial sensor in each period, when the acceleration data in all other types of acceleration sequences at the acquisition moment where the fluctuation point is located are all fluctuation points, the acquisition moment is used as the fluctuation acquisition moment, and the mean value of the mutation indices of the fluctuation points in all types of acceleration sequences at the fluctuation acquisition moments of each period is calculated as the average mutation index of each inertial sensor at each fluctuation acquisition moment in each period.

[0026] Furthermore, the method for obtaining the violent behavior index is:

[0027] For each fluctuation acquisition moment of each cycle, use a clustering algorithm to cluster the average mutation indices of all inertial sensors to obtain each cluster; the cluster where the average mutation index of each inertial sensor is located is used as the target cluster of each inertial sensor.

[0028] For the acceleration data of each inertial sensor at each fluctuation acquisition moment in each acceleration sequence of each cycle, the sequence composed of all acceleration data in the local window of the acceleration data is used as the local window sequence of each fluctuation acquisition moment.

[0029] For each inertial sensor at each fluctuation acquisition moment of each cycle, calculate the mean of the distances between the local window sequence of each acceleration sequence and the local window sequences of the same type of acceleration sequences of all other inertial sensors in its target cluster as the trend difference of each acceleration sequence of each inertial sensor at each fluctuation acquisition moment of each cycle, and use the mean of the trend differences of all types of acceleration sequences of each inertial sensor at each fluctuation acquisition moment of each cycle as the trend dissimilarity index of each inertial sensor at each fluctuation acquisition moment of each cycle.

[0030] The calculation formula of the behavior intensity index is: ; where is the behavior intensity index of each inertial sensor at each fluctuation acquisition moment of each cycle; is the average mutation index of each inertial sensor at each fluctuation acquisition moment of each cycle; is the number of elements in the target cluster of each inertial sensor; is the trend dissimilarity index of each inertial sensor at each fluctuation acquisition moment of each cycle; χ is a preset tuning factor.

[0031] Furthermore, the method for obtaining the acceleration reflection coefficient is:

[0032] Use the behavior intensity indices of each inertial sensor at all fluctuation acquisition moments of each cycle as the input of the Otsu threshold segmentation algorithm, output the optimal segmentation threshold, use the fluctuation acquisition moments with behavior intensity indices greater than or equal to the optimal segmentation threshold as the severe fluctuation acquisition moments, and use the fluctuation acquisition moments with behavior intensity indices less than the optimal segmentation threshold as the stable fluctuation acquisition moments.

[0033] For each inertial sensor of each cycle, calculate the absolute value of the difference between the mean of the behavior intensity indices of all severe fluctuation acquisition moments and the mean of the behavior intensity indices of all stable fluctuation acquisition moments as the behavior difference index of each inertial sensor of each cycle.

[0034] For each inertial sensor of each cycle, calculate the ratio of the number of stable fluctuation acquisition moments to the number of all acquisition moments in the cycle as the stable duration of each inertial sensor of each cycle.

[0035] For each inertial sensor in each period, calculate the product of the mean of the behavior severity indices and the behavior difference indices at all severely fluctuating acquisition times, and use the ratio of the product to the stable duration as the acceleration reflection coefficient of each inertial sensor in each period.

[0036] Furthermore, the method for obtaining the attitude angle reflection coefficient is as follows:

[0037] For each inertial sensor in each period, calculate the mean of the first pitch angle and the second pitch angle at each acquisition time as the initial pitch angle of each inertial sensor in each period at each acquisition time, calculate the mean of the first roll angle and the second roll angle at each acquisition time as the initial roll angle of each inertial sensor in each period at each acquisition time, and calculate the mean of the first yaw angle and the second yaw angle at each acquisition time as the initial yaw angle of each inertial sensor in each period at each acquisition time;

[0038] The calculation formula for the attitude angle reflection coefficient is: ; where is the attitude angle reflection coefficient of the i-th inertial sensor in each period; M is the number of severely fluctuating acquisition times of the i-th inertial sensor in each period; , , are respectively the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t-th severely fluctuating acquisition time in each period, , , are respectively the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the (t + 1)-th severely fluctuating acquisition time in each period, are respectively the sequence numbers of the t-th and (t + 1)-th severely fluctuating acquisition times of the i-th inertial sensor among all acquisition times in each period.

[0039] Furthermore, the method for obtaining the improved value of the noise covariance matrix is:

[0040] The calculation formula for the improved value of the noise covariance matrix is: ; where is the improved value of the noise covariance matrix of the i-th inertial sensor in each period; represents the preset initial value of the noise covariance matrix; is the acceleration reflection coefficient of the i-th inertial sensor in each period; is the attitude angle reflection coefficient of the i-th inertial sensor in each period; is the normalization function.

[0041] In a second aspect, an embodiment of the present application further provides a motion capture system based on an inertial sensor, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above methods are implemented.

[0042] The present application has at least the following beneficial effects:

[0043] The present application takes into account that the random actions of the observed person have a great impact on the cumulative error of the motion data. By constructing a mutation index, the mutation degree of the behavior of the observed person at different acquisition moments is evaluated; then, considering that if the behavior of the observed person is very violent, it will cause the data of multiple inertial sensors to change synergistically, a violent behavior index is constructed to further evaluate the violent degree of the behavior of the observed person; finally, considering that if the number of violent behaviors is very frequent, it will further increase the cumulative error, and by constructing an acceleration reflection coefficient and an attitude angle reflection coefficient, the overall cumulative error degree is evaluated.

[0044] The present application takes into account that in the existing Kalman filter fusion method, the noise covariance matrix uses a fixed value, because the influence of the actions of the observed person on the error is not considered, resulting in the problem of low motion capture accuracy; the present application constructs an acceleration reflection coefficient and an attitude angle reflection coefficient to dynamically adjust the noise covariance matrix, which can effectively reduce the cumulative error of each inertial sensor, thereby improving the accuracy and stability of motion capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0046] Figure 1 It is a flowchart of the steps of a motion capture method based on an inertial sensor provided by an embodiment of the present application;

[0047] Figure 2 It is a flowchart for obtaining the improved value of the noise covariance matrix provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] To further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, detail the specific implementation manners, structures, features, and effects of the motion capture method and system based on inertial sensors proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs.

[0050] The following will specifically describe the specific solutions of the motion capture method and system based on inertial sensors provided by the present application in conjunction with the accompanying drawings.

[0051] Please refer to Figure 1 , which shows the step flow chart of the motion capture method based on inertial sensors provided by an embodiment of the present application. The method includes the following steps:

[0052] Step S1, collect the roll angle, pitch angle, and yaw angle of each inertial sensor at each acquisition moment in each period, and each acceleration sequence of each inertial sensor in each period.

[0053] An inertial sensor, also known as an Inertial Measurement Unit (IMU for short), where the inertial sensor consists of three parts: a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, and is bound to key parts of the human body and moves with the observed person. By performing attitude calculation on the collected motion data, the attitude angles of each part of the human body are calculated, and the human body attitude data is rendered by the upper computer to obtain a visual animation of human motion.

[0054] First, bind t inertial sensors to the head, back, waist, left and right upper arms, left and right forearms, left and right hands, left and right thighs, left and right calves, and left and right feet of the observed person respectively. In this embodiment, the value of t is 15, and the implementer can select other values according to the actual situation.

[0055] Furthermore, make the observed person stand still, perform initial calibration on the inertial sensors, align the inertial sensor coordinate system with the carrier coordinate system, and at the same time use mathematical transformation to convert the measured values in the carrier coordinate system into measured values in the geographical coordinate system, and describe the attitude information of the observed person in the geographical coordinate system. The calibration of the inertial sensor, coordinate alignment, and data coordinate conversion are all well-known technologies in the field of motion capture and will not be elaborated here.

[0056] Obtain the acceleration data of each inertial sensor at each acquisition moment in each cycle during the movement of the observed person. Further, for each inertial sensor, obtain the pitch angle and roll angle at each acquisition moment through a three-axis accelerometer as the first pitch angle and the first roll angle at each acquisition moment, and obtain the yaw angle at each acquisition moment through a three-axis magnetometer as the first yaw angle at each acquisition moment; additionally, obtain the pitch angle, roll angle, and yaw angle at each acquisition moment through a three-axis gyroscope as the second pitch angle, the second roll angle, and the second yaw angle at each acquisition moment. Among them, the methods for obtaining the roll angle, pitch angle, and yaw angle are well-known technologies and will not be elaborated in this embodiment.

[0057] Specifically, the data acquisition frequency is 50Hz, the data acquisition cycle duration is 10 seconds, and all the acquired data is subjected to Z-score standardization to eliminate the influence of dimensions. Among them, Z-score standardization is a well-known technology and will not be elaborated in this embodiment.

[0058] For each inertial sensor, construct each acceleration sequence of each inertial sensor in each cycle according to the time sequence of the acceleration data collected by the three-axis accelerometer on the x-axis, y-axis, and z-axis respectively. Among them, there are three types of acceleration sequences, namely the x-axis acceleration sequence, the y-axis acceleration sequence, and the z-axis acceleration sequence.

[0059] Step S2: Based on the differences between the fluctuation points in the acceleration sequence, the difference situation of the acceleration sequence difference, and the differences between the means of each local acceleration sequence, obtain the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; obtain the average sudden change index of each inertial sensor at each fluctuating acquisition moment in each cycle based on the average situation of the sudden change index; based on the distance between the local acceleration data sequences and the average sudden change index, obtain the behavior intensity index of each inertial sensor at each fluctuating acquisition moment in each cycle.

[0060] During the motion capture process, since a single inertial sensor is easily affected by noise interference, resulting in a decrease in measurement accuracy, therefore, usually, the method of multi-inertial sensor data fusion is used to improve the accuracy of the measurement data. As a common data fusion method, the Kalman filter algorithm describes the statistical characteristics of the process noise by introducing the process noise covariance matrix, thereby achieving high-precision data fusion.

[0061] However, the actions of the observed person in motion capture are usually random, and sudden changes in actions will lead to sudden changes in the measurement values of inertial sensors. If the degree of mutation is large, it will cause transient errors, thus affecting the accuracy of attitude angle calculation. At the same time, when the action of the observed person changes, their attitude angle will also change accordingly; if the behavior of the observed person changes more frequently and to a greater extent, it will cause the attitude angle to be recalculated continuously, resulting in an increase in noise and cumulative error, and ultimately reducing the accuracy of the next motion capture.

[0062] Therefore, if the behavior of the observed person changes more suddenly and the number of sudden change behaviors is more frequent, the greater the impact on the process noise, and the more necessary it is to increase the value of the process noise covariance matrix to reduce the impact of noise error.

[0063] The acceleration data collected can reflect the degree of behavior change of the observed person. If the acceleration is greater, it means that the current behavior speed of the observed person is faster; if the acceleration suddenly increases or decreases instantaneously, it reflects that the possibility of the observed person's action having instantaneous motion or instantaneous stop is greater.

[0064] If the behavior of the observed person undergoes a sudden change, it will inevitably lead to a change in acceleration, resulting in peaks or valleys. According to the above analysis, for each acceleration sequence of each inertial sensor in each period, the peak-valley detection algorithm is used to obtain all the peaks and valleys in each acceleration sequence of each period as each fluctuation point; for each fluctuation point, the absolute value of the difference between the acceleration data of the fluctuation point and the acceleration data at the previous adjacent acquisition moment is calculated as the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each period.

[0065] Among them, the peak-valley detection algorithm selected in this embodiment is the automatic multi-scale peak search algorithm. Implementers can select other peak-valley detection algorithms according to actual situations. The peak-valley detection algorithm is a well-known technology and will not be elaborated in this embodiment.

[0066] Furthermore, for each acceleration sequence of each inertial sensor in each period, the absolute value of the first-order difference value of all elements in the acceleration sequence is calculated, and the maximum value of the absolute value of the first-order difference value of all elements in the acceleration sequence is used as the maximum difference value of the acceleration sequence to reflect the absolute change degree of the acceleration data at the current acquisition moment compared with the previous acquisition moment.

[0067] For each fluctuation point in each acceleration sequence of each inertial sensor in each period, the absolute value of the difference between the acceleration data of the fluctuation point and the maximum difference value is calculated as the clustering index of each fluctuation point; the smaller the clustering index, the more significant the forward difference index of each fluctuation point is reflected, indicating that the possibility of the behavior of the observed person changing violently at the corresponding acquisition moment of each fluctuation point is greater.

[0068] Further, for each fluctuation point in each acceleration sequence of each inertial sensor in each period, a window with a preset size is constructed centered on the fluctuation point as the local window of each fluctuation point, and the absolute value of the difference between the mean value of all acceleration data before the fluctuation point in the local window and the mean value of all acceleration data after the fluctuation point in the local window is used as the front-back mean difference of each fluctuation point; in this embodiment, the value of the preset size is 9, and the implementer can select other values according to the actual situation.

[0069] The greater the front-back mean difference, the greater the change in the acceleration mean value at the corresponding acquisition moment of each fluctuation point, and the greater the possibility of a sudden change in behavior. It should be noted that when the window center is near both ends of the sequence and the window exceeds the sequence range, the part where the window exceeds is filled, and the filled data is the mean value of the x-axis acceleration sequence.

[0070] Further, based on the forward difference index, the gregariousness index, and the front-back mean difference, the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each period is obtained, and the calculation formula is: ; in the formula, is the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each period; is the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each period, is the gregariousness index of each fluctuation point in each acceleration sequence of each inertial sensor in each period, is the front-back mean difference of each fluctuation point in each acceleration sequence of each inertial sensor in each period; is the preset tuning parameter coefficient. To avoid the denominator being 0, the value in this embodiment is 1, and the implementer can select other values according to the actual situation.

[0071] It should be noted that if the forward difference index is larger, it reflects that the absolute acceleration difference between each fluctuation point and the acceleration data at the previous adjacent acquisition moment is larger; if the gregariousness index is smaller, it reflects that the forward difference index of each fluctuation point is more significant among all elements; if the front-back mean difference is larger, it indicates that the average acceleration of the observed person has changed greatly before and after each fluctuation point, that is, the possibility of inconsistent behavior is greater. Therefore, if the sudden change index is larger, it indicates that the possibility of a drastic change in the behavior of the observed person at the acquisition moment of each fluctuation point in each acceleration sequence of each period is greater.

[0072] So far, the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each period is obtained.

[0073] For each fluctuation point in each acceleration sequence of each inertial sensor in each period, when the acceleration data in all other types of acceleration sequences at the acquisition moment where the fluctuation point is located are all fluctuation points, the acquisition moment is taken as the fluctuation acquisition moment, and the mean value of the mutation indices of the fluctuation points of all types of acceleration sequences at the fluctuation acquisition moments of each period is calculated as the average mutation index of each inertial sensor at each fluctuation acquisition moment of each period. If the average mutation index is larger, it reflects that the motion data collected by each inertial sensor has a greater degree of mutation at each fluctuation acquisition moment, and the greater the possibility that the behavior of the observed person shows a mutation.

[0074] Furthermore, if the behavior of the observed person is relatively intense, it will simultaneously cause significant changes in the data of multiple inertial sensors. For example, high knee lifts will cause the inertial sensors on the legs to change simultaneously. Therefore, the degree of co-variation of the data between each inertial sensor and other inertial sensors at the same acquisition moment can be used to further reflect the intensity of the behavior of the observed person.

[0075] If the number of inertial sensors with significantly changed acquisition data is larger at the same acquisition moment, it can reflect that the behavior of the observed person at this acquisition moment is more intense.

[0076] For each fluctuation acquisition moment of each period, the K-means clustering algorithm is used to cluster the average mutation indices of all inertial sensors to obtain each cluster. In this embodiment, the K-means clustering algorithm is selected, and the number of clusters is obtained according to the elbow method. The K-means clustering algorithm and the elbow method are well-known technologies and will not be elaborated in this embodiment. The cluster where the average mutation index of each inertial sensor is located is used as the target cluster of each inertial sensor.

[0077] For the acceleration data of each inertial sensor in each type of acceleration sequence at each fluctuation acquisition moment in each period, the sequence composed of all acceleration data in the local window of the acceleration data is used as the local window sequence at each fluctuation acquisition moment.

[0078] Furthermore, for each inertial sensor at each fluctuation acquisition moment of each period, the mean value of the DTW distances between the local window sequence of each type of acceleration sequence and the local window sequences of the same type of acceleration sequences of all other inertial sensors in its target cluster is calculated as the trend difference of each type of acceleration sequence of each inertial sensor at each fluctuation acquisition moment of each period. The mean value of the trend differences of all types of acceleration sequences of each inertial sensor at each fluctuation acquisition moment of each period is used as the trend dissimilarity index of each inertial sensor at each fluctuation acquisition moment of each period. Among them, the calculation method of the DTW distance is a well-known technology and will not be elaborated in this embodiment.

[0079] It should be noted that the smaller the trend difference index is, the greater the data coordination degree between each inertial sensor and the remaining inertial sensors in the target cluster is near each fluctuation acquisition moment of each inertial sensor in each cycle.

[0080] Furthermore, in order to reflect the severity of the behavior changes of the observed person, based on the average mutation index, the number of elements in the target cluster where the inertial sensor is located, and the trend difference index, calculate the behavior severity index of each inertial sensor at each fluctuation acquisition moment in each cycle. The calculation formula is: ; where is the behavior severity index of each inertial sensor at each fluctuation acquisition moment in each cycle; is the average mutation index of each inertial sensor at each fluctuation acquisition moment in each cycle; is the number of elements in the target cluster of each inertial sensor; is the trend difference index of each inertial sensor at each fluctuation acquisition moment in each cycle; χ is a preset tuning factor to prevent the denominator from being 0. In this embodiment, the value of χ is 1, and the implementer can select other values according to the actual situation.

[0081] It should be noted that if the average mutation index of each inertial sensor at each fluctuation acquisition moment in each cycle is larger, it reflects that the mutation degree of the motion data at each fluctuation acquisition moment of each inertial sensor in each cycle is larger; if the number of elements in the target cluster is larger, it reflects that there are more inertial sensors with significant changes in the acquired data at each fluctuation acquisition moment of each inertial sensor in each cycle; if the trend difference index is smaller, it reflects that the data change degrees of each inertial sensor in the target cluster are more consistent near each fluctuation acquisition moment of each inertial sensor in each cycle. Therefore, if the behavior severity index is larger, it reflects that there are more inertial sensors with significant changes in the acquired data at each fluctuation acquisition moment of each inertial sensor in each cycle, and the data change degrees are more consistent, indicating that the behavior changes of the observed person are more severe at each fluctuation acquisition moment of each inertial sensor in each cycle, and it is more necessary to increase the noise covariance matrix to correct the error.

[0082] Step S3, obtain the acceleration response coefficient of each inertial sensor in each cycle based on the behavior severity index; obtain the attitude angle response coefficient of each inertial sensor in each cycle based on the roll angle, pitch angle, and yaw angle.

[0083] Furthermore, high-frequency violent actions such as punching multiple times, sudden stops, and sharp turns will significantly increase the influence of transient errors, resulting in a decline in the performance of the filter in the attitude angle calculation process and seriously affecting the observation accuracy.

[0084] Based on the above analysis, the behavior severity index at all fluctuation acquisition moments of each inertial sensor in each period is used as the input of the Otsu threshold segmentation algorithm to output the optimal segmentation threshold. The fluctuation acquisition moments with a behavior severity index greater than or equal to the optimal segmentation threshold are regarded as severe fluctuation acquisition moments, and the fluctuation acquisition moments with a behavior severity index less than the optimal segmentation threshold are regarded as stable fluctuation acquisition moments.

[0085] Furthermore, for each inertial sensor in each period, calculate the absolute value of the difference between the mean of the behavior severity indices at all severe fluctuation acquisition moments and the mean of the behavior severity indices at all stable fluctuation acquisition moments, which is used as the behavior difference index of each inertial sensor in each period. The larger the behavior difference index, the greater the difference between the stable behavior and the non-stable behavior of the observed person reflected in the collected motion data.

[0086] Furthermore, for each inertial sensor in each period, calculate the ratio of the number of stable fluctuation acquisition moments to the number of all acquisition moments in the period, which is used as the stable duration of each inertial sensor in each period. The larger the stable duration, the more the duration of the stable behavior appears, and the greater the possibility that the overall behavior actions of the observed person change stably.

[0087] Based on the average of the behavior severity index, the behavior difference index, and the stable duration, obtain the acceleration response coefficient of the inertial sensor in each period. The calculation formula is: ; In the formula, is the acceleration response coefficient of each inertial sensor in each period; is the mean of the behavior severity indices at all severe fluctuation acquisition moments of each inertial sensor in each period; is the behavior difference index of each inertial sensor in each period; P is the stable duration of each inertial sensor in each period.

[0088] It should be noted that if has a larger value, it reflects that the behavior of the observed person is more severe at each severe fluctuation acquisition moment; if the behavior difference index L is larger, it indicates that the difference between the stable actions and the non-stable actions of the observed person is greater, reflecting that the severity of the observed person's non-stable actions is greater; if the stable duration P is smaller, it reflects that the duration of the stable actions of the observed person is shorter in the collected data. Therefore, if the acceleration response coefficient is larger, it means that during the process of collecting data by the inertial sensor in each period, the severity of the behavior actions of the observed person is greater, and the cumulative error brought is greater.

[0089] Furthermore, for each inertial sensor in each period, the average of the first pitch angle and the second pitch angle at each collection moment is calculated as the initial pitch angle of each inertial sensor in each period at each collection moment, the average of the first roll angle and the second roll angle at each collection moment is calculated as the initial roll angle of each inertial sensor in each period at each collection moment, and the average of the first yaw angle and the second yaw angle at each collection moment is calculated as the initial yaw angle of each inertial sensor in each period at each collection moment.

[0090] Furthermore, by analyzing the differences between the attitude angles at different violent fluctuation collection moments, the degree of behavioral change of the observed person is reflected, thereby further optimizing the noise covariance matrix. Based on the differences between the initial attitude angles at adjacent violent fluctuation collection moments, the attitude angle reflection coefficient of each inertial sensor in each cycle is obtained. The calculation formula is: ; In the formula, is the attitude angle reflection coefficient of the i-th inertial sensor in each cycle; M is the number of violent fluctuation collection moments of the i-th inertial sensor in each cycle; , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t-th violent fluctuation collection moment of each cycle, , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t+1-th violent fluctuation collection time of each cycle, They are respectively the position sequences of the t-th and t+1-th violent fluctuation collection moments of the i-th inertial sensor in all collection moments in the cycle.

[0091] It should be noted that if the difference in attitude angle between the t+1th violent fluctuation collection time and the tth violent fluctuation collection time is greater, it can be reflected that the attitude change of the i-th inertial sensor between the two violent fluctuation collection times is greater; if The smaller the value, the smaller the time interval between the t+1th violent fluctuation collection moment and the tth violent fluctuation collection moment. Therefore, if the attitude angle reflection coefficient at the tth violent fluctuation collection moment is larger, it means that the attitude change of the i-th inertial sensor between the tth and t+1th violent fluctuation collection moments is larger, the time interval is shorter, the more drastic the behavior change of the observed person is, and the greater the error caused. At the same time, if there are more violent fluctuation collection moments and M is larger, it will further lead to an increase in the error, resulting in a continuous increase in the cumulative error, so it is necessary to further increase the noise covariance matrix to eliminate the noise.

[0092] Step S4: Based on the acceleration reflection coefficient and the attitude angle reflection coefficient, obtain the improved value of the noise covariance matrix of each inertial sensor in each period. Use the improved value of the noise covariance matrix of each inertial sensor in each period as the value of the noise covariance matrix when using the Kalman filter for attitude calculation in the next adjacent period of each inertial sensor, and correct the error of the inertial sensor.

[0093] Specifically, in the existing Kalman filter fusion method, the noise covariance matrix is a matrix of 1 That is, a fixed value is adopted. The value of the noise covariance matrix will affect the calculation of the Kalman gain. If the behavior of the observed person is violent and the overall error is large, then increase the value of the noise covariance matrix and decrease the Kalman gain, making the Kalman filter more dependent on the previous data; on the contrary, if the behavior of the observed person is stable and the error is small, then decrease the value of the noise covariance matrix and increase the Kalman gain, making the Kalman filter more dependent on the current data.

[0094] According to the acceleration reflection coefficient of the i-th inertial sensor And the attitude angle reflection coefficient Improve the value of the noise covariance matrix of the Kalman filter, and obtain the improved value of the noise covariance matrix of the i-th inertial sensor in each period. The calculation formula is: ; In the formula, Is the improved value of the noise covariance matrix of the i-th inertial sensor in each period; Represents the preset initial value of the noise covariance matrix, and the value in this embodiment is 100. The implementer can select other values according to the actual situation; Is the acceleration reflection coefficient of the i-th inertial sensor in each period; Is the attitude angle reflection coefficient of the i-th inertial sensor in each period; Is the normalization function. Among them, the flowchart for obtaining the improved value of the noise covariance matrix is as Figure 2 Shown.

[0095] So far, obtain the improved value of the noise covariance matrix of each inertial sensor in each period. Use the improved value of the noise covariance matrix of each inertial sensor in each period as the value of the noise covariance matrix when using the Kalman filter for attitude calculation in the next adjacent period of each inertial sensor, and correct the error of the inertial sensor.

[0096] According to the above analysis, correct the error of each inertial sensor in each period until the observed person stops motion capture, thereby improving the accuracy of motion capture of the observed person.

[0097] Based on the same inventive concept as the above method, an embodiment of the present application further provides a motion capture system based on an inertial sensor, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned motion capture methods based on an inertial sensor are implemented.

[0098] It should be noted that: the above order of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above description of specific embodiments of this specification is given. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] The various embodiments in the present application are described in a progressive manner. The same or similar parts among the various embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized.

[0100] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A motion capture method based on an inertial sensor, characterized in that: The method comprises the following steps: Collecting the roll angle, pitch angle and yaw angle of each inertial sensor at each collection moment in each cycle and each acceleration sequence of each inertial sensor in each cycle; Based on the differences between the fluctuation points in the acceleration sequence, the differences of the acceleration sequence and the differences between the average values ​​of each local acceleration sequence, the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each period is obtained; Based on the average of the sudden change index, the average sudden change index of each inertial sensor at each fluctuation collection moment of each cycle is obtained; Based on the distance between the local acceleration data sequences and the average sudden change index, the behavior intensity index of each inertial sensor at each fluctuation collection moment in each cycle is obtained; Based on the behavior intensity index, the acceleration response coefficient of each inertial sensor in each cycle is obtained; Obtain the attitude angle reflection coefficient of each inertial sensor in each cycle based on the roll angle, pitch angle and yaw angle; Based on the acceleration reflection coefficient and the attitude angle reflection coefficient, the improved value of the noise covariance matrix of each inertial sensor in each period is obtained, and the improved value of the noise covariance matrix of each inertial sensor in each period is used as the value of the noise covariance matrix of each inertial sensor when using Kalman filtering for attitude solution in the next adjacent period of each period, so as to correct the error of the inertial sensor.

2. The motion capture method based on inertial sensor according to claim 1, characterized in that: The roll angle, pitch angle and yaw angle include: For each inertial sensor, the pitch angle and roll angle at each collection moment are obtained through a three-axis accelerometer as the first pitch angle and the first roll angle at each collection moment, and the yaw angle at each collection moment is obtained through a three-axis magnetometer as the first yaw angle at each collection moment; the pitch angle, roll angle and yaw angle at each collection moment are obtained through a three-axis gyroscope as the second pitch angle, the second roll angle and the second yaw angle at each collection moment.

3. The motion capture method based on inertial sensor according to claim 1, characterized in that: The method for obtaining the sudden change index is: For each acceleration sequence of each inertial sensor in each cycle, a peak and valley detection algorithm is used to obtain all peak values ​​and valley values ​​in each acceleration sequence of each cycle as each fluctuation point; for each fluctuation point, the absolute value of the difference between the acceleration data of the fluctuation point and the acceleration data of the previous adjacent acquisition time is calculated as the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; Obtain the sociability index of each fluctuation point based on the difference of the acceleration data sequence; For each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, a window of a preset size is constructed with the fluctuation point as the center as the local window of each fluctuation point, and the mean of all acceleration data before the fluctuation point in the local window is calculated, and the absolute value of the difference between the mean of all acceleration data after the fluctuation point in the local window is taken as the mean difference before and after each fluctuation point; The calculation formula of the sudden change index is: ; In the formula, is the sudden change index of each fluctuation point of each inertial sensor in each acceleration sequence of each cycle; is the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, is the group index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, is the difference between the mean values ​​before and after each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; is the preset tuning coefficient.

4. The motion capture method based on inertial sensor as claimed in claim 3, characterized in that: The method for obtaining the sociability index is: For each acceleration sequence of each inertial sensor in each cycle, the absolute value of the first-order difference value of all elements in the acceleration sequence is calculated, and the maximum value of the absolute value of the first-order difference value of all elements in the acceleration sequence is taken as the maximum value of the difference of the acceleration sequence; For each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, the absolute value of the difference between the acceleration data of the fluctuation point and the maximum difference value is calculated as the group index of each fluctuation point.

5. The motion capture method based on inertial sensor as claimed in claim 1, characterized in that: The method for obtaining the average sudden change index is: For each fluctuation point in each acceleration sequence of each inertial sensor in each period, when the acceleration data in all other types of acceleration sequences at the collection time where the fluctuation point is located are all fluctuation points, the collection time is taken as the fluctuation collection time, and the average of the sudden change indexes of the fluctuation points of all types of acceleration sequences at the fluctuation collection time of each period is calculated as the average sudden change index of each inertial sensor at each fluctuation collection time of each period.

6. The motion capture method based on inertial sensor as claimed in claim 3, characterized in that: The method for obtaining the behavior intensity index is: For each fluctuation collection moment of each cycle, a clustering algorithm is used to cluster the average sudden change index of all inertial sensors to obtain each cluster cluster; the cluster cluster where the average sudden change index of each inertial sensor is located is used as the target cluster of each inertial sensor; For the acceleration data of each inertial sensor at each fluctuation collection moment in each acceleration sequence of each cycle, a sequence composed of all acceleration data in a local window of the acceleration data is used as a local window sequence at each fluctuation collection moment; For each inertial sensor at each fluctuation collection time of each cycle, the average of the distances between the local window sequence of each acceleration sequence and the local window sequences of the same type of acceleration sequences of all other inertial sensors in its target cluster is calculated as the trend difference of each acceleration sequence of each inertial sensor at each fluctuation collection time of each cycle, and the average of the trend differences of all types of acceleration sequences of each inertial sensor at each fluctuation collection time of each cycle is used as the trend difference index of each inertial sensor at each fluctuation collection time of each cycle; The calculation formula of the behavior intensity index is: ; In the formula, is the behavior intensity index of each inertial sensor at each fluctuation collection moment in each cycle; is the average sudden change index of each inertial sensor at each fluctuation collection moment in each cycle; is the number of elements in the target cluster of each inertial sensor; is the trend difference index of each inertial sensor at each fluctuation collection moment in each cycle; χ is the preset parameter adjustment factor.

7. The motion capture method based on inertial sensor as claimed in claim 2, characterized in that: The method for obtaining the acceleration reflection coefficient is: The behavior intensity index of all fluctuation collection moments of each inertial sensor in each cycle is used as the input of the Otsu threshold segmentation algorithm, and the optimal segmentation threshold is output. The fluctuation collection moment when the behavior intensity index is greater than or equal to the optimal segmentation threshold is used as the violent fluctuation collection moment, and the fluctuation collection moment when the behavior intensity index is less than the optimal segmentation threshold is used as the stable fluctuation collection moment; For each inertial sensor in each period, the absolute value of the difference between the mean of the behavior intensity index at all violent fluctuation collection moments and the mean of the behavior intensity index at all stable fluctuation collection moments is calculated as the behavior difference index of each inertial sensor in each period; For each inertial sensor in each cycle, the ratio of the number of stable fluctuation collection moments to the number of all collection moments in the cycle is calculated as the stable duration of each inertial sensor in each cycle; For each inertial sensor in each cycle, the product of the average of the behavior intensity index at all the violent fluctuation collection moments and the behavior difference index is calculated, and the ratio of the product to the stable duration is used as the acceleration reflection coefficient of each inertial sensor in each cycle.

8. The motion capture method based on inertial sensor as claimed in claim 7, characterized in that: The method for obtaining the attitude angle reflection coefficient is: For each inertial sensor in each period, the average of the first pitch angle and the second pitch angle at each acquisition moment is calculated as the initial pitch angle of each inertial sensor in each period at each acquisition moment, the average of the first roll angle and the second roll angle at each acquisition moment is calculated as the initial roll angle of each inertial sensor in each period at each acquisition moment, and the average of the first yaw angle and the second yaw angle at each acquisition moment is calculated as the initial yaw angle of each inertial sensor in each period at each acquisition moment; The calculation formula of the attitude angle reflection coefficient is: ; In the formula, is the attitude angle reflection coefficient of the i-th inertial sensor in each cycle; M is the number of violent fluctuation collection moments of the i-th inertial sensor in each cycle; , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t-th violent fluctuation collection moment of each cycle, , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t+1-th violent fluctuation collection time of each cycle, They are respectively the position sequences of the t-th and t+1-th violent fluctuation collection moments of the i-th inertial sensor in all collection moments in the cycle.

9. The motion capture method based on inertial sensor according to claim 1, characterized in that: The method for obtaining the improved value of the noise covariance matrix is: The calculation formula of the improved value of the noise covariance matrix is: ; In the formula, is the improved value of the noise covariance matrix of the i-th inertial sensor in each cycle; Represents the preset initial value of the noise covariance matrix; is the acceleration reflection coefficient of the i-th inertial sensor in each cycle; is the attitude angle reflection coefficient of the i-th inertial sensor in each cycle; is the normalization function.

10. A motion capture system based on an inertial sensor, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the motion capture method based on an inertial sensor as claimed in any one of claims 1 to 9 are implemented.

Images