Motion capturing method and system based on inertial sensor

Abstract

The invention relates to the technical field of motion capture, in particular to a motion capture method and system based on inertial sensors, and the method comprises the steps: collecting a roll angle, a pitch angle and a yaw angle of each inertial sensor at each collection moment of each period and each acceleration sequence of each inertial sensor in each period; acquiring a sudden change index of each fluctuation point in each acceleration sequence of each period of each inertial sensor; acquiring an average sudden change index of each inertial sensor at each fluctuation acquisition moment of each period; acquiring a behavior violent index of each inertial sensor at each fluctuation acquisition moment of each period; acquiring an acceleration reflection coefficient and an attitude angle reflection coefficient of each inertial sensor in each period; and obtaining the improved value of the noise covariance matrix of each inertial sensor in each period, and correcting the error of the inertial sensor. According to the invention, the precision and stability of motion capture are improved.

Description

Technical Field

[0001] This application relates to the field of motion capture technology, and specifically to a motion capture method and system based on inertial sensors. Background Art

[0002] As an important means of obtaining digital attitude information, motion capture technology is widely used in fields such as rehabilitation medicine, virtual reality, intelligent robots, and sports training. Among them, the motion capture technology based on inertial sensors (IMUs) has become the mainstream technology of existing motion capture due to its advantages such as small size, low light requirements, and no need for external auxiliary equipment. However, since inertial sensors calculate position and attitude by measuring acceleration and integrating angular velocity, which is an integration process based on time, over time, large cumulative errors will occur in inertial sensors, affecting the capture accuracy of motion postures.

[0003] In the existing technologies for improving the motion capture accuracy of inertial sensors, the patent "CN114533039B" corrects the errors caused by the relative displacement between inertial sensors and bones during the motion process, but does not consider correcting the cumulative errors of inertial sensors over time. The patent "CN112957033B" obtains attitude and coordinate information through inertial sensors, obtains three-dimensional coordinate information through ultra-wideband positioning technology, and uses a filtering algorithm for fusion to calculate the full-body attitude; however, this method still does not solve the error problem of inertial sensors, and in an indoor environment, if the inertial sensor does not produce errors while the ultra-wideband positioning technology has errors, it will further increase the errors and affect the capture of motion postures.

[0004] Therefore, the existing technologies have not truly realized the correction of the errors of inertial sensors, and when improving the motion capture accuracy of inertial sensors, the existing Kalman filter is usually used for attitude calculation. However, the noise covariance matrix for error correction by the Kalman filter is a fixed value set according to prior knowledge, and it is not adaptively adjusted according to the actual scene characteristics. It does not consider that the actions of the observed person during the motion capture process are random, and the random variation degree of the actions will have a greater impact on the process noise, resulting in errors in the final attitude angle calculation. Summary of the Invention

[0005] In order to solve the above technical problems, the purpose of this application is to provide a motion capture method and system based on inertial sensors, and the specific technical solutions adopted are as follows: In a first aspect, an embodiment of this application provides a motion capture method based on inertial sensors, and the method includes the following steps: Collect the roll angle, pitch angle, and yaw angle of each inertial sensor at each acquisition moment in each cycle, as well as each acceleration sequence of each inertial sensor in each cycle; Based on the differences between the fluctuation points in the acceleration sequence, the difference of the acceleration sequence, and the differences between the means of each local acceleration sequence, obtain the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; Based on the average situation of the mutation index, obtain the average mutation index of each inertial sensor at each fluctuating acquisition moment in each cycle; Based on the distance between the local acceleration data sequences and the average mutation index, obtain the behavior intensity index of each inertial sensor at each fluctuating acquisition moment in each cycle; Based on the behavior intensity index, obtain the acceleration response coefficient of each inertial sensor in each cycle; Based on the roll angle, pitch angle, and yaw angle, obtain the attitude angle response coefficient of each inertial sensor in each cycle; Based on the acceleration response coefficient and the attitude angle response coefficient, obtain the improved value of the noise covariance matrix of each inertial sensor in each cycle, and use the improved value of the noise covariance matrix of each inertial sensor in each cycle as the value of the noise covariance matrix when using the Kalman filter for attitude solution in the next adjacent cycle of each inertial sensor, so as to correct the error of the inertial sensor.

[0006] Furthermore, the roll angle, pitch angle, and yaw angle include: For each inertial sensor, obtain the pitch angle and roll angle at each acquisition moment through a three-axis accelerometer as the first pitch angle and the first roll angle at each acquisition moment, and obtain the yaw angle at each acquisition moment through a three-axis magnetometer as the first yaw angle at each acquisition moment; obtain the pitch angle, roll angle, and yaw angle at each acquisition moment through a three-axis gyroscope as the second pitch angle, the second roll angle, and the second yaw angle at each acquisition moment.

[0007] Furthermore, the method for obtaining the mutation index is: For each acceleration sequence of each inertial sensor in each cycle, use the peak-valley detection algorithm to obtain all the peaks and valleys in each acceleration sequence of each cycle as each fluctuation point; for each fluctuation point, calculate the absolute value of the difference between the acceleration data of the fluctuation point and the acceleration data at the previous adjacent acquisition moment as the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; Based on the difference situation of the acceleration data sequence, obtain the gregarious index of each fluctuation point; For each fluctuation point in each acceleration sequence of each inertial sensor in each period, a window with a preset size is constructed centered on the fluctuation point as the local window of each fluctuation point, and the absolute value of the difference between the mean of all acceleration data before the fluctuation point in the local window and the mean of all acceleration data after the fluctuation point in the local window is used as the front-back mean difference of each fluctuation point; The calculation formula of the mutation index is: ; where is the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each period; is the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each period, is the clustering index of each fluctuation point in each acceleration sequence of each inertial sensor in each period, is the front-back mean difference of each fluctuation point in each acceleration sequence of each inertial sensor in each period; is a preset tuning parameter.

[0008] Furthermore, the method for obtaining the clustering index is: For each acceleration sequence of each inertial sensor in each period, calculate the absolute value of the first-order difference value of all elements in the acceleration sequence, and take the maximum value of the absolute value of the first-order difference value of all elements in the acceleration sequence as the maximum difference value of the acceleration sequence; For each fluctuation point in each acceleration sequence of each inertial sensor in each period, calculate the absolute value of the difference between the acceleration data of the fluctuation point and the maximum difference value as the clustering index of each fluctuation point.

[0009] Furthermore, the method for obtaining the average mutation index is: For each fluctuation point in each acceleration sequence of each inertial sensor in each period, when the acceleration data in all other types of acceleration sequences at the acquisition time where the fluctuation point is located are all fluctuation points, the acquisition time is used as the fluctuation acquisition time, and the mean of the mutation indexes of the fluctuation points of all types of acceleration sequences at the fluctuation acquisition time of each period is calculated as the average mutation index of each inertial sensor at each fluctuation acquisition time of each period.

[0010] Furthermore, the method for obtaining the behavior intensity index is: For each fluctuation acquisition time of each period, use the clustering algorithm to cluster the average mutation indexes of all inertial sensors to obtain each cluster; the cluster where the average mutation index of each inertial sensor is located is used as the target cluster of each inertial sensor; For the acceleration data of each inertial sensor at each fluctuation acquisition moment in each acceleration sequence of each cycle, the sequence composed of all acceleration data in the local window of the acceleration data is used as the local window sequence at each fluctuation acquisition moment; For each inertial sensor at each fluctuation acquisition moment in each cycle, calculate the mean of the distances between the local window sequence of each acceleration sequence and the local window sequences of the same type of acceleration sequences of all other inertial sensors in its target cluster as the trend difference of each inertial sensor's each acceleration sequence at each fluctuation acquisition moment in each cycle, and take the mean of the trend differences of all types of acceleration sequences of each inertial sensor at each fluctuation acquisition moment in each cycle as the trend dissimilarity index of each inertial sensor at each fluctuation acquisition moment in each cycle; The calculation formula of the behavior intensity index is: ; where is the behavior intensity index of each inertial sensor at each fluctuation acquisition moment in each cycle; is the average mutation index of each inertial sensor at each fluctuation acquisition moment in each cycle; is the number of elements in the target cluster of each inertial sensor; is the trend dissimilarity index of each inertial sensor at each fluctuation acquisition moment in each cycle; χ is a preset tuning factor.

[0011] Furthermore, the method for obtaining the acceleration reflection coefficient is: Take the behavior intensity index of each inertial sensor at all fluctuation acquisition moments in each cycle as the input of the Otsu threshold segmentation algorithm, output the optimal segmentation threshold, regard the fluctuation acquisition moments with the behavior intensity index greater than or equal to the optimal segmentation threshold as the severe fluctuation acquisition moments, and regard the fluctuation acquisition moments with the behavior intensity index less than the optimal segmentation threshold as the stable fluctuation acquisition moments; For each inertial sensor in each cycle, calculate the absolute value of the difference between the mean of the behavior intensity indices of all severe fluctuation acquisition moments and the mean of the behavior intensity indices of all stable fluctuation acquisition moments as the behavior difference index of each inertial sensor in each cycle; For each inertial sensor in each cycle, calculate the ratio of the number of stable fluctuation acquisition moments to the number of all acquisition moments in the cycle as the stable duration of each inertial sensor in each cycle; For each inertial sensor in each cycle, calculate the product of the mean of the behavior intensity indices of all severe fluctuation acquisition moments and the behavior difference index, and take the ratio of the product to the stable duration as the acceleration reflection coefficient of each inertial sensor in each cycle.

[0012] Furthermore, the method for obtaining the attitude angle reflection coefficient is: For each inertial sensor in each period, calculate the mean of the first pitch angle and the second pitch angle at each acquisition moment as the initial pitch angle of each inertial sensor in each period at each acquisition moment, calculate the mean of the first roll angle and the second roll angle at each acquisition moment as the initial roll angle of each inertial sensor in each period at each acquisition moment, and calculate the mean of the first yaw angle and the second yaw angle at each acquisition moment as the initial yaw angle of each inertial sensor in each period at each acquisition moment; The calculation formula of the attitude angle reflection coefficient is: ; In the formula, is the attitude angle reflection coefficient of the i-th inertial sensor in each period; M is the number of acquisition moments with severe fluctuations in each period of the i-th inertial sensor; , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t-th severely fluctuating acquisition moment in each period respectively, , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the (t + 1)-th severely fluctuating acquisition moment in each period respectively, are the sequence numbers of the t-th and (t + 1)-th severely fluctuating acquisition moments of the i-th inertial sensor in all acquisition moments in each period respectively.

[0013] Furthermore, the method for obtaining the improved value of the noise covariance matrix is: The calculation formula of the improved value of the noise covariance matrix is: ; In the formula, is the improved value of the noise covariance matrix of the i-th inertial sensor in each period; represents the preset initial value of the noise covariance matrix; is the acceleration reflection coefficient of the i-th inertial sensor in each period; is the attitude angle reflection coefficient of the i-th inertial sensor in each period; is the normalization function.

[0014] In a second aspect, an embodiment of the present application further provides a motion capture system based on an inertial sensor, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0015] The present application has at least the following beneficial effects: In view of the fact that the random actions of the observed person have a great impact on the cumulative error of the motion data, this application constructs a mutation index to evaluate the degree of mutation of the behavior of the observed person at different acquisition times. Then, considering that if the behavior of the observed person is very intense, it will cause the data of multiple inertial sensors to change synergistically, a behavior intensity index is constructed to further evaluate the intensity of the behavior of the observed person. Finally, considering that if the number of intense behaviors is very frequent, it will further increase the cumulative error. By constructing an acceleration reflection coefficient and an attitude angle reflection coefficient, the overall degree of cumulative error is evaluated.

[0016] This application takes into account that in the existing Kalman filter fusion method, the noise covariance matrix uses a fixed value, which leads to the problem of low motion capture accuracy because the influence of the actions of the observed person on the error is not considered. This application constructs an acceleration reflection coefficient and an attitude angle reflection coefficient to dynamically adjust the noise covariance matrix, which can effectively reduce the cumulative error of each inertial sensor, thereby improving the accuracy and stability of motion capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a flowchart of the steps of a motion capture method based on inertial sensors provided by an embodiment of the present application; Figure 2 It is a flowchart for obtaining the improved value of the noise covariance matrix provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] In order to further elaborate on the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of the motion capture method and system based on inertial sensors proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs.

[0021] The following specifically describes the specific solutions of the motion capture method and system based on inertial sensors provided by this application in conjunction with the accompanying drawings.

[0022] Please refer to Figure 1 , which shows a flowchart of the steps of a motion capture method based on inertial sensors provided by an embodiment of this application. The method includes the following steps: Step S1, collect the roll angle, pitch angle, and yaw angle of each inertial sensor at each acquisition moment in each cycle, and each acceleration sequence of each inertial sensor in each cycle.

[0023] An inertial sensor, also known as an Inertial Measurement Unit (IMU for short), where the inertial sensor consists of three parts: a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. It is bound to key parts of the human body and moves with the observed person. By performing attitude calculation on the collected motion data, the attitude angles of each part of the human body are calculated, and the human body attitude data is rendered by an upper computer to obtain a visual animation of human motion.

[0024] First, bind t inertial sensors to the head, back, waist, left and right upper arms, left and right forearms, left and right hands, left and right thighs, left and right calves, and left and right feet of the observed person respectively. In this embodiment, the value of t is 15, and the implementer can select other values according to the actual situation.

[0025] Furthermore, make the observed person stand still, perform initial calibration on the inertial sensors, align the inertial sensor coordinate system with the carrier coordinate system, and at the same time use mathematical transformation to convert the measured values in the carrier coordinate system into measured values in the geographical coordinate system, and describe the attitude information of the observed person in the geographical coordinate system. The calibration of inertial sensors, coordinate alignment, and data coordinate conversion are all well-known technologies in the field of motion capture and will not be elaborated here.

[0026] Obtain the acceleration data of each inertial sensor at each acquisition moment in each cycle during the movement of the observed person. Furthermore, for each inertial sensor, obtain the pitch angle and roll angle at each acquisition moment through the three-axis accelerometer as the first pitch angle and the first roll angle at each acquisition moment, and obtain the yaw angle at each acquisition moment through the three-axis magnetometer as the first yaw angle at each acquisition moment; in addition, obtain the pitch angle, roll angle, and yaw angle at each acquisition moment through the three-axis gyroscope as the second pitch angle, the second roll angle, and the second yaw angle at each acquisition moment. Among them, the methods for obtaining the roll angle, pitch angle, and yaw angle are well-known technologies and will not be elaborated in this embodiment.

[0027] Specifically, the data acquisition frequency is 50 Hz, and the data acquisition cycle duration is 10 seconds. All the acquired data is standardized by Z-score to eliminate the influence of dimension. Among them, Z-score standardization is a well-known technology, which will not be elaborated in this embodiment.

[0028] For each inertial sensor, the acceleration data collected by the triaxial accelerometer on the x-axis, y-axis, and z-axis respectively is used to construct each acceleration sequence of each inertial sensor in each cycle according to the time sequence of data acquisition. Among them, there are three types of acceleration sequences, namely, the x-axis acceleration sequence, the y-axis acceleration sequence, and the z-axis acceleration sequence.

[0029] Step S2: Based on the differences between the fluctuation points in the acceleration sequence, the differences in the differences of the acceleration sequence, and the differences between the means of the local acceleration sequences, obtain the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; based on the average situation of the sudden change index, obtain the average sudden change index of each inertial sensor at each fluctuating acquisition moment in each cycle; based on the distance between the local acceleration data sequences and the average sudden change index, obtain the behavior intensity index of each inertial sensor at each fluctuating acquisition moment in each cycle.

[0030] During the motion capture process, since a single inertial sensor is easily affected by noise interference, resulting in a decrease in measurement accuracy, usually, the method of multi-inertial sensor data fusion is used to improve the accuracy of the measurement data. As a common data fusion method, the Kalman filter algorithm describes the statistical characteristics of the process noise by introducing the process noise covariance matrix, thereby realizing high-precision data fusion.

[0031] However, the actions of the observed person during motion capture are usually random, and the sudden change of the action will cause the sudden change of the measurement value of the inertial sensor. If the degree of sudden change is large, it will cause transient errors, thus affecting the accuracy of attitude angle calculation. At the same time, when the action of the observed person changes, his attitude angle will also change accordingly; if the behavior of the observed person changes more frequently and more violently, it will cause the attitude angle to be recalculated continuously, resulting in an increase in noise and cumulative error, and ultimately reducing the accuracy of the next motion capture.

[0032] Therefore, if the behavior of the observed person changes more suddenly and the number of sudden change behaviors is more frequent, the greater the impact on the process noise, and the more it is necessary to increase the value of the process noise covariance matrix to reduce the impact of noise errors.

[0033] The acquired acceleration data can reflect the degree of behavior change of the observed person. If the acceleration is greater, it means that the current behavior speed of the observed person is faster; if the acceleration increases or decreases instantaneously, it reflects that the possibility of the observed person's action having instantaneous motion or instantaneous stop is greater.

[0034] If the behavior of the observed person undergoes a sudden change, it will inevitably lead to a change in acceleration, resulting in peaks or valleys. According to the above analysis, for each acceleration sequence of each inertial sensor in each period, the peak-valley detection algorithm is used to obtain all the peaks and valleys in each acceleration sequence of each period as each fluctuation point; for each fluctuation point, the absolute value of the difference between the acceleration data of the fluctuation point and the acceleration data at the previous adjacent acquisition moment is calculated as the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each period.

[0035] Among them, the peak-valley detection algorithm selected in this embodiment is the automatic multi-scale peak search algorithm. The implementer can select other peak-valley detection algorithms according to the actual situation. The peak-valley detection algorithm is a well-known technology and will not be elaborated in this embodiment.

[0036] Furthermore, for each acceleration sequence of each inertial sensor in each period, the absolute value of the first-order difference value of all elements in the acceleration sequence is calculated, and the maximum value of the absolute value of the first-order difference value of all elements in the acceleration sequence is used as the maximum difference value of the acceleration sequence to reflect the absolute change degree of the acceleration data at the current acquisition moment compared with the previous acquisition moment.

[0037] For each fluctuation point in each acceleration sequence of each inertial sensor in each period, the absolute value of the difference between the acceleration data of the fluctuation point and the maximum difference value is calculated as the clustering index of each fluctuation point; the smaller the clustering index, the more significant the forward difference index of each fluctuation point is reflected, indicating that the possibility of a sudden change in the behavior of the observed person at the corresponding acquisition moment of each fluctuation point is greater.

[0038] Furthermore, for each fluctuation point in each acceleration sequence of each inertial sensor in each period, a window with a preset size centered on the fluctuation point is constructed as the local window of each fluctuation point, and the absolute value of the difference between the mean value of all acceleration data before the fluctuation point in the local window and the mean value of all acceleration data after the fluctuation point in the local window is calculated as the front-back mean difference of each fluctuation point; in this embodiment, the value of the preset size is 9, and the implementer can select other values according to the actual situation.

[0039] The greater the front-back mean difference, the greater the change in the acceleration mean value at the corresponding acquisition moment of each fluctuation point is reflected, and the greater the possibility of a sudden behavior change is. It should be noted that when the window center is near both ends of the sequence and the window exceeds the sequence range, the exceeded part of the window is filled, and the filled data is the mean value of the x-axis acceleration sequence.

[0040] Further, based on the forward difference index, the gregariousness index, and the mean difference before and after, the mutation index of each fluctuation point in each acceleration sequence of each cycle is obtained. The calculation formula is as follows: ; In the formula, is the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; is the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, is the gregariousness index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, is the mean difference before and after of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; is a preset parameter adjustment coefficient. To avoid the denominator being 0, the value in this embodiment is 1, and the implementer can select other values according to the actual situation.

[0041] It should be noted that if the forward difference index is larger, it reflects that the absolute acceleration difference between each fluctuation point and the acceleration data at the previous adjacent acquisition moment is larger; if the gregariousness index is smaller, it reflects that the forward difference index of each fluctuation point is more significant among all elements; if the mean difference before and after is larger, it indicates that the average acceleration of the observed person has changed greatly before and after each fluctuation point, that is, the possibility of inconsistent behavior is greater. Therefore, if the mutation index is larger, it indicates that the possibility of a drastic change in the behavior of the observed person at the acquisition moment of each fluctuation point in each acceleration sequence of each cycle is greater.

[0042] Thus, the mutation index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle is obtained.

[0043] For each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, when the acceleration data in all other types of acceleration sequences at the acquisition moment where the fluctuation point is located are all fluctuation points, the acquisition moment is used as the fluctuation acquisition moment, and the mean value of the mutation indices of the fluctuation points of all types of acceleration sequences at the fluctuation acquisition moment of each cycle is calculated as the average mutation index of each inertial sensor at each fluctuation acquisition moment of each cycle. If the average mutation index is larger, it reflects that the degree of mutation of the motion data collected by each inertial sensor at each fluctuation acquisition moment is larger, and the possibility of a mutation in the behavior of the observed person is greater.

[0044] Further, if the behavior of the observed person is relatively intense, it will simultaneously cause significant changes in the data of multiple inertial sensors. For example, high knee lifts will cause the inertial sensors on the legs to change simultaneously. Therefore, the degree of co-variation of the data between each inertial sensor and other inertial sensors at the same acquisition moment can be used to further reflect the intensity of the behavior of the observed person.

[0045] If, at the same acquisition moment, the number of inertial sensors with significantly changing acquisition data is larger, it can reflect that the behavior of the observed person at that acquisition moment is more intense.

[0046] For each fluctuating acquisition moment in each period, use the clustering algorithm to cluster the average mutation indices of all inertial sensors to obtain each cluster. In this embodiment, the K-means clustering algorithm is selected for the clustering algorithm, and the number of clusters is obtained according to the elbow method. Among them, the K-means clustering algorithm and the elbow method are well-known technologies and will not be elaborated in this embodiment. Take the cluster where the average mutation index of each inertial sensor is located as the target cluster of each inertial sensor.

[0047] For the acceleration data of each inertial sensor at each fluctuating acquisition moment in each acceleration sequence in each period, the sequence composed of all acceleration data in the local window of the acceleration data is used as the local window sequence at each fluctuating acquisition moment.

[0048] Further, for each inertial sensor at each fluctuating acquisition moment in each period, calculate the mean of the DTW distances between the local window sequence of each acceleration sequence and the local window sequences of the same type of acceleration sequences of all other inertial sensors in its target cluster as the trend difference of each type of acceleration sequence of each inertial sensor at each fluctuating acquisition moment in each period. Take the mean of the trend differences of all types of acceleration sequences of each inertial sensor at each fluctuating acquisition moment in each period as the trend dissimilarity index of each inertial sensor at each fluctuating acquisition moment in each period. Among them, the calculation method of the DTW distance is a well-known technology and will not be elaborated in this embodiment.

[0049] It should be noted that the smaller the trend dissimilarity index, the greater the degree of data coordination between each inertial sensor and the remaining inertial sensors in the target cluster near each fluctuating acquisition moment of each inertial sensor in each period.

[0050] Further, in order to reflect the intensity of the behavior change of the observed person, based on the average mutation index, the number of elements in the target cluster where the inertial sensor is located, and the trend dissimilarity index, calculate the behavior intensity index of each inertial sensor at each fluctuating acquisition moment in each period. The calculation formula is: ; where is the behavior intensity index of each inertial sensor at each fluctuating acquisition moment in each period; is the average mutation index of each inertial sensor at each fluctuation acquisition moment in each period; is the number of elements in the target cluster of each inertial sensor; is the trend dissimilarity index of each inertial sensor at each fluctuation acquisition moment in each period; χ is a preset tuning factor to prevent the denominator from being 0. In this embodiment, the value of χ is 1, and the implementer can select other values according to the actual situation.

[0051] It should be noted that if the average mutation index of each inertial sensor at each fluctuation acquisition moment in each period is larger, it reflects that the mutation degree of the motion data at each fluctuation acquisition moment of each inertial sensor is greater; if the number of elements in the target cluster is larger, it reflects that more inertial sensors with significantly changed acquisition data are present at each fluctuation acquisition moment of each inertial sensor in each period; if the trend dissimilarity index is smaller, it reflects that the data change degrees of each inertial sensor in the target cluster are relatively consistent at the vicinity of each fluctuation acquisition moment of each inertial sensor in each period. Therefore, if the behavior intensity index is larger, it reflects that more inertial sensors with significantly changed acquisition data are present at each fluctuation acquisition moment of each inertial sensor in each period, and the data change degrees are more consistent, indicating that the behavior change of the observed person is more intense at each fluctuation acquisition moment of each inertial sensor in each period, and it is more necessary to increase the noise covariance matrix to correct the error.

[0052] Step S3, obtaining the acceleration response coefficient of each inertial sensor in each period based on the behavior intensity index; obtaining the attitude angle response coefficient of each inertial sensor in each period based on the roll angle, pitch angle, and yaw angle.

[0053] Furthermore, high-frequency intense actions such as multiple punches, sudden stops, and sharp turns will significantly increase the influence of transient errors, resulting in a decline in the performance of the filter during the attitude angle calculation process, seriously affecting the observation accuracy.

[0054] Based on the above analysis, taking the behavior intensity index of all fluctuation acquisition moments of each inertial sensor in each period as the input of the Otsu threshold segmentation algorithm, outputting the optimal segmentation threshold, taking the fluctuation acquisition moments with the behavior intensity index greater than or equal to the optimal segmentation threshold as the intense fluctuation acquisition moments, and taking the fluctuation acquisition moments with the behavior intensity index less than the optimal segmentation threshold as the stable fluctuation acquisition moments.

[0055] Further, for each inertial sensor in each period, calculate the absolute value of the difference between the mean of the behavior severity indices at all severely fluctuating acquisition moments and the mean of the behavior severity indices at all smoothly fluctuating acquisition moments, as the behavior difference index of each inertial sensor in each period; the larger the behavior difference index, the greater the difference between the stable behavior and the non-stable behavior of the observed person reflected in the collected motion data.

[0056] Further, for each inertial sensor in each period, calculate the ratio of the number of smoothly fluctuating acquisition moments to the number of all acquisition moments in the period, as the stable duration of each inertial sensor in each period. The larger the stable duration, the more the duration of the stable behavior appears, and the greater the possibility that the overall behavior actions of the observed person change stably.

[0057] Based on the average situation of the behavior severity index, the behavior difference index, and the stable duration, obtain the acceleration response coefficient of the inertial sensor in each period. The calculation formula is: ; In the formula, is the acceleration response coefficient of each inertial sensor in each period; is the mean of the behavior severity indices at all severely fluctuating acquisition moments of each inertial sensor in each period; is the behavior difference index of each inertial sensor in each period; P is the stable duration of each inertial sensor in each period.

[0058] It should be noted that if has a larger value, it reflects that the behavior of the observed person is more severe at each severely fluctuating acquisition moment; if the behavior difference index L is larger, it indicates that the difference between the stable actions and the non-stable actions of the observed person is greater, reflecting that the severity of the observed person's non-stable actions is greater; if the stable duration P is smaller, it reflects that the duration of the stable actions of the observed person is shorter in the collected data. Therefore, if the acceleration response coefficient is larger, it indicates that during the data acquisition process of the inertial sensor in each period, the severity of the behavior actions of the observed person is greater, and the resulting cumulative error is greater.

[0059] Further, for each inertial sensor in each period, calculate the mean of the first pitch angle and the second pitch angle at each acquisition moment as the initial pitch angle of each inertial sensor in each period at each acquisition moment, calculate the mean of the first roll angle and the second roll angle at each acquisition moment as the initial roll angle of each inertial sensor in each period at each acquisition moment, and calculate the mean of the first yaw angle and the second yaw angle at each acquisition moment as the initial yaw angle of each inertial sensor in each period at each acquisition moment.

[0060] Furthermore, by analyzing the differences between the attitude angles at different violent fluctuation collection moments, the degree of behavioral change of the observed person is reflected, thereby further optimizing the noise covariance matrix. Based on the differences between the initial attitude angles at adjacent violent fluctuation collection moments, the attitude angle reflection coefficient of each inertial sensor in each cycle is obtained. The calculation formula is: ; In the formula, is the attitude angle reflection coefficient of the i-th inertial sensor in each cycle; M is the number of violent fluctuation collection moments of the i-th inertial sensor in each cycle; , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t-th violent fluctuation collection moment of each cycle, , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t+1-th violent fluctuation collection time of each cycle, They are respectively the position sequences of the t-th and t+1-th violent fluctuation collection moments of the i-th inertial sensor in all collection moments in the cycle.

[0061] It should be noted that if the difference in attitude angle between the t+1th violent fluctuation collection time and the tth violent fluctuation collection time is greater, it can be reflected that the attitude change of the i-th inertial sensor between the two violent fluctuation collection times is greater; if The smaller the value, the smaller the time interval between the t+1th violent fluctuation collection moment and the tth violent fluctuation collection moment. Therefore, if the attitude angle reflection coefficient at the tth violent fluctuation collection moment is larger, it means that the attitude change of the i-th inertial sensor between the tth and t+1th violent fluctuation collection moments is larger, the time interval is shorter, the more drastic the behavior change of the observed person is, and the greater the error caused. At the same time, if there are more violent fluctuation collection moments and M is larger, it will further lead to an increase in the error, resulting in a continuous increase in the cumulative error, so it is necessary to further increase the noise covariance matrix to eliminate the noise.

[0062] Step S4, based on the acceleration reflection coefficient and the attitude angle reflection coefficient, the improved value of the noise covariance matrix of each inertial sensor in each period is obtained, and the improved value of the noise covariance matrix of each inertial sensor in each period is used as the value of the noise covariance matrix of each inertial sensor when using Kalman filtering for attitude solution in the next adjacent period of each period, so as to correct the error of the inertial sensor.

[0063] Specifically, the noise covariance matrix in the existing Kalman filter fusion method is 1 A matrix of 1, that is, using fixed values. The value of the noise covariance matrix affects the calculation of the Kalman gain. If the behavior of the observed person is violent and the overall error is large, the value of the noise covariance matrix is increased and the Kalman gain is decreased, making the Kalman filter more dependent on previous data. On the contrary, if the behavior of the observed person is stable and the error is small, the value of the noise covariance matrix is decreased and the Kalman gain is increased, making the Kalman filter more dependent on current data.

[0064] According to the acceleration reflection coefficient of the i-th inertial sensor And the attitude angle reflection coefficient Improve the value of the noise covariance matrix of the Kalman filter, and obtain the improved value of the noise covariance matrix of the i-th inertial sensor in each period. The calculation formula is: ; In the formula, Is the improved value of the noise covariance matrix of the i-th inertial sensor in each period; Represents the preset initial value of the noise covariance matrix. The value in this embodiment is 100, and the implementer can select other values according to the actual situation; Is the acceleration reflection coefficient of the i-th inertial sensor in each period; Is the attitude angle reflection coefficient of the i-th inertial sensor in each period; Is the normalization function. Among them, the flowchart for obtaining the improved value of the noise covariance matrix is as Figure 2 Shown.

[0065] So far, obtain the improved value of the noise covariance matrix of each inertial sensor in each period, and use it as the value of the noise covariance matrix when using the Kalman filter for attitude calculation in the next adjacent period of each inertial sensor, so as to correct the error of the inertial sensor.

[0066] According to the above analysis, correct the errors of each inertial sensor in each period until the observed person stops motion capture, thereby improving the accuracy of motion capture of the observed person.

[0067] Based on the same inventive concept as the above method, the embodiment of the present application also provides a motion capture system based on inertial sensors, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-mentioned motion capture methods based on inertial sensors.

[0068] It should be noted that: the above order of the embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the specific embodiments of this specification have been described above. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0069] Each embodiment in the present application is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

[0070] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A motion capture method based on an inertial sensor, characterized in that: The method comprises the following steps: Collecting the roll angle, pitch angle and yaw angle of each inertial sensor at each collection moment in each cycle and each acceleration sequence of each inertial sensor in each cycle; Based on the differences between the fluctuation points in the acceleration sequence, the differences of the acceleration sequence and the differences between the average values ​​of each local acceleration sequence, the sudden change index of each fluctuation point in each acceleration sequence of each inertial sensor in each period is obtained; Based on the average of the sudden change index, the average sudden change index of each inertial sensor at each fluctuation collection moment of each cycle is obtained; Based on the distance between the local acceleration data sequences and the average sudden change index, the behavior intensity index of each inertial sensor at each fluctuation collection moment in each cycle is obtained; Based on the behavior intensity index, the acceleration response coefficient of each inertial sensor in each cycle is obtained; Obtain the attitude angle reflection coefficient of each inertial sensor in each cycle based on the roll angle, pitch angle and yaw angle; Based on the acceleration reflection coefficient and the attitude angle reflection coefficient, the improved value of the noise covariance matrix of each inertial sensor in each period is obtained, and the improved value of the noise covariance matrix of each inertial sensor in each period is used as the value of the noise covariance matrix of each inertial sensor when using Kalman filtering for attitude solution in the next adjacent period of each period, so as to correct the error of the inertial sensor.

2. The motion capture method based on inertial sensor according to claim 1, characterized in that: The roll angle, pitch angle and yaw angle include: For each inertial sensor, the pitch angle and roll angle at each collection moment are obtained through a three-axis accelerometer as the first pitch angle and the first roll angle at each collection moment, and the yaw angle at each collection moment is obtained through a three-axis magnetometer as the first yaw angle at each collection moment; the pitch angle, roll angle and yaw angle at each collection moment are obtained through a three-axis gyroscope as the second pitch angle, the second roll angle and the second yaw angle at each collection moment.

3. The motion capture method based on inertial sensor according to claim 1, characterized in that: The method for obtaining the sudden change index is: For each acceleration sequence of each inertial sensor in each cycle, a peak and valley detection algorithm is used to obtain all peak values ​​and valley values ​​in each acceleration sequence of each cycle as each fluctuation point; for each fluctuation point, the absolute value of the difference between the acceleration data of the fluctuation point and the acceleration data of the previous adjacent acquisition time is calculated as the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; Obtain the sociability index of each fluctuation point based on the difference of the acceleration data sequence; For each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, a window of a preset size is constructed with the fluctuation point as the center as the local window of each fluctuation point, and the mean of all acceleration data before the fluctuation point in the local window is calculated, and the absolute value of the difference between the mean of all acceleration data after the fluctuation point in the local window is taken as the mean difference before and after each fluctuation point; The calculation formula of the sudden change index is: ; In the formula, is the sudden change index of each fluctuation point of each inertial sensor in each acceleration sequence of each cycle; is the forward difference index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, is the group index of each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, is the difference between the mean values ​​before and after each fluctuation point in each acceleration sequence of each inertial sensor in each cycle; is the preset tuning coefficient.

4. The motion capture method based on inertial sensor as claimed in claim 3, characterized in that: The method for obtaining the sociability index is: For each acceleration sequence of each inertial sensor in each cycle, the absolute value of the first-order difference value of all elements in the acceleration sequence is calculated, and the maximum value of the absolute value of the first-order difference value of all elements in the acceleration sequence is taken as the maximum value of the difference of the acceleration sequence; For each fluctuation point in each acceleration sequence of each inertial sensor in each cycle, the absolute value of the difference between the acceleration data of the fluctuation point and the maximum difference value is calculated as the group index of each fluctuation point.

5. The motion capture method based on inertial sensor as claimed in claim 1, characterized in that: The method for obtaining the average sudden change index is: For each fluctuation point in each acceleration sequence of each inertial sensor in each period, when the acceleration data in all other types of acceleration sequences at the collection time where the fluctuation point is located are all fluctuation points, the collection time is taken as the fluctuation collection time, and the average of the sudden change indexes of the fluctuation points of all types of acceleration sequences at the fluctuation collection time of each period is calculated as the average sudden change index of each inertial sensor at each fluctuation collection time of each period.

6. The motion capture method based on inertial sensor as claimed in claim 3, characterized in that: The method for obtaining the behavior intensity index is: For each fluctuation collection moment of each cycle, a clustering algorithm is used to cluster the average sudden change index of all inertial sensors to obtain each cluster cluster; the cluster cluster where the average sudden change index of each inertial sensor is located is used as the target cluster of each inertial sensor; For the acceleration data of each inertial sensor at each fluctuation collection moment in each acceleration sequence of each cycle, a sequence composed of all acceleration data in a local window of the acceleration data is used as a local window sequence at each fluctuation collection moment; For each inertial sensor at each fluctuation collection time of each cycle, the average of the distances between the local window sequence of each acceleration sequence and the local window sequences of the same type of acceleration sequences of all other inertial sensors in its target cluster is calculated as the trend difference of each acceleration sequence of each inertial sensor at each fluctuation collection time of each cycle, and the average of the trend differences of all types of acceleration sequences of each inertial sensor at each fluctuation collection time of each cycle is used as the trend difference index of each inertial sensor at each fluctuation collection time of each cycle; The calculation formula of the behavior intensity index is: ; In the formula, is the behavior intensity index of each inertial sensor at each fluctuation collection moment in each cycle; is the average sudden change index of each inertial sensor at each fluctuation collection moment in each cycle; is the number of elements in the target cluster of each inertial sensor; is the trend difference index of each inertial sensor at each fluctuation collection moment in each cycle; χ is the preset parameter adjustment factor.

7. The motion capture method based on inertial sensor as claimed in claim 2, characterized in that: The method for obtaining the acceleration reflection coefficient is: The behavior intensity index of all fluctuation collection moments of each inertial sensor in each cycle is used as the input of the Otsu threshold segmentation algorithm, and the optimal segmentation threshold is output. The fluctuation collection moment when the behavior intensity index is greater than or equal to the optimal segmentation threshold is used as the violent fluctuation collection moment, and the fluctuation collection moment when the behavior intensity index is less than the optimal segmentation threshold is used as the stable fluctuation collection moment; For each inertial sensor in each period, the absolute value of the difference between the mean of the behavior intensity index at all violent fluctuation collection moments and the mean of the behavior intensity index at all stable fluctuation collection moments is calculated as the behavior difference index of each inertial sensor in each period; For each inertial sensor in each cycle, the ratio of the number of stable fluctuation collection moments to the number of all collection moments in the cycle is calculated as the stable duration of each inertial sensor in each cycle; For each inertial sensor in each cycle, the product of the average of the behavior intensity index at all the violent fluctuation collection moments and the behavior difference index is calculated, and the ratio of the product to the stable duration is used as the acceleration reflection coefficient of each inertial sensor in each cycle.

8. The motion capture method based on inertial sensor as claimed in claim 7, characterized in that: The method for obtaining the attitude angle reflection coefficient is: For each inertial sensor in each period, the average of the first pitch angle and the second pitch angle at each acquisition moment is calculated as the initial pitch angle of each inertial sensor in each period at each acquisition moment, the average of the first roll angle and the second roll angle at each acquisition moment is calculated as the initial roll angle of each inertial sensor in each period at each acquisition moment, and the average of the first yaw angle and the second yaw angle at each acquisition moment is calculated as the initial yaw angle of each inertial sensor in each period at each acquisition moment; The calculation formula of the attitude angle reflection coefficient is: ; In the formula, is the attitude angle reflection coefficient of the i-th inertial sensor in each cycle; M is the number of violent fluctuation collection moments of the i-th inertial sensor in each cycle; , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t-th violent fluctuation collection moment of each cycle, , , are the initial pitch angle, initial roll angle and initial yaw angle of the i-th inertial sensor at the t+1-th violent fluctuation collection time of each cycle, They are respectively the position sequences of the t-th and t+1-th violent fluctuation collection moments of the i-th inertial sensor in all collection moments in the cycle.

9. The motion capture method based on inertial sensor as claimed in claim 1, characterized in that: The method for obtaining the improved value of the noise covariance matrix is: The calculation formula of the improved value of the noise covariance matrix is: ; In the formula, is the improved value of the noise covariance matrix of the i-th inertial sensor in each cycle; Represents the preset initial value of the noise covariance matrix; is the acceleration reflection coefficient of the i-th inertial sensor in each cycle; is the attitude angle reflection coefficient of the i-th inertial sensor in each cycle; is the normalization function.

10. A motion capture system based on an inertial sensor, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the motion capture method based on an inertial sensor as claimed in any one of claims 1 to 9 are implemented.

Images