Multi-sensor angle fusion joint measurement method and system

Abstract

The invention provides a joint measurement method and system based on multi-sensor angle fusion, and the method comprises the steps: collecting the first angle data of a target joint of a patient based on an IMU sensor, carrying out the timestamp alignment of the first angle data, carrying out the incremental angle calibration of the first angle data, and obtaining the first angle data of the target joint of the patient; performing fusion to obtain second angle data, performing dynamic weight fusion based on the weight of the IMU sensor and the second angle data to obtain corresponding third angle data, performing sliding window filtering on the third angle data, and responding to the condition that the filtered third angle data exceeds a preset biomechanical limiting condition; and if yes, fusing the biomechanical limiting condition and the third angle data to carry out biomechanical correction to obtain target angle data of the target joint.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and in particular, to a joint measurement method and system for multi-sensor angle fusion. Background Technique

[0002] The technology of joint angles is increasingly widely used in fields such as medical rehabilitation, robot control, sports science, and virtual reality, among which the measurement of joint angles is involved.

[0003] At present, the methods of measuring joint angles using multi-sensors mainly include dual-IMU (inertial sensor) joint measurement and four-IMU spine measurement. Among them, dual-IMU joint measurement usually fixes an IMU sensor on each of the upper arm and forearm (or the thigh and calf of the lower limb) of the upper limb, and uses gyroscope integration for angle measurement. And four-IMU spine measurement is to arrange multiple sensors at key parts such as the thoracic vertebra, lumbar vertebra, and cervical vertebra, and calculate the relative movement between each segment by establishing a spine kinematic chain model and using inverse kinematics algorithm for angle measurement.

[0004] However, since gyroscope data depends on integral calculation of angles, any small error (such as temperature drift, noise) will be amplified over time, resulting in the problem of integral drift, and it cannot adapt to changes in motion states, resulting in over-smoothing at low speeds and lag at high speeds, with poor dynamic response. In addition, there is a lack of spatio-temporal synchronization between multiple IMUs, resulting in deviations when fusing multi-angle data. Summary of the Invention

[0005] In view of this, the purpose of the present application is to provide a joint measurement method and system for multi-sensor angle fusion, which collects angle data through IMU sensors and aligns the hardware timestamps, and performs dynamic weight fusion after self-calibrating the angle data, and corrects the angle data through biomechanical constraints to obtain the final target angle data, solving the problem of integral drift. The dynamic weight can adapt to changes in motion states, thereby improving the dynamic responsiveness. In addition, through the spatio-temporal synchronization and self-calibration of timestamp alignment, the deviation during multi-angle data fusion is reduced, and more accurate high-precision data acquisition is achieved.

[0006] In a first aspect, an embodiment of the present application provides a joint measurement method for multi-sensor angle fusion, which is applied to a joint measurement system for multi-sensor angle fusion. The joint measurement system for multi-sensor angle fusion includes multiple IMU sensors, and each IMU sensor is configured with an independent adaptive Kalman filter; the method includes:

[0007] For the target joint of the patient, collect the first angle data of the target joint of the patient based on the IMU sensor, and align the timestamps of the first angle data;

[0008] Perform incremental angle calibration on the first angle data, fuse to obtain second angle data, and perform dynamic weight fusion based on the weights of the IMU sensors and the second angle data to obtain corresponding third angle data;

[0009] Perform sliding window filtering on the third angle data. In response to the filtered third angle data exceeding a preset biomechanical limit condition, fuse the biomechanical limit condition and the third angle data for biomechanical correction to obtain the target angle data of the target joint.

[0010] In a possible implementation manner, the method further includes:

[0011] Real-time monitor the sensor parameters of the IMU sensors; wherein, the sensor parameters at least include signal-to-noise ratio and motion acceleration;

[0012] Dynamically adjust the weights of each sensor based on the sensor parameters.

[0013] In a possible implementation manner, the method further includes:

[0014] Perform validity verification on the first angle data, compare the timestamps between all the first angle data, and determine the valid data and invalid data in the first angle data based on the comparison result of the timestamps;

[0015] In response to discarding the invalid data in the first angle data, perform timestamp alignment on the first angle data.

[0016] In a possible implementation manner, each IMU sensor embeds an independent high-precision clock module with target accuracy; the performing timestamp alignment on the first angle data includes:

[0017] When collecting the first angle data of the target joint of the patient based on the IMU sensors, add corresponding original hardware timestamps to each frame of the first angle data through the high-precision clock module;

[0018] In the initialization stage when collecting the first angle data of the target joint of the patient based on the IMU sensors, perform clock deviation calibration based on a preset system unified reference master clock and the original hardware timestamps to perform timestamp alignment on the first angle data.

[0019] In a possible implementation manner, the performing dynamic weight fusion based on the weights of the IMU sensors and the second angle data includes:

[0020] Adjust the weights of the IMU sensors based on the second angle data to obtain the adjusted weights of the IMU sensors;

[0021] Perform dynamic weight fusion on the adjusted weights of the IMU sensors and the second angle data.

[0022] In a possible implementation manner, each IMU sensor includes a high-precision multi-axis motion capture device with a target accuracy, and the multi-axis motion capture device has a target attitude solution accuracy;

[0023] The multi-axis motion capture device includes a target accelerometer, a target gyroscope, and a target Gaussian magnetometer; the target accelerometer, the target gyroscope, and the target Gaussian magnetometer achieve the target attitude solution accuracy of the multi-axis motion capture device.

[0024] In a possible implementation manner, the multi-sensor angle fusion joint measurement system includes a transmission device, and the transmission device includes a target wireless transmission module and a target low-power Bluetooth module. The method further includes:

[0025] After the first angle data is collected, the wireless transmission module transmits the first angle data based on a preset first transmission frequency; or,

[0026] After the first angle data is collected, the target low-power Bluetooth module transmits the first angle data based on a preset second transmission frequency; wherein, the transmission delay of the target low-power Bluetooth module at the second transmission frequency meets a preset transmission delay standard.

[0027] In a possible implementation manner, the method further includes:

[0028] Determine whether the patient is wearing the IMU sensor for the first time;

[0029] If the patient is wearing the IMU sensor for the first time, perform biomechanical adjustment on the patient's IMU sensor to calibrate the IMU sensor.

[0030] In a possible implementation manner, the method further includes:

[0031] Determine a corresponding joint evaluation quantization index based on the target angle data of the target joint of the patient;

[0032] Evaluate the target joint of the patient based on the joint evaluation quantization index to obtain an evaluation result of the target joint.

[0033] In a second aspect, an embodiment of the present application further provides a joint measurement system for multi-sensor angle fusion. The joint measurement system for multi-sensor angle fusion includes a plurality of IMU sensors, and each IMU sensor is configured with an independent adaptive Kalman filter;

[0034] The joint measurement system for multi-sensor angle fusion is used to execute the joint measurement method for multi-sensor angle fusion provided in the embodiment of the first aspect.

[0035] A joint measurement method and system for multi-sensor angle fusion provided by an embodiment of the present application, for a target joint of a patient, based on an IMU sensor, collects first angle data of the target joint of the patient, performs timestamp alignment on the first angle data, performs incremental angle calibration on the first angle data, fuses to obtain second angle data, and performs dynamic weight fusion based on the weights of the IMU sensors and the second angle data to obtain corresponding third angle data, performs sliding window filtering on the third angle data, and in response to the filtered third angle data exceeding a preset biomechanical limit condition, fuses the biomechanical limit condition and the third angle data to perform biomechanical correction to obtain target angle data of the target joint. In the present application, angle data is collected through an IMU sensor and hardware timestamp alignment is performed, and after self-calibrating the angle data, dynamic weight fusion is performed, and biomechanical correction is performed on the angle data through biomechanical limit conditions to obtain the final target angle data, solving the problem of integral drift. The dynamic weight can adapt to changes in the motion state, thereby improving the dynamic responsiveness. In addition, through the spatio-temporal synchronization and self-calibration of timestamp alignment, the deviation during multi-angle data fusion is reduced, and more accurate high-precision data acquisition is achieved.

[0036] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1 is a flowchart of a joint measurement method for multi-sensor angle fusion provided by an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of the joint measurement process for multi-sensor angle fusion. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the objectives, technical solutions and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purposes of illustration and description, and are not used to limit the protection scope of this application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0041] In addition, the described embodiments are only some embodiments of this application, rather than all embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application to be protected, but only represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of this application.

[0042] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0043] Considering that the joint angle technology is becoming more and more widely used in fields such as medical rehabilitation, robot control, sports science, and virtual reality, among which the measurement of joint angles is involved.

[0044] Currently, the methods for measuring joint angles using multiple sensors mainly include dual-IMU (inertial sensor) joint measurement and four-IMU spine measurement. Among them, dual-IMU joint measurement usually fixes an IMU sensor on each of the upper arm and forearm (or the thigh and calf of the lower limb) of the upper limb, and uses gyroscope integration for angle measurement, while four-IMU spine measurement is to arrange multiple sensors at key parts such as the thoracic vertebrae, lumbar vertebrae, and cervical vertebrae, and calculate the relative movement between each segment by establishing a spine kinematic chain model and using an inverse kinematics algorithm for angle measurement.

[0045] However, since gyroscope data relies on integral calculation of angles, any tiny errors (such as temperature drift, noise) will be amplified over time, leading to the problem of integral drift, and it cannot adapt to changes in motion states, resulting in over-smoothing at low speeds and lag at high speeds, with poor dynamic response. Additionally, the lack of spatio-temporal synchronization between multiple IMUs leads to deviations when fusing multi-angle data.

[0046] To address this problem, the present application provides a joint measurement method and system for multi-sensor angle fusion. Angle data is collected through IMU sensors and hardware timestamp alignment is performed. After self-calibrating the angle data, dynamic weight fusion is carried out, and biomechanical correction is performed on the angle data through biomechanical constraint conditions to obtain the final target angle data, solving the problem of integral drift. The dynamic weights can adapt to changes in motion states, thereby improving the dynamic responsiveness. In addition, through timestamp alignment for spatio-temporal synchronization and self-calibration, the deviation during multi-angle data fusion is reduced, achieving more accurate high-precision data acquisition.

[0047] Figure 1 It is a flowchart of the joint measurement method for multi-sensor angle fusion provided by an embodiment of the present application.

[0048] The joint measurement method for multi-sensor angle fusion in an embodiment of the present application is applied to a joint measurement system for multi-sensor angle fusion. The joint measurement system for multi-sensor angle fusion includes multiple IMU (Inertial Measurement Unit) sensors, and each IMU sensor is configured with an independent adaptive Kalman filter.

[0049] As Figure 1 shown, the joint measurement method for multi-sensor angle fusion in an embodiment of the present application may specifically include:

[0050] S101. For the target joint of the patient, based on the IMU sensor, collect the first angle data of the patient's target joint and perform timestamp alignment on the first angle data.

[0051] S102. Perform incremental angle calibration on the first angle data, fuse to obtain the second angle data, and perform dynamic weight fusion based on the weights of the IMU sensors and the second angle data to obtain the corresponding third angle data.

[0052] S103. Perform sliding window filtering on the third angle data. In response to the filtered third angle data exceeding the preset biomechanical constraint conditions, fuse the biomechanical constraint conditions and the third angle data for biomechanical correction to obtain the target angle data of the target joint.

[0053] In the above-mentioned joint measurement method for multi-sensor angle fusion, angle data is collected through an IMU sensor and hardware timestamp alignment is performed. After self-calibrating the angle data, dynamic weight fusion is carried out, and biomechanical correction is performed on the angle data through biomechanical constraint conditions to obtain the final target angle data, solving the problem of integral drift. The dynamic weight can adapt to changes in the motion state, thereby improving the dynamic responsiveness. In addition, through the spatio-temporal synchronization and self-calibration of timestamp alignment, the deviation during multi-angle data fusion is reduced, achieving more accurate high-precision data collection.

[0054] The above exemplary steps of the embodiments of the present application will be described below in conjunction with specific examples:

[0055] S101, for the target joint of the patient, based on the IMU sensor, collect the first angle data of the target joint of the patient and perform timestamp alignment on the first angle data.

[0056] It can be added that the adaptive Kalman filter based on the IMU sensor optimizes the quality of the first angle data in real time at a preset frequency, that is, the adaptive Kalman filter in the IMU sensor is used for embedded filtering acquisition to improve the data quality. Here, the present application adopts a distributed dynamic filtering architecture, and each IMU is equipped with an independent adaptive Kalman filter to optimize the data quality of each sensor in real time. For example, 125Hz filtering acquisition is achieved through the adaptive Kalman filter.

[0057] In the embodiments of the present application, the target joint is the joint for which the patient's angle is measured. For example, the spine, and the first angle data is the angle data of the target joint collected by the IMU sensor. The first angle data includes the angle data collected by each IMU sensor. When measuring the angle of the target joint of the patient, the IMU sensor is arranged at the target position corresponding to the target joint (for example, the spine joint corresponds to four points, that is, 4 positions, and the IMU sensor is arranged at these positions), and the first angle data of the target joint of the patient is collected through the IMU sensor, and timestamp alignment is performed on the first angle data for subsequent processing. For example, as Figure 2 shown, specifically through the adaptive Kalman filter of the IMU sensor for embedded filtering acquisition.

[0058] It should be noted that each IMU sensor includes a high-precision multi-axis motion capture device with a target accuracy, and the multi-axis motion capture device has a target attitude solution accuracy; the multi-axis motion capture device includes a target accelerometer, a target gyroscope, and a target Gaussian magnetometer; the target accelerometer, the target gyroscope, and the target Gaussian magnetometer achieve the target attitude solution accuracy of the multi-axis motion capture device.

[0059] For example, each IMU sensor includes a high-precision 9-axis motion capture device, which uses ICM-42688 (±16g accelerometer / ±2000dps gyroscope) + QMC5883L (±8 gauss magnetometer) to achieve an attitude solution accuracy of 0.1°.

[0060] Thus, the present application collects data through the IMU sensor, realizes the popularization of low-cost motion capture, and reduces the cost compared with the conventional optical motion capture system.

[0061] S102. Perform incremental angle calibration on the first angle data, fuse to obtain the second angle data, and perform dynamic weight fusion based on the weight of the IMU sensor and the second angle data to obtain the corresponding third angle data.

[0062] In the embodiment of the present application, each IMU sensor has a different weight, and the weight of each IMU sensor is determined by the positional relationship of the IMU sensor relative to the target joint; perform incremental angle calibration on the first angle data collected by each IMU sensor in step S101 to obtain the second angle data, and perform dynamic weight fusion on the weight of the IMU sensor and the second angle data to obtain the corresponding third angle data for subsequent processing. For example, as Figure 2 shown, after data collection, incremental angle calibration and dynamic weight fusion are performed.

[0063] Optionally, when performing dynamic weight fusion based on the weight of the IMU sensor and the second angle data, adjust the weight of the IMU sensor based on the second angle data to obtain the adjusted weight of the IMU sensor; perform dynamic weight fusion on the adjusted weight of the IMU sensor and the second angle data.

[0064] Specifically, the weight of the IMU sensor is dynamically adjusted by the second angle data of the above embodiment, and the adjusted weight of the IMU sensor and the second angle data are dynamically weight-fused.

[0065] Thus, by dynamically adjusting the fusion weights of each sensor, the adaptability problem of the fixed weight algorithm is solved.

[0066] S103. Perform sliding window filtering on the third angle data. In response to the filtered third angle data exceeding the preset biomechanical limit condition, fuse the biomechanical limit condition and the third angle data for biomechanical correction to obtain the target angle data of the target joint.

[0067] In the embodiments of the present application, the target angle data is the final angle data of the target joint of the patient. The biomechanical constraint conditions at least include the natural movement range of the human joints, the anatomical constraints of each joint, and the constraints on the joint rotation angle. The filtered third angle data is obtained by performing a sliding window filter on the third angle data obtained after the dynamic weight fusion. When the filtered third angle data exceeds the biomechanical constraint conditions, the biomechanical constraint conditions and the third angle data are fused to perform biomechanical correction to obtain the target angle data of the target joint. For example, as Figure 2 shown, after performing a sliding window filter on the angle data obtained after the dynamic weight fusion, a determination is made based on the biomechanical constraint conditions as to whether it exceeds the limit. If it exceeds the limit, biomechanical correction is performed.

[0068] Specifically, the present application embeds the biomechanical constraint conditions into the data fusion to realize the optimization of kinematic constraints. By simulating the natural movement range of the human joints and the anatomical constraints of each joint, it is ensured that the calculated angle changes do not exceed the biologically reasonable range. Constraints on the joint rotation angle are added to the fusion to avoid angle changes that do not conform to the actual biomechanics and to avoid situations of excessive bending or stretching. When calculating the joint angle changes, the system will limit the calculation results based on the movement limits of each joint, so that the angle changes of each joint are within the acceptable biomechanical range. For example, the joint angle error is made <2° through the biomechanical constraint conditions.

[0069] Therefore, the present application introduces a joint constraint model (the natural movement range of the human joints, the anatomical constraints of each joint, and the constraints on the joint rotation angle) to perform real-time biomechanical correction on the third angle data, realizing the automatic correction of abnormal angle data without the need for manual correction through complex data processing software, and solving the problems of low efficiency and dependence on the operator's experience.

[0070] Optionally, in response to the filtered third angle data not exceeding the preset biomechanical constraint conditions, the third angle data is directly output as the target angle data of the target joint. For example, as Figure 2 shown, when the filtered angle data does not exceed the biomechanical constraint conditions, the angle data is directly output.

[0071] The joint measurement method for multi-sensor angle fusion provided by the embodiments of the present application is directed to the target joint of a patient. Based on the IMU sensor, the first angle data of the target joint of the patient is collected, and the first angle data is aligned with timestamps. The first angle data is incrementally angle calibrated, and the second angle data is obtained by fusion. Then, dynamic weight fusion is performed based on the weight of the IMU sensor and the second angle data to obtain the corresponding third angle data. The third angle data is filtered by a sliding window. In response to the filtered third angle data exceeding the preset biomechanical limit condition, the biomechanical limit condition and the third angle data are fused for biomechanical correction to obtain the target angle data of the target joint. The joint measurement method for multi-sensor angle fusion of the present application collects angle data through the IMU sensor and performs hardware timestamp alignment. After self-calibrating the angle data, dynamic weight fusion is performed, and biomechanical correction is performed on the angle data through the biomechanical limit condition to obtain the final target angle data, which solves the problem of integral drift. The dynamic weight can adapt to changes in the motion state, thereby improving the dynamic responsiveness. In addition, through the spatio-temporal synchronization and self-calibration of timestamp alignment, the deviation during multi-angle data fusion is reduced, and more accurate high-precision data acquisition is achieved.

[0072] Further, the sensor parameters of the IMU sensor are monitored in real time; the weights of each sensor are dynamically adjusted based on the sensor parameters.

[0073] Among them, the sensor parameters include at least but are not limited to signal-to-noise ratio and motion acceleration.

[0074] Specifically, by monitoring parameters such as the signal-to-noise ratio and motion acceleration of each sensor in real time, the sensor fusion weights are dynamically adjusted to achieve the sensor data fusion with dynamic weights, which can adapt to dynamic motion scenarios.

[0075] Further, validity verification is performed on the first angle data. The timestamps between all the first angle data are compared, and the valid data and invalid data in the first angle data are determined based on the comparison result of the timestamps. In response to discarding the invalid data in the first angle data, the first angle data is aligned with timestamps.

[0076] Among them, the valid data is the data in the first angle data with timestamps that are far apart and do not meet the preset standard, and the invalid data is the data in the first angle data with timestamps that are close and meet the preset standard.

[0077] Specifically, as Figure 2 shown, validity verification is performed on the collected angle data, and the invalid data is discarded during verification, the valid data is retained, and the valid data is aligned with timestamps.

[0078] Optionally, each IMU sensor is embedded with a high-precision clock module with independent target accuracy. When aligning the timestamps of the first angle data, when collecting the first angle data of the target joint of the patient based on the IMU sensor, the high-precision clock module adds the corresponding original hardware timestamp to each frame of the first angle data; in the initialization stage when collecting the first angle data of the target joint of the patient based on the IMU sensor, clock deviation calibration is performed based on the preset system unified reference master clock and the original hardware timestamp to align the timestamps of the first angle data.

[0079] Specifically, when each IMU sensor collects data, its own high-precision clock module adds the corresponding original hardware timestamp (not the received timestamp) to each frame of the first angle data. The system sets a unified reference master clock (such as the host computer system time or the main control board RTC), and clock deviation calibration is performed in the initialization stage (calibration stage) at the start of data collection. Thus, the timestamp alignment of the first angle data is achieved. For example, the spatio-temporal alignment error of multiple sensors is less than 0.5 ms.

[0080] Thus, the multi-sensor delay error is eliminated through hardware timestamp alignment, and spatio-temporal synchronization compensation is achieved. Further, the multi-sensor angle fusion joint measurement system includes a transmission device, and the transmission device includes a target wireless transmission module and a target low-power Bluetooth module. After the first angle data is collected, the wireless transmission module transmits the first angle data based on a preset first transmission frequency; or, after the first angle data is collected, the target low-power Bluetooth module transmits the first angle data based on a preset second transmission frequency. Among them, the transmission delay of the target low-power Bluetooth module at the second transmission frequency meets the preset transmission delay standard.

[0081] For example, in this application, real-time wireless transmission or synchronous sampling at 33 Hz under the BLE 5.2 (a Bluetooth low-power technology) protocol of nRF52832 (a low-power Bluetooth system-on-chip) is performed. At this time, the transmission frequency of data upload is 33 Hz. According to work experience, the transmission delay is less than 15 ms.

[0082] In addition, this application can use a high-precision clock of 400 kHz for the I2C (Inter-Integrated Circuit bus) bus to ensure zero-conflict acquisition of sensor data.

[0083] Further, it is determined whether the patient is wearing the IMU sensor for the first time; if the patient is wearing the IMU sensor for the first time, biomechanical adjustment is performed on the patient's IMU sensor to calibrate the IMU sensor.

[0084] Thus, this application can perform appropriate biomechanical adjustment when the sensor is worn for the first time and maintain consistency during subsequent movements, ensuring the accuracy and practicality of the results.

[0085] Further, corresponding joint evaluation quantization indexes are determined based on the target angle data of the patient's target joint; the patient's target joint is evaluated based on the joint evaluation quantization indexes to obtain the evaluation result of the target joint. Among them, the joint evaluation quantization index is the quantization index for evaluating the patient's target joint.

[0086] For example, quantization indexes such as the maximum joint angle and the minimum joint angle of scoliosis in the process of the patient's bending movement are determined according to the target angle data of the patient's spine, and the scoliosis of the patient's spine is evaluated according to these quantization indexes.

[0087] Further, an intelligent environment anti-interference module can be obtained, and deep learning is used to identify and remove abnormal data such as electromagnetic interference in the angle data.

[0088] In summary, through multi-sensor spatio-temporal alignment and kinematic constraint optimization, and combined with an adaptive Kalman filter, the present application realizes real-time high-precision joint measurement, providing reliable data support for motion rehabilitation. The present application focuses on the "dynamic priority fusion algorithm" and the "self-calibrated joint motion model", that is, the dynamic weight fusion and a series of calibrations of the angle data. First, by real-time monitoring parameters such as the signal-to-noise ratio and motion acceleration of each sensor, the fusion weight is dynamically adjusted instead of adopting a fixed weight strategy; second, a motion model including the physiological limitations of human joints is established to automatically correct the angle data beyond the normal range. This idea of software and hardware collaborative optimization breaks through the limitation of the traditional solution that only focuses on a single technical link.

[0089] The embodiment of the present application also provides a joint measurement system for multi-sensor angle fusion. Among them, the joint measurement system for multi-sensor angle fusion includes a plurality of IMU sensors, and each IMU sensor is configured with an independent adaptive Kalman filter.

[0090] The above-mentioned joint measurement system for multi-sensor angle fusion is used to execute the above-mentioned joint measurement method for multi-sensor angle fusion.

[0091] The joint measurement system with multi-sensor angle fusion provided by the embodiments of the present application aims at the target joint of a patient. Based on the IMU sensor, it collects the first angle data of the patient's target joint, aligns the time stamps of the first angle data, performs incremental angle calibration on the first angle data, fuses to obtain the second angle data, and performs dynamic weight fusion based on the weight of the IMU sensor and the second angle data to obtain the corresponding third angle data. It performs sliding window filtering on the third angle data. In response to the filtered third angle data exceeding the preset biomechanical limit condition, it fuses the biomechanical limit condition and the third angle data for biomechanical correction to obtain the target angle data of the target joint. The joint measurement system with multi-sensor angle fusion of the present application collects angle data and performs hardware time stamp alignment through the IMU sensor, performs dynamic weight fusion after self-calibrating the angle data, and performs biomechanical correction on the angle data through the biomechanical limit condition to obtain the final target angle data, solving the problem of integral drift. The dynamic weight can adapt to changes in the motion state, thereby improving the dynamic responsiveness. In addition, through the space-time synchronization and self-calibration of time stamp alignment, the deviation during multi-angle data fusion is reduced, realizing more accurate high-precision data collection.

[0092] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, which will not be repeated in the present application. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or modules can be in an electrical, mechanical, or other form.

[0093] The modules described as separate components may or may not be physically separated. The components displayed as modules may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0095] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the deployment method described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0096] The above are only specific implementation manners of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A joint measurement method for multi-sensor angle fusion, applied to a joint measurement system for multi-sensor angle fusion. The joint measurement system for multi-sensor angle fusion includes multiple IMU sensors, and each IMU sensor is configured with an independent adaptive Kalman filter; characterized in that, The method includes: For the target joint of the patient, based on the IMU sensor, collect the first angle data of the target joint of the patient, and perform timestamp alignment on the first angle data; Perform incremental angle calibration on the first angle data, fuse to obtain second angle data, and perform dynamic weight fusion based on the weight of the IMU sensor and the second angle data to obtain the corresponding third angle data; Perform sliding window filtering on the third angle data. In response to the filtered third angle data exceeding the preset biomechanical limit condition, fuse the biomechanical limit condition and the third angle data to perform biomechanical correction to obtain the target angle data of the target joint.

2. The method according to claim 1, wherein The method further includes: Real-time monitor the sensor parameters of the IMU sensor; wherein, the sensor parameters at least include signal-to-noise ratio and motion acceleration; Dynamically adjust the weights of each sensor based on the sensor parameters.

3. The method according to claim 2, wherein The method further includes: Perform validity verification on the first angle data, compare the timestamps between all the first angle data, and determine the valid data and invalid data in the first angle data based on the comparison result of the timestamps; In response to discarding the invalid data in the first angle data, perform timestamp alignment on the first angle data.

4. The method according to claim 3, characterized in that, Each IMU sensor is embedded with an independent high-precision clock module with target precision; the performing timestamp alignment on the first angle data includes: When collecting the first angle data of the target joint of the patient based on the IMU sensor, add the corresponding original hardware timestamp to each frame of the first angle data through the high-precision clock module; In the initialization stage when collecting the first angle data of the target joint of the patient based on the IMU sensor, perform clock deviation calibration based on the preset system unified reference master clock and the original hardware timestamp to perform timestamp alignment on the first angle data.

5. The method according to claim 4, wherein The performing dynamic weight fusion based on the weight of the IMU sensor and the second angle data includes: Adjust the weight of the IMU sensor based on the second angle data to obtain the adjusted weight of the IMU sensor; Perform dynamic weight fusion on the adjusted weight of the IMU sensor and the second angle data.

6. The method according to claim 5, characterized in that, Each IMU sensor includes a high-precision multi-axis motion capture device with target precision, and the multi-axis motion capture device has target attitude solution precision; The multi-axis motion capture device includes a target accelerometer, a target gyroscope, and a target Gaussian magnetometer; the target accelerometer, the target gyroscope, and the target Gaussian magnetometer achieve the target attitude solution precision of the multi-axis motion capture device.

7. The method according to claim 6, characterized in that The multi-sensor angle fusion joint measurement system includes a transmission device, and the transmission device includes a target wireless transmission module and a target low-power Bluetooth module. The method further includes: After collecting the first angle data, the wireless transmission module transmits the first angle data based on a preset first transmission frequency; or, After collecting the first angle data, the target low-power Bluetooth module transmits the first angle data based on a preset second transmission frequency; wherein, the transmission delay of the target low-power Bluetooth module at the second transmission frequency meets a preset transmission delay standard.

8. The method according to claim 7, characterized in that, The method further includes: Determining whether the patient is wearing the IMU sensor for the first time; If the patient is wearing the IMU sensor for the first time, biomechanically adjusting the patient's IMU sensor to calibrate the IMU sensor.

9. The method according to claim 8, wherein The method further includes: Determining a corresponding joint evaluation quantification index based on the target angle data of the target joint of the patient; Evaluating the target joint of the patient based on the joint evaluation quantification index to obtain an evaluation result of the target joint.

10. A joint measurement system with multi-sensor angle fusion, characterized in that, The multi-sensor angle fusion joint measurement system includes multiple IMU sensors, and each IMU sensor is configured with an independent adaptive Kalman filter; The multi-sensor angle fusion joint measurement system is used to execute the multi-sensor angle fusion joint measurement method according to any one of claims 1-9.

Images