Method for realizing real-time dynamic capture of three-dimensional space of human spine by applying IMU (Inertial Measurement Unit)

Abstract

The invention relates to the technical field of motion capture and data processing, in particular to a method for realizing real-time dynamic capture of a three-dimensional space of a human spine by applying an IMU (Inertial Measurement Unit). According to the method, a personalized three-dimensional dynamic spine model is constructed based on CT image data of a user, and a mathematical model of the spine is established by fusing a physiological anatomical structure of the spine and a robot mechanism motion theory so as to accurately describe motion characteristics of the spine. An IMU (inertial measurement unit) sensor is adhered to a key part of the back of a user, an individualized spine model is driven by acquiring and processing IMU data in real time, real-time reconstruction of dynamic movement of the spine is realized, and space angles among vertebral sections of the spine are accurately calculated. The dynamic change of the spine in the movement process can be accurately captured, the movement posture of the spine can be deeply analyzed, and the movement effect of the spine can be accurately and visually tracked and quantified in real time.

Description

Technical Field

[0001] The present invention mainly relates to motion capture and data processing technology methods, in particular to a method for using IMU to achieve real-time dynamic capture of the human spine in three-dimensional space. Background Art

[0002] With the continuous development of modern sports science, the relationship between spinal health and athletic performance is gaining increasing attention. The spine is not only a supporting structure for the human body but also directly affects athletic ability, posture, and health. In recent years, athletes have increasingly demanded dynamic spinal monitoring to help optimize posture, improve performance, and avoid ineffective training. However, current sports monitoring technologies mostly focus on single static posture analysis or rely on traditional imaging techniques, which have certain limitations in their application to dynamic sports.

[0003] Traditional methods of dynamic spinal monitoring rely primarily on costly imaging techniques (such as X-rays, CT scans, and MRIs). While these techniques can provide detailed information about spinal structure, they are difficult to achieve real-time dynamic monitoring during actual motion and impose significant limitations and inconvenience on users. Furthermore, some surface marker-based motion analysis methods, such as VICON, while capable of monitoring spinal changes during motion to a certain extent, suffer from errors due to the influence of the external environment, marker placement, and device accuracy, making it difficult to fully and accurately capture the dynamic changes of the spine during complex motion. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for measuring angles based on a dynamic spine model to solve the above problems.

[0005] A method for real-time dynamic capture of the three-dimensional space of the human spine using an IMU is characterized by: first, to achieve personalized three-dimensional modeling of the spine, a three-dimensional model of the human spine is reconstructed using CT, and based on the reconstructed three-dimensional model, relevant features of the spine are extracted, including the upper and lower end faces of the vertebral body, the center of the vertebral body, the center of mass of the spine, the vertebral body height, the intervertebral disc height and the spatial position of the spine of the reconstructed spine model are extracted, and the reconstructed three-dimensional spine model is input into a real-time dynamic spine interaction system. Secondly, the relevant angles of the human back are obtained using the IMU, and the IMU data processing method based on human body characteristics is applied to obtain the spatial rotation angle of each spine. Then, a multi-series bionic personalized spine model based on robotics is constructed, and the height and angle information extracted from the reconstructed spine model are converted into structural parameters of a mechanical arm, corresponding to the rod length and the ball pair angle respectively. The spine is abstracted as a specific mechanical arm model, and the angle information obtained using the IMU is input into the mechanical arm model to drive the overall mechanical arm model movement, thereby obtaining the spatial position of each rod member. Finally, the spatial position information of the corresponding vertebra is input into the real-time dynamic spine interaction system to control the movement of each spine in space and obtain a real-time three-dimensional dynamic spine model. The advantages of the present invention are that it can establish a real-time visual three-dimensional dynamic model of the spine and accurately evaluate spine-related parameters. It does not require any ionizing radiation, is low-cost, and is convenient and easy to deploy, making it suitable for spinal training and evaluation scenarios.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] The method is characterized in that, first, in order to realize personalized three-dimensional modeling of the spine, anatomical information of the spine is extracted using CT images. The image output by CT is a grayscale image, and the theoretical range of grayscale values is: usually -1000HU to 3000+HU (in the absence of metal implants), which depends on the model and calibration of the CT equipment. The value range for bone tissue is generally 300 to 2000+HU. By limiting the grayscale value to the range of 300 to 1000+HU, all bone tissues in the CT image are segmented and extracted, and noise reduction processing is performed. Excess bone tissues, such as the skull, ribs, cervical vertebrae and pelvis, are removed, and finally a complete spinal skeletal model is obtained. Subsequently, the spinal model is segmented, a single segment of the spine is extracted, and the mesh is optimized and smoothed to improve the morphological accuracy and continuity of the model.

[0008] A uniform downsampling method is used to reduce the density of the spine data while retaining the characteristic points on the vertebral surface. The upper and lower end faces of the vertebra are then extracted, which usually correspond to the areas with high density distribution in the point cloud.

[0009] The density-based spatial clustering algorithm (DBSCAN) is used to perform cluster analysis on the point cloud and filter it based on geometric features, as shown in Figure (c). First, the normal vector of the vertebral surface is used as the initial screening condition. The black line represents the point cloud vector V of each point in the point cloud. cloud , calculated by the least squares method in the neighborhood. Establish a reference vector, which is defined as the line connecting the center points of adjacent vertebral point clouds, which is the reference vector V reference .

[0010] Analyze point cloud vector V cloud With the reference vector V reference The angle λ between them.

[0011]

[0012] According to the geometric characteristics of the upper and lower end surfaces of the vertebral body, points with angles close to 0° (normal vector facing the center point of the spine) or 180° (normal vector facing away from the center point of the spine) are screened out to capture the characteristic areas of the upper and lower end surfaces.

[0013] DBSCAN is applied to the selected point set to distinguish the upper and lower end surface point cloud areas from other point cloud areas. Subsequently, the two clusters with the largest number of points are extracted through the DBSCAN clustering results, corresponding to the point sets of the upper and lower end surfaces of the vertebral body, respectively, as shown in Figure (d). For the i-th segment of the spine, the value of i ranges from 1 to 18, corresponding to the order of each vertebra in the spine. Specifically, i = 18 corresponds to the sacrum (S1), i = 17 corresponds to the 5th lumbar vertebra (L5), i = 16 corresponds to the 4th lumbar vertebra (L4), i = 15 corresponds to the 3rd lumbar vertebra (L3), i = 14 corresponds to the 2nd lumbar vertebra (L2), i = 13 corresponds to the 1st lumbar vertebra (L1), i = 12 corresponds to the 12th thoracic vertebra (T12), i = 11 corresponds to the 11th thoracic vertebra (T11), i = 10 corresponds to the 10th thoracic vertebra (T10), i=9 corresponds to the 9th thoracic vertebra (T9), i=8 corresponds to the 8th thoracic vertebra (T8), i=7 corresponds to the 7th thoracic vertebra (T7), i=6 corresponds to the 6th thoracic vertebra (T6), i=5 corresponds to the 5th thoracic vertebra (T5), i=4 corresponds to the 4th thoracic vertebra (T4), i=3 corresponds to the 3rd thoracic vertebra (T3), i=2 corresponds to the 2nd thoracic vertebra (T2), and i=1 corresponds to the 1st thoracic vertebra (T1).

[0014] Calculate the centroid of the upper end face and the centroid of the lower end face

[0015]

[0016] Among them, n1 is the number of points concentrated on the upper end surface, p e-up is the coordinate of the e-th point on the upper end surface, (c up-x ,c up-y ,c up-z ) is the spatial coordinate of the centroid of the upper end face point set.

[0017]

[0018] Among them, n2 is the number of points concentrated on the lower end surface, p h-down is the coordinate of the hth point on the lower end surface, (c down-x ,c down-y ,c down-z ) is the spatial coordinate of the centroid of the lower end face point set.

[0019] By calculating the line connecting the centroids of the upper and lower end faces, the vector of the i-th segment of the spine is obtained.

[0020]

[0021] Thus, the model pose M of the i-th spine in the initial state is obtained i ,

[0022]

[0023] The invention is characterized in that, secondly, an IMU data processing method based on human body characteristics is applied, and six IMU devices are attached to the corresponding spinal positions on the back of the user to record the spinal movement status in real time. The IMU miniaturized sensor integrates an accelerometer, a gyroscope and a magnetometer, and can measure linear acceleration, angular velocity and magnetic field strength. During the acquisition process, six IMU devices are attached to key positions on the midline of the spine (the back area corresponding to the spinous processes of the sacrum (S1), the third lumbar vertebra (L3), the 12th thoracic vertebra (T12), the eighth thoracic vertebra (T8), the fourth thoracic vertebra (T4) and the first thoracic vertebra (T1) are selected as the area for fixing the IMU), and the IMU data processing method based on human body characteristics is applied to process the acquired IMU data.

[0024] S1 is the bottom of the spine, connecting the spine to the pelvis. It has high stability, so as a fixed point, it can provide a stable reference. L3 is in the lumbar area and is a more flexible but supportive part of the spine. By monitoring this segment, the movement pattern of the waist can be effectively tracked. T12 is the transition area between the thoracic and lumbar vertebrae. Its position is relatively fixed and it is connected to the thorax, which has a greater impact on the rotation and bending of the spine. T8, T4, and T1 are several key segments in the thoracic spine. They are located at different thoracic levels, which help to capture the movement characteristics of the upper spine (especially the thoracic spine). The thoracic spine is relatively stable, and rotation and scoliosis of the spine mainly occur in these areas.

[0025] Since the data processing for any two adjacent IMUs is identical, only the data processing for the first and second IMUs will be described. A spine consisting of five vertebrae is used as an example, with the first IMU attached to the top and the second IMU attached to the bottom. Quaternion interpolation is used to obtain the rotation angle of the spine when the IMU is not being worn. Quaternions are processed using the spherical linear interpolation (SLerp) method.

[0026] The rotation angle information quaternion q is obtained through the first IMU imu1 , obtain the rotation angle information quaternion q through the second IMU imu2 . Use spherical linear interpolation to obtain the intermediate variables of the quaternion:

[0027]

[0028] in The value of vtb-i ranges from 1 to 3.

[0029] The value range of t in the formula is 0-1, which represents the relative position of the interpolation between the initial value and the final value. The distance between the centers of the lower end faces of the vertebrae of each segment of the spine is constructed. The distance between the centers of the lower end faces of the vertebrae between the kth segment and the k+1th segment is l k (k=1-4)

[0030]

[0031] Find the t value:

[0032]

[0033] Based on this, each of the 17 spine segments corresponds to a change quaternion. For the fixed IMU sensor, the quaternion obtained by the IMU sensor is the corresponding change quaternion. For example, the 12th thoracic vertebra (T12) is fixed with the third IMU, that is,

[0034] For the i-th segment of the spine, the transformation quaternion can be expressed as:

[0035]

[0036] in, represents the real part of the transformed quaternion, is the imaginary part of the transformed quaternion.

[0037] Variation Quaternion Can be converted into rotation angles around three orthogonal axes,

[0038]

[0039] in Represents winding z s The angle of axis rotation, Represents rotation around y s The angle of axis rotation, Represents around x s The angle of axis rotation.

[0040] Similarly, the deformation angle of the intervertebral disc slice after the intervertebral disc is divided can be obtained. Taking the intervertebral disc sandwiched by the two vertebrae at the bottom as an example, it is divided into 7 layers at equal intervals according to the height of the intervertebral disc. Since the distance of each layer is set to be equal, the quaternion interpolation parameters of each layer can be directly obtained. From bottom to top they are Repeat the above steps to obtain the rotation angle of each layer of intervertebral disc around the three orthogonal axes. Represents the rotation angle of the jth layer of the intervertebral disc at the lower end of the i-th vertebra, where j ranges from 0 to 6.

[0041] A dynamic three-dimensional spinal model was constructed using previously collected and fused data. A multi-chain bionic personalized spinal model, built based on robotics, abstracts the spinal bones and intervertebral disc structures into corresponding joints in the robot. The spinal vertebrae correspond to the rods in the robot, and each intervertebral disc corresponds to a structure with six ball joints in series. The entire spine contains 16 intervertebral discs, modeled as an open-chain robotic arm mechanism consisting of 16 × 6 = 96 ball joints in series. The exponential product method is used to construct the robot's forward kinematics.

[0042] For the convenience of calculation, any spherical pair in space is further transformed into three orthogonal rotation pairs, with the base coordinate system {s} as the origin, {s} fixed at the center of the sacrum of the spine, and the jth spherical pair SP of the i-th vertebra i-j The three orthogonal rotation pairs R i-j-1 ,R i-j-2 ,R i-j-3 , the rotation angles corresponding to these three revolute pairs are

[0043] To rotate the pair R i-j-1 For example, its spin is S i-j-1 =(ω i-j-1 v i-j-1 ), where ω i-j-1 It is the revolute pair R i-j-1 The space vector of the axis of rotation, which is the unit angular velocity, v i-j-1 is the unit linear velocity. in R is the revolute pair i-j-1 The space vector of the axis of rotation is x s ,y s ,z s The projection,

[0044] v i-j-1 =-ω i-j-1 ×q i-j-1 ,

[0045] where q i-j-1 R is the revolute pair i-j-1 The spatial coordinates of any point on the axis of rotation. For three orthogonal revolute pairs, the intersection of the axis of rotation is selected as q i-j-1

[0046] Screw S i-j-1 Expressed in the form of Lie algebra, its matrix representation is

[0047]

[0048] in ω i-j-1 The antisymmetric matrix form of

[0049] Substitute into the revolute pair R i-j-1 Corresponding joint angle Then there is

[0050]

[0051] in According to the Rodriguez formula,

[0052]

[0053] in is the identity matrix.

[0054] Further combining the above formula to obtain the forward kinematics of the robot based on the exponential product method, substituting the joint angles corresponding to each revolute pair, for the i-th segment of the spine,

[0055]

[0056] Where T i It represents the new configuration of the i-th vertebra, that is, the position of the i-th vertebra in the base coordinate system {s}. When the human body stands naturally, the position of the vertebra is the initial position M i .

[0057] The real-time updated position of the spine T i Input into the real-time dynamic spine interactive system to obtain a real-time updated three-dimensional dynamic spine model.

[0058] The present invention and its preferred embodiment can be applied to the field of spinal motion monitoring and assessment training. It has the advantages of high precision, dynamic three-dimensional visualization of the spine, low cost, and ease of use. It can replace existing spinal three-dimensional modeling methods using X-rays, CT, MRI, three-dimensional laser scanners, and raster stereo imagers. It has the following advantages and uses:

[0059] 1. Realize 3D dynamic visualization of the spine. It can intuitively present the 3D structure and movement of the spine, allowing users to intuitively understand the movement and deformation of the spine, thus better assisting training.

[0060] 2. A personalized, precise spinal model. This system models and analyzes each user's unique spinal structure. By importing the user's CT scan data, the model accurately reconstructs each individual's spinal geometry, including the shape and size of the vertebrae, as well as the position and morphology of the intervertebral discs. This allows the spinal model to not only reflect the overall morphology of the spine but also to capture the details of each vertebral segment.

[0061] 3. Low cost and easy to use. This invention only requires six IMU sensors and an ordinary laptop computer, which is lower in cost than systems using 3D ultrasound probes, 3D scanners, and other equipment. Furthermore, one only needs to install Python software on the laptop computer, import the project package of this invention, import the user's CT model, and attach the IMU to the back to automatically achieve 3D spine visualization, eliminating the need for tedious manual operations. The product is highly deployable and universally applicable. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is the implementation process of the dynamic spine model method.

[0063] Figure 2 It is a CT-based spine feature extraction process.

[0064] Figure 3 This is a schematic diagram of the IMU data processing method based on human body characteristics.

[0065] Figure 4 It is a real-time dynamic spine interaction system. DETAILED DESCRIPTION

[0066] The present invention is described in detail below with reference to the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in this field based on these embodiments are all within the scope of protection of the present invention.

[0067] refer to Figures 1 to 4 As shown, Figure 1 This is the implementation process of the dynamic spine model method. Figure 2 It is a CT-based spine feature extraction process. Figure 3 This is a schematic diagram of the IMU data processing method based on human body characteristics. Figure 4 It is a real-time dynamic spine interaction system.

[0068] The origin of the base coordinate {s} is located at the center of mass of the sacrum o s , vertically upward is defined as z s The positive direction of the axis, parallel to the coronal axis and away from the front of the human body, is x s Axis positive direction, y s Axis perpendicular to x s o s z s Surface passes through o s The point pointing to the left hand is y s The positive direction of the axis.

[0069] The overall implementation process is divided into four steps: data collection, multimodal data fusion, dynamic spine model, and scenario application.

[0070] First, the user's CT image is obtained, and the CT spine feature extraction method is applied. It is imported into the real-time dynamic spine interactive system to generate a three-dimensional static spine to complete the imaging data collection.

[0071] refer to Figure 2 ,In order to realize personalized 3D modeling of the spine, CT images are used to extract the anatomical information of the spine, such as Figure 2As shown in (a). The image output by CT is a grayscale image. The theoretical range of grayscale values is usually -1000HU to 3000+HU (in the absence of metal implants), which depends on the model and calibration of the CT equipment. The value range for bone tissue is generally 300 to 2000+HU. By limiting the grayscale value to the range of 300 to 1000+HU, all bone tissues in the CT image are segmented and extracted, and noise reduction is performed. Excessive bone tissue, such as the skull, ribs, cervical vertebrae and pelvis, is removed to finally obtain a complete spinal skeletal model. Subsequently, the spinal model is segmented, a single segment of the spine is extracted, and the mesh is optimized and smoothed to improve the morphological accuracy and continuity of the model.

[0072] like Figure 2 As shown in (b), a uniform downsampling method is used to reduce the density of the spine data while retaining the feature points on the vertebral surface. The upper and lower end faces of the vertebra are then extracted, which usually correspond to areas with high density distribution in the point cloud.

[0073] The density-based clustering algorithm (DBSCAN) is used to perform cluster analysis on the point cloud and filter it in combination with geometric features, such as Figure 2 (c) First, the normal vector of the vertebral surface is used as the initial screening condition, and the black line represents the point cloud vector V of each point in the point cloud. cloud , calculated by the least squares method in the neighborhood. Establish a reference vector, which is defined as the line connecting the center points of adjacent vertebral point clouds, which is the reference vector V reference .

[0074] Analyze point cloud vector V cloud With the reference vector V reference The angle λ between them.

[0075]

[0076] According to the geometric characteristics of the upper and lower end surfaces of the vertebral body, points with angles close to 0° (normal vector facing the center point of the spine) or 180° (normal vector facing away from the center point of the spine) are screened out to capture the characteristic areas of the upper and lower end surfaces.

[0077] Apply DBSCAN to the selected point set to distinguish the upper and lower end face point cloud areas from other point cloud areas. Then, the two clusters with the largest number of points are extracted through the DBSCAN clustering results, corresponding to the point sets of the upper and lower end faces of the vertebra, respectively, as shown in the figure below. Figure 2As shown in (d), for the i-th vertebra, the value of i ranges from 1 to 17, corresponding to the order of the vertebrae in the spine. Specifically, i = 17 corresponds to the first lumbar vertebra (L1), i = 16 corresponds to the second lumbar vertebra (L2), i = 15 corresponds to the third lumbar vertebra (L3), i = 14 corresponds to the fourth lumbar vertebra (L4), i = 13 corresponds to the fifth lumbar vertebra (L5), i = 12 corresponds to the twelfth thoracic vertebra (T12), i = 11 corresponds to the eleventh thoracic vertebra (T11), and i = 10 corresponds to the tenth thoracic vertebra (T10). ), i=9 corresponds to the 9th thoracic vertebra (T9), i=8 corresponds to the 8th thoracic vertebra (T8), i=7 corresponds to the 7th thoracic vertebra (T7), i=6 corresponds to the 6th thoracic vertebra (T6), i=5 corresponds to the 5th thoracic vertebra (T5), i=4 corresponds to the 4th thoracic vertebra (T4), i=3 corresponds to the 3rd thoracic vertebra (T3), i=2 corresponds to the 2nd thoracic vertebra (T2), and i=1 corresponds to the 1st thoracic vertebra (T1).

[0078] Calculate the centroid of the upper end face and the centroid of the lower end face

[0079]

[0080] Among them, n1 is the number of points concentrated on the upper end surface, p e-up is the coordinate of the e-th point on the upper end surface, (c up-x ,c up-y ,c up-z ) is the spatial coordinate of the centroid of the upper end face point set.

[0081]

[0082] Among them, n2 is the number of points concentrated on the lower end surface, p h-down is the coordinate of the hth point on the lower end surface, (c down-x ,c down-y ,c down-z ) is the spatial coordinate of the centroid of the lower end face point set.

[0083] By calculating the line connecting the centroids of the upper and lower end faces, the vector of the i-th segment of the spine is obtained.

[0084]

[0085] Thus, the model pose M of the i-th spine in the initial state is obtained i ,

[0086]

[0087] At the same time, six IMU devices are attached to the user's back at corresponding spinal locations to record spinal motion in real time. The IMU's miniaturized sensor integrates an accelerometer, gyroscope, and magnetometer, capable of measuring linear acceleration, angular velocity, and magnetic field strength. During the acquisition process, the six IMU devices are attached to key locations along the spinal midline (the sacrum (S1), the third lumbar vertebra (L3), the 12th thoracic vertebra (T12), the eighth thoracic vertebra (T8), the fourth thoracic vertebra (T4), and the first thoracic vertebra (T1) are selected as the dorsal regions corresponding to the spinous processes of the vertebrae). The acquired IMU data is processed using an IMU data processing method based on human characteristics.

[0088] S1 is the bottom of the spine, connecting the spine to the pelvis. It has high stability, so as a fixed point, it can provide a stable reference. L3 is in the lumbar area and is a more flexible but supportive part of the spine. By monitoring this segment, the movement pattern of the waist can be effectively tracked. T12 is the transition area between the thoracic and lumbar vertebrae. Its position is relatively fixed and it is connected to the thorax, which has a greater impact on the rotation and bending of the spine. T8, T4, and T1 are several key segments in the thoracic spine. They are located at different thoracic levels, which help to capture the movement characteristics of the upper spine (especially the thoracic spine). The thoracic spine is relatively stable, and rotation and scoliosis of the spine mainly occur in these areas.

[0089] Since the data processing of any two adjacent IMUs is the same, only the implementation method for the first and second IMU data is introduced. Figure 3 ,like Figure 3 As shown in (a), a five-vertebra spine is shown, with the first IMU fixed at the top and the second at the bottom. Quaternion interpolation is used to obtain the rotation angle of the spine without the IMU. Spherical Linear Interpolation (SLerp) is used to process the quaternions.

[0090] The rotation angle information quaternion q is obtained through the first IMU imu1 , obtain the rotation angle information quaternion q through the second IMU imu2 . Use spherical linear interpolation to obtain the intermediate variables of the quaternion:

[0091]

[0092] in The value of vtb-i ranges from 1 to 3.

[0093] The value range of t in the formula is 0-1, which represents the relative position of the interpolation between the initial value and the final value. The distance between the centers of the lower end faces of the vertebrae of each segment of the spine is constructed. The distance between the centers of the lower end faces of the vertebrae between the kth segment and the k+1th segment is l k (k=1-4)

[0094]

[0095] Find the t value:

[0096]

[0097] Based on this, each of the 17 spine segments corresponds to a change quaternion. For the fixed IMU sensor, the quaternion obtained by the IMU sensor is the corresponding change quaternion. For example, the 12th thoracic vertebra (T12) is fixed with the third IMU, that is,

[0098] For the i-th segment of the spine, the transformation quaternion can be expressed as:

[0099]

[0100] in, represents the real part of the transformed quaternion, is the imaginary part of the transformed quaternion.

[0101] Variation Quaternion Can be converted into rotation angles around three orthogonal axes,

[0102]

[0103] in Represents winding z s The angle of axis rotation, Represents rotation around y s The angle of axis rotation, Represents around x s The angle of axis rotation.

[0104] Similarly, the deformation angle of the intervertebral disc slice after the intervertebral disc is divided can be obtained. Take the intervertebral disc sandwiched by the two vertebrae at the bottom as an example. Figure 3 As shown in (c), the intervertebral disc is divided into 7 layers at equal distances according to its height. Since the distance between each layer is set equal, the quaternion interpolation parameters of each layer can be directly obtained. From bottom to top they are Repeat the above steps to obtain the rotation angle of each layer of intervertebral disc around the three orthogonal axes. Represents the rotation angle of the jth layer of the intervertebral disc at the lower end of the i-th vertebra, where j ranges from 0 to 6.

[0105] Multimodal data fusion involves two main aspects: real-time processing of sensor data and processing of CT imaging data. This combined with feature modeling of CT imaging data creates a complete data fusion framework. Sensor data provides dynamic motion information about the spine, while CT imaging data provides precise static structural information.

[0106] A dynamic three-dimensional spinal model was constructed using previously collected and fused data. A multi-chain bionic personalized spinal model, built based on robotics, abstracts the spinal bones and intervertebral disc structures into corresponding joints in the robot. The spinal vertebrae correspond to the rods in the robot, and each intervertebral disc corresponds to a structure with six ball joints in series. The entire spine contains 16 intervertebral discs, modeled as an open-chain robotic arm mechanism consisting of 16 × 6 = 96 ball joints in series. The exponential product method is used to construct the robot's forward kinematics.

[0107] For the convenience of calculation, any spherical pair in space is further transformed into three orthogonal rotation pairs, with the base coordinate system {s} as the origin, {s} fixed at the center of the sacrum of the spine, and the jth spherical pair SP of the i-th vertebra i-j The three orthogonal rotation pairs R i-j-1 ,R i-j-2 ,R i-j-3 , the rotation angles corresponding to these three revolute pairs are

[0108] To rotate the pair R i-j-1 For example, its spin is S i-j-1 =(ω i-j-1 v i-j-1 ), where ω i-j-1 It is the revolute pair R i-j-1 The space vector of the axis of rotation, which is the unit angular velocity, v i-j-1 is the unit linear velocity. in R is the revolute pair i-j-1 The space vector of the axis of rotation is x s ,y s ,z s The projection,

[0109] v i-j-1 =-ω i-j-1 ×q i-j-1 ,

[0110] where q i-j-1 R is the revolute pair i-j-1 The spatial coordinates of any point on the axis of rotation. For three orthogonal revolute pairs, the intersection of the axis of rotation is selected as q i-j-1

[0111] Screw S i-j-1 Expressed in the form of Lie algebra, its matrix representation is

[0112]

[0113] in ω i-j-1 The antisymmetric matrix form of

[0114] Substitute into the revolute pair R i-j-1 Corresponding joint angle Then there is

[0115]

[0116] in According to the Rodriguez formula,

[0117]

[0118] in is the identity matrix.

[0119] Further combining the above formula to obtain the forward kinematics of the robot based on the exponential product method, substituting the joint angles corresponding to the revolute pairs, for the i-th segment of the spine,

[0120]

[0121] Where T i , represents the new configuration of the i-th vertebra, and represents the position of the i-th vertebra in the base coordinate system {s}. When the human body stands naturally, the position of the vertebra is the initial position M i .

[0122] refer to Figure 4 , Figure 4This is a real-time dynamic spinal interactive system, primarily divided into model visualization, dynamic data display, and operation areas. The implementation process is as follows: First, place the sensor in the charging dock and turn on the device. Then, click "Connect" to connect the sensor. Click "Reset_2" to calibrate the sensor. As the subject performs the prescribed movements, click "Start" to begin recording. After the movement is complete, click "Stop" to stop recording, and then click "Save" to save the data. The data is saved in CSV format, grouped by sensor. After the experiment is complete, click "Disconnect" to disconnect the sensors. To review historical data, use the buttons in the lower-right corner of the interface, click "Browse" to select previously stored data, and finally click "Replay" to replay and pause the data. The vertebral list on the right allows users to select specific spinal segments. This function not only provides information on the spatial angles between selected segments, but also allows for further analysis of the spine's motion and posture.

[0123] Among them, the "Reset_2" function uses statistical methods to establish a spinal model, and analyzes the changing angles of the corresponding positions of the spine and back in different postures of the human body (such as lying, standing, and flexed postures). It can effectively solve the errors caused by the reconstruction of the three-dimensional model by CT and the inference of in-vivo features by surface features. Through statistical analysis of a large amount of human spine data, the angle change rules under different postures are extracted, and the correction angle parameters from lying to standing are derived. At the same time, combined with the mapping relationship between the surface features of the back and the internal features of the spine, the angle correction parameters of the back mapped to the body are further proposed to optimize the accuracy of the model. This method not only makes up for the errors caused by the limitations of the scanning posture during CT reconstruction, but also improves the accuracy of inferring the spinal morphology in the body through surface features.

[0124] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

[0125] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present invention. The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and equivalent transformations or substitutions in function, method, or structure made by ordinary technicians in this field based on these embodiments are all within the scope of protection of the present invention.

Claims

1. A method for real-time dynamic capture of the three-dimensional space of the human spine using IMU, characterized in that include: Design of a multi-series bionic personalized spine model based on robotics; IMU data processing method based on human body characteristics; Real-time dynamic spine interaction program; The multi-serial bionic personalized spine model, built based on robotics, constructs a static 3D human spine model based on the user's CT data. By combining the forward kinematics theory of robotics with the anatomical structure of the multi-segmented spine, a highly bionic mathematical model is constructed. This model fully considers the multi-degree-of-freedom motion characteristics of the spine and the flexible connection relationships between the various segments. Through parametric design, it describes the posture and spatial position of the spine in both static and dynamic states. The human body-feature-based IMU data processing method performs spherical linear interpolation on the quaternion data output by the IMU, combines it with the physiological characteristics of each segment of the spine, and proposes personalized interpolation parameters. The quaternion information related to the movement of 17 vertebral segments and intervertebral discs is converted from 6 IMU data. This information is then applied to a multi-series bionic personalized spine model built based on robotics to drive the movement of the static model and further optimize the dynamic position of the spine and the IMU data. The real-time dynamic spine interaction program visualizes the multi-series bionic personalized spine model built based on robotics to achieve real-time monitoring of spinal posture and data analysis; the program uses IMU sensors to capture the spatial angle changes of the spine and converts them into a dynamic three-dimensional model to intuitively display the posture and angle of the spine in motion; by measuring the displacement of the first thoracic vertebra relative to the sacrum, the range of motion of the spine is quantified, compared with standard movements, and training movements are adjusted.

2. The method of claim 1 for realizing real-time dynamic capture of the three-dimensional space of the human spine using an IMU, characterized in that: A multi-series bionic personalized spine model based on robotics abstracts the spinal bones and intervertebral disc structures into corresponding joints in the robot. The spinal vertebrae correspond to the rods in the robot, and each intervertebral disc corresponds to a structure with six ball pairs in series. The entire spine contains 16 intervertebral discs, which are modeled as an open-chain robotic arm mechanism consisting of 16 × 6 = 96 ball pairs in series. The exponential product method is used to construct the robot's forward kinematics. For the convenience of calculation, any spherical pair in space is converted into three orthogonal rotation pairs, with the base coordinate system {s} as the origin, {s} fixed at the center of the sacrum of the spine, and the jth spherical pair SP of the i-th vertebra i-j The three orthogonal rotation pairs R i-j-1 ,R i-j-2 ,R i-j-3 , the rotation angles corresponding to these three revolute pairs are The value of i is 1-18 Specifically, i=18 corresponds to the sacrum (S1), i=17 corresponds to the fifth lumbar vertebra (L5), i=16 corresponds to the fourth lumbar vertebra (L4), i=15 corresponds to the third lumbar vertebra (L3), i=14 corresponds to the second lumbar vertebra (L2), i=13 corresponds to the first lumbar vertebra (L1), i=12 corresponds to the twelfth thoracic vertebra (T12), i=11 corresponds to the eleventh thoracic vertebra (T11), and i=10 corresponds to the tenth thoracic vertebra (T10). , i = 9 corresponds to the 9th thoracic vertebra (T9), i = 8 corresponds to the 8th thoracic vertebra (T8), i = 7 corresponds to the 7th thoracic vertebra (T7), i = 6 corresponds to the 6th thoracic vertebra (T6), i = 5 corresponds to the 5th thoracic vertebra (T5), i = 4 corresponds to the 4th thoracic vertebra (T4), i = 3 corresponds to the 3rd thoracic vertebra (T3), i = 2 corresponds to the 2nd thoracic vertebra (T2), i = 1 corresponds to the 1st thoracic vertebra (T1); the base coordinate {s} is located at the center of mass of the sacrum o s , vertically upward is defined as z s The positive direction of the axis, parallel to the coronal axis and away from the front of the human body, is x s Axis positive direction, y s Axis perpendicular to x s o s z s Surface passes through o s The point pointing to the left hand is y s The positive direction of the axis; For the revolute pair R i-j-1 , whose spin is S i-j-1 =(ω i-j-1 v i-j-1 ), where ω i-j-1 It is the revolute pair R i-j-1 The space vector of the axis of rotation, which is the unit angular velocity, v i-j-1 is the unit linear velocity; in R is the revolute pair i-j-1 The space vector of the axis of rotation is x s ,y s ,z s The projection of v i-j-1 =-ω i-j-1 ×q i-j-1 , where q i-j-1 R is the revolute pair i-j-1 The spatial coordinates of any point on the axis of rotation. For three orthogonal revolute pairs, the intersection of the axis of rotation is selected as q i-j-1 Screw S i-j-1 Expressed in the form of Lie algebra, its matrix representation is in ω i-j-1 The antisymmetric matrix form of Substitute into the revolute pair R i-j-1 Corresponding joint angle Then there is in According to the Rodriguez formula, in is the identity matrix; Further combining the above formula to obtain the forward kinematics of the robot based on the exponential product method, substituting the joint angles corresponding to each revolute pair, for the i-th segment of the spine, Where T i is the new configuration of the i-th vertebra, indicating the position of the i-th vertebra in the base coordinate system {s}; when the human body stands naturally, the position of the vertebra is the initial position M i .

3. The method of claim 1 for realizing real-time dynamic capture of the human spine in three dimensions using an IMU, characterized in that: Six IMU sensors were used to collect motion data from key parts of the spine. Through mathematical modeling and interpolation calculations, the posture changes of 18 major vertebral segments, including the thoracic, lumbar, and sacrum, during dynamic motion were calculated. During the acquisition process, six IMU devices were attached to key locations along the midline of the spine; the dorsal regions corresponding to the spinous processes of the sacrum (S1), the third lumbar vertebra (L3), the 12th thoracic vertebra (T12), the eighth thoracic vertebra (T8), the fourth thoracic vertebra (T4), and the first thoracic vertebra (T1) were selected as the areas for IMU fixation. Since the data processing for any two adjacent IMUs is the same, only the data from the first and second IMUs are described. A spine consisting of five vertebrae is used as an example, where the first IMU is fixed to the first vertebra at the top and the second IMU is fixed to the fifth vertebra at the bottom. Quaternion interpolation is used to obtain the rotation angle of the spine without the IMU. The spherical linear interpolation (SLerp) method is selected to process the quaternion. The rotation angle information quaternion q is obtained through the first IMU imu1 , obtain the rotation angle information quaternion q through the second IMU imu2 ; Use spherical linear interpolation to obtain the intermediate variables of the quaternion: in The value of vtb-i is 1-3; The value range of t in the formula is 0-1, which represents the relative position of the interpolation between the initial value and the final value, and the distance between the centers of the lower end faces of the vertebrae of each segment of the spine is constructed. The center of the lower end face of the vertebrae between the kth segment and the k+1th segment is The distance between the lines is l k (k=1-4) Find the t value: Based on this, each of the 17 spine segments corresponds to a change quaternion. For the fixed IMU sensor, the quaternion obtained by the IMU sensor is the corresponding change quaternion. For example, the 12th thoracic vertebra (T12) is fixed with the third IMU, that is, For the i-th segment of the spine, the transformation quaternion is expressed as: in, represents the real part of the transformed quaternion, is the imaginary part of the transformed quaternion; Variation Quaternion Converted into rotation angles around three orthogonal axes, in Represents winding z s The angle of axis rotation, Represents rotation around y s The angle of axis rotation, Represents around x s Angle of axis rotation; Similarly, the deformation angle of the intervertebral disc slice after the intervertebral disc is divided is obtained. Taking the intervertebral disc sandwiched by the two vertebrae at the bottom as an example, it is divided into 7 layers at equal intervals according to the height of the intervertebral disc. Since the distance of each layer is set to be equal, the quaternion interpolation parameters of each layer are directly obtained. From bottom to top they are Similarly, the rotation angles of the transformed quaternions of each layer of the intervertebral disc around the three orthogonal axes are obtained. Represents the rotation angle of the jth layer of the intervertebral disc at the lower end of the i-th vertebra, where j ranges from 0 to 6.

Images