Spinal alignment evaluation method and system based on optical motion capture and ultrasound imaging

Abstract

The application discloses a spine alignment evaluation method and system based on optical motion capture and ultrasonic imaging, relates to the cross field of medical imaging technology and motion tracking technology, and comprises the following steps: detecting the spine region of a patient to be measured based on an ultrasonic imaging device, collecting ultrasonic images of the spine region of the patient to be measured to obtain patient spine ultrasonic image information; acquiring an industrial camera array, the industrial camera array comprising a plurality of industrial cameras, fixedly tracking a preset optical reflective marker point on an ultrasonic probe of the ultrasonic imaging device based on the plurality of industrial cameras, and outputting six-degree-of-freedom pose information of the ultrasonic probe; and acquiring a calibration body, the ultrasonic imaging device performing multi-angle scanning on the calibration body to obtain multiple sets of calibration body ultrasonic image information, and synchronously triggering the industrial camera array to collect the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image collection scanning to obtain multiple probe poses To. The application has the effect of improving the spine alignment evaluation efficiency.

Description

Technical Field

[0001] This invention relates to the intersection of medical imaging technology and motion tracking technology, and in particular to a method and system for assessing spinal alignment based on optical motion capture and ultrasound imaging. Background Technology

[0002] Currently, the spine, as a vital support structure of the human body, directly impacts quality of life. Scoliosis, one of the most common spinal deformities, can lead to impaired cardiopulmonary function and chronic pain if left untreated. Commonly used diagnostic methods include X-ray films, CT scans, and MRI. However, X-rays and CT scans pose ionizing radiation risks, and prolonged and frequent use may increase the risk of cancer. While MRI is radiation-free, its high cost and complex operation make it unsuitable for large-scale screening. In recent years, three-dimensional ultrasound imaging has gained attention due to its safety, convenience, and repeatability. However, its inherent spatial positioning limitations—the inability to directly obtain the precise pose information of the ultrasound probe in three-dimensional space—lead to cumulative errors when stitching together multiple ultrasound images, severely affecting the accuracy and reliability of three-dimensional spinal reconstruction. Combining optical motion capture technology with ultrasound imaging technology improves the assessment of spinal alignment in patients. Therefore, spinal alignment assessment based on optical motion capture and ultrasound imaging is crucial.

[0003] In existing technologies, spinal alignment assessment methods based on optical motion capture and ultrasound imaging involve introducing optical motion capture systems into the field of ultrasound imaging, attempting to compensate for spatial position information by tracking markers on the probe. However, these solutions generally face the critical technical bottleneck of "hand-eye calibration": because the ultrasound image coordinate system and the optical coordinate system are heterogeneous coordinate systems, the transformation relationship between them is difficult to model accurately, resulting in large calibration errors, which in turn affect the accuracy of subsequent image registration and 3D reconstruction. In addition, existing ultrasound imaging in spinal alignment assessment only uses a simple rigid body transformation model, without considering the nonlinear errors caused by soft tissue deformation. Therefore, to address the above problems, there is an urgent need for a spinal alignment assessment method and system based on optical motion capture and ultrasound imaging. Summary of the Invention

[0004] To improve the efficiency of spinal alignment assessment, this invention provides a spinal alignment assessment method and system based on optical motion capture and ultrasound imaging.

[0005] In a first aspect, the spinal alignment assessment method based on optical motion capture and ultrasound imaging provided by the present invention adopts the following technical solution:

[0006] A spinal alignment assessment method based on optical motion capture and ultrasound imaging includes the following steps:

[0007] Step S1: Based on the ultrasound imaging equipment, the spinal region of the patient to be tested is detected, and ultrasound images of the spinal region of the patient to be tested are acquired to obtain the patient's spinal ultrasound image information.

[0008] Step S2: Obtain an industrial camera array, which includes multiple industrial cameras. Based on the multiple industrial cameras, perform fixed tracking on the preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device, and output the six-degree-of-freedom pose information of the ultrasonic probe.

[0009] Step S3: Obtain the calibration body. The ultrasonic imaging device performs multi-angle scanning on the calibration body to obtain multiple sets of ultrasonic image information of the calibration body. Simultaneously, the industrial camera array is triggered to acquire the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image acquisition and scanning to obtain multiple probe poses To. Feature extraction is performed on the calibration body to obtain feature point coordinates Pu. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasonic image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the theoretical world coordinate system coordinates Pw′ are obtained. The theoretical world coordinate system coordinates Pw′ are registered with the real world coordinates Pw of the calibration body, and the optimal rigid transformation matrix Tu is obtained.

[0010] Step S4: Based on the redundant feature point coordinates Pu, repeat the above steps to iteratively optimize the optimal rigid transformation matrix Tu, then perform cross-validation to update the optimal rigid transformation matrix Tu. Using the probe pose To and the optimal rigid transformation matrix Tu, project the patient's spinal ultrasound image information onto the world coordinate system to complete the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information, and generate a three-dimensional model of the patient's spine.

[0011] Step S5: Based on the three-dimensional model of the patient's spine, key clinical parameters of the patient's spine are automatically identified to obtain characteristic clinical parameter information. Based on the characteristic clinical parameter information, the spinal alignment status of the patient is evaluated and a spinal alignment evaluation report of the patient is generated.

[0012] Step S6: Obtain the visualization operation platform. The visualization operation platform displays the three-dimensional model of the patient's spine, the characteristic clinical parameter information, and the assessment results of the patient's spine alignment status in real time, and archives the spinal alignment assessment report of the patient to be tested.

[0013] Preferably, the system guides the patient to adopt a preset standard position, and outputs a signal indicating that the patient is ready when the patient is in the preset standard position.

[0014] An ultrasound imaging device is obtained, the ultrasound imaging device includes an ultrasound probe. When a patient's ready signal is received, the device detects whether the preset optical reflective markers on the ultrasound probe are clean and unobstructed. If they are clean and unobstructed, a sufficient amount of coupling agent is evenly applied to the ultrasound probe. After the application is completed, a probe ready signal is output.

[0015] Upon receiving both the patient's and probe's readiness signals, the ultrasound probe is placed longitudinally on the midline of the patient's spine, with the probe pressed firmly against the skin, maintaining stable and moderate contact pressure. It is then smoothly moved along the long axis of the spine from the cervical segment to the sacral segment. Based on dynamic focusing technology, ultrasound images of the patient's spinal region are acquired to obtain the patient's spinal ultrasound image information.

[0016] Preferably, the ultrasound imaging device supports Doppler mode to distinguish between vascular parts and static tissues. When acquiring ultrasound images of the spinal region of the patient to be tested, Doppler mode is turned on to identify and delete vascular parts and static tissue parts in the ultrasound image to obtain the patient's spinal ultrasound image information.

[0017] Preferably, an industrial camera array is acquired, the industrial camera array comprising multiple industrial cameras;

[0018] Once the signal indicating that the patient is ready is received, multiple industrial cameras will be evenly deployed around the patient at multiple angles.

[0019] The industrial camera array captures images from each industrial camera, and then tracks and fixes preset optical reflective markers on the ultrasonic probe based on these images, outputting the six-degree-of-freedom pose information of the ultrasonic probe.

[0020] Preferably, a calibration body is obtained, wherein multiple feature points are arranged on the calibration body;

[0021] Obtain a rigid support, mount and fix the calibration body on the rigid support, and install the rigid support within the field of view of the industrial camera array.

[0022] The operator holds the ultrasonic probe and scans the calibration body in multiple spatial postures to acquire multiple sets of ultrasonic images of the calibration body and obtain multiple sets of ultrasonic image information of the calibration body.

[0023] Multiple industrial cameras in the industrial camera array are synchronously triggered to identify the calibration object, and the six-degree-of-freedom pose of the ultrasonic probe is recorded each time an ultrasonic image acquisition scan is performed to obtain multiple probe poses To.

[0024] Preferably, the multiple sets of ultrasound image information of the calibration bodies are preprocessed to obtain preprocessed ultrasound image information;

[0025] The coordinates of the preset feature points on the calibration body in the ultrasound image coordinate system are extracted by using automatic thresholding or manual annotation to obtain the feature point coordinates Pu.

[0026] The feature point coordinates Pu are transformed to the probe coordinate system using a rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system to be calibrated.

[0027] Based on the probe pose To, the rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the corresponding world coordinate system coordinates, i.e. the theoretical world coordinate system coordinates Pw′, are calculated.

[0028] Register the theoretical world coordinate system coordinates Pw′ with the real world coordinates Pw of the calibration body, denote the number of feature points as N, and construct the objective function:

[0029] ,

[0030] Where N is the number of feature points, Pw is the real-world coordinates of the calibration volume, and To... （i） The probe pose is the location of each feature point, Tu is the rigid transformation matrix from the ultrasound image coordinate system to the probe coordinate system to be calibrated, and Pu is the coordinate of the feature point.

[0031] The optimal rigid transformation matrix Tu is obtained by solving the rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system based on the least squares method or singular value decomposition (SVD) algorithm.

[0032] Preferably, based on the redundant feature point coordinates Pu, the above steps are repeated to iteratively optimize the optimal rigid transformation matrix Tu, and the Levenberg-Marquardt algorithm is used to reduce nonlinear errors.

[0033] The feature point coordinates of each feature point are divided into a training set and a test set. The optimal rigid transformation matrix Tu is solved using the training set. The root mean square error between the predicted coordinates and the true coordinates of the feature points is calculated in the test set. If the root mean square error is less than the preset root mean square error threshold, the calibration is deemed valid and the optimal rigid transformation matrix Tu is updated. Otherwise, the position of the calibration body is readjusted or the number of data acquisitions is increased until the root mean square error is less than the preset root mean square error threshold.

[0034] The optimal rigid transformation matrix Tu is stored. During the actual scanning process, the system acquires the probe pose To in real time. Using the probe pose To and the optimal rigid transformation matrix Tu, the patient's spinal ultrasound image information is projected onto the world coordinate system. Through volume rendering technology, the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information are completed to generate a three-dimensional model of the patient's spine.

[0035] Preferably, a convolutional neural network (CNN) model is obtained, and a signal connection link between the CNN model and the patient to be tested is established;

[0036] The three-dimensional model of the patient's spine is input into the convolutional neural network (CNN) model to identify the vertebral body boundaries of the patient's spine and obtain vertebral body boundary information. At the same time, the key parameters of the patient's spine are calculated to obtain Cobb angle information and spinal rotation angle information.

[0037] The vertebral body boundary information, the Cobb angle information, and the spinal rotation angle information are combined to form the characteristic clinical parameter information of the patient to be tested;

[0038] Based on vertebral body boundary information, Cobb angle information, and spinal rotation angle information, the spinal alignment status of the patient under test is evaluated to obtain the patient's spinal alignment status evaluation result.

[0039] Based on vertebral boundary information, Cobb angle information, spinal rotation angle information, and the patient's spinal alignment status assessment results, a spinal alignment assessment report for the patient to be tested is generated.

[0040] Preferably, a visualization operation platform is obtained, and the patient's three-dimensional spinal model, the characteristic clinical parameter information, and the patient's spinal alignment status assessment results are displayed in real time on the visualization operation platform based on a three-dimensional rendering engine;

[0041] Operators can connect to the visualization platform via touch or mouse, and can rotate, scale, and cut the three-dimensional model of the patient's spine.

[0042] The spinal alignment assessment report of the patients under test was exported in DICOM format for clinical archiving.

[0043] Secondly, the present invention provides a spinal alignment assessment system based on optical motion capture and ultrasound imaging, employing the following technical solution:

[0044] A spinal alignment assessment system based on optical motion capture and ultrasound imaging includes:

[0045] The three-dimensional ultrasound imaging module is configured to detect the spinal region of the patient under test based on the ultrasound imaging device, and acquire ultrasound images of the spinal region of the patient under test to obtain the patient's spinal ultrasound image information.

[0046] An optical motion capture module is configured to acquire an industrial camera array, which includes multiple industrial cameras. Based on the multiple industrial cameras, the module performs fixed tracking of preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device and outputs the six-degree-of-freedom pose information of the ultrasonic probe.

[0047] The hand-eye calibration module is configured to acquire a calibration body. The ultrasonic imaging device performs multi-angle scanning on the calibration body to obtain multiple sets of ultrasonic image information of the calibration body. Simultaneously, the industrial camera array is triggered to acquire the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image acquisition scan to obtain multiple probe poses To. Feature point coordinates Pu are obtained by extracting features from the calibration body. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasonic image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the theoretical world coordinate system coordinates Pw′ are obtained. The theoretical world coordinate system coordinates Pw′ are registered with the real world coordinates Pw of the calibration body, and the optimal rigid transformation matrix Tu is obtained.

[0048] The three-dimensional reconstruction module is configured to perform iterative optimization of the optimal rigid transformation matrix Tu based on redundant feature point coordinates Pu. After repeating the above steps, cross-validation is performed to update the optimal rigid transformation matrix Tu. The patient's spinal ultrasound image information is projected onto the world coordinate system through the probe pose To and the optimal rigid transformation matrix Tu, thereby completing the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information and generating a three-dimensional model of the patient's spine.

[0049] The key clinical parameter identification module is configured to automatically identify key clinical parameters of the patient's spine based on the patient's three-dimensional spinal model to obtain characteristic clinical parameter information, evaluate the spinal alignment status of the patient based on the characteristic clinical parameter information, and generate a spinal alignment assessment report for the patient.

[0050] The interactive archiving module is configured to acquire a visualization operation platform, which displays the patient's three-dimensional spinal model, the characteristic clinical parameter information, and the patient's spinal alignment status assessment results in real time, and archives the spinal alignment assessment report of the patient to be tested.

[0051] In summary, the present invention has the following beneficial technical effects:

[0052] Ultrasonic images of the patient's spine are acquired using an ultrasonic imaging device. Based on an industrial camera array, preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device are tracked and fixed, and the six-degree-of-freedom pose information of the ultrasonic probe is output. By constructing a precise hand-eye calibration mechanism, accurate mapping between the ultrasonic probe image coordinate system and the optical coordinate system is achieved, which improves the accuracy of the three-dimensional reconstruction of the patient's spine and thus improves the efficiency of the patient's spine alignment assessment. Attached Figure Description

[0053] Figure 1 This embodiment is a flowchart illustrating the spinal alignment assessment method based on optical motion capture and ultrasound imaging.

[0054] Figure 2 This embodiment mainly illustrates the module diagram of the spinal alignment assessment system based on optical motion capture and ultrasound imaging.

[0055] Figure labels: 1. Three-dimensional ultrasound imaging module; 2. Optical motion capture module; 3. Hand-eye calibration module; 4. Three-dimensional reconstruction module; 5. Key clinical parameter identification module; 6. Interactive archiving module. Detailed Implementation

[0056] The present invention will be further described in detail below with reference to the accompanying drawings.

[0057] This invention discloses a method for assessing spinal alignment based on optical motion capture and ultrasound imaging.

[0058] A spinal alignment assessment method based on optical motion capture and ultrasound imaging includes the following steps:

[0059] Reference Figure 1 Step S1 involves using ultrasound imaging equipment to detect the spinal region of the patient and acquiring ultrasound images of the spinal region to obtain spinal ultrasound image information. Step S1 specifically includes the following sub-steps:

[0060] The system guides the patient to adopt a preset standard posture. When the patient is in the preset standard posture, it outputs a signal indicating that the patient is ready to be tested.

[0061] An ultrasound imaging device is acquired, including an ultrasound probe. It should be noted that, in this embodiment, the ultrasound probe refers to a high-frequency linear array transducer or a high-frequency convex array transducer. Upon receiving a patient readiness signal, the device checks whether the preset optical reflective markers on the ultrasound probe are clean and unobstructed. If clean and unobstructed, a sufficient amount of coupling agent is evenly applied to the ultrasound probe to ensure good acoustic contact between the ultrasound probe and the patient's skin. After application, a probe readiness signal is output.

[0062] Upon receiving both the patient's and probe's readiness signals, the ultrasound probe is placed longitudinally along the midline of the patient's spine. It should be noted that in this embodiment, the detection begins at the seventh cervical vertebra or the first thoracic vertebra. The probe remains in close contact with the skin, maintaining stable and moderate pressure. Furthermore, this embodiment employs a uniform, slow, and continuous movement to avoid uneven force that could deform soft tissue and cause errors. The probe moves smoothly along the long axis of the spine, from the cervical segment to the sacral segment, ensuring coverage of the entire spinal region requiring evaluation. Based on dynamic focusing technology, ultrasound images of the patient's spine are acquired. It should be noted that the dynamic focusing technology in this embodiment is crucial for obtaining high-quality images. After emitting ultrasound waves, the preset receiver in the ultrasound imaging device dynamically adjusts the reception time of echo signals at different depths, ensuring high resolution throughout the entire scanning depth range. This guarantees that the vertebral structures of the patient's spine, from superficial to deep, are clearly discernible.

[0063] The ultrasound imaging equipment supports Doppler mode, which distinguishes between vascular regions and static tissues. When acquiring ultrasound images of the patient's spine, Doppler mode is activated to identify and remove vascular and static tissue regions from the ultrasound image, thus obtaining the patient's spinal ultrasound image information. This improves the accuracy of ultrasound detection of the patient's spinal region, thereby improving the accuracy of subsequent vertebral body boundary segmentation.

[0064] Reference Figure 1 Step S2 involves acquiring an industrial camera array, which includes multiple industrial cameras. Based on these multiple cameras, the array tracks and fixes preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device, outputting the six-degree-of-freedom pose information of the ultrasonic probe. Step S2 specifically includes the following sub-steps:

[0065] Acquire an industrial camera array, which consists of multiple industrial cameras.

[0066] It should be noted that the industrial camera in the embodiments of this application may be a high-precision industrial camera, such as the OptiTrackPrime series industrial camera.

[0067] Once the patient's readiness signal is received, multiple industrial cameras are evenly deployed around the patient at multiple angles.

[0068] The industrial camera array captures images from each industrial camera, and then tracks and fixes preset optical reflective markers on the ultrasonic probe based on these images, outputting the six-degree-of-freedom pose information of the ultrasonic probe.

[0069] It should be noted that the multiple industrial cameras in the industrial camera array in this application embodiment all have sub-millimeter spatial resolution and high frame rate tracking capability of ≥100fps, which can accurately capture the translation and rotation motion of the ultrasonic probe in three-dimensional space.

[0070] In practical applications, multiple industrial cameras are installed around the patient to ensure that the preset optical reflective markers on the ultrasound probe can be clearly captured by at least two cameras at any possible working position. This provides data support for three-dimensional triangulation calculation and avoids tracking failure due to the preset optical reflective markers on the ultrasound probe being blocked.

[0071] Reference Figure 1 Step S3 involves acquiring the calibration body. The ultrasonic imaging device performs multi-angle scanning on the calibration body to obtain multiple sets of ultrasonic image information. Simultaneously, an industrial camera array is triggered to acquire the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image acquisition scan, resulting in multiple probe poses To. Feature point coordinates Pu are extracted from the calibration body. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasonic image coordinate system to the probe coordinate system, and the feature point coordinates Pu, the theoretical world coordinate system coordinates Pw′ are obtained. The theoretical world coordinate system coordinates Pw′ are registered with the real world coordinates Pw of the calibration body, and the optimal rigid transformation matrix Tu is obtained. Step S3 specifically includes the following sub-steps:

[0072] A calibration body is obtained, on which multiple feature points are arranged. The calibration body can be a metal needle array (diameter ≤0.5mm, spacing ≥2mm), a plane scale with graduations, or a high-precision dot matrix spherical target.

[0073] Obtain a rigid support, mount and fix the calibration object onto the rigid support, and place the rigid support within the field of view of the industrial camera array. Ensure that all feature points on the calibration object can be detected and identified by the ultrasonic probe and the industrial cameras in the industrial camera array.

[0074] The operator holds an ultrasonic probe and scans the calibration body in multiple spatial postures (covering at least five different angles and positions in this embodiment) to acquire multiple sets of ultrasonic images of the calibration body and obtain multiple sets of ultrasonic image information of the calibration body.

[0075] Multiple industrial cameras in the synchronously triggered industrial camera array identify the calibration object, and the six-degree-of-freedom pose of the ultrasonic probe is recorded during each ultrasonic image acquisition scan to obtain multiple probe poses To. It should be noted that in the embodiments of this application, attention should be paid to maintaining a sufficiently large relative position change between the ultrasonic probe and the calibration object in order to improve calibration accuracy.

[0076] Step S3 further includes the following sub-steps:

[0077] Preprocessed ultrasound image information is obtained by preprocessing multiple sets of calibration body ultrasound image information.

[0078] Specifically, histogram equalization is performed on the ultrasound image information of the calibration body to enhance image contrast, and median filtering is performed to remove noise while preserving edge features, thereby enhancing the visibility of feature points in the ultrasound image information of the calibration body.

[0079] The coordinates of the feature points Pu are obtained by extracting the coordinates of the preset feature points on the calibration body in the ultrasound image coordinate system using automatic thresholding or manual annotation.

[0080] Specifically, when the calibration body is a metal needle array, automatic thresholding can be used to identify bright areas, and then the coordinates of feature points on the metal needle array can be accurately located through centroid calculation or template matching algorithms to obtain the feature point coordinates Pu. Manual annotation involves the operator manually clicking on the ultrasound image information of the calibration body to mark the location of the feature points and obtain the feature point coordinates Pu.

[0081] The feature point coordinates Pu are transformed to the probe coordinate system using a rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system to be calibrated.

[0082] Based on the probe pose To, the rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the corresponding world coordinate system coordinates, i.e., the theoretical world coordinate system coordinates Pw′, are calculated.

[0083] Register the theoretical world coordinate system coordinates Pw′ with the real world coordinates Pw of the calibration body, denote the number of feature points as N, and construct the objective function:

[0084] ,

[0085] Where N is the number of feature points, Pw is the real-world coordinates of the calibration volume, and To... （i） Let Tu be the probe pose at each feature point, Tu be the rigid transformation matrix from the ultrasound image coordinate system to the probe coordinate system to be calibrated, and Pu be the coordinates of the feature points.

[0086] The optimal rigid transformation matrix Tu is obtained by solving the rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system based on the least squares method or singular value decomposition (SVD) algorithm.

[0087] Reference Figure 1Step S4 involves iteratively optimizing the optimal rigid transformation matrix Tu based on redundant feature point coordinates Pu, followed by cross-validation to update the matrix. Using the probe pose To and the optimal rigid transformation matrix Tu, the patient's spinal ultrasound image information is projected onto the world coordinate system, completing the accurate registration and 3D reconstruction of multiple frames of patient spinal ultrasound images, and generating a 3D model of the patient's spine. Step S4 specifically includes the following sub-steps:

[0088] Based on the redundant feature point coordinates Pu, the above steps are repeated to iteratively optimize the optimal rigid transformation matrix Tu, and the Levenberg-Marquardt algorithm is used to reduce nonlinear errors.

[0089] The coordinates of each feature point are divided into a training set and a test set. The optimal rigid transformation matrix Tu is solved using the training set. The root mean square error (RMSE) between the predicted and true coordinates of the feature points is calculated using the test set. If the RMSE is less than a preset RMSE threshold, the calibration is deemed valid, and the optimal rigid transformation matrix Tu is updated. Otherwise, the calibration body position is readjusted or the number of data acquisitions is increased until the RMSE is less than the preset RMSE threshold. It should be noted that the RMSE threshold in this embodiment is 0.5 mm.

[0090] The optimal rigid transformation matrix Tu is stored for subsequent real-time scanning. During the actual scanning process, the system acquires the probe pose To in real time. Using the probe pose To and the optimal rigid transformation matrix Tu, the patient's spinal ultrasound image information is projected onto the world coordinate system. Through volume rendering technology, the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information are completed, generating a three-dimensional model of the patient's spine.

[0091] Reference Figure 1 Step S5 involves automatically identifying key clinical parameters of the patient's spine based on a 3D model of the patient's spine to obtain characteristic clinical parameter information. Based on this characteristic clinical parameter information, the spinal alignment status of the patient is assessed, and a spinal alignment assessment report is generated. Step S5 specifically includes the following sub-steps:

[0092] Obtain a convolutional neural network (CNN) model and establish a signal connection link between the CNN model and the patient to be tested.

[0093] The three-dimensional model of the patient's spine is input into a convolutional neural network (CNN) model to identify the vertebral body boundaries of the patient's spine and obtain vertebral body boundary information. At the same time, the key parameters of the patient's spine are calculated to obtain Cobb angle information and spinal rotation angle information.

[0094] In practical applications, the Cobb angle is an internationally standardized indicator used to measure the degree of scoliosis. The Cobb angle reflects the curvature of the spine and is an important basis for diagnosing and judging the severity of scoliosis.

[0095] Vertebral body boundary information, Cobb angle information, and spinal rotation angle information are combined to form characteristic clinical parameter information of the patient under test.

[0096] Based on vertebral boundary information, Cobb angle information, and spinal rotation angle information, the spinal alignment status of the patient under test is evaluated to obtain the patient's spinal alignment status assessment result.

[0097] Based on vertebral boundary information, Cobb angle information, spinal rotation angle information, and the patient's spinal alignment status assessment results, a spinal alignment assessment report for the patient to be tested is generated.

[0098] Reference Figure 1 Step S6 involves acquiring a visualization platform. This platform displays the patient's 3D spinal model, characteristic clinical parameters, and spinal alignment assessment results in real time, and archives the patient's spinal alignment assessment report. Step S6 specifically includes the following sub-steps:

[0099] A visualization platform is available, which uses a 3D rendering engine to display the patient's 3D spinal model, characteristic clinical parameters, and spinal alignment assessment results in real time.

[0100] Operators can connect to the visualization platform via touch or mouse, and can rotate, scale, and cut the 3D model of the patient's spine.

[0101] Export the spinal alignment assessment report of the patient to be tested in DICOM format for clinical archiving.

[0102] This invention also discloses a spinal alignment assessment system based on optical motion capture and ultrasound imaging.

[0103] Reference Figure 2 A spinal alignment assessment system based on optical motion capture and ultrasound imaging includes:

[0104] The three-dimensional ultrasound imaging module is configured to detect the spinal region of the patient under test using an ultrasound imaging device, and acquire ultrasound images of the spinal region of the patient under test to obtain the patient's spinal ultrasound image information.

[0105] The optical motion capture module is configured to acquire an industrial camera array, which includes multiple industrial cameras. Based on the multiple industrial cameras, it performs fixed tracking of preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device and outputs the six-degree-of-freedom pose information of the ultrasonic probe.

[0106] The hand-eye calibration module is configured to acquire a calibration body. The ultrasonic imaging device performs multi-angle scanning on the calibration body to obtain multiple sets of ultrasonic image information of the calibration body. Simultaneously, the industrial camera array is triggered to acquire the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image acquisition scan to obtain multiple probe poses To. Feature extraction is performed on the calibration body to obtain feature point coordinates Pu. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasonic image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the theoretical world coordinate system coordinates Pw′ are obtained. The theoretical world coordinate system coordinates Pw′ are registered with the real world coordinates Pw of the calibration body, and the optimal rigid transformation matrix Tu is obtained.

[0107] The 3D reconstruction module is configured to iteratively optimize the optimal rigid transformation matrix Tu based on redundant feature point coordinates Pu. After repeating the above steps, cross-validation is performed to update the optimal rigid transformation matrix Tu. The patient's spinal ultrasound image information is projected onto the world coordinate system using the probe pose To and the optimal rigid transformation matrix Tu, thus completing the accurate registration and 3D reconstruction of multiple frames of patient spinal ultrasound image information and generating a 3D model of the patient's spine.

[0108] The key clinical parameter identification module is configured to automatically identify key clinical parameters of the patient's spine based on the patient's three-dimensional spinal model to obtain characteristic clinical parameter information, assess the spinal alignment status of the patient based on the characteristic clinical parameter information, and generate a spinal alignment assessment report for the patient.

[0109] The interactive archiving module is configured to access a visualization operation platform. The visualization operation platform displays the patient's three-dimensional spinal model, characteristic clinical parameter information, and the patient's spinal alignment status assessment results in real time, and archives the spinal alignment assessment report of the patient to be tested.

[0110] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or systems. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] This invention is described with reference to flowchart illustrations and structural diagrams of methods and systems according to embodiments of the invention. It should be understood that the combination of each process and module in the flowchart and structural diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 One or more processes and structures Figure 1 A device for a function specified in one or more modules.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and structures Figure 1 The function specified in one or more modules.

[0113] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and structures Figure 1 The steps of a specified function in one or more modules.

[0114] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0115] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for assessing spinal alignment based on optical motion capture and ultrasound imaging, characterized in that, Includes the following steps: Step S1: Based on the ultrasound imaging equipment, the spinal region of the patient to be tested is detected, and ultrasound images of the spinal region of the patient to be tested are acquired to obtain the patient's spinal ultrasound image information. Step S2: Obtain an industrial camera array, which includes multiple industrial cameras. Based on the multiple industrial cameras, perform fixed tracking on the preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device, and output the six-degree-of-freedom pose information of the ultrasonic probe. Step S3: Obtain the calibration body. The ultrasonic imaging device performs multi-angle scanning on the calibration body to obtain multiple sets of ultrasonic image information of the calibration body. Simultaneously, the industrial camera array is triggered to acquire the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image acquisition and scanning to obtain multiple probe poses To. Feature extraction is performed on the calibration body to obtain feature point coordinates Pu. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasonic image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the theoretical world coordinate system coordinates Pw′ are obtained. The theoretical world coordinate system coordinates Pw′ are registered with the real world coordinates Pw of the calibration body, and the optimal rigid transformation matrix Tu is obtained. Step S4: Based on the redundant feature point coordinates Pu, repeat the above steps to iteratively optimize the optimal rigid transformation matrix Tu, then perform cross-validation to update the optimal rigid transformation matrix Tu. Using the probe pose To and the optimal rigid transformation matrix Tu, project the patient's spinal ultrasound image information onto the world coordinate system to complete the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information, and generate a three-dimensional model of the patient's spine. Step S5: Based on the three-dimensional model of the patient's spine, key clinical parameters of the patient's spine are automatically identified to obtain characteristic clinical parameter information. Based on the characteristic clinical parameter information, the spinal alignment status of the patient is evaluated and a spinal alignment evaluation report of the patient is generated. Step S6: Obtain the visualization operation platform. The visualization operation platform displays the three-dimensional model of the patient's spine, the characteristic clinical parameter information, and the assessment results of the patient's spine alignment status in real time, and archives the spinal alignment assessment report of the patient to be tested.

2. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 1, characterized in that, Step S1 specifically includes the following sub-steps: The system guides the patient to adopt a preset standard position. When the patient is in the preset standard position, it outputs a signal indicating that the patient is ready to be tested. An ultrasound imaging device is obtained, the ultrasound imaging device includes an ultrasound probe. When a patient's ready signal is received, the device detects whether the preset optical reflective markers on the ultrasound probe are clean and unobstructed. If they are clean and unobstructed, a sufficient amount of coupling agent is evenly applied to the ultrasound probe. After the application is completed, a probe ready signal is output. Upon receiving both the patient's and probe's readiness signals, the ultrasound probe is placed longitudinally on the midline of the patient's spine, with the probe pressed firmly against the skin and maintaining stable and moderate contact pressure. It is then smoothly moved along the long axis of the spine from the cervical segment to the sacral segment. Based on dynamic focusing technology, ultrasound images of the patient's spinal region are acquired to obtain the patient's spinal ultrasound image information.

3. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 2, characterized in that, The ultrasound imaging device supports Doppler mode, which distinguishes between vascular parts and static tissues. When acquiring ultrasound images of the spinal region of the patient to be tested, Doppler mode is turned on to identify and delete vascular parts and static tissue parts in the ultrasound image to obtain the patient's spinal ultrasound image information.

4. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 3, characterized in that, Step S2 specifically includes the following sub-steps: Acquire an industrial camera array, wherein the industrial camera array comprises multiple industrial cameras; Once the signal indicating that the patient is ready is received, multiple industrial cameras will be evenly deployed around the patient at multiple angles. The industrial camera array captures images from each industrial camera, and then tracks and fixes preset optical reflective markers on the ultrasonic probe based on these images, outputting the six-degree-of-freedom pose information of the ultrasonic probe.

5. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 4, characterized in that, Step S3 specifically includes the following sub-steps: A calibration body is obtained, on which multiple feature points are arranged; Obtain a rigid support, mount and fix the calibration body on the rigid support, and install the rigid support within the field of view of the industrial camera array. The operator holds the ultrasonic probe and scans the calibration body in multiple spatial postures to acquire multiple sets of ultrasonic images of the calibration body and obtain multiple sets of ultrasonic image information of the calibration body. Multiple industrial cameras in the industrial camera array are synchronously triggered to identify the calibration object, and the six-degree-of-freedom pose of the ultrasonic probe is recorded each time an ultrasonic image acquisition scan is performed to obtain multiple probe poses To.

6. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 5, characterized in that, Step S3 further includes the following sub-steps: Preprocessing is performed on the multiple sets of ultrasound images of the calibration bodies to obtain preprocessed ultrasound image information. The coordinates of the preset feature points on the calibration body in the ultrasound image coordinate system are extracted by using automatic thresholding or manual annotation to obtain the feature point coordinates Pu. The feature point coordinates Pu are transformed to the probe coordinate system using a rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system to be calibrated. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the corresponding world coordinate system coordinates, i.e. the theoretical world coordinate system coordinates Pw′, are calculated. Register the theoretical world coordinate system coordinates Pw′ with the real world coordinates Pw of the calibration body, denote the number of feature points as N, and construct the objective function: ， Where N is the number of feature points, Pw is the real-world coordinates of the calibration volume, and To... （i） The probe pose is the location of each feature point, Tu is the rigid transformation matrix from the ultrasound image coordinate system to the probe coordinate system to be calibrated, and Pu is the coordinate of the feature point. The optimal rigid transformation matrix Tu is obtained by solving the rigid transformation matrix Tu from the ultrasound image coordinate system to the probe coordinate system based on the least squares method or singular value decomposition (SVD) algorithm.

7. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 6, characterized in that, Step S4 specifically includes the following sub-steps: Based on the redundant feature point coordinates Pu, the above steps are repeated to iteratively optimize the optimal rigid transformation matrix Tu, and the Levenberg-Marquardt algorithm is used to reduce nonlinear error. The feature point coordinates of each feature point are divided into a training set and a test set. The optimal rigid transformation matrix Tu is solved using the training set. The root mean square error between the predicted coordinates and the true coordinates of the feature points is calculated in the test set. If the root mean square error is less than the preset root mean square error threshold, the calibration is deemed valid and the optimal rigid transformation matrix Tu is updated. Otherwise, the position of the calibration body is readjusted or the number of data acquisitions is increased until the root mean square error is less than the preset root mean square error threshold. The optimal rigid transformation matrix Tu is stored. During the actual scanning process, the system acquires the probe pose To in real time. Using the probe pose To and the optimal rigid transformation matrix Tu, the patient's spinal ultrasound image information is projected onto the world coordinate system. Through volume rendering technology, the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information are completed to generate a three-dimensional model of the patient's spine.

8. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 7, characterized in that, Step S5 specifically includes the following sub-steps: Obtain a convolutional neural network (CNN) model and establish a signal connection link between the CNN model and the patient to be tested; The three-dimensional model of the patient's spine is input into the convolutional neural network (CNN) model to identify the vertebral body boundaries of the patient's spine and obtain vertebral body boundary information. At the same time, the key parameters of the patient's spine are calculated to obtain Cobb angle information and spinal rotation angle information. The vertebral body boundary information, the Cobb angle information, and the spinal rotation angle information are combined to form the characteristic clinical parameter information of the patient to be tested; Based on vertebral body boundary information, Cobb angle information, and spinal rotation angle information, the spinal alignment status of the patient under test is evaluated to obtain the patient's spinal alignment status evaluation result. Based on vertebral boundary information, Cobb angle information, spinal rotation angle information, and the patient's spinal alignment status assessment results, a spinal alignment assessment report for the patient to be tested is generated.

9. The spinal alignment assessment method based on optical motion capture and ultrasound imaging according to claim 8, characterized in that, Step S6 specifically includes the following sub-steps: A visualization operation platform is obtained, and the patient's three-dimensional spinal model, the characteristic clinical parameter information, and the patient's spinal alignment status assessment results are displayed in real time on the visualization operation platform based on a three-dimensional rendering engine; Operators can connect to the visualization platform via touch or mouse, and can rotate, scale, and cut the three-dimensional model of the patient's spine. The spinal alignment assessment report of the patients under test was exported in DICOM format for clinical archiving.

10. A spinal alignment assessment system based on optical motion capture and ultrasound imaging, characterized in that, The spinal alignment assessment system based on optical motion capture and ultrasound imaging is used to implement the spinal alignment assessment method based on optical motion capture and ultrasound imaging as described in any one of claims 1-9, including: The three-dimensional ultrasound imaging module is configured to detect the spinal region of the patient under test based on the ultrasound imaging device, and acquire ultrasound images of the spinal region of the patient under test to obtain the patient's spinal ultrasound image information. An optical motion capture module is configured to acquire an industrial camera array, which includes multiple industrial cameras. Based on the multiple industrial cameras, the module performs fixed tracking of preset optical reflective markers on the ultrasonic probe of the ultrasonic imaging device and outputs the six-degree-of-freedom pose information of the ultrasonic probe. The hand-eye calibration module is configured to acquire a calibration body. The ultrasonic imaging device performs multi-angle scanning on the calibration body to obtain multiple sets of ultrasonic image information of the calibration body. Simultaneously, the industrial camera array is triggered to acquire the six-degree-of-freedom pose of the ultrasonic probe during each ultrasonic image acquisition scan to obtain multiple probe poses To. Feature point coordinates Pu are obtained by extracting features from the calibration body. Based on the probe pose To, the rigid transformation matrix Tu from the ultrasonic image coordinate system to the probe coordinate system to be calibrated, and the feature point coordinates Pu, the theoretical world coordinate system coordinates Pw′ are obtained. The theoretical world coordinate system coordinates Pw′ are registered with the real world coordinates Pw of the calibration body, and the optimal rigid transformation matrix Tu is obtained. The three-dimensional reconstruction module is configured to perform iterative optimization of the optimal rigid transformation matrix Tu based on redundant feature point coordinates Pu. After repeating the above steps, cross-validation is performed to update the optimal rigid transformation matrix Tu. The patient's spinal ultrasound image information is projected onto the world coordinate system through the probe pose To and the optimal rigid transformation matrix Tu, thereby completing the accurate registration and three-dimensional reconstruction of multiple frames of patient spinal ultrasound image information and generating a three-dimensional model of the patient's spine. The key clinical parameter identification module is configured to automatically identify key clinical parameters of the patient's spine based on the patient's three-dimensional spinal model to obtain characteristic clinical parameter information, evaluate the spinal alignment status of the patient based on the characteristic clinical parameter information, and generate a spinal alignment assessment report for the patient. The interactive archiving module is configured to acquire a visualization operation platform, which displays the patient's three-dimensional spinal model, the characteristic clinical parameter information, and the patient's spinal alignment status assessment results in real time, and archives the spinal alignment assessment report of the patient to be tested.

Images