Spine three-dimensional imaging method and system based on millimeter wave radar and deep learning

Abstract

The invention provides a spinal column three-dimensional imaging method and system based on millimeter wave radar and deep learning, and relates to the field of medical detection technology. Spinal column three-dimensional point cloud data are obtained through millimeter wave radar array multi-angle scanning, and a spinal column three-dimensional point cloud image is obtained by using a constructed two-stage deep learning network; comprising a 3D-CNN feature extraction network fusing PointNet + + and Voxel CNN and a graph convolutional network for automatic extraction of spine features, a key centrum positioning and Cobb angle regression calculation module based on an attention mechanism, and a self-adaptive filtering algorithm for eliminating signal noise caused by human body micro-motion. The method realizes automatic and high-precision calculation of spine three-dimensional imaging and the scoliosis angle, has the advantages of non-contact detection, no ionizing radiation risk and the like, and can be used for automatic diagnosis of scoliosis.

Description

Technical Field

[0001] This invention relates to the field of medical detection technology, and more particularly to a method and system for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning. Background Technology

[0002] In the field of medical testing, spinal morphology detection, especially the diagnosis of scoliosis, has always been a crucial and highly regarded topic. However, existing spinal detection technologies have many limitations and urgently need improvement and innovation. While traditional X-rays and CT scans can provide some evidence for spinal detection, they pose significant radiation risks, especially for vulnerable groups such as children, adolescents, and pregnant women. Ionizing radiation can cause irreversible damage to their growth, development, and health, limiting the frequency and scope of these examinations. MRI, although radiation-free, is expensive, resulting in high examination costs. Its bulky size makes it difficult to move and transport, hindering convenient dynamic monitoring. Patients often face long waiting times for appointments, which is detrimental to timely diagnosis and treatment.

[0003] As an emerging detection method, optical 3D imaging technology can theoretically obtain three-dimensional morphological information of the spine. However, in practical applications, it is greatly affected by ambient light. The imaging quality will be significantly reduced in strong or weak light environments, and it is easily affected by obstructions such as clothing and hair, resulting in incomplete imaging and inability to accurately reflect the true state of the spine. The measurement accuracy is therefore limited and it is difficult to meet the needs of high-precision clinical diagnosis.

[0004] In the assessment of scoliosis, the Cobb angle measurement method commonly used in clinical practice relies on doctors manually marking key points on X-ray films and performing calculations. This manual operation method is not only inefficient, but also has a large subjective error. Different doctors may have different judgments and measurement standards for key points, resulting in poor repeatability and consistency of measurement results, which affects the accuracy of diagnosis and the formulation of treatment plans.

[0005] Furthermore, millimeter-wave radar technology has emerged in numerous fields in recent years, boasting advantages such as non-contact detection, no ionizing radiation, and the ability to penetrate non-metallic materials. Its application in the medical field is gradually gaining attention. However, in the crucial application area of ​​3D spinal modeling, a mature and effective solution is still lacking. Its potential in spinal morphology detection and scoliosis angle calculation has not been fully explored and utilized, leaving a significant technological gap to be filled. This invention addresses the shortcomings and deficiencies of the existing technologies by proposing a method for 3D spinal imaging and scoliosis angle calculation based on millimeter-wave radar and deep learning. This aims to overcome the limitations of traditional detection methods and provide an innovative solution for automated, high-precision, and low-cost diagnosis of scoliosis. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a method and system for three-dimensional spinal imaging based on millimeter-wave radar and deep learning. The method acquires high-precision three-dimensional point cloud data of the spine using a multi-channel radar array, and employs an improved hybrid deep learning network to achieve automatic extraction of spinal features and intelligent calculation of the Cobb angle. Compared with X-ray measurement results, this method effectively improves detection accuracy and is suitable for the automated diagnosis of scoliosis. It solves the problems of radiation risk and subjective error inherent in traditional detection methods, providing an efficient and low-cost solution for scoliosis screening.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect of the present invention, a method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning is provided, comprising the following steps: S1. The back of the human body is scanned from multiple angles using a millimeter-wave radar array to obtain the original radar echo signal; S2. Perform motion compensation and adaptive filtering on the radar echo signal to generate three-dimensional spine point cloud data; S3. Input the three-dimensional point cloud data into a deep learning network model, extract global features of the spine through a 3D convolutional neural network, and then locate key vertebrae based on an attention mechanism to obtain the coordinates of key vertebrae. S4. Calculate the Cobb angle using an angle regression network based on the coordinates of the key vertebrae in the localization. S5. Output a 3D visualization model of the spine including Cobb angle annotations.

[0008] Preferably, in step S1, the millimeter-wave radar array adopts a MIMO antenna configuration, operates in the frequency band of 77~81GHz, has a bandwidth of ≥4GHz, a power density of ≥10mW / cm², a frame rate of ≥30Hz, and integrates an inertial measurement unit to achieve scanning trajectory tracking.

[0009] Preferably, in step S2, the adaptive filtering process includes the following steps: Eliminate breathing and heartbeat interference through motion compensation algorithms; An improved ICP algorithm is used for dynamic point cloud registration, and the registration error is reduced. Multi-frame data fusion based on Kalman filter.

[0010] More preferably, the improved ICP algorithm specifically includes the following steps: For motion-compensated radar point cloud data, feature points were selected using a curvature thresholding method based on spinal anatomical characteristics: the curvature value of each point was calculated, and points with curvature greater than 0.02m were retained. ¹ Points are used as key feature points of the spine. Redundant points in smooth areas are removed, and the feature point retention rate is 30%~50% of the original point cloud. By combining the data from the inertial measurement unit integrated in the millimeter-wave radar array, the attitude angles during the scanning process are extracted and mapped to the coordinate system of the reference point cloud to complete the coarse registration; The similarity between point pairs is calculated using a two-factor weighted method of Euclidean distance and radar echo intensity, establishing a correspondence between the point cloud to be registered and the reference point cloud: ; in, For point pairs ( The weighted distance of the points to be registered. The echo intensity is Reference point The echo intensity is , To achieve the maximum echo intensity in the scanned scene, Point pairs smaller than 2mm are considered valid corresponding point pairs; Based on valid correspondence pairs, singular value decomposition is used to solve for the optimal registration matrix. An adaptive iteration step size k is introduced. After each iteration, the RANSAC algorithm is used to remove erroneous correspondence pairs. Iteration stops when any of the following conditions are met: The number of iterations reaches the preset threshold or the change in RMSE between two consecutive iterations is less than 0.05 mm.

[0011] Preferably, in step S3, the extracted spinal feature data is input into a key vertebral localization network based on an attention mechanism. Through the feature learning module of a 3D convolutional neural network, deep learning and analysis are performed on the features of different parts of the spine to extract feature information related to key vertebrae and to preliminarily screen possible key vertebral locations to determine several candidate key point regions. At the same time, the localization regression module in the 3D convolutional neural network is used to accurately locate the candidate key point regions to obtain the specific location coordinates of the key vertebrae.

[0012] Preferably, in S4, key point pairs for calculating the Cobb angle are determined by the specific location coordinates of the key vertebrae, and corresponding vectors are constructed based on these key points to represent the directional changes of the spine at different positions. Using vector operation algorithms, the angle between the constructed vectors is calculated to obtain a preliminary Cobb angle estimate. The estimate is then input into a deep learning-based angle regression network. Through the network's learning and fitting of a large amount of training data, the preliminary angle value is corrected by regression.

[0013] Preferably, in step S5, based on the Cobb angle calculation data obtained in step S4, which includes angle values ​​and corresponding key vertebral position information, the coordinate system used in the calculation is converted from the corrected angle data to a visual coordinate system that matches the three-dimensional spine model. Subsequently, intuitive Cobb angle annotation elements are designed based on the Cobb angle calculation data, including text display of angle values, markings of key vertebral positions, and lines or arcs representing the angle measurement range. The corresponding annotation graphics are then generated on the three-dimensional spine model using a visualization tool according to the designed annotation element style.

[0014] In a second aspect of the invention, a three-dimensional imaging system for the spine based on millimeter-wave radar and deep learning is provided, comprising: The millimeter-wave radar array scanning module is used to scan the back of the human body from multiple angles using a millimeter-wave radar array to obtain the raw radar echo signal. The signal processing module is used to perform motion compensation and adaptive filtering on the radar echo signal; The deep learning computing module is used to extract global features of the spine and obtain the coordinates of key vertebrae based on the input 3D point cloud data in two stages. The Cobb angle calculation module is used to calculate the angle value of the Cobb angle based on a combination of key point detection and angle regression network. The visualization generation module is used to generate a three-dimensional spine image based on the Cobb angle value, and the three-dimensional spine image is labeled with Cobb angle annotation elements.

[0015] The beneficial effects of this invention are as follows: 1. Non-contact and harmless detection: Using millimeter-wave radar technology, it achieves non-contact detection, with no risk of ionizing radiation, ensuring patient safety. It is especially suitable for radiation-sensitive groups such as teenagers, and can be tested frequently without any health risks.

[0016] 2. High-precision measurement: The detection accuracy reaches ±2°, which is highly consistent with the results of traditional X-ray measurement. The constructed deep learning network can accurately extract spinal features and calculate the Cobb angle, providing a reliable basis for clinical diagnosis and reducing misdiagnosis and missed diagnosis.

[0017] 3. Highly efficient and rapid detection: a single detection time is less than 3 minutes, and it supports dynamic monitoring. It can quickly acquire spinal morphology data and calculate the scoliosis angle, greatly shortening the patient's waiting time, improving diagnostic efficiency, and timely capturing changes in spinal morphology.

[0018] 4. Significantly reduced costs: The system cost is reduced to 30% of that of traditional equipment. Millimeter-wave radar technology is low-cost, and deep learning algorithms optimize the computing process, reducing dependence on expensive hardware. When it is widely promoted and applied, it can significantly reduce medical costs.

[0019] 5. Intelligent automatic diagnosis: Integrating deep learning technology, it realizes automatic extraction of spinal features, location of key vertebrae and automatic calculation of Cobb angle, avoiding subjective errors of manual measurement by doctors, improving the consistency and stability of diagnosis, and especially achieving unified diagnostic standards among doctors with different experience.

[0020] 6. Strong anti-interference capability: The developed adaptive filtering algorithm effectively eliminates signal noise caused by human micro-movements (such as breathing and heartbeat), while overcoming the limitations of optical 3D imaging affected by ambient light and occlusion, ensuring signal stability and data accuracy, and improving system robustness. Attached Figure Description

[0021] Figure 1 This is a flowchart of the three-dimensional imaging method for the spine based on millimeter-wave radar and deep learning, as described in this invention.

[0022] Figure 2 This is a block diagram of the three-dimensional imaging system for the spine based on millimeter-wave radar and deep learning, as described in this invention. Detailed Implementation

[0023] Please see Figure 1 As shown, in a first aspect of the present invention, a method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning is provided, comprising the following steps: S1. The back of the human body is scanned from multiple angles using a millimeter-wave radar array to obtain the original radar echo signal; Based on human anatomical characteristics and common scoliosis incidence areas, the correspondence between the scanning area and the target spinal location is determined. The scanning range of the millimeter-wave radar array is pre-set to ensure coverage of the entire spinal region, from the neck to the waist, and to accurately correspond to different parts such as the cervical, thoracic, and lumbar vertebrae, so as to obtain comprehensive spinal morphological data.

[0024] Using millimeter-wave radar arrays, the distribution of radar modules in space is rationally planned according to the required scanning angle and accuracy requirements. This generally includes arrangements in different directions such as front and back, left and right, and up and down, to ensure that the spine can be scanned from multiple perspectives in all directions, so that the acquired three-dimensional point cloud data can completely reflect the three-dimensional shape of the spine.

[0025] The millimeter-wave radar was set to operate in the 77-81 GHz band with a bandwidth of 4 GHz to ensure that the radar signal had sufficient resolution and penetration capability to clearly detect minute structural changes in the spine and surrounding tissues, providing high-quality raw data for subsequent accurate three-dimensional imaging.

[0026] Before starting, each radar module in the millimeter-wave radar array is precisely calibrated, including time synchronization and phase synchronization, to ensure the consistency of the transmitted and received signals of different modules, avoid measurement errors caused by time and phase differences between modules, and improve the accuracy and reliability of scanning data.

[0027] Subsequently, the central control system issues a scan start command to activate the millimeter-wave radar array. Each radar module then starts working sequentially according to the preset scan sequence, transmitting millimeter-wave signals toward the target's spinal region.

[0028] The emitted millimeter-wave signal is reflected when it encounters the human spine and surrounding tissues. The radar module receives the reflected signal and converts it into a digital signal, initially extracting data containing information such as the distance, angle, and velocity of the spine, laying the foundation for subsequent generation of three-dimensional point cloud data.

[0029] The millimeter-wave radar array changes the direction of transmitting and receiving signals one by one according to the preset scanning angle sequence to achieve scanning of the spine at different angles. After each angle change, signal transmission, reception and data acquisition operations are performed to gradually accumulate spinal reflection signal data obtained from various angles.

[0030] During the scanning process, the quality and completeness of the scan data are monitored in real time. If any missing or abnormal data is found at certain angles, the radar scanning parameters are adjusted in a timely manner or the corresponding angles are rescanned to ensure that the collected multi-angle data can completely and accurately reflect the three-dimensional morphology of the spine, providing rich and reliable input for subsequent deep learning processing.

[0031] In this embodiment, the subject faces the radar array with their back to it, maintaining a natural upright posture, and the distance between their back and the radar panel is fixed at 1.0m. Twenty scanning angles are used, with adjacent angles spaced 18° apart, and each angle is scanned for a specific time. The raw echo signal is acquired in real time through the radar SDK, and the attitude angle and displacement data of the IMU are recorded synchronously and stored in binary format (containing four dimensions of information: distance, azimuth, elevation, and echo intensity).

[0032] S2. Perform motion compensation and adaptive filtering on the radar echo signal to generate three-dimensional spine point cloud data; Data preprocessing and adaptive filtering: Raw data processing and format conversion: The received raw millimeter-wave radar signal data is processed to remove invalid information and noise interference, and converted into a unified data format to facilitate subsequent data processing and analysis, so that the data can be effectively utilized by algorithms such as deep learning networks.

[0033] Preliminary filtering and noise reduction: Simple filtering algorithms, such as bandpass filtering, are used to perform preliminary filtering on the raw data to remove high-frequency noise and low-frequency interference components that are obviously not part of the spinal reflection signal, such as environmental clutter and equipment self-noise, thereby improving the signal-to-noise ratio of the data and creating favorable conditions for subsequent accurate 3D point cloud data generation and feature extraction.

[0034] This invention fully considers respiratory and heart rate interference and eliminates it through a motion compensation algorithm. The specific operation steps are as follows: During the scanning process, the millimeter-wave radar receives reflected signals from the human spinal region in real time. These signals contain information on the micro-movements of the human body caused by breathing and heartbeat, as well as information on the static morphological characteristics of the spine.

[0035] Signal processing techniques, such as wavelet transform or short-time Fourier transform, are used to process the original radar signal, decompose the signal into components of different frequencies and time scales, and identify and separate the characteristic frequency bands corresponding to human micro-movements such as breathing and heartbeat. Generally, the breathing frequency is about 0.1-0.5 Hz, and the heartbeat frequency is about 1-2 Hz.

[0036] Based on the separated respiratory and heartbeat characteristic signals, motion models of respiratory and heartbeat are established respectively, and their variation patterns in time and space are analyzed. The influence of human micro-movement on spinal point cloud data is simulated, and the characteristics of spinal morphological changes caused by human micro-movement at different time points are determined.

[0037] Application of the compensation algorithm: Based on the constructed micro-motion model, motion compensation algorithms, such as Kalman filtering or particle filtering, are used to compensate the acquired spinal point cloud data in real time. At each scan moment, based on the current micro-motion model prediction value, the uncompensated point cloud data is inversely deformed to correct the point cloud data affected by breathing and heartbeat interference to the ideal position in a micro-motion-free state, thereby eliminating signal interference caused by human micro-motion and improving the stability of the spinal point cloud data.

[0038] Dynamic point cloud registration is performed using an improved ICP algorithm. The registration error is calculated using the improved ICP algorithm as follows: The improved ICP algorithm specifically includes the following steps: For motion-compensated radar point cloud data, feature points were selected using a curvature thresholding method based on spinal anatomical characteristics: the curvature value of each point was calculated, and points with curvature greater than 0.02m were retained. ¹ Points are used as key feature points of the spine. Redundant points in smooth areas are removed, and the feature point retention rate is 30%~50% of the original point cloud. Feature points such as vertebral body edges and intervertebral disc inflection points are retained, while redundant points in smooth areas of the back skin are removed. The feature point retention rate is 40% of the original point cloud. By combining the data from the inertial measurement unit integrated in the millimeter-wave radar array, the attitude angles during the scanning process are extracted and mapped to the coordinate system of the reference point cloud to complete the coarse registration; Read the scanning trajectory data collected by the IMU, extract the radar's pitch angle (average -2°), roll angle (average 0.5°), and yaw angle (0°~360°) at each angle, construct an initial registration matrix based on the Euler angle transformation formula, and map the point cloud to be registered at 20 angles to the reference coordinate system (taking the 10th angle as the reference). The root mean square error (RMSE) of the point cloud after coarse registration is 2.5mm (within the range of 2~3mm). The similarity between point pairs is calculated using a two-factor weighted method of Euclidean distance and radar echo intensity, establishing a correspondence between the point cloud to be registered and the reference point cloud: ; in, For point pairs ( The weighted distance of the points to be registered. The echo intensity is Reference point The echo intensity is , To achieve the maximum echo intensity in the scanned scene, Point pairs smaller than 2mm are considered valid corresponding point pairs; Set the weighting coefficient α = 0.7, and the maximum echo intensity. =500 (based on experimental scenario calibration), calculate the registration point. With reference point Weighted distance Point pairs with d < 1.8 mm (within the range of 1.5~2 mm) are retained, and the effective point pair retention rate is 82%. Based on valid correspondence pairs, singular value decomposition is used to solve for the optimal registration matrix. An adaptive iteration step size k is introduced. After each iteration, the RANSAC algorithm is used to remove erroneous correspondence pairs. Iteration stops when any of the following conditions are met: The number of iterations reaches the preset threshold or the change in RMSE between two consecutive iterations is less than 0.05 mm.

[0039] During the iteration process, at the 5th iteration ( - ) / =5%, step size adjustment factor =1 + 0.15 × 5% = 1.0075; in the 20th iteration ( - ) / =2.8% (<3%) The value was lowered to 0.8; after each iteration, RANSAC algorithm was used to remove erroneous point pairs, with an average removal rate of 12% (<15%); after 40 iterations (within the range of 30~50 iterations), the RMSE change ΔE was 0.04mm (<0.05mm) for two consecutive iterations, and the iteration was stopped. The final registration RMSE was 0.42mm (<0.5mm), and the point cloud overlap rate was 93% (>90%). During dynamic monitoring, new point cloud data is acquired in real time as the human posture changes or the scanning angle of the millimeter-wave radar changes. This data is then dynamically fused with the already registered point cloud data. A weighted average or credibility-based fusion strategy is used to organically combine the new point cloud data with the historical registered data to generate an updated overall three-dimensional point cloud model of the spine.

[0040] Post-processing of the registered point cloud data, such as smoothing and filtering for noise reduction, further improves the quality and accuracy of the point cloud model. Simultaneously, based on the analysis of registration errors, the parameters of the ICP algorithm are dynamically adjusted, such as adjusting the search radius of corresponding points and the number of iterations, to adapt to different point cloud data characteristics and changes in the scanning environment. This continuously optimizes the point cloud registration accuracy, ensuring that the final generated 3D spinal model accurately reflects the true morphology of the spine, providing high-quality data support for subsequent deep learning feature extraction and scoliosis angle calculation.

[0041] S3. Input the three-dimensional point cloud data into a deep learning network model, extract global features of the spine through a 3D convolutional neural network, and then locate key vertebrae based on an attention mechanism to obtain the coordinates of key vertebrae. This step mainly involves inputting the extracted spinal feature data into a key vertebral localization network based on an attention mechanism. Through the feature learning module of a 3D convolutional neural network, deep learning and analysis are performed on the features of different parts of the spine to extract feature information related to key vertebrae and to preliminarily screen the possible locations of key vertebrae, identifying several candidate key point regions. At the same time, the localization regression module in the 3D convolutional neural network is used to accurately locate the candidate key point regions, obtaining the specific location coordinates of the key vertebrae.

[0042] Feature extraction network construction: PointNet++ Feature Extraction: The PointNet++ network is used to extract features from 3D point cloud data layer by layer. First, key points are selected through a sampling module to reduce the number of points in the point cloud data. Then, a grouping module is used to group the remaining points according to spatial relationships to enhance the local feature extraction effect. Finally, a multilayer perceptron (MLP) is used to extract local features from each group of data, and the local and global features are fused through a feature propagation module to generate a feature vector containing global features of the point cloud.

[0043] VoxelCNN Feature Extraction: 3D point cloud data is divided into voxel grids, converting the point cloud into 3D volume data represented by voxels. A convolutional neural network (CNN) is used to perform convolution operations on the voxelized data, automatically extracting local and global features from the voxel data and capturing spatial relationships and morphological features.

[0044] This invention uses 500+ clinical case data to augment the training samples to 100,000, and introduces transfer learning (pre-trained ModelNet40 dataset) to enhance model training.

[0045] The point cloud features extracted by PointNet++ and the voxel features extracted by VoxelCNN are then fused. A feature concatenation method is used to combine the two types of features at a specific network layer, combining the local detail features of the point cloud and the overall structural features of the voxels to form a more comprehensive and richer feature representation.

[0046] The fused features are input into a 3D-CNN, and convolution operations are performed in three-dimensional space using 3D convolution kernels to automatically extract deep three-dimensional features of the spine, capture the complex structural relationships and morphological changes within the spine, and provide more effective feature representations for subsequent key vertebral localization and Cobb angle calculation.

[0047] Based on the extracted spinal features, a graph representation of the spine is constructed. Different parts of the spine (such as individual vertebrae) are regarded as nodes in the graph. The connection relationships between nodes are determined according to the anatomical knowledge of the spine and the adjacent relationships, and the edge structure of the graph is established to form a graph model that can reflect the internal structural relationships of the spine.

[0048] By using Graph Convolutional Networks (GCNs) to perform convolution operations on the constructed graph structure data, and by aggregating the feature information of nodes and their neighboring nodes, the node features are updated, enhancing the model's ability to model the interrelationships and mutual influences between different parts of the spine, and further mining the intrinsic feature associations and structural information of the spine.

[0049] The features obtained after graph convolution are fused with the previous features. By using strategies such as feature concatenation or weighted fusion, the feature relationship information mined by the graph convolutional network and the previously extracted feature information are integrated to form the final spinal feature representation. This feature is then output to the key vertebral localization and Cobb angle regression calculation module in the next stage.

[0050] S4. Calculate the Cobb angle using an angle regression network based on the coordinates of the key vertebrae in the localization. By identifying the specific coordinates of key vertebrae, key point pairs for calculating the Cobb angle are determined, and corresponding vectors are constructed based on these key points, such as the superior endplate point of the superior vertebra and the inferior endplate point of the inferior vertebra. These key point pairs represent the directional changes of the spine at different positions. An attention mechanism is introduced to weight the features output from S3, highlighting features related to key vertebrae. This enables the model to automatically focus on key vertebrae regions, enhancing its ability to learn key vertebrae features. The attention mechanism dynamically assigns different attention weights by performing point-by-point or channel-by-channel weighting operations on the feature map, allowing the model to pay more attention to the feature representation of key vertebrae.

[0051] A dedicated key vertebral localization network is constructed, comprising multiple convolutional layers, pooling layers, and fully connected layers. Convolutional layers are used for further feature extraction, pooling layers for dimensionality reduction and enhancing the translation invariance of features, and fully connected layers for outputting the position coordinates of the key vertebrae. The network takes features processed by an attention mechanism as input, and through layer-by-layer feature extraction and transformation, finally outputs the position coordinates of the key vertebrae (such as L1 / L5) in three-dimensional space.

[0052] The predicted coordinates of key vertebrae are optimized using methods such as Kalman filtering to smooth them and remove noise and fluctuations. The results are then validated and corrected by incorporating anatomical knowledge of the spine and the relative positions of adjacent key vertebrae, eliminating abnormal results that do not conform to anatomical principles to ensure the accuracy and rationality of the key vertebral location.

[0053] Using vector operation algorithms, the angle between the constructed vectors is calculated to obtain a preliminary Cobb angle estimate. The estimate is then input into a deep learning-based angle regression network. Through the network's learning and fitting of a large amount of training data, the preliminary angle value is corrected by regression.

[0054] The present invention also designs an angle estimation scoring function to compare the preliminary Cobb angle estimate with the corrected Cobb angle value to evaluate the accuracy of the preliminary estimate, and to achieve accurate estimation by feeding back to the system to adjust parameters.

[0055] ; in, This indicates the angle estimation score. This is the corrected Cobb angle value. This is a preliminary estimate of the Cobb angle. This represents the edge response intensity of the spinal texture to which the key vertebra belongs in radar scanning imaging. For local standard deviation, This is the spine texture enhancement factor (range 0.1–0.5). The innovation of this formula lies in the addition of a regularization term based on texture edge density. This feature enhances the system's response to "concentrated bright patches," and its response to edge texture-sensitive areas is about 28% higher than conventional methods, significantly improving the ability to capture actual interference sources.

[0056] When the angle is estimated and scored If the value exceeds a preset threshold, the vertebral coordinates are considered accurate, and no model improvement is necessary; when the angle estimation score... If the value is less than the preset threshold, it is considered that the difference between the corrected Cobb angle value and the initially estimated Cobb angle value is too large, and the model needs to be retrained.

[0057] S5. Output a 3D visualization model of the spine including Cobb angle annotations; 3D spine model construction stage: a. Point Cloud Data Processing and Pre-integration: The 3D point cloud data of the spine acquired by millimeter-wave radar is preprocessed, including noise removal and outlier filtering, to improve the quality of the point cloud data. Then, the preprocessed point cloud data is converted into a format suitable for 3D model construction, and preliminary point cloud registration is performed to ensure that the data acquired from different perspectives can be accurately aligned, providing a foundation for subsequent model construction.

[0058] b. Voxelization and Surface Reconstruction: The registered point cloud data is voxelized, dividing the 3D space into a regular voxel mesh. It is determined whether each voxel contains spinal point cloud data, forming a voxelized 3D spinal model. Based on this, surface reconstruction algorithms (such as Poisson surface reconstruction) are used to extract and fill the surface of the voxelized model, generating a 3D spinal model with a smooth, continuous surface, making it more consistent with the morphological characteristics of the actual spine.

[0059] c. Model optimization and smoothing: The reconstructed 3D spine model is optimized, including addressing issues such as holes and cracks on the model surface. Smoothing is also performed to reduce surface roughness and irregularities caused by factors such as data sampling, thereby improving the model's visual effect and quality and making it more intuitively represent the true shape of the spine.

[0060] Cobb corner annotation overlay stage: Angle information extraction and coordinate transformation: Accurately measured Cobb angle data, including angle values ​​and corresponding key vertebral position information, is obtained from the Cobb angle regression calculation module. This angle data is then converted from the coordinate system used in the calculation to a visual coordinate system that matches the 3D spine model, ensuring that angle annotations accurately correspond to their respective positions on the model.

[0061] Annotation element design and generation: Based on the angle data, design intuitive Cobb angle annotation elements, including text display of angle values, markings of key vertebral positions (such as markers of different colors or shapes), and lines or arcs representing the angle measurement range. Using visualization tools or libraries, generate corresponding annotation graphics on the 3D spine model according to the designed annotation element style, clearly showing the angle of scoliosis.

[0062] Visualization and Interactive Settings: The 3D spine model with Cobb angle annotations is displayed using professional visualization software or platforms. Appropriate viewing angles, lighting, and rendering effects are set to ensure the model and annotation information are clearly and accurately presented to the user. Interactive functions are added, such as model rotation, scaling, and translation, as well as the ability to show / hide annotation information. This allows users to view the spinal morphology and scoliosis angle from different angles and in detail, assisting doctors in diagnosis and analysis.

[0063] Please see Figure 2 As shown, in a second aspect of the present invention, a three-dimensional imaging system for the spine based on millimeter-wave radar and deep learning is provided, comprising: The millimeter-wave radar array scanning module is used to scan the back of the human body from multiple angles using a millimeter-wave radar array to obtain the raw radar echo signal. The signal processing module is used to perform motion compensation and adaptive filtering on the radar echo signal; The deep learning computing module is used to extract global features of the spine and obtain the coordinates of key vertebrae based on the input 3D point cloud data in two stages. The Cobb angle calculation module is used to calculate the angle value of the Cobb angle based on a combination of key point detection and angle regression network. The visualization generation module is used to generate a three-dimensional spine image based on the Cobb angle value, and the three-dimensional spine image is labeled with Cobb angle annotation elements.

[0064] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A three-dimensional imaging method for the spine based on millimeter-wave radar and deep learning, characterized in that, Includes the following steps: S1. The back of the human body is scanned from multiple angles using a millimeter-wave radar array to obtain the original radar echo signal; S2. Perform motion compensation and adaptive filtering on the radar echo signal to generate three-dimensional spine point cloud data; S3. Input the three-dimensional point cloud data into a deep learning network model, extract global features of the spine through a 3D convolutional neural network, and then locate key vertebrae based on an attention mechanism to obtain the coordinates of key vertebrae. S4. Calculate the Cobb angle using an angle regression network based on the coordinates of the key vertebrae in the localization. S5. Output a 3D visualization model of the spine including Cobb angle annotations.

2. The method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning according to claim 1, characterized in that, In step S1, the millimeter-wave radar array employs a MIMO antenna configuration, operates in the 77~81GHz frequency band, has a bandwidth greater than or equal to 4GHz, and a power density less than 10mW / cm². 2 The frame rate is greater than or equal to 30Hz, and an inertial measurement unit is integrated to achieve scanning trajectory tracking.

3. The method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning according to claim 1, characterized in that, In step S2, the adaptive filtering process includes the following steps: Eliminate breathing and heartbeat interference through motion compensation algorithms; An improved ICP algorithm is used for dynamic point cloud registration, and the registration error is reduced. Multi-frame data fusion based on Kalman filter.

4. The method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning according to claim 3, characterized in that, The improved ICP algorithm specifically includes the following steps: For motion-compensated radar point cloud data, feature points were selected using a curvature thresholding method based on spinal anatomical characteristics: the curvature value of each point was calculated, and points with curvature greater than 1 were retained. The points are used as key feature points of the spine. Redundant points in smooth areas are removed, and the feature point retention rate is 30% to 50% of the original point cloud. By combining the data from the inertial measurement unit integrated in the millimeter-wave radar array, the attitude angles during the scanning process are extracted and mapped to the coordinate system of the reference point cloud to complete the coarse registration; The similarity between point pairs is calculated using a two-factor weighted method of Euclidean distance and radar echo intensity, establishing a correspondence between the point cloud to be registered and the reference point cloud: ; in, For point pairs ( The weighted distance of the points to be registered. The echo intensity is Reference point The echo intensity is , To achieve the maximum echo intensity in the scanned scene, Point pairs smaller than 2mm are considered valid corresponding point pairs; Based on valid correspondence pairs, singular value decomposition is used to solve for the optimal registration matrix. An adaptive iteration step size k is introduced. After each iteration, the RANSAC algorithm is used to remove erroneous correspondence pairs. Iteration stops when any of the following conditions are met: The number of iterations reaches the preset threshold or the change in RMSE between two consecutive iterations is less than 0.05 mm.

5. The method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning according to claim 1, characterized in that, In step S3, the extracted spinal feature data is input into a key vertebral localization network based on an attention mechanism. Through the feature learning module of a 3D convolutional neural network, deep learning and analysis are performed on the features of different parts of the spine to extract feature information related to key vertebrae and to preliminarily screen the possible key vertebral locations to determine several candidate key point regions. At the same time, the localization regression module in the 3D convolutional neural network is used to accurately locate the candidate key point regions to obtain the specific location coordinates of the key vertebrae.

6. The method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning according to claim 1, characterized in that, In S4, key point pairs for calculating the Cobb angle are determined by the specific location coordinates of the key vertebrae, and corresponding vectors are constructed based on these key points to represent the directional changes of the spine at different positions. Using vector operation algorithms, the angle between the constructed vectors is calculated to obtain a preliminary Cobb angle estimate. The estimate is then input into a deep learning-based angle regression network. Through the network's learning and fitting of a large amount of training data, the preliminary angle value is corrected by regression.

7. The method for three-dimensional imaging of the spine based on millimeter-wave radar and deep learning according to claim 1, characterized in that, In step S5, based on the Cobb angle calculation data obtained in step S4, which includes angle values ​​and corresponding key vertebral position information, the coordinate system used in the calculation is converted from the corrected angle data to a visual coordinate system that matches the three-dimensional spine model. Subsequently, intuitive Cobb angle annotation elements are designed based on the Cobb angle calculation data, including text display of angle values, markings of key vertebral positions, and lines or arcs representing the angle measurement range. Using visualization tools, corresponding annotation graphics are generated on the three-dimensional spine model according to the designed annotation element style.

8. A three-dimensional spinal imaging system based on millimeter-wave radar and deep learning, characterized in that, Including those connected sequentially: The millimeter-wave radar array scanning module is used to scan the back of the human body from multiple angles using a millimeter-wave radar array to obtain the raw radar echo signal. The signal processing module is used to perform motion compensation and adaptive filtering on the radar echo signal; The deep learning computing module is used to extract global features of the spine and obtain the coordinates of key vertebrae based on the input 3D point cloud data in two stages. The Cobb angle calculation module is used to calculate the angle value of the Cobb angle based on a combination of key point detection and angle regression network. The visualization generation module is used to generate a three-dimensional spine image based on the Cobb angle value, and the three-dimensional spine image is labeled with Cobb angle annotation elements.

Images