A method for detecting scoliosis in adolescents based on three-dimensional imaging

Abstract

The present application belongs to the technical field of image processing, and mainly relates to a kind of detection method of adolescent scoliosis based on three-dimensional imaging, the method obtains the multi-modal data such as the back depth map, temperature distribution, attitude angle and foot bottom pressure matrix of the detected person by depth camera, infrared imaging equipment, IMU sensor and foot bottom pressure distribution sensor;Cloud platform generates three-dimensional point cloud and obtains back three-dimensional surface model using Poisson surface reconstruction algorithm, overall posture correction is carried out in combination with attitude angle and pressure gravity center offset, and thermal force correction model of local deformation is constructed based on thermal force diagram;The landmark point of spinal segment is automatically identified by deep learning algorithm, the spinal midline is fitted, and the Cobb angle is calculated, the degree of scoliosis is judged, and the analysis report is output;The present application does not need X-ray imaging, and the detection safety is high, can fuse posture, mechanics and thermal information, reduce artificial posture interference, and is suitable for early screening and dynamic review of adolescent.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and mainly relates to a method for detecting adolescent scoliosis based on three-dimensional imaging. Background Technology

[0002] Adolescent scoliosis is a common spinal deformity during growth and development, characterized by an abnormal lateral curvature of the spine in the coronal plane. It is often accompanied by spinal rotation and trunk asymmetry, and may lead to a series of problems such as skeletal deformities, limited cardiopulmonary function, and psychological disorders.

[0003] In the Chinese patent application CN202410197322.3, an image processing method and system for detecting scoliosis in adolescents is disclosed. This invention relates to the field of image processing technology and includes a method and system for detecting scoliosis in adolescents. The method includes obtaining a training set for a classification model, training the classification model to obtain a trained model, inputting the scoliosis image to be classified into the trained model, and classifying the scoliosis image. The method also includes inputting the scoliosis image to be predicted into a trained deep learning model to predict the center point of the spine. The scoliosis images include images of a single scoliosis, a double scoliosis, and a triple scoliosis. Different models are selected based on different image structures to predict the center point of the spine, making the output center point more accurate. This overcomes the difficulty of predicting the center point due to different structures of the scoliosis segments, thereby improving the accuracy of Cobb angle calculation.

[0004] The aforementioned methods can relatively accurately assess scoliosis by analyzing images of adolescent spines using image processing techniques and calculating the Cobb angle to evaluate the degree of curvature. However, current image processing technologies typically rely on X-ray images of adolescent backs as input data. Although the single radiation dose is small and the detection frequency is limited, there is still a certain radiation risk, especially for adolescents who are in the growth and development stage, where the potential impact of radiation cannot be ignored. Furthermore, existing detection methods often rely on single image analysis. In reality, adolescents may actively conceal their true spinal abnormalities through subtle means, such as adjusting their center of gravity and body posture, due to psychological reasons. This can make the degree of curvature less obvious in the images, ultimately affecting the accuracy of diagnosis and even missing the optimal time for intervention. Therefore, there is an urgent need for a scoliosis detection method based on three-dimensional imaging, without the need for X-ray image support, and capable of integrating multi-source data to improve detection accuracy, so as to more safely and efficiently serve the early screening and dynamic assessment of adolescents. Summary of the Invention

[0005] This invention provides a method for detecting adolescent scoliosis based on three-dimensional imaging, aiming to solve the problems of radiation risks and image analysis that are easily interfered with by the subject's active posture adjustment in existing detection methods that rely on X-ray imaging.

[0006] To solve the above problems, the present invention employs the following technology:

[0007] A method for detecting adolescent scoliosis based on three-dimensional imaging:

[0008] Multiple sensors are set up to acquire the body data of the subject, and the subject is guided to perform data collection in a standard posture to obtain raw data. The raw data obtained by the sensors is then uploaded to the cloud platform.

[0009] The cloud platform preprocesses the raw data obtained by the sensors and transforms it into standardized data; it then analyzes the standardized data to obtain a 3D point cloud dataset, a back heat map, attitude correction data, and pressure correction data for modeling.

[0010] Further effective point selection was performed based on the 3D point cloud dataset, and a 3D back surface model was reconstructed using the Poisson surface reconstruction algorithm. Posture correction data was then used to correct the posture of the 3D back surface model. A back thermal correction model was constructed using the back heatmap. Based on the 3D back surface model, a marker point recognition algorithm was used to extract spinal segment markers, obtaining the spinal midline morphology. The spinal midline was then thermally corrected based on the generated spinal midline and the back thermal correction model. The spinal midline was divided into upper and lower vertebral bodies, the Cobb angle was calculated, the scoliosis was assessed, and a report was generated.

[0011] The construction of the back thermal correction model specifically includes:

[0012] By jointly calibrating the internal and external parameters and synchronizing the time of the depth camera and infrared device, a coordinate mapping relationship between the back heat map and the back three-dimensional curved surface model is established, and the temperature data of the back heat map is projected and interpolated to the vertices of the back three-dimensional curved surface model.

[0013] Temperature-driven normal deformation is defined at the vertices of the model corresponding to the abnormal region. A Laplace smoothing algorithm based on thermal weights is used for iterative optimization to generate a back thermal correction model.

[0014] As a preferred real-time method, the acquisition of raw data specifically includes:

[0015] Multiple types of sensors are set up at the detection point, including depth cameras, infrared imaging devices, IMU sensors, and plantar pressure distribution sensors. The depth camera acquires the back depth map of the subject, the infrared imaging device acquires the back temperature data of the subject, the IMU sensor acquires the body posture data of the subject, and the plantar pressure distribution sensor acquires the plantar pressure data of the subject.

[0016] Guide the subject to perform data collection in a standard posture. The specific requirements are: the room temperature is below 28°C, guide the subject to stand on the foot pressure distribution sensor in the detection area with both feet parallel and apart, facing away from the depth camera and infrared imaging device, and wear the IMU sensor on the midline of the subject's back near the area from the seventh cervical vertebra to the first thoracic vertebra.

[0017] The acquired raw data is uploaded to the cloud platform via wireless network.

[0018] As a preferred real-time method, the conversion to standardized data specifically includes:

[0019] Image distortion correction is performed on the complete image data captured by the depth camera and the back temperature image data captured by the infrared thermal imaging device using an optical correction algorithm based on the intrinsic parameter matrix. The images are then scaled and cropped at the edges to remove background interference. A Gaussian filter image noise reduction algorithm is applied to reduce noise interference from the photosensitive device.

[0020] Kalman filtering algorithm is applied to the raw angular velocity and acceleration data in the body posture data collected by IMU sensor to reduce high frequency jitter and zero drift, and redundant data at the start of the acquisition process is cleared to unify the output dimensions and format.

[0021] Low-value noise points in the plantar pressure matrix of the plantar pressure distribution data are removed by thresholding to reduce interference in non-contact areas, and the left and right foot pressure maps are unified to the standard template coordinate system.

[0022] As a preferred real-time method, the analysis of standardized data specifically includes:

[0023] The depth map image data from the depth camera is processed, and the image depth is extracted using the camera intrinsic parameters to construct a 3D point cloud dataset of the back of the subject being detected.

[0024] By utilizing the difference between the human body and room temperature, a threshold is set, and point clouds of all points exceeding the threshold are extracted from the temperature image data to obtain a back heat map.

[0025] Complementary filtering is used to dynamically fuse the body posture data to obtain the posture angle of the current body posture in three-dimensional space; the deviation angle is calculated based on the angle between the acceleration vector and the standard gravity direction; and the posture correction data is obtained by merging the data.

[0026] For the plantar pressure distribution map, the plantar pressure distribution map is symmetrically divided into two parts, and the left and right contact areas of the plantar surface are calculated for each part to obtain pressure correction data including the left and right contact areas of the plantar surface and the distribution map.

[0027] As a preferred real-time method, the reconstruction of the back 3D curved surface model using the Poisson surface reconstruction algorithm specifically includes:

[0028] Based on the distance between the camera and the detection point, a depth range is set, the effective point set of the back of the subject is extracted, and statistical filtering is used to remove outliers from the effective point set.

[0029] The normal vector is calculated based on the filtered valid point set. Then, the position of the depth camera is used as the reference viewpoint, and the direction of the normal vector is adjusted to be consistent with the viewpoint.

[0030] An adaptive octree spatial structure is constructed, and a cube-shaped bounding space covering the entire point cloud range is initialized using the maximum boundary length of the point cloud data as the scale.

[0031] The cube is segmented, and the segmentation depth is dynamically adjusted according to the point cloud density.

[0032] For each segmented node, a corresponding third-order B-spline basis function is constructed, and the corresponding gradient vector field is calculated by numerical differentiation through the connection relationship between nodes.

[0033] For each sampling point in the input point cloud, multiple basis functions are selected in its neighborhood. Gaussian weights are used to assign weight coefficients based on the spatial relationship between the point and the center of each basis function. The weighted linear combination of the gradient vectors of all basis functions in the neighborhood of the currently processed point is used to obtain the approximate normal vector at that sampling point. This process is performed point by point on the entire point cloud to construct a complete normal vector field.

[0034] Based on the estimated normal vector field, a discrete system of the Poisson equation is constructed. A sparse matrix structure is used to assemble the coefficients of the gradient of the B-spline basis function. Zero normal flux boundary conditions are introduced to constrain boundary deformation, and zero mean constraints are superimposed to eliminate constant drift of the solution.

[0035] The linear system is solved iteratively using the conjugate gradient method to obtain a scalar potential function defined in three-dimensional space.

[0036] Based on the moving cube algorithm, isosurfaces are extracted from the potential function to generate a three-dimensional curved surface model of the back side.

[0037] As a preferred real-time method, the posture correction of the three-dimensional curved surface model of the back specifically includes:

[0038] The body posture data output by the IMU sensor is converted into the reconstructed coordinate system of the three-dimensional curved surface model of the back;

[0039] The attitude angles in the obtained attitude correction data are used as the basis for overall correction, and the back 3D curved surface model is corrected by performing an inverse transformation around the coordinate axis.

[0040] Using the deviation angle in the obtained attitude correction data as a global constraint, the attitude normalization process is performed on the back 3D curved surface model.

[0041] Based on the left and right contact areas of the sole in the pressure correction data, calculate the actual center of gravity, ignoring the specific gravity data, and calculate the geometric center of gravity of the sole.

[0042] Calculate the Euclidean distance between the actual centroid and the geometric centroid to determine the offset.

[0043] The difference between the actual center of gravity and the geometric center of gravity is converted into front-back and left-right components, and the back 3D curved surface model is corrected in reverse according to the difference components.

[0044] As a preferred real-time method, the extraction of spinal segment landmarks using a landmark recognition algorithm specifically includes:

[0045] The coordinates of the spinal segment landmarks at the top and bottom of the spine in three-dimensional space are identified using a landmark recognition algorithm.

[0046] The marker recognition algorithm directly processes 3D point cloud data through the PointNet network architecture. It independently encodes the features of each point using a shared multilayer perceptron and achieves permutation invariance global feature aggregation through max pooling, overcoming the unstructured nature of point clouds. It is pre-trained on a point cloud dataset containing markers to obtain a converged model that can identify the part of the point cloud data containing markers.

[0047] Based on the extracted landmark points, the midline of the spine is fitted using piecewise cubic B-spline curves to obtain the midline morphology of the spine in the three-dimensional curved surface model of the back, and the center points of each spinal segment landmark are marked.

[0048] As a preferred real-time method, the thermal correction of the spinal midline specifically includes:

[0049] Extract anomalous regions where the local temperature difference exceeds 1.5℃, and apply deformation displacement along the geometric normal vector of the corresponding vertex at the corresponding vertex.

[0050] The position is optimized by using thermal weighted Laplace smoothing, and the piecewise cubic B-spline curve is resolved based on the corrected control point set to obtain the midline of the spine that fits the thermal condition.

[0051] As a preferred real-time method, the determination of scoliosis and the output of a report specifically include:

[0052] Distinguish between the upper and lower vertebrae of the spine based on the markings along the midline of the spine.

[0053] For the upper and lower cones, calculate the inclination of each marked point;

[0054] For the markers with the greatest inclination of the upper and lower vertebrae, the Cobb angle is calculated. Using the two adjacent markers as auxiliary points, the inclination direction vectors of the upper and lower vertebrae are constructed respectively. Then, the Cobb angle of the spine in this segment is obtained by using the angle between these two vectors.

[0055] The process involves assessing scoliosis based on the Cobb angle, integrating the analysis, and outputting a report that includes analytical model images and scoliosis details.

[0056] The beneficial effects of this invention are:

[0057] 1. Improved detection safety: The non-radiation detection method based on three-dimensional imaging avoids the ionizing radiation risk of X-ray imaging and is more suitable for repeated testing of adolescents.

[0058] 2. Enhance the stability and accuracy of posture analysis: By integrating multi-source data such as posture angle, support force distribution and back heat map, the body posture of the subject is dynamically corrected, significantly reducing the interference of human posture control on the test results.

[0059] 3. Reconstructing the true morphology of the spine and quantifying the evaluation results: By using point cloud reconstruction and thermally guided model deformation correction, combined with automatically identified vertebral landmarks and midline fitting algorithms, a three-dimensional model of the spine is constructed, and the Cobb angle is automatically calculated to achieve objective quantitative analysis of the degree of scoliosis, which facilitates early clinical diagnosis and intervention.

[0060] Legend

[0061] Figure 1 This is a flowchart of a method for detecting adolescent scoliosis based on three-dimensional imaging, as described in this invention. Detailed Implementation

[0062] To make the technical means, creative features, and achieved objectives and effects of this invention easier to understand, the invention is further described below with reference to specific embodiments. However, the following embodiments are merely preferred embodiments of this invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are all within the protection scope of this invention. Unless otherwise specified, the experimental methods in the following embodiments are conventional methods, and the materials and reagents used in the following embodiments are commercially available unless otherwise specified.

[0063] Example 1 Figure 1 A flowchart of a method for detecting adolescent scoliosis based on three-dimensional imaging is shown. This embodiment provides a method for detecting adolescent scoliosis based on three-dimensional imaging, specifically including the following steps:

[0064] Step 1: Acquire multimodal data about the subject's spine using multiple sensors and upload the acquired data to the cloud platform:

[0065] The specific steps are as follows: various types of sensors are set up at the detection point, the subject is guided to perform data collection in a standard posture, and the acquired data is uploaded to the cloud platform via wireless network.

[0066] Specifically, setting up multiple types of sensors at the detection point means that the sensor types include depth cameras, infrared imaging devices, IMU sensors (inertial measurement units), and plantar pressure distribution sensors. The depth camera acquires the depth map of the subject's back; the infrared imaging device acquires the temperature data of the subject's back; the IMU sensor acquires the subject's body posture data, including three-axis posture angles and dynamic stability information; and the plantar pressure distribution sensor acquires the plantar pressure data of the subject.

[0067] Guiding the subject to perform data collection in a standard posture means: in an environment with a room temperature below 28°C, guiding the subject to stand on the foot pressure distribution sensor in the detection area with both feet parallel and apart; facing away from the depth camera and infrared imaging device, which can clearly obtain a complete image and thermal data of the subject's back; and wearing the IMU sensor on the midline of the subject's back near the seventh cervical vertebra to the first thoracic vertebra to obtain the subject's body posture data.

[0068] Step 2: After receiving the raw data, the cloud platform preprocesses the raw data to obtain standardized data, and then analyzes the standardized data to obtain modeling data.

[0069] Specifically, preprocessing the raw data involves: applying an optical correction algorithm based on an intrinsic parameter matrix to correct image distortion in the complete image data acquired by the depth camera and the back temperature image data acquired by the infrared thermal imaging device; performing aspect ratio unification and edge cropping on the images to remove background interference; applying a Gaussian filter image denoising algorithm to reduce noise interference from the photosensitive device; and applying a Kalman filter algorithm to the raw angular velocity and acceleration data in the body posture data acquired by the IMU sensor, establishing a state-space model including angle, angular velocity, and acceleration bias, and using state equations to predict the system state at the next moment. The system calculates the predicted covariance and Kalman gain based on the current sensor measurements. It then adjusts the weight ratio between the predicted and measured values ​​based on the Kalman gain, generating the optimal state estimate through weighted fusion. This effectively filters out high-frequency noise and compensates for sensor zero-point drift error in real time while preserving the true motion trend. Furthermore, it removes redundant data generated during the acquisition process and standardizes the output dimensions and format. Low-value noise points in the plantar pressure matrix of the plantar pressure distribution data are thresholded to reduce interference in non-contact areas, and the left and right foot pressure maps are unified into a standard template coordinate system, providing a foundation for subsequent matching or analysis.

[0070] The analysis of standardized data refers to the following steps: First, the depth map image data from the depth camera is processed to construct a 3D point cloud dataset of the subject's back. Each pixel in the image data output by the depth camera contains a depth value, which is the distance from the camera. The depth camera itself also contains an imaging parameter, including focal length and principal point coordinates. Based on the coordinate position of the pixel in the image, combined with the depth value, a 3D coordinate value of each pixel is formed, thus forming a complete 3D point cloud dataset for modeling to represent the surface structure of the subject's back.

[0071] Subsequently, using the back temperature image data obtained from the infrared imaging device, and taking advantage of the difference between the human body and room temperature, a threshold of 32℃ was set, and values ​​greater than 32℃ in the temperature image data were extracted to obtain a back thermal map.

[0072] Then, for the body posture data obtained by the IUM sensor, including acceleration and angular velocity, complementary filtering is used to dynamically fuse the acceleration and angular velocity data to obtain the posture angle of the current body posture in three-dimensional space. Next, the deviation angle is calculated. When the subject is standing still, the acceleration value collected by the IUM sensor is mainly composed of gravitational acceleration. Therefore, this acceleration vector can be used as an approximate reflection of the direction of the torso's center of gravity. The spatial angle between the direction of this acceleration vector in the sensor coordinate system and the standard gravity direction (i.e., the vertical downward direction) is calculated. The posture angle and the deviation angle are combined to obtain the posture correction data.

[0073] Finally, for the plantar pressure distribution data, the difference in support force between the left and right sides is calculated. Specifically, when a person is standing still, the contact pressure distribution between the feet and the ground can be regarded as a scalar field of a set of discrete sampling points. Each sampling point reflects the normal pressure value at that location. According to the principle of mechanics, the total support force of a single foot is equal to the integral of the pressure values ​​of all contact points of that foot and the corresponding area. Based on the standard posture plantar pressure distribution data, the left and right sides are symmetrically divided into two parts, and the integral of the corresponding areas is calculated to obtain the difference in support force between the left and right sides. The support area and distribution map are calculated. The plantar support area reflects the actual contact range between the foot and the ground. Insufficient contact area or significant differences between the left and right sides are usually related to habitual uneven loading of the standing posture, abnormal foot posture, etc. In the discrete sampling pressure field, when the pressure value of a certain point exceeds the minimum contact threshold, it can be regarded as an effective contact point. The specific minimum contact threshold is set according to the weight of the subject. The weight can be divided by the average plantar contact area of ​​subjects of the same weight to obtain the average unit pressure, and then multiplied by the empirical coefficient for processing similar weights as the threshold. Through the statistics of effective contact points, the support area and distribution map are obtained. These are combined to form pressure correction data.

[0074] Step 3: Construct a virtual back model of the subject based on the modeling data, refine the virtual back model using other data, and generate a thermally corrected back model by combining the back heatmap.

[0075] The specific steps are as follows: construct the back model of the subject based on the 3D point cloud dataset; load and analyze the posture correction data and pressure correction data to correct the back model; and construct the back thermal distribution model based on the back model image and the back heat map.

[0076] Specifically, constructing the back model of the subject involves: in the 3D point cloud dataset of the back image, each point has a depth value. A depth range is set based on the distance between the camera and the detection point. This depth range is fine-tuned according to the actual distance to ensure that the back of the subject can be preserved. Based on this depth range, the 3D point cloud dataset is filtered, and points within the range are retained as the valid point set. Subsequently, the valid point set is further processed using statistical filtering. For each point, a neighborhood is constructed by taking its 15 nearest neighbors. The mean distance of each point to other points in its neighborhood is calculated. Then, the distribution of the mean distance of all points is statistically analyzed. If the mean distance of a point exceeds the overall mean plus 1.5 times the standard deviation, the point is considered an outlier and is removed, thus retaining a densely distributed and structurally reasonable valid point set.

[0077] After removing outliers from the valid point set, the Poisson surface reconstruction algorithm is used to accurately reconstruct the 3D curved surface model of the subject's back. The specific process is as follows: First, the normal vector is calculated based on the filtered valid point set. For each point, its 15 nearest neighbors are selected, and the eigenvector corresponding to the smallest eigenvalue is determined through covariance matrix eigenvalue decomposition as the initial normal direction. Then, using the depth camera position as a reference viewpoint, the normal vector direction is adjusted to align with the viewpoint. The point being processed is connected to the viewpoint, and the angle between the normal vector and the line segment is calculated. If the angle is greater than 90°, it indicates that the normal vector points in the opposite direction, and it is reversed to avoid errors during surface reconstruction. To address directional ambiguity, an adaptive octree spatial structure is constructed. Using the maximum boundary length of the entire point cloud data as the scale, a cube-shaped bounding space covering the entire point cloud area is initialized and generated as the octree root node. The cube is then segmented, with the segmentation depth dynamically adjusted based on the point cloud density. An initial maximum depth of 8 is set to ensure fine representation of high-density regions and maintain a reasonable topology in low-density regions; specific settings can be modified according to actual conditions. Within the 3D space partitioned by the octree, for each subdivided leaf node, a locally supported third-order B-spline basis function is constructed in its corresponding space. This basis function exists only within the current node and its adjacent nodes. The basis functions have non-zero values ​​to ensure efficient local computation and numerical stability. Based on predefined octree node connections (such as topological relationships between parent-child and adjacent nodes), gradient differentiation is performed on each basis function. Specifically, numerical differentiation is used to calculate the gradient components of the basis function in the X, Y, and Z directions in three-dimensional space, thus obtaining the corresponding gradient vector field. Subsequently, for each sampling point in the input point cloud, multiple basis functions are selected in its neighborhood. Gaussian weights are used to assign weight coefficients based on the spatial relationship between the point and the centers of each basis function, ensuring that closer basis functions have a greater influence on the point. This process applies to all basis functions in the neighborhood of the currently processed point. The approximate normal vector at the sampling point is obtained by weighted linear combination of the gradient vectors. This process is performed point by point on the entire point cloud to construct a complete normal vector field. Based on the estimated normal vector field, a discrete system of the Poisson equation is constructed. The coefficients of the B-spline basis function gradients are assembled using a sparse matrix structure. A zero normal flux boundary condition is introduced to constrain boundary deformation, and a zero mean constraint is superimposed to eliminate constant drift of the solution, thereby ensuring the uniqueness and stability of the Poisson equation solution. The zero normal flux boundary condition means that in the boundary region of the point cloud, the component of the corresponding basis function gradient in the boundary normal direction is set to zero, indicating that the flow field does not penetrate at the boundary.This is achieved by adding a linear constraint with zero normal components to the boundary sampling points. The zero-mean constraint means adding an additional condition to the overall solution space so that the average of all estimated normals is zero, setting the sum of the normal vectors of all points to zero, and adding it as a linear constraint equation to the optimization objective. The constructed linear Poisson system is iteratively solved using the conjugate gradient method to obtain a scalar potential function field defined in three-dimensional space. Subsequently, based on the moving cube algorithm, specific isosurfaces are extracted from this potential function. The isosurface threshold is determined by automatically calculating the median of the potential function values ​​corresponding to all point clouds, thereby ensuring that the extracted isosurfaces are closest to the original data distribution, and finally generating a smooth, closed, and topologically reasonable back 3D curved surface model.

[0078] Loading and analyzing attitude correction data and pressure correction data to correct the back model involves: first, converting the roll, pitch, and yaw angle data output by the IMU sensor into the reconstructed coordinate system of the 3D back model; then, using the attitude angles in the obtained attitude correction data as the basis for overall correction, performing an inverse transformation around the coordinate axes to correct the overall rotation, tilt, or distortion caused by posture deviations during the detection process, thereby adjusting the model's attitude to the standard reference attitude; finally, using the deviation angles in the obtained attitude correction data as global constraints, performing attitude normalization processing on the 3D surface model, i.e., performing a reverse rotation of the model around the coordinate axes. The process involves a rotational maneuver to gradually eliminate postural distortions such as tilt and rotation caused by the subject's body position deviation, ensuring that the model's shape remains consistent with the actual back structure in a natural standing position. Subsequently, based on the left and right contact areas of the soles and the position of the pressure center of gravity in the pressure correction data, the actual center of gravity is calculated. This actual center of gravity can be obtained by weighted averaging the pressure values ​​of each measurement point on the sole pressure distribution map with their corresponding coordinates. The pressure value of each sole region is multiplied by its lateral and longitudinal coordinates to obtain the sum of the lateral and longitudinal moments of all regions. Dividing this sum by the total pressure yields the center of gravity of the human body. The horizontal and vertical coordinates on the plane are used to determine the average projected position of gravity acting on the ground when a person is standing still, i.e., the actual center of gravity. Next, based on the contact area map in the pressure correction data, the geometric center of gravity is obtained. Each point within these contact areas is treated as a pixel unit with uniform mass, and their coordinate values ​​in the horizontal and vertical directions are calculated separately. Then, the horizontal coordinates of all these points are averaged to obtain the center position of the contact area in the horizontal direction, and the vertical coordinates are averaged to obtain the center position in the vertical direction. Finally, combining the average coordinates in these two directions yields the geometric center of gravity of the entire sole contour area. The center of gravity is determined; then, the Euclidean distance between the actual center of gravity and the geometric center of gravity is calculated. When it exceeds a set threshold, a correction is triggered. The initial threshold is set to 0.7cm, which is adjusted according to the actual weight of the person being tested. If the weight is too high, the threshold can be appropriately increased. Then, the difference is converted into a correction value. Specifically, the difference is decomposed into anterior-posterior and lateral components, which correspond to the posture offsets in the sagittal and coronal planes of the spine, respectively. Based on the offset, the three-dimensional curved surface model of the back is corrected in reverse to restore the natural spinal shape of the person being tested. This results in a three-dimensional curved surface model of the back that has been largely free from human active control.

[0079] Combining the back thermal map with the construction of the back thermal correction model refers to: establishing the coordinate mapping relationship between the back thermal map and the back 3D curved surface model; completing the emissivity and radiometric calibration of the thermal image data through joint calibration and time synchronization of the intrinsic and extrinsic parameters of the depth camera and infrared device; using a target that can be recognized by both devices simultaneously for cross-modal registration; projecting and interpolating the temperature data of the back thermal map to the vertices of the back 3D curved surface model; performing visibility judgment for occluded and out-of-view areas and filling in void areas; determining the areas requiring deformation based on temperature anomaly detection: using the temperature difference of the left and right symmetrical ROIs along the spine as the judgment criterion. The model uses the deviation of the current temperature from the average temperature difference as a criterion, initially using 1.5℃ as the upper and lower limits to identify abnormal temperature regions. For detected abnormal regions, a temperature-driven normal deformation is defined at the corresponding vertex of the model. The displacement of the normal deformation is proportional to the deviation of the vertex temperature relative to the back temperature baseline (the average of the mapped region). Next, an iterative optimization is performed using a Laplace smoothing algorithm based on thermal weights. Greater smoothing weights are assigned to regions with smaller temperature differences to suppress noise, while smoothing is reduced in feature regions with significant temperature differences to preserve deformation details. An iterative convergence threshold is set to ensure the stability of the optimization process. Ultimately, the generated back thermal correction model not only preserves the consistency of the back bony structure but also demonstrates the asymmetric changes in muscle tension through a back heatmap.

[0080] Step 4: Using the generated virtual model, locate the midline of the subject's spine, correct the shape of the midline using a thermal correction model, analyze the subject's scoliosis based on this, and output a scoliosis report.

[0081] Specifically, analyzing scoliosis involves: First, locating the midline of the spine based on a 3D back surface model; obtaining the coordinates of spinal segment landmarks (such as C7 and L5 / S1) at the top and bottom of the spine in 3D space, and identifying these key landmarks using a landmark recognition algorithm; the landmark recognition algorithm directly processes 3D point cloud data through the PointNet network architecture, independently encoding the features of each point using a shared multilayer perceptron, and achieving permutation-invariant global feature aggregation through max pooling to overcome the unstructured nature of point clouds; based on this deep learning method, an end-to-end training process is constructed: first, a dataset containing the 3D coordinates of spinal segment landmarks such as C7 / L5 / S1 and their corresponding point clouds is prepared, and the point coordinates are... Using the normal vector as network input, the predicted position is output after forward propagation. The deviation between the predicted value and the true coordinate is calculated using a smoothed L1 loss function, and spatial vector consistency is constrained by directional cosine similarity. The network parameters are dynamically optimized through backpropagation until convergence. The back point cloud data of the subject is input into the PointNet model to generate the coordinates of the marker points. After obtaining the coordinates of the spinal segment markers C7 and L5 / S1, the key points of the spinal segment markers T6 / T12 / L3 and other vertebral bodies are identified as control nodes in the same way. The midline of the spine is fitted with a piecewise cubic B-spline curve to obtain the midline shape of the spine in the three-dimensional curved surface model of the back, in which the center point of each spinal segment marker is marked as a marker point.

[0082] Next, based on the generated thermal correction model of the spinal midline and back, the midline geometric correction is driven by the mapping relationship between the back thermal map temperature distribution and the vertex of the three-dimensional surface model: abnormal areas with local high and low temperature differences exceeding 1.5℃ (usually corresponding to muscle tension imbalance zones) are extracted, and deformation displacement along the geometric normal vector of the corresponding vertex is applied at the corresponding vertex. The displacement amount in the deformation displacement is proportional to the temperature difference, and the initial preset proportionality coefficient is 0.2mm / ℃, with an upper limit constraint of 3mm. For the vertebral control points affected by thermal deformation, thermal weighted Laplacian smoothing is used for position optimization. In the gentle area with a temperature difference of less than 0.5°, strong smoothing weight is applied to maintain the stability of the bony structure, while in the abnormal zone area with a temperature difference greater than 1.5°, weak smoothing weight is used to preserve the deformation characteristics. Finally, based on the corrected control point set, the piecewise cubic B-spline curve is resolved, and the spinal midline with fitted thermal conditions is obtained through analysis.

[0083] Finally, the scoliosis was analyzed based on the obtained spinal midline. Markings on the midline distinguished between the upper and lower vertebrae of the spine; specifically, L1-L3 were the upper vertebrae, and the rest were the lower vertebrae. For both the upper and lower vertebrae, the inclination at each marker point was calculated. Specifically, the first derivative of a cubic B-spline curve was calculated at each marker point to obtain the tangent vector direction. The angle between this tangent vector and the vertical unit vector was calculated, and the marker point with the largest angle was recorded as having the largest inclination. For the marker points with the largest inclination in both the upper and lower vertebrae, the Cobb angle was calculated, using the two adjacent marker points as auxiliary points. The tilt direction vectors of the upper and lower vertebrae are constructed separately. Then, the angle between these two vectors is used as the basis for calculating the Cobb angle. That is, the angle between these two direction vectors is calculated by using the inverse cosine function to obtain the degree of curvature of the spine in this scoliosis segment and determine whether there is a problem with scoliosis. If the angle exceeds the clinically defined threshold of 10°, it is determined to be scoliosis. At the same time, the severity is assessed according to the size of the angle: Cobb angle of 10°-25° is mild, 25°-45° is moderate, and more than 45° is severe. After obtaining the results, a report including analysis model images and scoliosis information is output.

[0084] Example 2:

[0085] In order to conduct early screening and dynamic follow-up for adolescent scoliosis, a hospital deployed the adolescent scoliosis detection system based on three-dimensional imaging described in this invention. The room temperature at the deployment site was maintained at approximately 22°C, with uniform lighting and no strong shadows or reflective areas.

[0086] Before the test, medical staff first fixed a foot pressure distribution sensor pad on the ground in the testing area. A depth camera and an infrared thermal imaging device were installed on one side about 2 meters above the ground. The two were kept in a fixed position by a fixed bracket and were jointly calibrated. After being disinfected, the IMU sensor was worn on the back of the adolescent being tested, between the seventh cervical vertebra and the first thoracic vertebra, and fixed in place to prevent slippage.

[0087] The subject stands in the center of the foot pressure distribution sensing pad as instructed, with feet parallel and shoulder-width apart, facing away from the camera and thermal imaging equipment. Raw body data is collected, and then uploaded to the cloud platform via wireless network.

[0088] On the cloud platform, the received raw data was first preprocessed. Then, the cloud platform further analyzed the data to obtain a 3D point cloud dataset, and used the Poisson surface reconstruction algorithm to obtain a smooth, closed 3D curved surface model of the back. Combining IMU posture angle data and plantar pressure distribution data analysis, it was found that due to the light weight of the subject, the plantar pressure distribution data analysis revealed that the subject had a certain center of gravity shift, which may lead to inaccurate identification of spinal segments and errors, resulting in the masking of scoliosis. Therefore, the model was modified according to the center of gravity shift to reduce the error in spinal detection results caused by the subject's instability or incorrect posture.

[0089] Subsequently, the back thermal map is projected onto the vertex of the 3D curved surface model. Based on the symmetrical temperature difference distribution, thermally driven normal deformation is applied to the local area, and the back thermal correction model is formed through Laplace smoothing optimization of thermal weights.

[0090] After the model is established, the landmark recognition algorithm is called to identify the spinal segment landmarks of the model, extract the coordinate points of key vertebrae such as C7, L5 / S1 and T6 / T12 / L3, and use piecewise cubic B-spline curves to fit the midline of the spine; for areas where the local high and low temperature difference exceeds 1.5℃, midline geometric correction is performed to obtain the final fitted midline of the spine.

[0091] Finally, the maximum tilt point of the upper and lower vertebrae is calculated, and the Cobb angle is calculated based on the maximum tilt point, and a test report is generated. The report shows that the Cobb angle of a certain subject's spine is 13.7°, which is determined to be mild scoliosis. The report also includes a three-dimensional back model, a midline morphology diagram of the spine, and a mechanical and thermal distribution analysis diagram, which can be used by doctors to formulate subsequent rehabilitation training plans and follow-up examination plans.

[0092] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting scoliosis of adolescents based on three-dimensional imaging, characterized by: setting up various sensors to obtain body data of the detected person, guiding the detected person to collect data in a standard posture, obtaining raw data, and uploading the raw data obtained by the sensors to a cloud platform; the cloud platform pre-processes the raw data obtained by the sensors and converts them into standardized data; analyzes the standardized data to obtain a three-dimensional point cloud data set for modeling, a back heat map, posture correction data and pressure correction data; further effective point screening is performed according to the three-dimensional point cloud data set, and a back three-dimensional surface model is reconstructed by using a Poisson surface reconstruction algorithm; the back three-dimensional surface model is corrected in posture in combination with the posture correction data; a back heat correction model is constructed in combination with the back heat map; and a spinal segment marker is extracted by using a landmark recognition algorithm according to the back three-dimensional surface model to obtain a spinal midline shape; the spinal midline is corrected in heat based on the generated spinal midline and back heat correction model; the spinal midline is divided into upper and lower vertebral bodies, the Cobb angle is calculated, the scoliosis condition is judged, and a report is output; the back heat correction model specifically includes: the coordinate mapping relationship between the back heat map and the back three-dimensional surface model is established through joint calibration of the internal and external parameters of the depth camera and the infrared device and time synchronization, the temperature data of the back heat map is projected and interpolated to the vertices of the back three-dimensional surface model; a temperature-driven normal deformation is defined at the vertices of the model corresponding to the abnormal area, and a Laplace smoothing algorithm based on heat weight is used for iterative optimization to generate a back heat correction model.

2. The method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: the raw data is specifically obtained by: setting up various types of sensors at the detection points, the sensor types including a depth camera, an infrared imaging device, an IMU sensor and a plantar pressure distribution sensor, obtaining a back depth map of the detected person by the depth camera, obtaining back temperature data of the detected person by the infrared imaging device, obtaining body posture data of the detected person by the IMU sensor, and obtaining plantar pressure data of the detected person by the plantar pressure distribution sensor; the detected person is guided to collect data in a standard posture, and the specific requirements are that the room temperature is lower than 28℃, the detected person is guided to stand on the plantar pressure distribution sensor in the detection area, the feet are parallel and apart, the back is against the depth camera and the infrared imaging device, and the IMU sensor is worn on the detected person's back midline near the seventh cervical vertebra to the first thoracic vertebra region; the obtained raw data is uploaded to the cloud platform through a wireless network.

3. A method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: the standardized data is specifically converted by: applying an optical correction algorithm based on an internal parameter matrix to correct the image distortion of the complete image data collected by the depth camera and the back temperature image data collected by the infrared thermal imaging device, uniformly scaling the image and edge cropping, eliminating background interference, and applying a Gaussian filter image noise reduction algorithm to reduce the noise interference of the photosensitive device; The Kalman filtering algorithm is applied to original angular velocity and acceleration data in body posture data collected by the IMU sensor, high-frequency jitter and zero-point drift are reduced, and redundant data during collection is removed, and the output dimension and format are unified; Threshold rejection is performed on low-value noise points in the plantar pressure matrix in the plantar pressure distribution data, non-contact area interference is reduced, and left and right foot pressure maps are unified to a standard template coordinate system.

4. The method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: The analysis of the standardized data specifically includes: The depth map image data of the depth camera is processed, the image depth is extracted using the camera internal parameters, and a three-dimensional point cloud dataset of the back of the detected person is constructed; A threshold is set using the difference between the human body and the room temperature, all point clouds in the temperature image data that exceed the threshold are extracted, and a back thermal map is obtained; The body posture data is dynamically fused using a complementary filtering method, and the posture angle of the current body posture in the three-dimensional space is obtained; the deviation angle is calculated according to the included angle between the acceleration vector and the standard gravity direction; and the posture correction data is obtained by merging. For the plantar pressure distribution map, the plantar pressure distribution map is symmetrically divided into two parts, the left and right contact areas of the plantar pressure distribution map are calculated for each part, and the pressure correction data including the left and right contact areas of the plantar pressure distribution map is obtained.

5. The method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: The Poisson surface reconstruction algorithm specifically includes: According to the distance between the camera and the detection point, a depth range is set, the effective point set of the detected person's back is extracted, and statistical filtering is used to remove outliers in the effective point set; Based on the filtered effective point set, the normal vector is calculated, and then the depth camera position is taken as the reference viewpoint, and the normal vector direction is adjusted to be consistent with the viewpoint; An adaptive octree spatial structure is constructed, and the maximum boundary length of the point cloud data as a whole is taken as the scale to initialize the generation of a cubic enclosing space covering the entire point cloud range; The cube is segmented, and the segmentation depth is dynamically adjusted according to the point cloud density; For each node obtained by segmentation, a corresponding third-order B-spline basis function is constructed, and the corresponding gradient vector field is calculated by numerical differentiation through the connection relationship between nodes; For each sampling point of the input point cloud, a plurality of basis functions are selected in its neighborhood, a weight coefficient is assigned to the sampling point according to the spatial position relationship between the point and each basis function center using Gaussian weight, and the approximate normal vector at the sampling point is obtained by weighted linear combination of all basis function gradient vectors in the neighborhood of the current processing point. The process is executed point by point on the entire point cloud to construct a complete normal vector field. Based on the estimated normal vector field, a discrete system of Poisson equation is constructed, the B-spline basis function gradient is assembled using a sparse matrix structure, a zero normal flux boundary condition is introduced to constrain the boundary deformation, and a zero mean constraint is added to eliminate the constant drift of the solution. The conjugate gradient method is used to iteratively solve the linear system to obtain a scalar potential function defined in a three-dimensional space; Based on the moving cube algorithm, an isosurface is extracted from the potential function to generate a three-dimensional curved surface model of the back.

6. A method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 4, characterized in that: The posture correction of the three-dimensional curved surface model of the back specifically includes: The body posture data output by the IMU sensor is converted to the reconstruction coordinate system in which the three-dimensional curved surface model of the back is located. The posture angle in the obtained posture correction data is taken as the overall correction basis, and the back three-dimensional surface model is corrected through inverse transformation around the coordinate axis; The deviation angle in the obtained posture correction data is taken as the global constraint, and the posture of the back three-dimensional surface model is normalized; According to the left and right contact areas of the foot bottom in the pressure correction data, the actual gravity center point is calculated, the specific gravity data is ignored, and the geometric gravity center point of the foot bottom is calculated; The Euclidean distance between the actual gravity center point and the geometric gravity center point is calculated to determine the deviation; The difference between the actual gravity center point and the geometric gravity center point is converted into front-back and left-right components, and the back three-dimensional surface model is corrected in the reverse direction according to the difference components.

7. The method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: The specific steps of the landmark point recognition algorithm for extracting the spinal segment landmarks include: The coordinates of the spinal segment landmarks at the top and bottom of the spine in the three-dimensional space are identified through the landmark point recognition algorithm; The landmark point recognition algorithm directly processes three-dimensional point cloud data through the PointNet network architecture, independently encodes each point feature using a shared multi-layer perception machine, realizes global feature aggregation through maximum pooling, overcomes the unstructured characteristics of point cloud, and pre-trains through a point cloud data set containing landmark points to obtain a converged model to identify the part containing landmark points in the point cloud data; According to the extracted landmark points, a segmented cubic B-spline curve is used to fit the spinal midline to obtain the spinal midline shape of the back three-dimensional surface model, and the center points of each spinal segment landmark are marked.

8. The method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: The specific steps of the thermal force correction of the spinal midline include: Abnormal areas with a local high-low temperature difference exceeding 1.5℃ are extracted, and a deformation displacement along the geometric normal vector of the corresponding vertex is applied at the corresponding vertex; Position optimization is performed using thermal force weight Laplace smoothing, and a segmented cubic B-spline curve is re-solved based on the corrected control point set to obtain a spinal midline that fits the thermal force situation.

9. The method for detecting scoliosis in adolescents based on three-dimensional imaging according to claim 1, characterized in that: The specific steps of judging the scoliosis situation and outputting the report include: According to the marker points marked on the spinal midline, the upper and lower end cones of the spine are distinguished For the upper and lower end cones, the inclination of each marker point is calculated respectively; For the marker points with the largest inclination in the upper and lower end cones, Cobb angle calculation is performed, and the respective adjacent two marker points are used as auxiliary to construct the upper end vertebrae inclination direction vector and the lower end vertebrae inclination direction vector, and then the included angle between the two vectors is used to obtain the Cobb angle of the spine in the segment; According to the Cobb angle, the scoliosis situation is judged, the analysis process is integrated, and a report including the analysis model picture and the scoliosis situation is output.

Images