Intelligent scoliosis screening method and system

Abstract

The invention discloses an intelligent scoliosis screening method and system, and the method comprises the steps: carrying out the three-dimensional scanning of the back of a subject through depth sensing equipment, and obtaining the 3D point cloud data of the geometric shape of the back and the quantized surface appearance; collecting data of a static standing posture and a dynamic forward bending posture of the subject; respectively extracting a back key point feature vector, a spine curve feature vector and a skin tension feature vector; generating a spine biomechanical feature vector according to the static standing posture and the dynamic forward bending posture data; inputting the back key point feature vector, the spine curve feature vector, the skin tension feature vector and the spine biomechanical feature vector into a trained scoliosis screening model, outputting the scoliosis degree and the corresponding risk score of the subject, and generating a scoliosis screening report. The scoliosis can be screened only through the 3D point cloud data collected by the depth sensing device, and the screening efficiency and accuracy are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of medical devices and medical assistance technologies, and particularly relates to an intelligent scoliosis screening method and system, and a computing device. Background Art

[0002] Scoliosis is a common skeletal deformity, manifested as a lateral curvature of the spine. Early detection and intervention are crucial for preventing the progression of the disease. Currently, scoliosis screening methods mainly include visual inspection, X-ray examination, and CT scanning methods, etc. Among them, visual inspection is the most common preliminary screening method. Doctors or professionals judge whether scoliosis exists by observing the standing and forward flexion postures of the subjects, but it is easily subject to the subjective influence of medical experience and the screening efficiency is low. The X-ray examination method uses X-rays as the gold standard for diagnosing scoliosis (including parameters such as the Cobb angle). However, X-ray examination has the risk of radiation exposure, especially not suitable for populations such as children and adolescents who need frequent screening. In addition, X-ray images need to be interpreted by professional doctors, with low efficiency and difficult to meet the needs of large-scale screening. CT scanning evaluates the severity and type of scoliosis through three-dimensional bone structure images. However, the radiation dose of CT scanning is higher than that of X-rays, not suitable for routine screening, requires specialized equipment and professional personnel to operate, and is not convenient for popularization.

[0003] With the development of artificial intelligence and image technology, some methods based on neural network models have emerged. However, since they often only focus on the geometric features of the back surface and ignore important information such as skin tension and spinal biomechanics, it is difficult to distinguish between structural and functional scoliosis, resulting in the need to improve the accuracy and reliability of screening.

[0004] To solve the above problems, there is an urgent need for a non-invasive, efficient, and accurate scoliosis screening method to make up for the deficiencies of the existing technology. Based on this, the present invention proposes an intelligent scoliosis screening method that can screen scoliosis only through the 3D point cloud data collected by a depth sensing device, so as to improve the screening efficiency and accuracy. Summary of the Invention

[0005] In view of the above problems, the present invention provides an intelligent scoliosis screening method and system, and a computing device.

[0006] According to one aspect of the present invention, there is provided an intelligent scoliosis screening method, including:

[0007] Performing three-dimensional scanning on the back of a subject through a depth sensing device to obtain 3D point cloud data including the back geometry and quantified surface appearance; and collecting the static standing posture and dynamic forward flexion posture data of the subject;

[0008] Extract the back key point feature vector, spinal curve feature vector, and skin tension feature vector respectively according to the 3D point cloud data; generate a spinal biomechanical feature vector according to the static standing posture and dynamic forward flexion posture data;

[0009] Input the back key point feature vector, spinal curve feature vector, skin tension feature vector, and spinal biomechanical feature vector into the trained scoliosis screening model, and output the scoliosis degree and corresponding risk score of the subject, where the scoliosis degree includes normal, abnormal posture, and suspected scoliosis;

[0010] Generate a scoliosis screening report according to the back geometry, spinal curve, scoliosis degree, and corresponding risk score of the subject.

[0011] In an optional manner, the method for extracting the skin tension feature vector further includes:

[0012] Calculate the curvature and normal vector change rate of each point neighborhood according to the 3D point cloud data, and construct a strain tensor field representing the micro-deformation of the skin surface; wherein, the strain tensor field reflects the stretching and compression degrees in different directions of the skin surface to reveal the tension distribution of potential soft tissues;

[0013] Perform three-layer decomposition on the strain tensor field according to the Daubechies wavelet basis method, and extract the approximation coefficients and detail coefficients of the skin texture features;

[0014] Calculate the entropy features of the approximation coefficients and detail coefficients respectively and fuse them to obtain the skin tension feature vector; wherein, the entropy features include information entropy, energy entropy, and singular value entropy.

[0015] In an optional manner, the method for extracting the spinal biomechanical feature vector further includes:

[0016] Construct a multi-rigid body spinal dynamics model according to the static and dynamic posture data, wherein the multi-rigid body spinal dynamics model divides the spine into multiple rigid segments;

[0017] Calculate the torque and angle acting on each spinal segment during the dynamic forward flexion process by the inverse dynamics method;

[0018] Estimate the muscle strength of the erector spinae and rectus abdominis muscles in the back according to the simulated electromyogram signals, and calculate the stability parameters of the spine according to the torque, angle, and muscle strength; wherein, the stability parameters include spinal bending stiffness, bending strength, and flexion mode;

[0019] Combine the torque, angle, muscle strength, and spinal stability parameters to form the spinal biomechanical feature vector.

[0020] In an alternative embodiment, the method further comprises:

[0021] evaluating the influence degree of soft tissue on the spinal morphology according to the skin tension feature vector to distinguish structural scoliosis and functional scoliosis;

[0022] Specifically, a soft tissue model including back skin, muscles and adipose soft tissues is constructed, and the skin tension feature vector is input into the soft tissue model to output the pressure distribution of the soft tissue on the spine and the bending degree of the spine under different pressure distributions; wherein, if the soft tissue tension distribution exceeds a preset abnormal threshold and causes spinal curvature, it is determined as functional scoliosis, otherwise it is determined as structural scoliosis.

[0023] In an alternative embodiment, further comprising constructing a strain tensor field characterizing the minute deformation of the skin surface:

[0024] For each point p in the 3D point cloud data i , find its k nearest neighbor points;

[0025] Calculate the local coordinate system centered at point p i , transform the k nearest neighbor points into the local coordinate system and perform regression fitting to obtain a local tangent plane; wherein, the normal vector n of point p i is in the z-axis direction; i

[0026] Construct a strain tensor field E according to the gradient of point p i on the local tangent plane i .

[0027] In an alternative embodiment, further comprising calculating the stability parameter of the spine according to the torque, angle and muscle strength:

[0028] Calculate the bending stiffness of the spine according to the torque and angle of the spinal segment, and the calculation formula of the bending stiffness of the spine is:

[0029]

[0030] wherein, M i is the torque of the i-th spinal segment; θ i is the angle of the i-th spinal segment; θ 0 is the reference angle threshold; n is the total number of spinal segments; α is a dynamic adjustment coefficient;

[0031] Calculate the bending strength of the spine according to the bending stiffness of the spine, and the calculation formula of the bending strength of the spine is:

[0032]

[0033] Among them, K d (x) is the dynamic bending stiffness of the spine at position x; β is the bending strength adjustment coefficient; L is the total length of the spine; C(x) is the curvature of the spine at position x;

[0034] According to the bending strength of the spine, calculate the flexion mode of the spine. The calculation formula for the flexion mode of the spine is:

[0035]

[0036] Among them, ρ is the spine density; A is the cross-sectional area of the spine; γ is the pressure distribution adjustment coefficient; P(x) is the pressure on the spine at position x.

[0037] In an alternative manner, the scoliosis screening model includes:

[0038] An angular offset tracking module, which is used to extract and track the angular offset changes of the center of the spinal vertebrae in three-dimensional space to capture the subtle bending and rotation information of the spine;

[0039] An ROI module, which is used to locate and crop the point cloud data containing the spine area;

[0040] A downsampling module, which includes a convolutional layer and a pooling layer, and is used to extract the deep features of the input features;

[0041] A U-Net module, which includes an encoder-decoder part. Among them, the encoder part is composed of multiple downsampling modules, and the decoder part is composed of multiple upsampling modules. The features of the corresponding levels of the encoder and decoder are fused through a skip connection structure;

[0042] An upsampling module, which uses a transposed convolutional network layer to locate the position and degree of scoliosis;

[0043] A curvature perception module, which is used to extract the curvature features of the spine surface;

[0044] An attitude fusion module, which includes a multi-layer perceptron, and is used to fuse the biomechanical feature vectors extracted from the static standing posture and the dynamic forward flexion posture, learn the association between different posture features, and output a comprehensive representation of the posture features;

[0045] A prediction output module, which is used to output the degree of scoliosis and the corresponding risk score of the subject.

[0046] In an alternative manner, the loss function of the scoliosis screening model is:

[0047]

[0048] Among them, N 1is the total number of all pixels in the heat map; h is the height of the heat map; w is the width of the heat map; y ij is the ideal spinal surface depth value of the pixel at coordinates (i, j); p ij is the actual spinal surface depth value of the pixel at (i, j); N 2 is the number of key points; O xi is the offset of the i-th key point in the x direction; O yi is the offset of the i-th key point in the y direction.

[0049] According to another aspect of the present invention, there is provided an intelligent scoliosis screening system, including:

[0050] An attitude acquisition module, configured to perform three-dimensional scanning on the back of a subject through a depth sensing device to obtain 3D point cloud data including the back geometry and quantified surface appearance; and collect static standing attitude and dynamic forward flexion attitude data of the subject;

[0051] A feature extraction module, configured to extract a back key point feature vector, a spinal curve feature vector, and a skin tension feature vector respectively according to the 3D point cloud data; generate a spinal biomechanical feature vector according to the static standing attitude and dynamic forward flexion attitude data;

[0052] A screening and evaluation module, configured to input the back key point feature vector, the spinal curve feature vector, the skin tension feature vector, and the spinal biomechanical feature vector into a trained scoliosis screening model, and output the scoliosis degree and corresponding risk score of the subject, wherein the scoliosis degree includes normal, abnormal posture, and suspected scoliosis;

[0053] A report generation module, configured to generate a scoliosis screening report according to the back geometry, spinal curve, scoliosis degree, and corresponding risk score of the subject.

[0054] According to still another aspect of the present invention, there is provided a computing device, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0055] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the operations corresponding to the above intelligent scoliosis screening method.

[0056] According to the solution provided by the present invention, a three-dimensional scan of the back of a subject is performed by a depth sensing device to obtain 3D point cloud data including the back geometry and the quantified surface appearance; and, static standing posture and dynamic forward flexion posture data of the subject are collected; according to the 3D point cloud data, a back key point feature vector, a spinal curve feature vector, and a skin tension feature vector are respectively extracted; according to the static standing posture and dynamic forward flexion posture data, a spinal biomechanical feature vector is generated; the back key point feature vector, the spinal curve feature vector, the skin tension feature vector, and the spinal biomechanical feature vector are input into a trained scoliosis screening model, and the scoliosis degree and the corresponding risk score of the subject are output, wherein the scoliosis degree includes normal, abnormal posture, and suspected scoliosis; according to the back geometry, spinal curve, scoliosis degree, and the corresponding risk score of the subject, a scoliosis screening report is generated. The present invention can screen for scoliosis only through the 3D point cloud data collected by the depth sensing device, significantly improving the screening efficiency and accuracy.

[0057] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present invention more obvious and understandable, the following specifically describes the embodiments of the present invention. Brief Description of the Drawings

[0058] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0059] Figure 1 A flowchart showing the intelligent scoliosis screening method according to an embodiment of the present invention is shown;

[0060] Figure 2 A schematic diagram of dataset annotation according to an embodiment of the present invention is shown;

[0061] Figure 3 A schematic diagram of angle offset prediction according to an embodiment of the present invention is shown;

[0062] Figure 4 A schematic diagram of scoliosis diagnosis according to an embodiment of the present invention is shown;

[0063] Figure 5 A schematic diagram of the framework of the intelligent scoliosis screening system according to an embodiment of the present invention is shown;

[0064] Figure 6 A schematic diagram of the structure of the computing device according to an embodiment of the present invention is shown. Detailed implementation manners

[0065] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0066] Figure 1 The flowchart of the intelligent scoliosis screening method according to the embodiment of the present invention is shown. Specifically, as Figure 1 shown, it includes the following steps:

[0067] Step S101, performing three-dimensional scanning on the back of the subject through a depth sensing device to obtain 3D point cloud data including the back geometry and the quantified surface appearance; and collecting the static standing posture and dynamic forward flexion posture data of the subject.

[0068] In this embodiment, depth sensing is a non-invasive and non-radiative examination method, which is particularly suitable for the screening of adolescents and children, avoiding the potential risk of X-ray radiation. The three-dimensional point cloud data provides objective spinal geometric parameters (such as Cobb angle, vertebral rotation angle, vertebral offset distance, etc.), avoiding the subjectivity of manual measurement, and can detect mild spinal deformities and muscle imbalances, improving the sensitivity of screening. The entire scanning process usually only takes a few seconds or minutes, shortening the screening time and being suitable for large-scale screening. In addition, the three-dimensional point cloud data can be three-dimensionally visualized, facilitating doctors to intuitively observe the geometry of the spine.

[0069] Step S102, respectively extracting the back key point feature vector, spinal curve feature vector and skin tension feature vector according to the 3D point cloud data; generating a spinal biomechanical feature vector according to the static standing posture and dynamic forward flexion posture data.

[0070] In this embodiment, the 3D point cloud data contains rich surface geometric information and can extract features such as back key points, spinal curves, and skin tension. Combining the static standing and dynamic forward flexion postures to evaluate the performance of the spine in different states is closer to real life and helps to detect early or mild scoliosis.

[0071] Specifically, a 3D scanner is used to obtain the back point cloud data of the subject during static standing and dynamic forward flexion. During the dynamic forward flexion process, continuous scanning or interval scanning methods are adopted. As shown in Table 1, the back key points such as the inferior angle of the scapula point, the iliac crest point, and the spinal spinous process point are detected by using point cloud processing algorithms (such as curvature analysis, normal vector analysis, or PointNet++), and the distances, angles, and relative positions between the key points are calculated to construct feature vectors. Based on the spinal spinous process point, methods such as spline curves and B-spline curves are used to fit the spinal curve. The curvature, torsion, length, and curvature radius of the spinal curve are calculated to construct feature vectors. Based on information such as the density of the point cloud and the change of the normal vector, the skin tension distribution is estimated. The statistical features (such as mean, variance, maximum value, minimum value) or spatial distribution features of the skin tension are extracted to construct feature vectors. Based on the point cloud data or the synchronized video, the standing and the maximum amplitude of forward flexion during static standing and dynamic forward flexion are identified. A biomechanical model of the spine is established according to finite element or multibody dynamics. The biomechanical parameters such as the moment, stress, strain, and joint range of motion of the spine are calculated in different postures. The above biomechanical parameters are combined into a spinal biomechanical feature vector.

[0072] Table 1

[0073]

[0074]

[0075]

[0076]

[0077]

[0078] In an alternative manner, the method for extracting the skin tension feature vector further includes:

[0079] According to the 3D point cloud data, the curvature and the rate of change of the normal vector of each point neighborhood are calculated to construct a strain tensor field characterizing the small deformation of the skin surface; wherein, the strain tensor field reflects the stretching and compression degrees in different directions of the skin surface to reveal the tension distribution of potential soft tissues;

[0080] The strain tensor field is decomposed into three layers according to the Daubechies wavelet basis method, and the approximation coefficients and detail coefficients of the skin texture features are extracted;

[0081] The entropy features of the approximation coefficients and the detail coefficients are calculated respectively and fused to obtain the skin tension feature vector; wherein, the entropy features include information entropy, energy entropy, and singular value entropy.

[0082] In this embodiment, in scoliosis screening, it is crucial to detect and evaluate the condition at an early stage. Utilizing skin tension information can more sensitively capture potential abnormalities in soft tissue and skeletal structures, quantitatively evaluate the severity of scoliosis, and thus improve the accuracy of screening. Since skin tension changes may occur before structural changes, this embodiment can achieve early warning of scoliosis.

[0083] Specifically, for each point in the 3D point cloud data, its neighborhood points are determined. The neighborhood can be determined by the k-nearest neighbor algorithm or fixed-radius neighborhood search. Calculate the curvature (such as Gaussian curvature, mean curvature) and normal vector change rate of each point and its neighborhood to reflect the bending degree and change trend of the skin surface. Construct a strain tensor based on the curvature and normal vector change rate. For example, the strain tensor is a 2x2 matrix representing the stretching and compression degrees of the skin surface in different directions, where εxx and εy represent stretching / compression in the x and y directions respectively, and εxy and εyx represent shear deformation.

[0084] Select a suitable Daubechies wavelet basis (such as db4) to perform three-layer Daubechies wavelet decomposition on each component of the strain tensor field. After decomposition, an approximation coefficient and a detail coefficient matrix are obtained. Calculate the information entropy, energy entropy, and singular value entropy of the approximation coefficient matrix. Among them, a higher entropy value indicates a more uniform information distribution, the energy entropy reflects the concentration degree of the signal, and the singular value entropy is obtained by decomposing the approximation coefficient matrix based on singular value decomposition (SVD) to get the entropy of the singular values. Similarly, calculate the information entropy, energy entropy, and singular value entropy of the detail coefficient matrix.

[0085] In an alternative way, the construction of the strain tensor field characterizing the minute deformation of the skin surface further includes:

[0086] For each point p in the 3D point cloud data i , find its k nearest neighbor points;

[0087] Calculate the local coordinate system centered at point p i , transform the k nearest neighbor points into the local coordinate system and perform regression fitting to obtain the local tangent plane; where the normal vector n of point p i is in the z-axis direction; i

[0088] Construct the strain tensor field E based on the gradient of point p i on the local tangent plane i .

[0089] In this embodiment, fitting the local tangent plane using k-nearest neighbor points can better adapt to the complex curvature changes of the skin surface and avoid the errors caused by global fitting. The strain tensor is directly constructed using the gradient on the local tangent plane, which can more clearly express the direction and intensity of the skin surface deformation and improve the sensitivity to minute deformations. Among them, the local tangent plane and the gradient can more accurately capture the subtle skin deformations caused by early or mild changes in scoliosis, thereby improving the early detection rate of screening.

[0090] Specifically, for each point p in the 3D point cloud data i , use algorithms such as k-d tree or ball tree to find its k nearest neighbor points N i = {p 1 , p 2 ,..., p k}, calculate the centroid of the neighborhood point set centroid = (1 / k) × ∑p j , j = 1 to k, and move the neighborhood points and the center to the origin: p j ′ = p j - centroid, j = 1 to k.

[0091] Use principal component analysis (PCA) to estimate the normal vector n i of point p i , and form the covariance matrix with the point set N i ′ = {p 1 ′ , p 2 ′ ,..., p k ′} Calculate the eigenvectors and eigenvalues of the covariance matrix C, where the eigenvector corresponding to the smallest eigenvalue is the normal vector n i .

[0092] Define the local coordinate system, where the z-axis is the normal vector n i , the x-axis can arbitrarily select a vector perpendicular to the z-axis. For example, select a vector v that is not parallel to n i , and then calculate x i = normalize(v - dot(v, n i ) × n i ) (where normalize means unit normalization). Calculate the y-axis y i = cross(z i , x i ), and the local coordinate system is composed of the vectors (x i , y i , z i)composition

[0093] Convert the k nearest neighbor points N i from the global coordinate system to the local coordinate system centered at point p i Fit a plane in the local coordinate system. The goal is to minimize the squared error to obtain the plane equation z = ax + by + c, where a and b represent the gradients in the x and y directions respectively. Construct the strain tensor E according to the gradients i ,

[0094] In an alternative approach, the method further includes:

[0095] Evaluate the degree of influence of soft tissues on spinal morphology based on the skin tension eigenvector to distinguish between structural scoliosis and functional scoliosis;

[0096] Specifically, construct a soft tissue model including back skin, muscles and adipose soft tissues. Input the skin tension eigenvector into the soft tissue model, and output the pressure distribution of the soft tissues on the spine and the degree of spinal curvature under different pressure distributions; among them, if the soft tissue tension distribution exceeds the preset abnormal threshold and causes spinal curvature, it is judged as functional scoliosis, otherwise it is judged as structural scoliosis.

[0097] In this embodiment, the pressure distribution and curvature degree of soft tissues on the spine are quantified through the soft tissue model, and the parameters of the soft tissue model can be adjusted according to the individual characteristics (age, height, weight) of the patient to achieve personalized evaluation.

[0098] Specifically, use finite element analysis (FEA) software to construct a geometric model including back skin, muscles and adipose soft tissues, and assign corresponding material properties (such as elastic modulus, Poisson's ratio) to different soft tissues. Define the boundary conditions of the model. For example, the vertebrae at the bottom of the spine can be fixed to restrict their displacement and rotation.

[0099] Convert the skin tension eigenvector into the pressure distribution on the surface of the soft tissue model. For example, associate each component of the eigenvector with the pressure magnitude in the corresponding area of the model surface, and use interpolation methods to map the information of the eigenvector to the entire model surface. Adjust the pressure distribution according to different individual situations (consider the pressure influence of adipose tissue for obese individuals).

[0100] Use finite element analysis software to calculate the stress, strain, and displacement of the soft tissue model under a given pressure distribution. Extract the degree of spinal curvature from the analysis results using the Cobb angle or other spinal curvature metrics. Analyze the pressure distribution of the soft tissue on the spine and evaluate which areas have excessive or insufficient pressure. By statistically presetting the abnormal thresholds of the soft tissue tension distribution and the degree of spinal curvature based on data from a large number of healthy individuals and scoliosis patients, if the soft tissue tension distribution exceeds the preset abnormal threshold and causes spinal curvature, it is judged as functional scoliosis. If the soft tissue tension distribution is within the normal range or even if it exceeds the threshold, it cannot explain the degree of spinal curvature, it is judged as structural scoliosis.

[0101] In an alternative approach, the method for extracting the spinal biomechanical feature vector further includes:

[0102] Construct a multi-rigid-body spinal dynamics model based on the static and dynamic posture data, where the multi-rigid-body spinal dynamics model divides the spine into multiple rigid segments;

[0103] Calculate the moments and angles acting on each spinal segment during the dynamic forward flexion process through the inverse dynamics method;

[0104] Estimate the muscle forces of the erector spinae muscles and rectus abdominis muscles based on the simulated electromyogram signals, and calculate the spinal stability parameters based on the moments, angles, and muscle forces; where the stability parameters include spinal curvature stiffness, bending strength, and flexion mode;

[0105] Combine the moments, angles, muscle forces, and spinal stability parameters to form the spinal biomechanical feature vector.

[0106] In this embodiment, not only relying on surface posture information, but by constructing a multi-rigid-body dynamics model to calculate the moments and angles received by the spinal segments, thereby deeply understanding the force conditions of the spine during movement. The spinal curvature stiffness, bending strength, and flexion mode stability parameters can directly reflect the spine's ability to resist deformation and injury.

[0107] Specifically, a motion capture system (such as Vicon) or an inertial measurement unit (IMU) is used to record the pose data (including the movement trajectories and angular changes of spinal segments) of the subject during dynamic forward flexion. A surface electromyography (sEMG) sensor is used to record the electromyography signals of the erector spinae and rectus abdominis muscles of the subject during dynamic forward flexion. The spine is divided into multiple rigid segments such as the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, and sacral vertebrae, and each segment is connected by joints. The mass and moment of inertia mechanical parameters of each rigid segment are determined according to the height, weight, and bone size parameters of the subject. Incorporating the rotational range of the intervertebral discs and the physiological constraints of joint mobility, based on the dynamic pose data, the inverse dynamics method (Newton-Euler equations or Lagrange equations) is used to calculate the moments and angles acting on each spinal segment during dynamic forward flexion. According to the recorded electromyography signals and the established relationship model, the muscle forces of the erector spinae and rectus abdominis muscles are estimated (such as methods like the linear envelope method and the optimization method based on physiological models). The spinal bending stiffness is calculated by analyzing the moment-angle relationship, the maximum bending moment that the spine can withstand is estimated, and the flexion mode of the spine is determined through finite element analysis to reflect the deformation mode of the spine during instability.

[0108] In this embodiment, calculating the stability parameter of the spine according to the moment, angle, and muscle force further includes:

[0109] According to the moment and angle of the spinal segment, calculate the bending stiffness of the spine, and the calculation formula for the bending stiffness of the spine is:

[0110]

[0111] where M i is the moment of the i-th spinal segment; θ i is the angle of the i-th spinal segment; θ 0 is the reference angle threshold; n is the total number of spinal segments; α is the dynamic adjustment coefficient;

[0112] According to the spinal bending stiffness, calculate the bending strength of the spine, and the calculation formula for the bending strength of the spine is:

[0113]

[0114] where K d (x) is the dynamic bending stiffness of the spine at position x; β is the bending strength adjustment coefficient; L is the total length of the spine; C(x) is the curvature of the spine at position x;

[0115] According to the spinal bending strength, calculate the flexion mode of the spine, and the calculation formula for the flexion mode of the spine is:

[0116]

[0117] Among them, ρ is the spinal density; A is the cross-sectional area of the spine; γ is the pressure distribution adjustment coefficient; P(x) is the pressure on the spine at position x.

[0118] In step S103, input the back key point feature vector, spinal curve feature vector, skin tension feature vector, and spinal biomechanical feature vector into the trained scoliosis screening model, and output the degree of scoliosis and the corresponding risk score of the subject. Among them, the degree of scoliosis includes normal, abnormal posture, and suspected scoliosis.

[0119] In this embodiment, the back key point feature vector, spinal curve feature vector, skin tension feature vector, and spinal biomechanical feature vector respectively reflect the state of the spine from different perspectives. Among them, the back key points and spinal curve features focus on morphology, the skin tension reflects muscle imbalance, and the biomechanical features reflect the function of the spine (for example, even if there are small errors in posture detection, the biomechanical features still provide key information). In the early stage of scoliosis, there are only slight abnormalities in posture or asymmetry in skin tension, while the biomechanical indicators may have changed, which is conducive to the early detection of scoliosis, so as to achieve early diagnosis and early treatment.

[0120] In an optional manner, the scoliosis screening model includes:

[0121] An angular offset tracking module, which is used to extract and track the angular offset changes of the centers of spinal vertebrae in three-dimensional space to capture the subtle bending and rotation information of the spine;

[0122] An ROI module, which is used to locate and crop the point cloud data containing the spinal region;

[0123] A downsampling module, which includes a convolutional layer and a pooling layer, and is used to extract the deep features of the input features;

[0124] A U-Net module, which includes an encoder-decoder part. Among them, the encoder part is composed of multiple downsampling modules, and the decoder part is composed of multiple upsampling modules. The features of the corresponding levels of the encoder and decoder are fused through a skip connection structure;

[0125] An upsampling module, which uses a transposed convolutional network layer to locate the position and degree of scoliosis;

[0126] A curvature perception module, which is used to extract the curvature features of the spinal surface;

[0127] A posture fusion module, which includes a multi-layer perceptron, and is used to fuse the biomechanical feature vectors extracted from the static standing posture and the dynamic forward flexion posture, learn the association between different posture features, and output a comprehensive representation of the posture features;

[0128] A prediction output module for outputting the scoliosis degree of a subject and the corresponding risk score.

[0129] In this embodiment, by integrating the biomechanical characteristics of static standing postures and dynamic forward bending postures, the subtle changes of the spine in different postures are learned. The angular offset tracking module and the curvature perception module capture the subtle bending, rotation, and surface curvature changes of the spine in three-dimensional space, which are crucial for the early detection and diagnosis of scoliosis, especially in the early stages of the disease when it is difficult to detect with the naked eye. Among them, as Figure 3 shown, the ROI module locates and crops the point cloud data of the spine region containing the spine from the cervical vertebra to the lumbar vertebra, and restores the center points of the vertebral bodies to the vertebral ROI. The angular offset tracking module extracts and tracks the angular offset changes of the center points of the vertebral bodies of the spine in three-dimensional space. The curvature perception module extracts the curvature features of the spine surface. The downsampling module uses convolutional layers and pooling layers to extract the deep features of the input features. The U-Net module captures the subtle changes of the spine through an encoder-decoder structure and fuses the features of the corresponding levels of the encoder and decoder through a skip connection structure. The upsampling module uses a transposed convolutional network layer to locate the position and degree of scoliosis. The posture fusion module fuses the angular offset information, curvature features, and the positioning information output by the upsampling module. The prediction output module outputs the scoliosis degree and the corresponding risk score. As Figure 2 shown, to prevent the model from labeling the cervical vertebra region as the vertebral ground truth during training, it is necessary to mask the non-vertebral region before Cobb angle measurement and make a dataset of the center points of the vertebral bodies and a dataset of the vertebral position anchors. The image processing process is as Figure 4 shown. The image is input into the model to obtain the image of the center points of the vertebral bodies and the image of the vertebral position anchors (the two images provide the position information of the key points). The cervical vertebra part in the center points of the vertebral bodies is filtered using the vertebral position anchors to obtain the image of the center points of the vertebral bodies of the spine.

[0130] In an alternative manner, the loss function of the scoliosis screening model is:

[0131]

[0132] where N 1 is the total number of all pixels in the heat map; h is the height of the heat map; w is the width of the heat map; y ij is the ideal spine surface depth value of the pixel with coordinates (i, j); p ij is the actual spine surface depth value of the pixel at (i, j); N 2 is the number of key points; O xi is the offset of the i-th key point in the x direction; O yi is the offset of the i-th key point in the y direction.

[0133] In this embodiment, the heatmap cross-entropy loss is used to learn and predict whether each pixel belongs to the spinal surface, and the key point offset loss is used to learn and predict the offset of the spinal key points (such as the vertebral body center), so as to more accurately measure the Cobb angle index of the spine. Specifically, for each training sample (spinal point cloud data), a corresponding heatmap is generated, and each pixel of the heatmap represents the probability that the pixel belongs to the spinal surface. The depth value of the spinal surface in the ideal state is generated using a Gaussian kernel function to form a smooth probability distribution around the key points. For example, the pixel value at the center position of the vertebral body is set to 1, and the values of the surrounding pixels decay according to the distance. Among them, the key point offset represents the difference between the predicted key point position and the real position, the heatmap cross-entropy loss measures the difference between the predicted heatmap and the real heatmap, and the key point offset loss measures the difference between the predicted key point position and the real position.

[0134] Step S104, generate a scoliosis screening report according to the back geometry, spinal curve, scoliosis degree and corresponding risk score of the subject.

[0135] For example, a subject's back shows obvious asymmetry, with the right side higher than the left side. The spinal curve presents an S shape, with a more obvious curve on the right side, and the right scoliosis angle is 15 degrees. The risk score is 75 points (out of 100 points in total), and the risk level is medium risk.

[0136] According to the solution provided by the present invention, a three-dimensional scan of the subject's back is performed by a depth sensing device to obtain 3D point cloud data including the back geometry and quantified surface appearance; and, static standing posture and dynamic forward flexion posture data of the subject are collected; according to the 3D point cloud data, back key point feature vectors, spinal curve feature vectors and skin tension feature vectors are respectively extracted; a spinal biomechanical feature vector is generated according to the static standing posture and dynamic forward flexion posture data; the back key point feature vector, spinal curve feature vector, skin tension feature vector and spinal biomechanical feature vector are input into a trained scoliosis screening model, and the scoliosis degree and corresponding risk score of the subject are output, where the scoliosis degree includes normal, abnormal posture and suspected scoliosis; a scoliosis screening report is generated according to the back geometry, spinal curve, scoliosis degree and corresponding risk score of the subject. The present invention can screen scoliosis only through the 3D point cloud data collected by the depth sensing device, significantly improving the screening efficiency and accuracy.

[0137] Figure 5 The framework schematic diagram of the intelligent scoliosis screening system according to the embodiment of the present invention is shown. The intelligent scoliosis screening system includes:

[0138] The posture acquisition module 510 is configured to perform three-dimensional scanning on the back of a subject through a depth sensing device to obtain 3D point cloud data including the back geometry and the quantified surface appearance; and collect the static standing posture and dynamic forward flexion posture data of the subject.

[0139] The feature extraction module 520 is configured to extract the back key point feature vector, spine curve feature vector, and skin tension feature vector respectively according to the 3D point cloud data; generate a spine biomechanics feature vector according to the static standing posture and dynamic forward flexion posture data.

[0140] The screening and evaluation module 530 is configured to input the back key point feature vector, spine curve feature vector, skin tension feature vector, and spine biomechanics feature vector into a trained scoliosis screening model, and output the scoliosis degree and the corresponding risk score of the subject, wherein the scoliosis degree includes normal, abnormal posture, and suspected scoliosis.

[0141] The report generation module 540 is configured to generate a scoliosis screening report according to the back geometry, spine curve, scoliosis degree, and the corresponding risk score of the subject.

[0142] Figure 6 The structural schematic diagram of the computing device embodiment of the present invention is shown. The specific implementation of the computing device in the specific embodiment of the present invention is not limited.

[0143] As Figure 6 shown, the computing device may include: a processor 602, a communications interface 604, a memory 606, and a communication bus 608.

[0144] Among them: the processor 602, the communications interface 604, and the memory 606 communicate with each other through the communication bus 608. The communications interface 604 is used to communicate with network elements of other devices such as clients or other servers. The processor 602 is configured to execute the program 610, and specifically may execute the relevant steps in the embodiment of the intelligent scoliosis screening method described above.

[0145] Specifically, the program 610 may include program codes, and the program codes include computer operation instructions.

[0146] The processor 602 may be a central processing unit (CPU), or a specific application integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computing device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0147] A memory 606 for storing a program 610. The memory 606 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk memory.

[0148] According to the solution provided by the present invention, a three-dimensional scan of the back of a subject is performed by a depth sensing device to obtain 3D point cloud data including the back geometry and quantified surface appearance; and, static standing posture and dynamic forward flexion posture data of the subject are collected; a back key point feature vector, a spinal curve feature vector, and a skin tension feature vector are respectively extracted from the 3D point cloud data; a spinal biomechanical feature vector is generated according to the static standing posture and dynamic forward flexion posture data; the back key point feature vector, the spinal curve feature vector, the skin tension feature vector, and the spinal biomechanical feature vector are input into a trained scoliosis screening model to output the scoliosis degree and corresponding risk score of the subject, wherein the scoliosis degree includes normal, abnormal posture, and suspected scoliosis; a scoliosis screening report is generated according to the back geometry, spinal curve, scoliosis degree, and corresponding risk score of the subject. The present invention can screen for scoliosis only through the 3D point cloud data collected by the depth sensing device, significantly improving the screening efficiency and accuracy.

[0149] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract and drawings), as well as all the processes or units of any method or device so disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose. In addition, those skilled in the art can understand that although some of the embodiments herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination. The present invention can be implemented by means of hardware including several different elements and by means of a properly programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same hardware item. The steps in the above embodiments, unless otherwise specified, should not be construed as a limitation on the execution order.

Claims

1. An intelligent scoliosis screening method, characterized in that: include: The subject's back is scanned in three dimensions using a depth sensing device to obtain 3D point cloud data including back geometry and quantitative surface appearance; and, collecting static standing posture and dynamic forward bending posture data of the subjects; Extract back key point feature vectors, spine curve feature vectors and skin tension feature vectors respectively according to the 3D point cloud data; generate spine biomechanics feature vectors according to the static standing posture and dynamic forward bending posture data; Inputting the back key point feature vector, spine curve feature vector, skin tension feature vector and spine biomechanics feature vector into a trained scoliosis screening model, and outputting the scoliosis degree and corresponding risk score of the subject, wherein the scoliosis degree includes normal, abnormal posture and suspected scoliosis; A scoliosis screening report is generated based on the subject's back geometry, spinal curve, degree of scoliosis and corresponding risk score.

2. The intelligent scoliosis screening method according to claim 1, characterized in that: The method for extracting the skin tension feature vector further comprises: Calculating the curvature and normal vector change rate of each point neighborhood according to the 3D point cloud data, and constructing a strain tensor field that characterizes the slight deformation of the skin surface; wherein the strain tensor field reflects the degree of stretching and compression of the skin surface in different directions to reveal the tension distribution of the underlying soft tissue; Decomposing the strain tensor field in three layers according to the Daubechies wavelet basis method to extract the approximate coefficients and detail coefficients of the skin texture characteristics; The entropy features of the approximate coefficient and the detail coefficient are calculated and fused respectively to obtain a skin tension feature vector; wherein the entropy features include information entropy, energy entropy and singular value entropy.

3. The intelligent scoliosis screening method according to claim 1, characterized in that: The method for extracting the spine biomechanical characteristic vector further comprises: Constructing a multi-rigid body spinal dynamics model according to the static and dynamic posture data, wherein the multi-rigid body spinal dynamics model divides the spine into a plurality of rigid segments; The moments and angles acting on each spinal segment during dynamic flexion were calculated using the inverse dynamics method; Estimate the muscle strength of the back erector spinae and rectus abdominis according to the simulated electromyographic signal, and calculate the stability parameters of the spine according to the torque, angle and muscle strength; wherein the stability parameters include the bending stiffness, bending strength and flexion mode of the spine; The torque, angle, muscle strength and spinal stability parameters are combined to form the spinal biomechanical characteristic vector.

4. The intelligent scoliosis screening method according to claim 1, characterized in that: The method further comprises: The influence of soft tissue on spinal morphology is evaluated according to the skin tension feature vector to distinguish structural scoliosis from functional scoliosis; Specifically, a soft tissue model including back skin, muscle and fat soft tissue is constructed, the skin tension feature vector is input into the soft tissue model, and the pressure distribution of the soft tissue on the spine and the degree of curvature of the spine under different pressure distributions are output; wherein, if the soft tissue tension distribution exceeds a preset abnormal threshold and causes spinal curvature, it is judged as functional scoliosis, otherwise it is judged as structural scoliosis.

5. The intelligent scoliosis screening method according to claim 2, characterized in that: The constructing of the strain tensor field characterizing the micro-deformation of the skin surface further comprises: For each point p in the 3D point cloud data i , find its k nearest neighbors; Calculate the point p i The k nearest neighbor points are transformed into the local coordinate system and regressed to get the local tangent plane. i The normal vector n i is the z-axis direction; According to point p i The gradient on the local tangent plane constructs the strain tensor field E i .

6. The intelligent scoliosis screening method according to claim 3, characterized in that: Calculating the stability parameters of the spine based on the torque, angle and muscle strength further includes: The bending stiffness of the spine is calculated according to the moment and angle of the spinal segment. The calculation formula of the bending stiffness of the spine is: Among them, M i is the moment of the i-th spinal segment; θ i is the angle of the i-th spinal segment; θ0 is the reference angle threshold; n is the total number of spinal segments; α is the dynamic adjustment coefficient; According to the bending stiffness of the spine, the bending strength of the spine is calculated. The calculation formula of the bending strength of the spine is: Among them, K d (x) is the dynamic bending stiffness of the spine at position x; β is the bending strength adjustment coefficient; L is the total length of the spine; C(x) is the curvature of the spine at position x; According to the spine bending strength, the flexion mode of the spine is calculated, and the calculation formula of the flexion mode of the spine is: Among them, ρ is the spine density; A is the cross-sectional area of ​​the spine; γ is the pressure distribution adjustment coefficient; P(x) is the pressure of the spine at position x.

7. The intelligent scoliosis screening method according to claim 1, characterized in that: The scoliosis screening model includes: An angular offset tracking module is used to extract and track the angular offset changes of the spinal vertebrae center in three-dimensional space to capture the subtle bending and rotation information of the spine; ROI module, used to locate and crop point cloud data containing the spine area; The downsampling module, which includes convolutional layers and pooling layers, is used to extract deep features of the input features; The U-Net module includes an encoder-decoder part, where the encoder part consists of multiple downsampling modules and the decoder part consists of multiple upsampling modules. The features of the corresponding levels of the encoder and decoder are fused through a skip connection structure. An upsampling module, using a deconvolutional network layer, is used to locate the location and degree of scoliosis; Curvature perception module, used to extract the curvature features of the spine surface; A posture fusion module, including a multi-layer perceptron, is used to fuse the biomechanical feature vectors extracted from the static standing posture and the dynamic forward bending posture, learn the association between different posture features and output a comprehensive representation of the posture features; The prediction output module is used to output the subject's scoliosis degree and the corresponding risk score.

8. The intelligent scoliosis screening method according to claim 1 or 7, characterized in that: The loss function of the scoliosis screening model is: Where N1 is the total number of pixels in the heat map; h is the height of the heat map; w is the width of the heat map; y is the ij is the ideal spinal surface depth value of the pixel with coordinates (i, j); p ij is the actual depth value of the spine surface of the pixel (i, j); N2 is the number of key points; O xi is the offset of the i-th key point in the x direction; yi is the offset of the i-th key point in the y direction.

9. An intelligent scoliosis screening system, characterized in that: include: A posture acquisition module, used to perform a three-dimensional scan of the subject's back through a depth sensing device to obtain 3D point cloud data including the back geometry and quantitative surface appearance; and, collecting static standing posture and dynamic forward bending posture data of the subjects; A feature extraction module is used to extract back key point feature vectors, spine curve feature vectors and skin tension feature vectors according to the 3D point cloud data; and generate a spine biomechanical feature vector according to the static standing posture and dynamic forward bending posture data; A screening and evaluation module, for inputting the back key point feature vector, the spinal curve feature vector, the skin tension feature vector and the spinal biomechanics feature vector into a trained scoliosis screening model, and outputting the scoliosis degree and the corresponding risk score of the subject, wherein the scoliosis degree includes normal, abnormal posture and suspected scoliosis; A report generating module is used to generate a scoliosis screening report based on the subject's back geometry, spinal curve, degree of scoliosis and corresponding risk score.

10. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned intelligent scoliosis screening method.

Images