A method and system for detecting spine key points and calculating Cobb angles

Abstract

This invention discloses a method and system for detecting key points of the spine and calculating Cobb angles, relating to the field of image processing technology. By constructing a trained key point detection model comprising a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence, after obtaining an X-ray image of the spine to be detected, the X-ray image can be directly input into the trained key point detection model to automatically obtain the coordinates of the key points of the spine to be detected, thereby automatically detecting the key points of the spine. Subsequently, the Cobb angle can be automatically calculated based on the coordinates of the automatically detected key points of the spine, thus realizing the automatic calculation of the Cobb angle. Compared with manual measurement methods, the calculation accuracy is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for detecting key points of the spine and calculating Cobb angles based on X-ray images. Background Technology

[0002] Adolescent idiopathic scoliosis (AIS) refers to axial rotation or lateral deviation of the spine occurring around puberty. Early detection and supportive treatment reduce the likelihood of subsequent surgery, and the Cobb angle is considered the gold standard for assessing scoliosis. Due to the ambiguity and variability of scoliosis X-rays, calculating the Cobb angle based on X-rays is challenging. Generally, clinicians manually select and measure key points on the spine, further choosing the most tilted vertebra to calculate the Cobb angle. However, this manual measurement method often leads to inconsistent results depending on the physician, causing calculation errors. With the prevalence and severity of adolescent scoliosis, automated detection has become a significant concern in the medical field. Considering that manual measurement of the Cobb angle for scoliosis is both time-consuming and unreliable in clinical practice, there is a need to develop automated methods to accurately calculate the Cobb angle from spinal X-rays. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for detecting key points of the spine and calculating the Cobb angle, which can automatically detect key points of the spine to further accurately calculate the Cobb angle.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for detecting key points in the spine, the method comprising:

[0006] Obtain X-ray images of the spine to be examined;

[0007] The X-ray image is input into a trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0008] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0009] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0010] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0011] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0012] In some embodiments, the step of predicting the predicted vertebral center point heatmap of the vertebra to be predicted using the initial heatmap of the target vertebral body's center point specifically includes:

[0013] The kernel size is determined based on the preset maximum offset angle and the preset maximum spacing; the offset angle is the angle between the line connecting the center point of the vertebra to be predicted and the center point of the target vertebra and the U-axis in the image coordinate system; the spacing is the distance between the center point of the vertebra to be predicted and the center point of the target vertebra.

[0014] For each target pixel of the target vertebra, a probability distribution formula for the spacing ratio of the target pixel is determined based on a preset average spacing ratio and a preset variance of the spacing ratio. Similarly, a probability distribution formula for the offset angle of the target pixel is determined based on a preset average offset angle and a preset variance of the offset angle. Finally, a convolutional kernel template for the target pixel is determined based on the spacing ratio probability distribution formula and the offset angle probability distribution formula. The spacing ratio is the proportion of the distance between the center point of the vertebra to be predicted and the center point of the target vertebra to the total distance. The total distance is the sum of the spacings of all adjacent vertebrae in the spine to be detected.

[0015] The initial heatmap of the vertebral center point of the target vertebra is convolved using the convolution kernel size and the convolution kernel template of each target pixel to obtain the predicted heatmap of the vertebral center point of the vertebra to be predicted.

[0016] In some embodiments, the kernel size is w*h, where w is the width of the kernel and h is the length of the kernel; w = h = 2*max(d max (k1, k2)*cos(|θ) max (k1, k2) - 90°|), d max (k1, k2)*sin(|θ) max(k1, k2)-90°|)); where k1 is the target vertebra; k2 is the vertebra to be predicted; d max (k1, k2) are the preset maximum spacing values; |θ max (k1, k2)-90°| is the preset maximum offset angle.

[0017] In some embodiments, the probability distribution formula for the spacing ratio is:

[0018]

[0019] Where g(p) is the probability distribution formula for the spacing proportion; p is the spacing proportion variable; μ dis The preset average spacing ratio; σ dis The pre-defined spacing percentage variance;

[0020]

[0021] Where (u, v) are the coordinates of the target pixel; (x2, y2) are the coordinates of the center point of the cone to be predicted; d sum This is the calculated total distance.

[0022] The formula for the probability distribution of the offset angle is:

[0023]

[0024] Where f(θ) is the probability distribution formula for the offset angle; θ is the offset angle variable; μ deg The preset average offset angle; σ deg The preset offset angle variance;

[0025]

[0026] The convolution kernel template is:

[0027] h k1 (u-x2, v-y2) = f(θ)*g(p);

[0028] Among them, h k1 (u-x2, v-y2) is the convolution kernel template.

[0029] In some embodiments, before inputting the X-ray image into the trained keypoint detection model to obtain the keypoint coordinates of the spine to be detected, the detection method further includes: training the initial keypoint detection model using a sample set to obtain a trained keypoint detection model; the sample set includes multiple samples; the samples include sample X-ray images and the true coordinates of the center point and the true coordinates of the corner points of each vertebra in the spine of the sample X-ray images;

[0030] The method for constructing the sample set includes: acquiring multiple samples to form an initial set; performing data augmentation on the initial set to obtain the sample set; the data augmentation methods include random horizontal flipping, random horizontal rotation, and multi-view distance shooting.

[0031] In some embodiments, the multi-view distance shooting specifically includes:

[0032] The sample X-ray film is padded and multiple position markers are set in the padded area to obtain the padded image;

[0033] Multiple images are obtained by taking multiple photos of the expanded image from different angles and distances using a mobile phone.

[0034] For each captured image, an extrinsic parameter matrix is ​​determined based on the first coordinates of each location marker in the expanded image and the second coordinates of each location marker in the captured image; the extrinsic parameter matrix is ​​used to determine the true values ​​of the center point coordinates and the true values ​​of the corner point coordinates of each vertebra in the spine of the captured image; the captured image and the true values ​​of the center point coordinates and the true values ​​of the corner point coordinates of each vertebra in the spine of the captured image constitute a new sample.

[0035] In some embodiments, the preset maximum offset angle, the preset maximum spacing, the preset average spacing percentage, the preset variance of spacing percentage, the preset average offset angle, and the preset variance of offset angle are pre-calculated based on the sample set.

[0036] A spinal key point detection system, the detection system comprising:

[0037] Image acquisition unit, used to acquire X-ray images of the spine to be detected;

[0038] The key point detection unit is used to input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0039] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0040] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0041] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0042] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0043] A method for calculating Cobb angles, the method comprising:

[0044] Obtain X-ray images of the spine to be examined;

[0045] The X-ray image is input into a trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0046] Fit the coordinates of the center points of all the vertebrae to obtain the fitting equation of the center point curve composed of the center points of all the vertebrae;

[0047] For each vertebra, a first equation for the upper edge line and a second equation for the lower edge line are calculated based on the corner coordinates of the vertebra; a first coordinate of the first intersection point of the center point curve and the upper edge line is calculated based on the fitted equation and the first equation; a second coordinate of the second intersection point of the center point curve and the lower edge line is calculated based on the fitted equation and the second equation; a third equation of the first perpendicular line of the tangent line passing through the first intersection point and tangent to the center point curve is calculated based on the fitted equation and the first coordinates; and a fourth equation of the second perpendicular line of the tangent line passing through the second intersection point and tangent to the center point curve is calculated based on the fitted equation and the second coordinates.

[0048] For each vertebra, the angle between the first perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra is calculated based on the third equation of the vertebra and the third equation of the neighboring vertebra. The angle between the first perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra is calculated based on the third equation of the vertebra and the fourth equation of the neighboring vertebra. The angle between the second perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra is calculated based on the fourth equation of the vertebra and the third equation of the neighboring vertebra. The angle between the second perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra is calculated based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra. The maximum value of all the angles is selected as the maximum angle corresponding to the vertebra. The neighboring vertebrae are the other vertebrae in the spine to be tested besides the vertebra in question.

[0049] The maximum value of the maximum included angle of all the vertebrae is selected as the Cobb angle;

[0050] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0051] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0052] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0053] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0054] A Cobb angle calculation system, the calculation system comprising:

[0055] Image acquisition unit, used to acquire X-ray images of the spine to be detected;

[0056] The key point detection unit is used to input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0057] The center point fitting unit is used to fit the coordinates of the center points of all the vertebrae to obtain the fitting equation of the center point curve composed of the center points of all the vertebrae.

[0058] The corner fitting unit is used to calculate, for each vertebra, a first equation for the upper edge line and a second equation for the lower edge line of the vertebra based on the corner coordinates of the vertebra; calculate the first coordinates of the first intersection point of the center point curve and the upper edge line based on the fitting equation and the first equation; calculate the second coordinates of the second intersection point of the center point curve and the lower edge line based on the fitting equation and the second equation; calculate the third equation of the first perpendicular line of the tangent line passing through the first intersection point and tangent to the center point curve based on the fitting equation and the first coordinates; and calculate the fourth equation of the second perpendicular line of the tangent line passing through the second intersection point and tangent to the center point curve based on the fitting equation and the second coordinates.

[0059] An angle calculation unit is used to calculate, for each vertebra, the angle between the first perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the third equation of the neighboring vertebra; the angle between the first perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the fourth equation of the neighboring vertebra; the angle between the second perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra; and the angle between the second perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra. The maximum value of all the angles is selected as the maximum angle corresponding to the vertebra; the neighboring vertebrae are the other vertebrae in the spine to be detected besides the vertebra in question.

[0060] The Cobb angle calculation unit is used to select the maximum value of the maximum included angle of all the vertebrae as the Cobb angle;

[0061] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0062] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0063] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0064] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0065] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0066] This invention provides a method and system for detecting key points of the spine and calculating Cobb angles. By constructing a trained key point detection model comprising a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence, after acquiring an X-ray image of the spine to be detected, the X-ray image can be directly input into the trained key point detection model to automatically obtain the coordinates of the key points of the spine to be detected, thereby automatically detecting the key points of the spine. Subsequently, the Cobb angle can be automatically calculated based on the coordinates of the automatically detected key points of the spine, thus realizing the automatic calculation of the Cobb angle. Compared with manual measurement methods, the calculation accuracy is greatly improved. Attached Figure Description

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 This is a flowchart of the detection method provided in Embodiment 1 of the present invention;

[0069] Figure 2 This is a schematic diagram of the detection method provided in Embodiment 1 of the present invention;

[0070] Figure 3 This is a schematic diagram of the structure of the heat map prediction module provided in Embodiment 1 of the present invention;

[0071] Figure 4 This is a system block diagram of the detection system provided in Embodiment 2 of the present invention;

[0072] Figure 5 This is a flowchart of the calculation method provided in Embodiment 3 of the present invention;

[0073] Figure 6 This is a schematic diagram of the calculation method provided in Embodiment 3 of the present invention;

[0074] Figure 7 This is a system block diagram of the computing system provided in Embodiment 4 of the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0076] The purpose of this invention is to provide a method and system for detecting key points of the spine and calculating the Cobb angle, which can automatically detect key points of the spine to further accurately calculate the Cobb angle.

[0077] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] Example 1:

[0079] Spinal landmark detection is essentially a regression problem. Regression-based methods can utilize global information from input X-ray images for spinal landmark detection. However, the dense mappings between regression points and latent features require significant parameters and computational costs. Therefore, the input X-ray images must be downsampled to a very low resolution to support training and inference. This results in the loss of high-resolution details from the original X-ray images, limiting the performance of regression-based methods and leading to low accuracy in spinal landmark detection. Therefore, this embodiment provides a novel spinal landmark detection method to improve the accuracy of spinal landmark detection.

[0080] This embodiment provides a method for detecting key points of the spine based on X-ray images, such as... Figure 1 and Figure 2 As shown, the detection method includes:

[0081] S1: Obtain an X-ray of the spine to be examined;

[0082] S2: Input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0083] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0084] (1) Heatmap prediction module:

[0085] The heatmap prediction module is used to predict the initial heatmap of the vertebral body center point and the vertebral body corner point heatmap for each vertebra in the spine to be detected, taking X-ray images as input. The initial heatmap of the vertebral body center point includes the probability value of each pixel in the initial heatmap of the vertebral body center point being the vertebral body center point. The vertebral body corner point heatmap includes the probability value of each pixel in the vertebral body corner point heatmap being the vertebral body corner point. The vertebral body corner point heatmap may include 4 images, namely the upper left corner point heatmap, the lower left corner point heatmap, the upper right corner point heatmap, and the lower right corner point heatmap. The upper left corner point heatmap includes the probability value of each pixel in the upper left corner point heatmap being the upper left corner point of the vertebral body. The lower left corner point heatmap includes the probability value of each pixel in the lower left corner point heatmap being the lower left corner point of the vertebral body. The upper right corner point heatmap includes the probability value of each pixel in the upper right corner point heatmap being the upper right corner point of the vertebral body. The lower right corner point heatmap includes the probability value of each pixel in the lower right corner point heatmap being the lower right corner point of the vertebral body.

[0086] like Figure 3 As shown, the heatmap prediction module in this embodiment can employ a U-net deep learning network, including a downsampling encoder, skip connections, and an upsampling decoder. Specifically, the downsampling encoder can be ResNet34's conv1-5 (i.e., convolutional layers 1-5). The downsampling encoder extracts high-level semantic features from the input image, and skip connections combine deep and shallow features. By utilizing both high-level semantic features and low-level detail features, the gradient vanishing problem caused by increasing depth in deep neural networks is mitigated. The upsampling decoder constructs initial heatmaps of the vertebral center point and vertebral corner points for each vertebra.

[0087] (2) Heatmap weighting module:

[0088] The heatmap weighting module is used to predict the center point of the vertebra to be predicted for each vertebra to be predicted by using the initial heatmap of the center point of the target vertebra. The module also performs a weighted summation of the initial heatmap of the center point of the vertebra to be predicted and all predicted heatmaps of the center points of the vertebra to be predicted to obtain the final heatmap of the center point of the vertebra to be predicted. The vertebra to be predicted is any vertebra in the spine to be detected, and the target vertebra is any other vertebra in the spine to be detected except for the vertebra to be predicted.

[0089] According to medical diagnosis, key points of 17 vertebrae in the spine need to be detected, including 12 thoracic vertebrae and 5 lumbar vertebrae. Therefore, this embodiment requires detection of 17 vertebrae. For each vertebra, a predicted heatmap of the vertebral center point needs to be obtained based on the initial heatmap of the vertebral center point of the other 16 vertebrae. Then, the initial heatmap of the vertebral center point and the predicted heatmaps of the 16 vertebral center points are weighted and summed according to a certain ratio to obtain the final heatmap of the vertebral center point. Finally, the final heatmap of the vertebral center point of all 17 vertebrae is obtained.

[0090] This embodiment specifically utilizes the continuity between vertebrae to predict the predicted vertebrae center point heatmap of the vertebra to be predicted based on the initial heatmap of the target vertebrae's center point obtained from the U-net network and the distance and angle constraints between vertebrae.

[0091] Intervertebral continuity refers to the uninterrupted continuity of the distance between two vertebrae within a certain range, representing the proportion of the distance between two vertebrae to the total intervertebral distances. The offset angle between two vertebrae is the distance between the center points of two vertebrae. The total intervertebral distances are the sum of the distances between all adjacent vertebrae. There are 16 segments of intervertebral distances in 17 vertebrae. The offset angle between two vertebrae is the angle between the line connecting the center points of the two vertebrae and the U-axis. The proportion of the distance between the i-th segment of the vertebral distance is p. i The calculation formula is: d i d represents the intervertebral distance of the i-th vertebral segment, that is, the distance between the center point of the i-th vertebral segment and the center point of the (i+1)-th vertebral segment. sum This represents the sum of the distances between the 16 vertebral bodies. Statistical analysis of the sample set revealed that the proportion of the distance between two vertebral bodies to the total distance between vertebral bodies follows a normal distribution, and the offset angle between two vertebral bodies follows a normal distribution within the range of 0° to 180°.

[0092] The distance and angle constraints between vertebrae refer to the following: To predict the center point of the k2 vertebra to be predicted based on the initial heatmap of the target vertebra's center point, some prior information is needed. This includes preset maximum offset angles, preset maximum spacing, preset average spacing percentages, preset variance of spacing percentages, preset average offset angles, and preset variance of offset angles. This prior information is obtained as follows: For each sample in the sample set, the distance between the center point of the target vertebra (k1) and the center point of the k2 vertebra to be predicted is calculated, yielding the spacing for each sample. Simultaneously, the sum of the spacings of all adjacent vertebrae in the sample is calculated, and the spacing is divided by this sum to obtain the spacing percentage for each sample. The angle between the line connecting the center point of the target vertebra (k1) and the center point of the k2 vertebra to be predicted is calculated and intersected with the U-axis, yielding the offset angle for each sample. Finally, the maximum spacing value across all samples is calculated to obtain the preset maximum spacing value d. max (k1, k2), calculate the proportion of intervals corresponding to all samples, and represent them using a normal distribution to obtain the preset mean and variance μ of the interval proportions. dis (k1, k2), σ dis (k1, k2), calculate the maximum value of the offset angle corresponding to all samples, and obtain the preset maximum value of the offset angle |θ. max (k1, k2) - 90°|, count the offset angles corresponding to all samples, and represent them using a normal distribution to obtain the preset mean and variance μ of the offset angles. deg (k1, k2), σ deg (k1, k2). Therefore, the preset maximum offset angle, preset maximum spacing, preset average spacing ratio, preset variance of spacing ratio, preset average offset angle, and preset variance of offset angle in this embodiment are all calculated in advance based on the sample set, and vary with the target vertebra and the vertebra to be predicted.

[0093] Specifically, predicting the predicted vertebral center point of the vertebra to be predicted using the initial heat map of the target vertebral body's center point can include:

[0094] 1) Determine the kernel size based on the preset maximum offset angle and the preset maximum spacing. The offset angle is the angle between the line connecting the center point of the vertebra to be predicted and the center point of the target vertebra and the U-axis in the image coordinate system. The spacing is the distance between the center point of the vertebra to be predicted and the center point of the target vertebra.

[0095] The kernel size is w*h, where w is the width of the kernel and h is the length of the kernel. w = h = 2*max(d) max (k1, k2)*cos(|θ) max(k1, k2) - 90°|), d max (k1, k2)*sin(|θ) max (k1, k2) - 9°|)), (x2 - w / 2, y2) are the origins of the convolution kernel. Where k1 is the target vertebra, k2 is the vertebra to be predicted, and d max (k1, k2) are the preset maximum spacing values, |θ max (k1, k2)-90°| is the preset maximum offset angle, and (x2, y2) is the coordinate of the center point of the vertebra k2 to be predicted, which is an unknown value during the prediction process.

[0096] 2) For each target pixel of the target vertebra, the probability distribution formula of the distance ratio of the target pixel is determined according to the preset average distance ratio and the preset variance of the distance ratio. The probability distribution formula of the offset angle of the target pixel is determined according to the preset average offset angle and the preset variance of the offset angle. The convolution kernel template of the target pixel is determined according to the probability distribution formula of the distance ratio and the probability distribution formula of the offset angle. The distance ratio is the proportion of the distance between the center point of the vertebra to be predicted and the center point of the target vertebra to the total distance. The total distance is the sum of the distances between all adjacent vertebrae in the spine to be detected.

[0097] The probability distribution formula for the proportion of the distance from (u, v) to (x2, y2) is:

[0098]

[0099] Where g(p) is the probability distribution formula for the spacing proportion, p is the spacing proportion variable, and μ dis σ is the preset average spacing ratio. dis This represents the preset spacing percentage variance.

[0100]

[0101] Where (u, v) are the coordinates of the target pixel, (x2, y2) are the coordinates of the center point of the cone to be predicted, and d sum The total distance is calculated as follows: when there are 17 vertebrae, it represents the sum of the distances between 16 adjacent vertebrae. The distances between adjacent vertebrae are calculated based on the initial heatmap of the vertebral center point of each vertebra predicted by the U-net network.

[0102] The formula for the probability distribution of the offset angle from (u,v) to (x2,y2) is:

[0103]

[0104] Where f(θ) is the probability distribution formula for the offset angle, θ is the offset angle variable, and μ deg σ is the preset average offset angle. degThis is the preset offset angle variance.

[0105]

[0106] The convolution kernel template, that is, the probability convolution kernel on the initial heatmap of the center point of the target vertebra corresponding to the vertebra body, that is, the target pixel point (u, v) is the center point of the vertebra body to be predicted, can be expressed as:

[0107] h k1 (u-x2, v-y2) = f(θ)*g(p);

[0108] Among them, h k1 (u-x2, v-y2) is the convolution kernel template.

[0109] 3) Convolve the initial heatmap of the center point of the target vertebra using the convolution kernel size and the convolution kernel template of each target pixel to obtain the predicted heatmap of the center point of the vertebra to be predicted.

[0110] The formula for calculating the predicted heatmap of the vertebral body center point is:

[0111] g k2 (x2, y2)=∑f k1 (x1, y1)h k1 (ux s ,v-y2);

[0112] Among them, g k2 (x2, y2) is the predicted heatmap of the vertebral body center point, and (x2, y2) are the coordinates of the center point of the vertebra to be predicted; f k1 (x1, y1) represents the initial heatmap of the center point of the target vertebra, and (x1, y1) represents the coordinates of the center point of the target vertebra; h k1 (u-x2, v-y2) is the convolution kernel template, (u, v) are the coordinates of the target pixel, and the summation symbol refers to summing the values ​​of the target pixels.

[0113] In this embodiment, a convolution kernel template is first calculated based on prior information such as statistical spacing, spacing ratio, and offset angle. Then, the convolution kernel template and the initial heatmap of the target vertebra's center point are convolved to obtain the predicted heatmap of the vertebra's center point. After obtaining the predicted heatmaps of the vertebra's center points for all target vertebrae, the final heatmap of the vertebra's center point is obtained by weighting the initial heatmap of the vertebra's center point and all predicted heatmaps of the vertebra's center points, using the following formula:

[0114]

[0115] in, For the final heatmap of the center point of the i-th vertebra to be predicted, β kFor the weight parameter, g(k, i) is the predicted heatmap of the center point of the i-th vertebra obtained from the prediction of the k-th vertebra. There are a total of 17 vertebrae, so k∈(0, 16). When k=i, the predicted heatmap of the center point of the vertebra at this time refers to the initial heatmap of the center point of the i-th vertebra.

[0116] (3) Coordinate Transformation Module

[0117] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0118] The coordinate transformation module in this embodiment can use soft-argmax to convert the final heatmap of the vertebral body center point into the coordinate output of the vertebral body center point. By obtaining the center point coordinates, the center point of the vertebral body is first located, and then the coordinates of the four corner points of the vertebral body are obtained based on the center point coordinates and the vertebral body corner point heatmap, so as to convert the vertebral body corner point heatmap into the vertebral body corner point coordinate output. This alleviates the problem of quantization accuracy loss, can directly learn the heatmap, has a high degree of integration, and can directly output the required coordinates.

[0119] Before inputting the X-ray images into the trained keypoint detection model to obtain the keypoint coordinates of the spine to be detected, the detection method in this embodiment further includes: training the initial keypoint detection model using a sample set to obtain a trained keypoint detection model. The sample set includes multiple samples, including sample X-ray images and the true coordinates of the center point and corner points of each vertebra in the spine of the sample X-ray images. The method for constructing the sample set includes: acquiring multiple samples to form an initial set, performing data augmentation on the initial set to obtain the sample set. Data augmentation methods include randomly horizontally flipping the sample X-ray images with a 50% probability, randomly horizontally rotating the sample X-ray images with a 50% probability, and taking multi-view distance shots.

[0120] In this embodiment, an initial set including sample X-ray images is first obtained directly, and data augmentation is performed to obtain the final sample set, thereby improving the diversity of the samples and better training the initial keypoint detection model.

[0121] Multi-view distance shooting can include:

[0122] (1) Padding is applied to the sample X-ray film, and multiple position markers are set in the expanded area to obtain the expanded image.

[0123] This embodiment utilizes the ReplicationPad2d function in PyTorch to pad and expand the sample X-ray film, adding position markers within the expanded area. Specifically, up to 12 position markers can be set. The expanded area refers to the region in the expanded image that is larger than the sample X-ray film.

[0124] (2) Use a mobile phone to take multiple shots of the enlarged image from different angles and distances to obtain multiple images.

[0125] (3) For each captured image, the extrinsic parameter matrix is ​​determined based on the first coordinate of each position marker in the expanded image and the second coordinate in the captured image. The extrinsic parameter matrix is ​​used to determine the true value of the center point coordinate and the true value of the corner point coordinate of each vertebra in the spine of the captured image. The captured image and the true value of the center point coordinate and the true value of the corner point coordinate of each vertebra in the spine of the captured image constitute a new sample.

[0126] This embodiment utilizes an edge detection algorithm to detect 12 location markers on a captured image, obtaining the two-dimensional coordinates of each location marker in the captured image. Using the coordinate system of the expanded image as the first coordinate system and the coordinate system of the captured image as the second coordinate system, the extrinsic parameter matrix (Ri) is calculated using the PNP algorithm based on the first coordinates of the 12 location markers in the first coordinate system and the second coordinates in the second coordinate system, the intrinsic parameter matrix, and the distortion coefficients. iraw , t iraw ), where R iraw Given a 3x3 rotation matrix, t iraw The translation matrix is ​​3*1. The intrinsic parameter matrix and distortion coefficients are known information that can be obtained when capturing images using a mobile phone. Based on the obtained extrinsic parameter matrix (R... iraw , t iraw The actual coordinates of the four corner points of each vertebra labeled in the expanded image are used to specifically calculate the extrinsic parameter matrix (R). iraw , t iraw The true corner coordinates of the four corner points of each vertebra in the augmented image are multiplied by the true corner coordinates of the captured image. The true center point coordinates are then calculated based on these true corner coordinates to determine the true center point and corner coordinates of each vertebra in the captured image. This method of expanding the dataset using sample X-ray images taken with a mobile phone is thus achieved.

[0127] To quickly identify the augmented image corresponding to the captured image, this embodiment can also set four image number markers in the augmented area of ​​the augmented image. The image number markers have eight colors, corresponding to values ​​from 0 to 7, so the augmented image can be numbered in octal. Subsequently, a four-digit octal number can be obtained based on the RGB pixel values ​​corresponding to the four image number markers on the captured image. Then, the octal number can be converted to a decimal number to determine the number of the augmented image corresponding to the captured image.

[0128] During training, the loss functions used in this embodiment include: Euclidean distance loss and inverse distance constraint. The Euclidean distance loss applies an L2 constraint to the predicted coordinates of the cone center point. The Euclidean distance loss is as follows:

[0129]

[0130] Where N is the total number of vertebrae; in this embodiment, N is 17, representing 17 predicted coordinates of the center points of the vertebrae; (x i y i ) represents the predicted coordinates of the center point of the i-th vertebra. The coordinates of the center point of the i-th vertebra are the actual values.

[0131] The reverse distance constraint is applied to the predicted coordinates of the vertebral center point, and the reverse distance constraint is as follows:

[0132]

[0133] Where, p i The percentage of the intervertebral space in the i-th segment. pts i This represents the predicted coordinates of the center point of the i-th vertebral segment, in pts. i+1 μ represents the predicted coordinates of the center point of the (i+1)th vertebral segment. i σ is the preset average proportion of the intervertebral spacing in the i-th segment; i The preset variance of the vertebral spacing ratio of the i-th segment is denoted as 'variance'. Both the preset mean and the preset variance of the vertebral spacing ratio of the i-th segment are calculated in advance based on the sample set.

[0134] Training is performed using the two loss functions mentioned above, with training parameters including the network parameters and weight parameter β in the U-net network. k This allows for the utilization of the continuity of the spine, further improving the accuracy of test results.

[0135] This embodiment discloses a method for detecting key points of the spine based on X-ray images. It utilizes the U-net deep learning network to predict the initial heatmap of the vertebral body center point and the heatmap of the vertebral body corner points for each vertebra. Leveraging the continuity between vertebrae, it predicts the vertebral body center point prediction heatmap based on the initial heatmap of the target vertebral body center point obtained from the U-net network and the distance and angle constraints between the vertebral body center points. The predicted vertebral body center point heatmap of the target vertebra and the initial heatmap of the vertebral body center point are weighted according to a certain ratio to obtain the final vertebral body center point heatmap of the vertebral body. This final heatmap is then processed using soft-argmax. The final heatmap of the vertebral body center point is converted into coordinate output to obtain the coordinates of the center point of each vertebra in the spine. In this way, the center point of the vertebra is first located, and then the coordinates of the four corner points of each vertebra are obtained based on the coordinates of the center point of the vertebra and the heatmap of the vertebral corner points to complete the detection process of the key points of the spine. The detection method of this embodiment can directly distinguish the center of different vertebrae. After locating the center of the vertebra, the key points of the vertebra are regressed using a convolutional layer, which maintains the orderliness of the spine. The key points of the X-ray are extracted using a neural network, which solves the instability of traditional manual methods, assists in the screening of scoliosis, and has high detection accuracy.

[0136] Example 2:

[0137] This embodiment provides a spinal key point detection system based on X-ray images, such as... Figure 4 As shown, the detection system includes:

[0138] Image acquisition unit M1 is used to acquire X-ray images of the spine to be detected;

[0139] The key point detection unit M2 is used to input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0140] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0141] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0142] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0143] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0144] Example 3:

[0145] This embodiment provides a method for calculating the Cobb angle based on X-ray images, such as... Figure 5 and Figure 6 As shown, the calculation method includes:

[0146] T1: Obtain an X-ray of the spine to be examined;

[0147] T2: Input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0148] T3: Fit the coordinates of the center points of all the vertebrae to obtain the fitting equation of the center point curve composed of the center points of all the vertebrae;

[0149] The least moving squares method is used to fit the center point curve composed of the center points of all vertebrae, and the fitting equation is obtained.

[0150] T4: For each vertebra, calculate the first equation of the upper edge line and the second equation of the lower edge line of the vertebra based on the corner coordinates of the vertebra; calculate the first coordinates of the first intersection point of the center point curve and the upper edge line based on the fitted equation and the first equation; calculate the second coordinates of the second intersection point of the center point curve and the lower edge line based on the fitted equation and the second equation; calculate the third equation of the first perpendicular line of the tangent line passing through the first intersection point and tangent to the center point curve based on the fitted equation and the first coordinates; calculate the fourth equation of the second perpendicular line of the tangent line passing through the second intersection point and tangent to the center point curve based on the fitted equation and the second coordinates.

[0151] T5: For each vertebra, calculate the angle between the first perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the third equation of the neighboring vertebra; calculate the angle between the first perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the fourth equation of the neighboring vertebra; calculate the angle between the second perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the third equation of the neighboring vertebra; calculate the angle between the second perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra; select the maximum value of all the angles as the maximum angle corresponding to the vertebra; the neighboring vertebrae are the other vertebrae in the spine to be tested besides the vertebra in question.

[0152] T6: Select the maximum value of the maximum included angle of all the vertebrae as the Cobb angle.

[0153] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0154] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0155] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0156] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0157] In this embodiment, the perpendicular line from the intersection of the upper and lower edges of the vertebral body and the center point curve to the tangent of the center point curve is calculated. The angle between the perpendicular line from the intersection of the upper and lower edges of the vertebral body and the center point curve to the tangent of the center point curve is calculated between the intersection of the upper and lower edges of the vertebral body and the center point curve to the tangent of the center point curve. After calculating the angle between each pair of vertebral bodies, the largest angle is taken as the Cobb angle. Based on the grayscale image of the spine X-ray, the coordinates of the center point and corner points of the corresponding vertebral body are output first, and the Cobb angle, which can measure the degree of spinal curvature, is calculated from this. This can be used to assist in the screening of scoliosis in adolescents.

[0158] This embodiment can also divide the scoliosis into regions based on the second derivative of the fitted equation. Starting from the first vertebra, the regions with the same positive and negative values ​​of the second derivative are divided into one region, and the regions after the change are divided into another region. This can help determine whether the scoliosis is a C-curve or an S-curve, thus aiding in the determination of the type of scoliosis.

[0159] Example 4:

[0160] This embodiment provides a Cobb angle calculation system based on X-ray images, such as... Figure 7 As shown, the computing system includes:

[0161] Image acquisition unit M1 is used to acquire X-ray images of the spine to be detected;

[0162] The key point detection unit M2 is used to input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected.

[0163] The center point fitting unit M3 is used to fit the coordinates of the center points of all the vertebrae to obtain the fitting equation of the center point curve composed of the center points of all the vertebrae.

[0164] The corner fitting unit M4 is used to calculate, for each vertebra, a first equation for the upper edge line and a second equation for the lower edge line of the vertebra based on the corner coordinates of the vertebra; calculate the first coordinates of the first intersection point of the center point curve and the upper edge line based on the fitting equation and the first equation; calculate the second coordinates of the second intersection point of the center point curve and the lower edge line based on the fitting equation and the second equation; calculate the third equation of the first perpendicular line of the tangent line passing through the first intersection point and tangent to the center point curve based on the fitting equation and the first coordinates; and calculate the fourth equation of the second perpendicular line of the tangent line passing through the second intersection point and tangent to the center point curve based on the fitting equation and the second coordinates.

[0165] Angle calculation unit M5 is used to calculate, for each vertebra, the angle between the first perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the third equation of the neighboring vertebra; the angle between the first perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the fourth equation of the neighboring vertebra; the angle between the second perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra; and the angle between the second perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra. The maximum value of all the angles is selected as the maximum angle corresponding to the vertebra; the neighboring vertebrae are the other vertebrae in the spine to be tested besides the vertebra mentioned above.

[0166] The Cobb angle calculation unit M6 is used to select the maximum value of the maximum included angle of all the vertebrae as the Cobb angle.

[0167] The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence.

[0168] The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point.

[0169] The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted;

[0170] The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebra to obtain the coordinates of the key points of the spine to be detected.

[0171] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0172] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting key points in the spine, characterized in that, The detection method includes: Obtain X-ray images of the spine to be examined; The X-ray image is input into a trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected. The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence. The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point. The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted; The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebral body to obtain the coordinates of the key points of the spine to be detected. The specific steps of using the initial heatmap of the target vertebral body's center point to predict the predicted heatmap of the vertebral body's center point include: The kernel size is determined based on the preset maximum offset angle and the preset maximum spacing; the offset angle is the angle between the line connecting the center point of the vertebra to be predicted and the center point of the target vertebra and the U-axis in the image coordinate system; the spacing is the distance between the center point of the vertebra to be predicted and the center point of the target vertebra. For each target pixel of the target vertebra, a probability distribution formula for the spacing ratio of the target pixel is determined based on a preset average spacing ratio and a preset variance of the spacing ratio. Similarly, a probability distribution formula for the offset angle of the target pixel is determined based on a preset average offset angle and a preset variance of the offset angle. Finally, a convolutional kernel template for the target pixel is determined based on the spacing ratio probability distribution formula and the offset angle probability distribution formula. The spacing ratio is the proportion of the distance between the center point of the vertebra to be predicted and the center point of the target vertebra to the total distance. The total distance is the sum of the spacings of all adjacent vertebrae in the spine to be detected. The initial heatmap of the vertebral center point of the target vertebra is convolved using the convolution kernel size and the convolution kernel template of each target pixel to obtain the predicted heatmap of the vertebral center point of the vertebra to be predicted. The convolution kernel size is w*h, where w is the width of the convolution kernel and h is the length of the convolution kernel; w = h = 2*max(d max (k1,k2)*cos(|θ max (k1,k2)-90°|),d max (k1,k2)*sin(|θ max (k1,k2)-90°|));where k1 is the target vertebra; k2 is the vertebra to be predicted; d max (k1,k2) represents the preset maximum spacing; |θ max (k1,k2)-90°| is the preset maximum offset angle.

2. The detection method according to claim 1, characterized in that, The formula for the probability distribution of the spacing ratio is: Where g(p) is the probability distribution formula for the spacing proportion; p is the spacing proportion variable; μ dis The preset average spacing ratio; σ dis The pre-defined spacing percentage variance; Where (u, v) are the coordinates of the target pixel; (x2, y2) are the coordinates of the center point of the cone to be predicted; d sum This is the calculated total distance. The formula for the probability distribution of the offset angle is: Where f(θ) is the probability distribution formula for the offset angle; θ is the offset angle variable; μ deg The preset average offset angle; σ deg The preset offset angle variance; The convolution kernel template is: h k1 (u-x2,v-y2)=f(θ)*g(p); Among them, h k1 (u-x2,v-y2) is the convolution kernel template.

3. The detection method according to claim 1, characterized in that, Before inputting the X-ray image into the trained keypoint detection model to obtain the keypoint coordinates of the spine to be detected, the detection method further includes: training the initial keypoint detection model using a sample set to obtain a trained keypoint detection model; the sample set includes multiple samples; the samples include sample X-ray images and the true coordinates of the center point and the true coordinates of the corner points of each vertebra in the spine of the sample X-ray images; The method for constructing the sample set includes: acquiring multiple samples to form an initial set; performing data augmentation on the initial set to obtain the sample set; the data augmentation methods include random horizontal flipping, random horizontal rotation, and multi-view distance shooting.

4. The detection method according to claim 3, characterized in that, The multi-view distance shooting specifically includes: The sample X-ray film is padded and multiple position markers are set in the padded area to obtain the padded image; Multiple images are obtained by taking multiple photos of the expanded image from different angles and distances using a mobile phone. For each captured image, an extrinsic parameter matrix is ​​determined based on the first coordinates of each location marker in the expanded image and the second coordinates of each location marker in the captured image; the extrinsic parameter matrix is ​​used to determine the true values ​​of the center point coordinates and the true values ​​of the corner point coordinates of each vertebra in the spine of the captured image; the captured image and the true values ​​of the center point coordinates and the true values ​​of the corner point coordinates of each vertebra in the spine of the captured image constitute a new sample.

5. The detection method according to claim 3, characterized in that, The preset maximum offset angle, the preset maximum spacing, the preset average spacing ratio, the preset variance of spacing ratio, the preset average offset angle, and the preset variance of offset angle are pre-calculated based on the sample set.

6. A spinal key point detection system, characterized in that, The detection system includes: Image acquisition unit, used to acquire X-ray images of the spine to be detected; The key point detection unit is used to input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected. The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence. The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point. The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted; The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebral body to obtain the coordinates of the key points of the spine to be detected. The specific steps of using the initial heatmap of the target vertebral body's center point to predict the predicted heatmap of the vertebral body's center point include: The kernel size is determined based on the preset maximum offset angle and the preset maximum spacing; the offset angle is the angle between the line connecting the center point of the vertebra to be predicted and the center point of the target vertebra and the U-axis in the image coordinate system; the spacing is the distance between the center point of the vertebra to be predicted and the center point of the target vertebra. For each target pixel of the target vertebra, a probability distribution formula for the spacing ratio of the target pixel is determined based on a preset average spacing ratio and a preset variance of the spacing ratio. Similarly, a probability distribution formula for the offset angle of the target pixel is determined based on a preset average offset angle and a preset variance of the offset angle. Finally, a convolutional kernel template for the target pixel is determined based on the spacing ratio probability distribution formula and the offset angle probability distribution formula. The spacing ratio is the proportion of the distance between the center point of the vertebra to be predicted and the center point of the target vertebra to the total distance. The total distance is the sum of the spacings of all adjacent vertebrae in the spine to be detected. The initial heatmap of the vertebral center point of the target vertebra is convolved using the convolution kernel size and the convolution kernel template of each target pixel to obtain the predicted heatmap of the vertebral center point of the vertebra to be predicted. The convolution kernel size is w*h, where w is the width of the convolution kernel and h is the length of the convolution kernel; w = h = 2*max(d max (k1,k2)*cos(|θ max (k1,k2)-90°|),d max (k1,k2)*sin(|θ max (k1,k2)-90°|));where k1 is the target vertebra; k2 is the vertebra to be predicted; d max (k1,k2) represents the preset maximum spacing; |θ max (k1,k2)-90°| is the preset maximum offset angle.

7. A method for calculating Cobb angles, characterized in that, The calculation method includes: Obtain X-ray images of the spine to be examined; The X-ray image is input into a trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected. Fit the coordinates of the center points of all the vertebrae to obtain the fitting equation of the center point curve composed of the center points of all the vertebrae; For each vertebra, a first equation for the upper edge line and a second equation for the lower edge line are calculated based on the corner coordinates of the vertebra; a first coordinate of the first intersection point of the center point curve and the upper edge line is calculated based on the fitted equation and the first equation; a second coordinate of the second intersection point of the center point curve and the lower edge line is calculated based on the fitted equation and the second equation; a third equation of the first perpendicular line of the tangent line passing through the first intersection point and tangent to the center point curve is calculated based on the fitted equation and the first coordinates; and a fourth equation of the second perpendicular line of the tangent line passing through the second intersection point and tangent to the center point curve is calculated based on the fitted equation and the second coordinates. For each vertebra, the angle between the first perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra is calculated based on the third equation of the vertebra and the third equation of the neighboring vertebra. The angle between the first perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra is calculated based on the third equation of the vertebra and the fourth equation of the neighboring vertebra. The angle between the second perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra is calculated based on the fourth equation of the vertebra and the third equation of the neighboring vertebra. The angle between the second perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra is calculated based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra. The maximum value of all the angles is selected as the maximum angle corresponding to the vertebra. The neighboring vertebrae are the other vertebrae in the spine to be tested besides the vertebra in question. The maximum value of the maximum included angle of all the vertebrae is selected as the Cobb angle; The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence. The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point. The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted; The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebral body to obtain the coordinates of the key points of the spine to be detected. The specific steps of using the initial heatmap of the target vertebral body's center point to predict the predicted heatmap of the vertebral body's center point include: The kernel size is determined based on the preset maximum offset angle and the preset maximum spacing; the offset angle is the angle between the line connecting the center point of the vertebra to be predicted and the center point of the target vertebra and the U-axis in the image coordinate system; the spacing is the distance between the center point of the vertebra to be predicted and the center point of the target vertebra. For each target pixel of the target vertebra, a probability distribution formula for the spacing ratio of the target pixel is determined based on a preset average spacing ratio and a preset variance of the spacing ratio. Similarly, a probability distribution formula for the offset angle of the target pixel is determined based on a preset average offset angle and a preset variance of the offset angle. Finally, a convolutional kernel template for the target pixel is determined based on the spacing ratio probability distribution formula and the offset angle probability distribution formula. The spacing ratio is the proportion of the distance between the center point of the vertebra to be predicted and the center point of the target vertebra to the total distance. The total distance is the sum of the spacings of all adjacent vertebrae in the spine to be detected. The initial heatmap of the vertebral center point of the target vertebra is convolved using the convolution kernel size and the convolution kernel template of each target pixel to obtain the predicted heatmap of the vertebral center point of the vertebra to be predicted. The convolution kernel size is w*h, where w is the width of the convolution kernel and h is the length of the convolution kernel; w = h = 2*max(d max (k1,k2)*cos(|θ max (k1,k2)-90°|),d max (k1,k2)*sin(|θ max (k1,k2)-90°|));where k1 is the target vertebra; k2 is the vertebra to be predicted; d max (k1,k2) represents the preset maximum spacing; |θ max (k1,k2)-90°| is the preset maximum offset angle.

8. A Cobb angle calculation system, characterized in that, The computing system includes: Image acquisition unit, used to acquire X-ray images of the spine to be detected; The key point detection unit is used to input the X-ray image into the trained key point detection model to obtain the key point coordinates of the spine to be detected; the key point coordinates include the center point coordinates and corner point coordinates of each vertebra in the spine to be detected. The center point fitting unit is used to fit the coordinates of the center points of all the vertebrae to obtain the fitting equation of the center point curve composed of the center points of all the vertebrae. The corner fitting unit is used to calculate, for each vertebra, a first equation for the upper edge line and a second equation for the lower edge line of the vertebra based on the corner coordinates of the vertebra; calculate the first coordinates of the first intersection point of the center point curve and the upper edge line based on the fitting equation and the first equation; calculate the second coordinates of the second intersection point of the center point curve and the lower edge line based on the fitting equation and the second equation; calculate the third equation of the first perpendicular line of the tangent line passing through the first intersection point and tangent to the center point curve based on the fitting equation and the first coordinates; and calculate the fourth equation of the second perpendicular line of the tangent line passing through the second intersection point and tangent to the center point curve based on the fitting equation and the second coordinates. An angle calculation unit is used to calculate, for each vertebra, the angle between the first perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the third equation of the neighboring vertebra; the angle between the first perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the third equation of the vertebra and the fourth equation of the neighboring vertebra; the angle between the second perpendicular line of the vertebra and the first perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra; and the angle between the second perpendicular line of the vertebra and the second perpendicular line of the neighboring vertebra based on the fourth equation of the vertebra and the fourth equation of the neighboring vertebra. The maximum value of all the angles is selected as the maximum angle corresponding to the vertebra; the neighboring vertebrae are the other vertebrae in the spine to be detected besides the vertebra in question. The Cobb angle calculation unit is used to select the maximum value of the maximum included angle of all the vertebrae as the Cobb angle; The trained keypoint detection model includes a heatmap prediction module, a heatmap weighting module, and a coordinate transformation module connected in sequence. The heat map prediction module is used to predict the initial heat map of the vertebral body center point and the vertebral body corner point of each vertebra in the spine to be detected, using the X-ray as input; the initial heat map of the vertebral body center point includes the probability value of each pixel in the initial heat map of the vertebral body center point being the vertebral body center point; the vertebral body corner point heat map includes the probability value of each pixel in the vertebral body corner point heat map being the vertebral body corner point. The heatmap weighting module is used to predict the center point of the vertebral body to be predicted for each vertebral body to be predicted using the initial heatmap of the center point of the target vertebral body, and to perform a weighted summation of the initial heatmap of the center point of the vertebral body to be predicted and all the predicted heatmaps of the center point of the vertebral body to be predicted to obtain the final heatmap of the center point of the vertebral body to be predicted; the vertebral body to be predicted is any vertebral body in the spine to be detected, and the target vertebral body is any other vertebral body in the spine to be detected except for the vertebral body to be predicted; The coordinate transformation module is used to perform coordinate transformation on the final heat map of the vertebral center point and the vertebral corner point heat map of each vertebral body to obtain the coordinates of the key points of the spine to be detected. The specific steps of using the initial heatmap of the target vertebral body's center point to predict the predicted heatmap of the vertebral body's center point include: The kernel size is determined based on the preset maximum offset angle and the preset maximum spacing; the offset angle is the angle between the line connecting the center point of the vertebra to be predicted and the center point of the target vertebra and the U-axis in the image coordinate system; the spacing is the distance between the center point of the vertebra to be predicted and the center point of the target vertebra. For each target pixel of the target vertebra, a probability distribution formula for the spacing ratio of the target pixel is determined based on a preset average spacing ratio and a preset variance of the spacing ratio. Similarly, a probability distribution formula for the offset angle of the target pixel is determined based on a preset average offset angle and a preset variance of the offset angle. Finally, a convolutional kernel template for the target pixel is determined based on the spacing ratio probability distribution formula and the offset angle probability distribution formula. The spacing ratio is the proportion of the distance between the center point of the vertebra to be predicted and the center point of the target vertebra to the total distance. The total distance is the sum of the spacings of all adjacent vertebrae in the spine to be detected. The initial heatmap of the vertebral center point of the target vertebra is convolved using the convolution kernel size and the convolution kernel template of each target pixel to obtain the predicted heatmap of the vertebral center point of the vertebra to be predicted. The convolution kernel size is w*h, where w is the width of the convolution kernel and h is the length of the convolution kernel; w = h = 2*max(d max (k1,k2)*cos(|θ max (k1,k2)-90°|),d max (k1,k2)*sin(|θ max (k1,k2)-90°|));where k1 is the target vertebra; k2 is the vertebra to be predicted; d max (k1,k2) represents the preset maximum spacing; |θ max (k1,k2)-90°| is the preset maximum offset angle.

Images