Scoliosis parameter automatic measurement method based on X-ray film key point detection

Abstract

The invention belongs to the technical field of image processing, and relates to an automatic scoliosis parameter measurement method based on X-ray film key point detection, which is characterized in that a key point thermodynamic diagram and an offset diagram of a spine centrum are predicted at the same time based on a multi-task deep learning network, and bilaterally symmetric feature information of a spine structure is captured in a priori explicit manner by using a symmetric offset distance. The centrum center point is accurately recognized, correct pairing of centrum angular points is completed, various abnormal detection conditions such as missing points, redundant points and outliers can be effectively solved, the accuracy of centrum key point detection is guaranteed, full-automatic measurement of scoliosis core parameters such as Cobb angular parameters is achieved according to clinical measurement criteria, the calculation complexity is low, the real-time performance is high, and the method is suitable for large-scale popularization and application. And a foundation is laid for subsequent intelligent diagnosis and treatment of spinal diseases.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and relates to an automatic measurement method for scoliosis parameters based on key point detection of X-ray films. Background Art

[0002] Modern lifestyles pose a severe challenge to spinal health. Long periods of sitting and bad daily habits lead to tight back muscles and improper postures, and lack of sufficient exercise weakens the back and abdominal muscles, greatly increasing the risk of spinal diseases. In order to ease the pressure of seeking medical treatment and improve the diagnostic efficiency of clinicians, spinal disease auxiliary diagnosis and treatment technologies based on artificial intelligence (AI) have developed rapidly.

[0003] At present, in the clinical and AI-assisted diagnosis of spinal diseases, X-rays are widely used in the diagnosis of spinal deformity diseases, such as adolescent idiopathic scoliosis and lumbar degenerative scoliosis. According to the diagnostic needs, the spinal X-rays taken include 24 vertebrae (cervical 1-7, thoracic 1-12, lumbar 1-5), sacrum and ilium. Clinical spine surgeons measure a variety of medical indicators based on the anatomical structure of the spine, such as Cobb angle, C7 plumb line, sacral median vertical line, apical vertebra offset, sacral inclination, coronal plane balance, trunk inclination, etc., and classify the disease according to the commonly used classification criteria, so as to formulate a reasonable treatment plan. Among them, the measurement of Cobb angle is crucial and the most complex.

[0004] In the Cobb angle measurement, the physician needs to determine the end vertebra of the scoliosis based on experience, so as to determine the upper end plate of the upper end vertebra and the lower end plate of the lower end vertebra. The angle between these two straight lines is the Cobb angle. Depending on the severity of the patient's scoliosis, there may be 1-3 Cobb angles, namely: upper thoracic curve, main thoracic curve and thoracolumbar curve. The existing fully automatic Cobb angle measurement technology is mainly divided into two categories: one is to directly predict the Cobb angle by deep regression network, but this method lacks a clear measurement method display and has little clinical guidance significance; the other is to use deep learning image processing technology to obtain morphological information such as spinal structure, such as vertebral mask, vertebral coordinates, vertebral identification information, etc. Based on this information, the Cobb angle is parsed and calculated. This method can provide detailed display consistent with clinical measurement, so it has become the current mainstream method. Although different methods use different means to predict the results, the ultimate goal is to accurately predict or measure the inclination slope of each vertebra to calculate the Cobb angle. Although the method of using vertebral masks and coordinate fields to calculate the Cobb angle has achieved certain success, when encountering vertebral deformity, damage or poor X-ray imaging quality, the prediction of vertebral coordinate points often has the problem of missing points, multiple points and wrong points, and the coordinate points cannot be effectively identified, which makes it impossible to accurately obtain the vertebral tilt vector in subsequent processing. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art, and to provide an automatic measurement method for scoliosis parameters based on X-ray key point detection. The method simultaneously predicts the key point heat map and offset map of the spinal vertebrae based on a multi-task deep learning network, and uses the symmetric offset distance prior to explicitly capture the left-right symmetric feature information of the spinal structure, accurately identify the vertebral center point and complete the correct pairing of the vertebral corner points. It can effectively solve various abnormal detection situations such as missing points, redundant points, and outliers, ensure the accuracy of vertebral key point detection, and realize the fully automatic measurement of scoliosis core parameters such as Cobb angle parameters according to clinical measurement criteria.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for automatically measuring scoliosis parameters based on X-ray key point detection comprises the following steps:

[0008] S1. Dataset collection: Obtain the spine X-ray dataset (SpineX-C dataset), annotate the spine X-rays and then perform Gaussian filtering to obtain the corner mask map of the spine area;

[0009] S2, data processing: based on the corner point mask map, the cone center point mask map is obtained, and the center point adjacent offset vector is constructed by top-down and bottom-up methods respectively. Then, the corner point offset vector corresponding to each center point is constructed by using the matching relationship between the center point and the corner point.

[0010] S3, deep neural network construction: construct a deep neural network (KLAP-Net), and add a symmetric offset distance limit loss to constrain the offset distance of the key points in the deep neural network;

[0011] S4. Spine key point detection: Use the deep neural network constructed in step S3 to train the AISpine-XC dataset. Use the trained deep neural network model to obtain four key point feature maps of the vertebral center point, corner point, center point adjacent offset vector field and corner point offset vector in the predicted X-ray for any input spinal X-ray.

[0012] S5. Adaptive correction of vertebral center points: Using the adjacent offset field information of each center point, the ACIC algorithm is used to automatically correct the missing points, multiple points and wrong points of the vertebral center points to obtain the corrected center points;

[0013] S6, corner point correction and matching: using the CFOF algorithm to correct the predicted corner points to obtain corrected corner points, and at the same time completing the matching between the center point and the corner point;

[0014] S7, automatic measurement of Cobb angle: Based on the information of vertebral center and corner points, construct the vertebral position constraint matrix, find the maximum tilt angle in the matrix, and gradually modify the vertebral position index to finally obtain the three largest angles. According to the positional relationship of the three angles, the values ​​of the Cobb angles of thoracic scoliosis, main thoracic curve and thoracolumbar curve are finally determined;

[0015] S8. Measurement of other parameters: Based on the positions of the vertebral center and corner points, according to the clinical measurement method of scoliosis parameters, measure the C7 plumb line, sacral midline, apical vertebral offset, sacral inclination, coronal plane balance, and trunk inclination.

[0016] As a further technical solution of the present invention, the spinal X-ray film in step S1 includes a coronal X-ray film and a left and right oblique X-ray film. When marking the spinal X-ray film, the corner points of 17 vertebrae from T1 to L5 are marked, and the four corner points of the 17 vertebrae from T1 to L5 are marked respectively, so that a total of 68 vertebrae are marked. m=1,2,...,68, and Gaussian filtering is performed on the key points to obtain the corner mask map C={C1,C2,C3,C4} of the spine area, where C1, C2, C3 and C4 correspond to the upper left, upper right, lower left and lower right corner points respectively.

[0017] As a further technical solution of the present invention, the specific process of step S2 is:

[0018] S21, first obtain the coordinates of the center point of the vertebra n=0,1,...,N, where N+1 is the number of vertebral center points. Then, Gaussian filtering is performed on the vertebral center points to obtain the vertebral center point mask Q cen ;

[0019] S22. Construct a bidirectional center point adjacent offset vector (AOF) to obtain a top-down offset vector V TD and the bottom-up offset vector V BU , the specific calculation method is: j=0,1,...,N-1;

[0020] S23, by constructing the corner point offset vector O m To achieve the matching of center point and corner point, Figure 2 As shown in (d), the corner point offset vector O m The calculation process is: in represents the corner point coordinates, Represents the corresponding center point coordinates, m=1,2,...,68.

[0021] As a further technical solution of the present invention, the working process of the deep neural network (KLAP-Net) constructed in step S3 is as follows: the spinal X-ray is input into the deep neural network, firstly the low-level image features are extracted through two layers of convolution operations, and then the encoder and decoder are designed with a U-shaped architecture, wherein the encoder (Encoders) adopts a multi-scale residual downsampling module to extract the key point features of the spinal X-ray, and adds a self-attention (Self Attention, SA) module in the highest dimension to enhance the expression of spinal semantic information; the decoder (Decoders) adopts a residual upsampling fusion module, and finally passes through a fully connected layer to obtain a 17-layer feature map; then four different feature mapping heads are set based on the feature map, and the four feature mapping heads all include convolution, batch normalization, and Relu activation functions, and the convolution kernels are 1×1, 3×3, 7×7, and 3×3, respectively. After feature mapping, four key point feature maps of vertebral center points, vertebral corner points, center point adjacent offset vector fields, and corner point offset vector fields predicted by the network are obtained.

[0022] As a further technical solution of the present invention, the process of limiting the offset distance of the loss constraint key point by the symmetric offset distance in step S3 is as follows:

[0023] First define the predicted vertebral center and corner loss function L h : in, is the predicted vertebral corner feature, i = 1, 2, 3, 4, is the predicted vertebral center feature;

[0024] Then design the offset loss L0 for the corner point offset vector and the center point adjacent offset vector:

[0025]

[0026] in, and are the predicted corner point offset vector and the center point adjacent offset vector, respectively, O and V are the corner point offset vector and the center point adjacent offset vector obtained in step S2, respectively;

[0027] Then, the symmetric relationship between the vertebral corners and the center point is used to design the symmetric offset distance limit loss L s :

[0028]

[0029] Where eps = 1 × e -4 , For the predicted vertebral center point, the final comprehensive loss is set as the weighted loss function: L hos =λ1L h+λ2L0+λ3L s , where λ1, λ2 and λ3 are weight coefficients.

[0030] As a further technical solution of the present invention, in the process of training the deep neural network in step S4, random flipping, random scaling, and contrast change are used for data enhancement. The training uses the Adam optimizer, the learning rate is set to 0.001, the number of batch samples is 8, and a total of 400 rounds of training are performed.

[0031] As a further technical solution of the present invention, the specific process of step S5 is:

[0032] S51, according to the center point adjacent offset vector Get the average adjacent distance D of the center point adj : Redundant points are deleted from top to bottom according to the average adjacent distance of the center points. If the distance between two adjacent center points is less than D thr =τD adj , then delete the following redundant points; where τ is the threshold setting parameter;

[0033] S52, using the top-down center point adjacent offset vector predicted in step S4 Calculate the current point The corresponding lower adjacent point Q i ′ +1 , If the predicted neighboring points With Q i ′ +1 The distance d between them is less than τD adj , then it is believed that If the prediction is correct, keep this point unchanged and use it as the new current point. Otherwise, it is considered that there is a missed point. The missing points are filled by comprehensively using the center point adjacent offset vectors from top to bottom and bottom to top to obtain a new point: Update the indexes of all Q vectors and take the new point as the current point. Repeat the above process until the detection is completed and the corrected center point Q is obtained. c .

[0034] As a further technical solution of the present invention, the specific process of step S6 is: through the trained deep neural network (KLAP-Net), the positions of the four corner points are obtained. At the same time, the confidence of each corner point prediction can be obtained according to the network output The predicted corner offset vector based on the predicted confidence value Make adjustments to get the adjusted corner point offset vector m=1,2…,M;then based on the corrected center point Qc and the adjusted corner offset vector Calculate the corrected corner point position K ′ m : Then the predicted corner point position and the corrected corner point position K ′ m Perform comprehensive weighting to obtain the final corner point position K m : Where α is the weighted weight; all predicted corner points are traversed and verified, and the matching of each vertebral center point and corner point is completed during the verification process, and the positions of all predicted corner points are corrected to obtain the accurate detection positions of all corner points and mark them.

[0035] As a further technical solution of the present invention, the specific process of step S7 is: constructing the angle matrix A of 17 vertebrae and the vertebrae position constraint matrix T according to the center point corrected in step S5 and the corner point information corrected in step S6, wherein the vertebrae position constraint matrix T is:

[0036]

[0037] Where i, j represent the index of the center point of the first and last vertebra in a bend, tanθ represents the inclination of each vertebra; the angle matrix A is: A[i, j] = |θ i -θ j |,θ i ,θ j is the inclination angle of the i,j vertebral body. The angle matrix A and the vertebral position constraint matrix T are operated to obtain a new angle matrix A new =A*T, according to the new angle matrix, the three maximum inclination angles are retrieved, namely the Cobb angles corresponding to the thoracic scoliosis, the main thoracic curve and the thoracolumbar curve.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) For coronal, left and right oblique spinal X-ray images, the present invention can achieve accurate positioning and identification matching of the center points and corner points of 17 vertebrae (first thoracic vertebra to fifth lumbar vertebra) without any human-machine interaction;

[0040] (2) The present invention adopts an automatic correction algorithm for key points that integrates vertebral structure priors, which improves the robustness of key point detection and ensures the feasibility of subsequent parameter measurement. It is particularly suitable for spinal images with occlusion, blur and large inclination;

[0041] (3) The present invention is combined with clinical judgment criteria, with low computational complexity and high real-time performance, laying a foundation for the subsequent intelligent diagnosis and treatment of spinal diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The present invention provides a flowchart of a method for automatically measuring scoliosis parameters based on X-ray film key point detection.

[0043] Figure 2 Schematic diagram of vertebral key points and structural information according to an embodiment of the present invention.

[0044] Figure 3 This is a structural diagram of a deep neural network framework according to an embodiment of the present invention.

[0045] Figure 4 Schematic diagram of the vertebral center point adaptive correction process according to an embodiment of the present invention.

[0046] Figure 5 This is an example diagram of vertebral key point detection results according to an embodiment of the present invention.

[0047] Figure 6 This is an example diagram of the vertebral Cobb angle and core parameter measurement results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Example:

[0050] like Figure 1 As shown, a method for automatically measuring scoliosis parameters based on X-ray key point detection includes the following steps:

[0051] S1. Dataset collection:

[0052] A spine X-ray dataset (SpineX-C dataset) is obtained, and the spine X-rays are annotated and then Gaussian filtered to obtain a corner mask map of the spine area; the dataset of this embodiment is provided by the spine surgery department of the hospital, and contains a total of 294 spine X-rays. The spinal curvature posture and clarity in each spine X-ray are different. The spine surgeon annotates the dataset images and annotates the four corner points of the 17 vertebrae respectively, obtaining a total of 68 vertebrae annotated key point positions. m=1,2,...,68, and perform Gaussian filtering on the key points to obtain the true value mask map C={C1,C2,C3,C4} of the corner point area, corresponding to the upper left, upper right, lower left, and lower right corner points respectively, such as Figure 2(a)

[0053] S2. Data processing:

[0054] Based on the corner point mask map, the cone center point mask map is obtained, and the center point adjacent offset vector is constructed in two ways: top-down and bottom-up. Then, the corner point offset vector corresponding to each center point is constructed using the matching relationship between the center point and the corner point. Specifically:

[0055] Taking into account the rigid structure characteristics of the spine, auxiliary structure information is constructed based on the marked corner key points. The auxiliary structure information includes the vertebral center point, the offset vector field of the two-way adjacent center points, and the corner point offset vector. The position coordinates of the vertebral center point are n=0,1,...N, the acquisition method is:

[0056]

[0057] Where n = 0, 1, 2, ... N, N + 1 is the number of vertebral center points, and Gaussian filtering is performed on the vertebral center points to obtain the center point mask image Q cen ,like Figure 2 (b) In order to make full use of the structural similarity between vertebrae, a bidirectional adjacent center point offset vector field (Adjacent Offset Field, AOF) is constructed to obtain the top-down offset vector V TD and the bottom-up offset vector V BU ,like Figure 2 (c) is shown in the figure. The specific calculation method is as follows: j=0,1,...,N-1; To achieve accurate pairing of center point and corner point, such as Figure 2 As shown in (d), construct the corner point offset vector O m , calculated as follows: in represents the corner point coordinates, Represents the corresponding center point coordinates, m = 1, 2, ..., 68, and the corner point offset vector contains the direction information of the four corner points relative to the center point.

[0058] S3. Deep neural network construction:

[0059] like Figure 3As shown in the figure, the working process of the constructed deep neural network (KLAP-Net) is as follows: the spinal X-ray is input into the deep neural network, and the low-level image features are extracted through two layers of convolution operations, and then the encoder and decoder are designed with a U-shaped architecture. The encoder (Encoders) uses a multi-scale residual downsampling module to extract the key point features of the spinal X-ray, and adds a self-attention (Self Attention, SA) module in the highest dimension to enhance the expression of spinal semantic information; the decoder (Decoders) uses a residual upsampling fusion module, and finally passes through a fully connected layer to obtain a 17-layer feature map; then four different feature mapping heads are set based on the feature map. The four feature mapping heads all include convolution, batch normalization, and Relu activation functions. The convolution kernels are 1×1, 3×3, 7×7, and 3×3 respectively. After feature mapping, four key point feature maps of vertebral center points, vertebral corner points, center point adjacent offset vector fields, and corner point offset vector fields predicted by the network are obtained.

[0060] At the same time, a symmetric offset distance limit loss is added to the deep neural network to constrain the offset distance of the key points, so that the network can explicitly learn the prior knowledge of the vertebral structure. First, the loss function L for center point and corner point prediction is defined. h , the specific calculation is as follows: in, is the predicted vertebral corner feature, i = 1, 2, 3, 4, is the predicted vertebral center feature;

[0061] Then design the offset loss L0 for the corner point offset vector and the center point adjacent offset vector:

[0062]

[0063] in, and are the predicted corner point offset vector and the center point adjacent offset vector, respectively, O and V are the corner point offset vector and the center point adjacent offset vector obtained in step S2, respectively;

[0064] Then, in order to make full use of the symmetric relationship between the vertebral corners and the center point, the symmetric offset distance limit loss L is designed. s : Where eps = 1 × e -4 , For the predicted vertebral center point, the final comprehensive loss is set as the weighted loss function: L hos =λ1L h +λ2L0+λ3L s , where λ1, λ2 and λ3 are weight coefficients.

[0065] S4. Spine key point detection:

[0066] The deep neural network constructed in step S3 of the AISpine-XC dataset training was used. Random flipping, random scaling, and contrast change were used for data enhancement during training. The Adam optimizer was used for training, the learning rate was set to 0.001, the number of batch samples was 8, and a total of 400 rounds of training were performed. Using the trained deep neural network model, for any input original spinal X-ray, four key point feature maps of the predicted vertebral center point, corner point, center point adjacent offset vector field, and corner point offset vector in the X-ray were obtained.

[0067] S5, Vertebral center point adaptive correction:

[0068] By using the adjacent offset field information of each center point, the ACIC algorithm is used to automatically correct the missing points, multiple points and wrong points of the vertebral center point to obtain the corrected center point; specifically:

[0069] S51, according to the center point adjacent offset vector Get the average adjacent distance D of the center point adj : Redundant points are deleted from top to bottom according to the average adjacent distance of the center points. If the distance between two adjacent center points is less than τD adj , then delete the following redundant points; where τ is the threshold setting parameter.

[0070] S52, using the top-down center point adjacent offset vector predicted in step S4 like Figure 4 As shown, calculate the current point The corresponding lower adjacent point Q i ′ +1 , If the predicted neighboring points With Q i ′ +1 The distance d between them is less than τD adj , then it is believed that If the prediction is correct, keep this point unchanged and use it as the new current point. Otherwise, it is considered that there is a missed point. The missing points are filled by comprehensively using the center point adjacent offset vectors from top to bottom and bottom to top to obtain a new point: Update the indexes of all Q vectors and take the new point as the current point. Repeat the above process until the detection is completed and obtain the corrected center point set Q c .

[0071] S6. Corner point correction and matching:

[0072] The predicted corner points are corrected using the CFOF algorithm to obtain the corrected corner points, and the center point and corner point matching are completed at the same time; specifically: the positions of the four corner points are obtained through KLAP-Net At the same time, the confidence of each corner point prediction can be obtained according to the network output The predicted corner offset vector based on the predicted confidence value Make adjustments to get the adjusted corner point offset vector m=1,2…,M;then based on the corrected center point Q c and the adjusted corner offset vector Calculate the corrected corner point position K ′ m : Then the predicted corner point position and the corrected corner point position K ′ m Perform comprehensive weighting to obtain the final corner point position K m : Where α is the weighted weight; all predicted corner points are traversed and verified, and the matching of each vertebral center point and corner point is completed during the verification process, and the positions of all predicted corner points are corrected to obtain the accurate detection positions of all corner points and mark them, such as Figure 5 shown.

[0073] S7, Cobb angle automatic measurement:

[0074] According to the center point corrected in step S5 and the corner point information corrected in step S6, the angle matrix A of 17 vertebrae and the vertebrae position constraint matrix T are constructed, where the vertebrae position constraint matrix T is:

[0075]

[0076] Where i, j represent the index of the center point of the first and last vertebra in a bend, tanθ represents the inclination of each vertebra; the angle matrix A is: A[i, j] = |θ i -θ j |,θ i ,θ j is the inclination angle of the i,j vertebral body. The angle matrix A and the vertebral position constraint matrix T are operated to obtain a new angle matrix A new =A*T, according to the new angle matrix, the three maximum inclination angles are retrieved, namely the Cobb angles corresponding to the thoracic scoliosis, the main thoracic curve and the thoracolumbar curve.

[0077] S8. Other parameter measurements:

[0078] Based on the center points and corner points of 17 vertebrae (C7-L5), multiple spinal structural parameters can be measured according to the clinical measurement method of scoliosis parameters, such as Figure 6 As shown in the figure, the automatic measurement parameters include Cobb angle, C7 plumb line, sacral median vertical line, apical vertebra translation, sacral inclination, coronal plane balance, trunk inclination, etc. For example, according to the center points of C7 and L5 vertebrae, the C7 plumb line and sacral median vertical line can be drawn, and the apical vertebra translation (AVT) can be calculated according to the relative position relationship between the apical vertebra and the C7 plumb line and the sacral median vertical line. The remaining parameters can also be calculated according to the corresponding key points according to the clinical measurement method.

[0079] The algorithms, systems, and network structures not described in detail in the present invention are all general technologies in the art.

[0080] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0081] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for automatically measuring scoliosis parameters based on key point detection of X-ray films, characterized in that: The following steps are involved: S1. Dataset collection: obtain a spinal X-ray dataset, annotate the spinal X-rays, and then perform Gaussian filtering to obtain a corner mask map of the spinal region; S2, data processing: based on the corner point mask map, the cone center point mask map is obtained, and the center point adjacent offset vector is constructed by top-down and bottom-up methods respectively. Then, the corner point offset vector corresponding to each center point is constructed by using the matching relationship between the center point and the corner point. S3, deep neural network construction: construct a deep neural network, and add a symmetric offset distance limit loss to constrain the offset distance of key points in the deep neural network; S4. Spine key point detection: Use the deep neural network constructed in step S3 to train the AISpine-XC dataset. Use the trained deep neural network model to obtain four key point feature maps of the vertebral center point, corner point, center point adjacent offset vector field and corner point offset vector in the predicted X-ray for any input spinal X-ray. S5. Adaptive correction of vertebral center points: Using the adjacent offset field information of each center point, the ACIC algorithm is used to automatically correct the missing points, multiple points and wrong points of the vertebral center points to obtain the corrected center points; S6, corner point correction and matching: using the CFOF algorithm to correct the predicted corner points to obtain corrected corner points, and at the same time completing the matching between the center point and the corner point; S7, automatic measurement of Cobb angle: Based on the information of vertebral center and corner points, construct the vertebral position constraint matrix, find the maximum tilt angle in the matrix, and gradually modify the vertebral position index to finally obtain the three largest angles. According to the positional relationship of the three angles, the values ​​of the Cobb angles of thoracic scoliosis, main thoracic curve and thoracolumbar curve are finally determined; S8. Measurement of other parameters: Based on the positions of the vertebral center and corner points, according to the clinical measurement method of scoliosis parameters, measure the C7 plumb line, sacral midline, apical vertebral offset, sacral inclination, coronal plane balance, and trunk inclination.

2. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 1, characterized in that: Step S1: The spinal X-ray film includes coronal and left and right oblique X-ray films. When marking the spinal X-ray film, the corner points of 17 vertebrae from T1 to L5 are marked. The four corner points of the 17 vertebrae from T1 to L5 are marked respectively, and a total of 68 vertebrae are marked. Key point positions m=1,2,...,68, and Gaussian filtering is performed on the key points to obtain the corner mask map C={C1,C2,C3,C4} of the spine area, where C1, C2, C3 and C4 correspond to the upper left, upper right, lower left and lower right corner points respectively.

3. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 2, characterized in that: The specific process of step S2 is: S21, first obtain the coordinates of the center point of the vertebra n=0,1,...,N, where N+1 is the number of vertebral center points. Then, Gaussian filtering is performed on the vertebral center points to obtain the vertebral center point mask Q cen ; S22. Construct a bidirectional center point adjacent offset vector (AOF) to obtain a top-down offset vector V TD and the bottom-up offset vector V BU , the specific calculation method is: j=0,1,...,N-1; S23, by constructing the corner point offset vector O m To achieve the matching between the center point and the corner point, as shown in Figure 2(d), the corner point offset vector O m The calculation process is: in represents the corner point coordinates, Represents the corresponding center point coordinates, m=1,2,...,68.

4. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 3, characterized in that: The working process of the deep neural network constructed in step S3 is as follows: the spinal X-ray is input into the deep neural network, and the low-level image features are extracted through two layers of convolution operations, and then the encoder and decoder are designed with a U-shaped architecture, wherein the encoder uses a multi-scale residual downsampling module to extract the key point features of the spinal X-ray, and adds a self-attention module in the highest dimension to enhance the expression of spinal semantic information; the decoder uses a residual upsampling fusion module, and finally passes through a fully connected layer to obtain a 17-layer feature map; then four different feature mapping heads are set based on the feature map, and the four feature mapping heads all include convolution, batch normalization, and Relu activation functions, and the convolution kernels are 1×1, 3×3, 7×7, and 3×3, respectively. After feature mapping, four key point feature maps of vertebral center point, vertebral corner point, center point adjacent offset vector field, and corner point offset vector field predicted by the network are obtained.

5. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 4, characterized in that: The process of the symmetric offset distance limiting loss constraint key point offset distance in step S3 is: First define the predicted vertebral center and corner loss function L h : in, is the predicted vertebral corner feature, i = 1, 2, 3, 4, is the predicted vertebral center feature; Then design the offset loss L0 for the corner point offset vector and the center point adjacent offset vector: in, and are the predicted corner point offset vector and the center point adjacent offset vector, respectively, O and V are the corner point offset vector and the center point adjacent offset vector obtained in step S2, respectively; Then, the symmetric relationship between the vertebral corners and the center point is used to design the symmetric offset distance limit loss L s : Where eps = 1 × e -4 , For the predicted vertebral center point, the final comprehensive loss is set as the weighted loss function: L hos =λ1L h +λ2L0+λ3L s , where λ1, λ2 and λ3 are weight coefficients.

6. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 5, characterized in that: In step S4, during the deep neural network training process, random flipping, random scaling, and contrast change are used for data enhancement. The Adam optimizer is used for training, the learning rate is set to 0.001, the number of batch samples is 8, and a total of 400 rounds of training are performed.

7. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 6, characterized in that: The specific process of step S5 is: S51, according to the center point adjacent offset vector Get the average adjacent distance of the center point Redundant points are deleted from top to bottom according to the average adjacent distance of the center points. If the distance between two adjacent center points is less than D thr =τD adj , then delete the following redundant points; where τ is the threshold setting parameter; S52, using the top-down center point adjacent offset vector predicted in step S4 Calculate the current point The corresponding lower adjacent point Q i ′ +1 , If the predicted neighboring points With Q i ′ +1 The distance d between them is less than τD adj , then it is believed that If the prediction is correct, keep this point unchanged and use it as the new current point. Otherwise, it is considered that there is a missed point. The missing points are filled by comprehensively using the center point adjacent offset vectors from top to bottom and bottom to top to obtain a new point: Update the indexes of all Q vectors and take the new point as the current point. Repeat the above process until the detection is completed and the corrected center point Q is obtained. c .

8. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 7, characterized in that: The specific process of step S6 is: through the trained deep neural network (KLAP-Net), the positions of the four corner points are obtained At the same time, the confidence of each corner point prediction can be obtained according to the network output The predicted corner offset vector based on the predicted confidence value Make adjustments to get the adjusted corner point offset vector m=1,2…,M;then based on the corrected center point Q c and the adjusted corner offset vector Calculate the corrected corner point position Then the predicted corner point position and the corrected corner point position K ′ m Perform comprehensive weighting to obtain the final corner point position K m : Where α is the weighted weight; all predicted corner points are traversed and verified, and the matching of each vertebral center point and corner point is completed during the verification process, and the positions of all predicted corner points are corrected to obtain the accurate detection positions of all corner points and mark them.

9. The method for automatically measuring scoliosis parameters based on X-ray key point detection according to claim 8, characterized in that: The specific process of step S7 is: construct the angle matrix A of 17 vertebrae and the vertebrae position constraint matrix T according to the center point corrected in step S5 and the corner point information corrected in step S6, wherein the vertebrae position constraint matrix T is: Where i, j represent the index of the center point of the first and last vertebra in a bend, tanθ represents the degree of inclination of each vertebra; the angle matrix A is: A[i, j] = |θ i -θ j |,θ i ,θ j is the inclination angle of the i,j vertebral body. The angle matrix A and the vertebral position constraint matrix T are operated to obtain a new angle matrix A new =A*T, according to the new angle matrix, the three maximum inclination angles are retrieved, namely the Cobb angles corresponding to the thoracic scoliosis, the main thoracic curve and the thoracolumbar curve.

Images