A spine key point detection method, system, electronic device and storage medium

Abstract

The application discloses a spine key point detection method and system, an electronic device and a storage medium. The method comprises the following steps: acquiring a to-be-detected spine image; inputting the to-be-detected spine image into a key point detection network to obtain the center point and four corner points of each vertebral body; determining the original direction vector of the end plate of each vertebral body according to the four corner points of each vertebral body, wherein the end plate is the connecting line of the upper or lower bony boundary line of each vertebral body; inputting the four corner points of each vertebral body into a direction prediction module to obtain a predicted direction vector of the end plate; performing symbol alignment on the predicted direction vector and the original direction vector while keeping the midpoint and length of the end plate unchanged, fusing the symbol-aligned predicted direction vector and the original direction vector to obtain a final direction vector of the end plate; reconstructing the four corner points of each vertebral body according to the final direction vector of the end plate; and calculating a Cobb angle according to the reconstructed four corner points. The application can improve the accuracy of Cobb angle measurement.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method, system, electronic device and storage medium for detecting key points of the spine. Background Technology

[0002] In the field of automatic measurement of Cobb angle in scoliosis, keypoint detection methods based on deep learning have become the mainstream technical approach. These methods generally employ a cascaded framework of "first detecting the vertebral body center, then regressing the four corner points": first, the center localization module outputs the center position of each vertebra; then, the corner point offset branch regresses the offset of each vertebra's four corner points relative to the center; finally, the endplate direction is calculated from the corner point coordinates, and the Cobb angle is derived. This approach reduces the difficulty of corner point detection by sharing the center position and avoids the class imbalance problem caused by directly detecting all keypoints, making it a widely adopted technical architecture in this field.

[0003] However, this type of cascade regression framework has a technical flaw that has not been fully recognized: there is a fundamental misalignment between the model's optimization objective and the clinical measurement objective.

[0004] Specifically, existing methods primarily use loss functions supervised by the Euclidean distance of corner coordinates, aiming to minimize pixel-level spatial location errors. However, the clinical measurement accuracy of the Cobb angle is highly dependent on the accuracy of the endplate orientation, which is determined by the line connecting the two endpoints of the upper (or lower) endplate of the same vertebral body. When the vertebral endplate is close to the horizontal direction, even a small offset of the corner point along the tangent of the endplate is sufficient to cause a significant change in the endplate orientation angle, leading to an intolerable deviation of several degrees in the Cobb angle.

[0005] This means that a prediction that converges well in coordinate space may be far from optimal in angle space. Existing corner loss functions based on Euclidean distance only reflect the displacement error of a single pixel, failing to effectively constrain this situation of "small displacement, large angle error," and even less able to directly guide the network's learning process at the orientation level.

[0006] Therefore, how to overcome the limitations of existing coordinate regression frameworks in terms of orientation perception and establish a detection mechanism that can balance corner positioning accuracy and end-board orientation consistency is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] To address the shortcomings mentioned above, this application provides a method, system, electronic device, and storage medium for detecting key points of the spine.

[0008] Firstly, to achieve the above objectives, this application provides a method for detecting key points of the spine, comprising: Acquire the image of the spine to be tested, input the image of the spine to be tested into the key point detection network, and obtain the center point and four corner points of each vertebra; Based on the four corner points of each vertebra, determine the original direction vector of the endplate of each vertebra. The endplate is the line connecting the bony boundary line of the upper or lower edge of each vertebra. Input the four corner points of each vertebra into the direction prediction module to obtain the predicted direction vector of the endplate of each vertebra. The predicted direction vector of the endplate is aligned with the corresponding original direction vector of the endplate, while keeping the center point of the vertebral body and the length of the endplate unchanged. The predicted direction vector of the endplate after alignment is fused with the original direction vector of the endplate to obtain the final direction vector of the endplate. Reconstruct the four corner points of each vertebra based on the final direction vector of the endplate of each vertebra; Based on the four reconstructed corner points, the Cobb angle is calculated.

[0009] Preferably, the symbol-aligned final board predicted direction vector is fused with the original final board direction vector to obtain the final board direction vector; including: The final direction vector of the end plate after symbol alignment is multiplied by the correction coefficient, and then added to the original direction vector of the end plate multiplied by the correction coefficient complement. The result of the addition is then normalized to obtain the final direction vector of the end plate. The sum of the correction factor and the complement of the correction factor is equal to 1.

[0010] Preferably, the correction coefficient is less than 0.5 and greater than 0.

[0011] Preferably, the key point detection network includes a feature extraction network and a corner prediction head connected to the feature extraction network. The feature extraction network is used to extract features from the spinal image to be tested, and the corner prediction head is used to output the offset of the four corner points of each vertebra relative to its center point based on the features.

[0012] Preferably, the endplate includes an upper endplate and a lower endplate, and the direction prediction module outputs the upper endplate prediction direction vector and the lower endplate prediction direction vector for each vertebra.

[0013] Preferably, the method further includes: when the angle between the predicted direction vector of the final board and the corresponding original direction vector of the final board is greater than 90 degrees, reversing the predicted direction vector of the final board to achieve sign alignment.

[0014] Preferably, the Cobb angles are calculated based on the four reconstructed corner points, including: Based on the four corner points after reconstruction, determine the direction of the endplate of each vertebra; The end vertebrae of the target curvature region are determined based on the endplate direction. The angle between the end vertebrae and the endplate direction is calculated to obtain the Cobb angle. The target curvature region includes at least one of the proximal thoracic curve, the main thoracic curve, and the thoracolumbar curve.

[0015] Secondly, this application also provides a spinal key point detection system, comprising: The acquisition module is used to acquire the image of the spine to be tested, input the image of the spine to be tested into the key point detection network, and obtain the center point and four corner points of each vertebra. The processing module is used to determine the original direction vector of the endplate of each vertebra based on the four corner points of each vertebra, wherein the endplate is the line connecting the upper or lower bony boundary lines of each vertebra; input the four corner points of each vertebra into the direction prediction module to obtain the predicted direction vector of the endplate of each vertebra; align the predicted direction vector of the endplate with the corresponding original direction vector of the endplate, while keeping the center point of the vertebra and the length of the endplate unchanged; fuse the sign-aligned predicted direction vector of the endplate with the original direction vector of the endplate to obtain the final direction vector of the endplate; and reconstruct the four corner points of each vertebra based on the final direction vector of the endplate of each vertebra. The execution module calculates the Cobb angle based on the reconstructed four corner points.

[0016] Thirdly, this application also provides an electronic device, including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit performs the above-described method.

[0017] Fourthly, this application also provides a storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the above-described method.

[0018] Compared with the prior art, the beneficial effects of this application are as follows: This application obtains the predicted direction vector of the final board by setting up a direction prediction module, and then aligns and fuses the predicted direction vector with the original direction vector calculated from the corner coordinates to obtain the final final board direction. This decouples the prediction of the final board direction from the corner regression task, realizing active correction of the final board direction. Compared with existing technologies that rely solely on corner coordinates to indirectly determine the final board direction, this application effectively avoids the problem of significant deviations in the final board direction caused by small errors in the corner coordinates, thus improving the accuracy of Cobb angle calculation. Attached Figure Description

[0019] Figure 1 This is a flowchart of the spinal key point detection method of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that in this application, "spine image" refers to a medical image containing the spinal region, preferably an anteroposterior X-ray image of the spine; "vertebral body" refers to the bony structure of each segment of the spine. This application uses 17 vertebral bodies as an example for illustration, but those skilled in the art will understand that the method of this application is also applicable to spinal images with other numbers of vertebral bodies.

[0022] The spinal keypoint detection method provided in this application includes a training phase and an inference phase. In the training phase, a basic keypoint detection network is first trained using an anteroposterior X-ray image of the spine with annotations of the vertebral body center and four corner points. Then, structural constraints are introduced into the corner point offset branch for fine-tuning. Finally, a direction prediction module is added, and only this newly added module is trained. In the inference phase, the vertebral body center and four corner points are first obtained, then the endplate direction is corrected based on the predicted endplate direction vector, and finally, the Cobb angle is calculated.

[0023] This application provides a method for detecting key points in the spine, including: The image of the spine to be tested is obtained and input into the key point detection network to obtain the center point and four corner points of each vertebra.

[0024] Specifically, an image of the spine to be tested is acquired, preferably an anteroposterior X-ray image. The image of the spine to be tested is scaled to a fixed size H×W and then grayscale normalized. In one embodiment, the input size is 1024×512 pixels.

[0025] Each image contains 17 vertebrae arranged from top to bottom. Each vertebra has four corner points: upper left, upper right, lower left, and lower right, denoted as _____. , , , The center of the i-th vertebra is defined as the average of its four corner points: ; The center point is used to determine the position and order of the vertebral bodies, while the four corner points are used to describe the boundaries of the vertebral bodies and the orientation of the upper and lower endplates. The upper endplate is the line connecting the upper left and upper right corner points, and the lower endplate is the line connecting the lower left and lower right corner points.

[0026] It is understood that the 17 vertebrae are only illustrative examples, and in other embodiments, different numbers of vertebrae may be detected as needed.

[0027] In a preferred embodiment of this application, the keypoint detection network includes a feature extraction network and a corner prediction head connected to the feature extraction network. The feature extraction network is used to extract features from the spinal image to be tested, and the corner prediction head is used to output the offset of the four corner points of each vertebra relative to its center point based on the features.

[0028] Furthermore, the feature extraction network is the high-resolution HRNet-W32. HRNet-W32 extracts features in parallel across multiple resolution branches and performs cross-resolution information exchange at different stages, thereby continuously preserving high-resolution spatial representations. To adapt to keypoint prediction, each resolution branch is projected to a uniform number of channels through 1×1 convolutions, and the low-resolution branches are then bilinearly upsampled to the target scale with a stride of 4. After concatenation, a unified feature map F is obtained by fusion through 3×3 convolutions. This feature map simultaneously serves center heatmap, center offset, corner offset, and final board orientation prediction.

[0029] Based on the fusion feature F, the center heatmap head outputs a vertebral body center probability map H; the center offset head outputs a 2-channel offset Δc to compensate for the quantization error caused by heatmap downsampling; and the corner offset head outputs an 8-channel offset Δp to represent the two-dimensional displacement of the four corner points relative to the vertebral body center. During inference, non-maximum suppression is applied to the center heatmap, and 17 center candidates are selected. The center position is then restored by combining the center offset, and the four corner points are restored by the corner offset. Finally, the ordered vertebral body sequence is obtained by sorting the data according to the center ordinate.

[0030] In another embodiment of this application, the feature extraction network may also employ other backbone networks such as ResNet.

[0031] Based on the four corner points of each vertebra, determine the original direction vector of the endplate of each vertebra. The endplate is the line connecting the bony boundary line of the upper or lower edge of each vertebra.

[0032] Specifically, the original direction vector of the upper endplate is determined by the line connecting the upper left and upper right corners of the vertebral body, and the original direction vector of the lower endplate is determined by the line connecting the lower left and lower right corners of the vertebral body. In one embodiment, the endplate direction vector is a unit direction vector.

[0033] Input the four corner points of each vertebra into the direction prediction module to obtain the predicted direction vector of the endplate of each vertebra.

[0034] The four corner points of each vertebra are input into the direction prediction module to obtain the predicted direction vector of the endplate of each vertebra. The direction prediction module is used to predict the direction of the endplate of the vertebra based on the positions of the four corner points of the vertebra.

[0035] In a preferred embodiment, the orientation prediction module is a convolutional prediction head. The convolutional prediction head has 4 output channels, which are used to output the upper endplate orientation vector and the lower endplate orientation vector, respectively. Specifically, channels 1-2 are used to output the upper endplate orientation vector, and channels 3-4 are used to output the lower endplate orientation vector.

[0036] When training this orientation prediction module, the base network and existing prediction heads are frozen, and only this newly added orientation head is trained. The supervision targets are obtained using Normalize(TR-TL) and Normalize(BR-BL), respectively: the upper endplate orientation vector is determined by the unit vector pointing from the upper left corner to the upper right corner, and the lower endplate orientation vector is determined by the unit vector pointing from the lower left corner to the lower right corner. The loss function can be either orientation cosine loss or smoothed L1 loss.

[0037] The predicted direction vector of the endplate is aligned with the corresponding original direction vector of the endplate, while keeping the center point of the vertebral body and the length of the endplate unchanged. The aligned predicted direction vector of the endplate is then fused with the original direction vector of the endplate to obtain the final direction vector of the endplate.

[0038] Specifically, since the direction prediction vectors predicted by the direction prediction module may have opposite directions, sign alignment is required. Specifically, the angle between the final board's predicted direction vector and the corresponding original final board direction vector is calculated. When the angle is greater than 90 degrees, the final board's predicted direction vector is reversed to achieve sign alignment. Sign alignment ensures that the predicted direction vector and the original direction vector point in the same direction.

[0039] In the embodiments of this application, the midpoint and length of the final plate are kept unchanged, and the predicted direction vector of the final plate after symbol alignment is fused with the original direction vector of the final plate to obtain the final direction vector of the final plate.

[0040] In one implementation, the fusion is a weighted fusion. Specifically, the symbol-aligned final board predicted direction vector is multiplied by a correction coefficient, and then added to the original final board direction vector multiplied by the complement of the correction coefficient. The sum is then normalized to obtain the final board direction vector. The sum of the correction coefficient and its complement equals 1. The correction coefficient is less than 0.5 and greater than 0.

[0041] In a preferred embodiment, the correction coefficient α = 0.048. That is, the predicted direction vector participates in the fusion with only a small weight, while the original direction vector is retained with a larger weight, thereby achieving conservative correction and avoiding over-correction of the final board direction due to excessive prediction deviation.

[0042] It should be noted that during the above fusion process, the midpoint and length of the final plate remain unchanged, and only the direction of the final plate is adjusted, so that the correction process improves the consistency of the final plate direction without destroying the original position structure of the corner points.

[0043] The four corner points of each vertebra are reconstructed based on the final direction vector of the endplate of each vertebra.

[0044] In some embodiments, during reconstruction, the upper end plate is extended by half its length along the corrected upper end plate direction vector, with the midpoint of the upper end plate as the center, to obtain the upper left and upper right corner points respectively; the lower end plate is extended by half its length along the corrected lower end plate direction vector, with the midpoint of the lower end plate as the center, to obtain the lower left and lower right corner points respectively.

[0045] Based on the four reconstructed corner points, the Cobb angle is calculated.

[0046] Specifically, the endplate orientation of each vertebra is determined based on the reconstructed four corner points. The end vertebrae of the target curvature region are then determined based on the endplate orientation, and the Cobb angle is calculated from the included angle of the endplate orientation of each end vertebra. The target curvature region includes at least one of the proximal thoracic curve (PT), the main thoracic curve (MT), and the thoracolumbar curve (TL).

[0047] The two vertebrae at the ends refer to the two vertebrae with the largest tilt angles in the target curvature region. The calculation of the Cobb angle follows the well-known rules for selecting the end vertebrae in the art.

[0048] In one implementation, the Cobb angle values ​​for the PT, MT, and TL regions are output, along with visualizations of the center point, four corner points, and endplate line of each vertebra, to aid in clinical diagnosis. Example 1

[0049] A spinal X-ray image is acquired, scaled to 1024×512 and normalized, and then input into the HRNet-W32 backbone network. The network outputs multi-resolution features, which are projected, upsampled, and fused to form a unified feature map. The center heatmap, center offset, and corner offset are then output separately. During inference, non-maximum suppression is applied to the center heatmap, selecting 17 center candidates. The center position is restored using the center offset, and the four corner points are restored using the corner offset, resulting in the center point and four corner points of each vertebra. During training, center heatmap focus loss, center offset L1 loss, and corner offset L1 loss are used for supervision. Based on Example 1, a pre-trained keypoint detection network is loaded. The original direction vectors of the superior and inferior endplates of each vertebra are determined based on the four corner points. The four corner points of each vertebra are input into a 4-channel convolutional prediction head to obtain the predicted direction vectors of the superior and inferior endplates for each vertebra. For each endplate, the angle between the predicted direction vector and the original direction vector is calculated. If the angle is greater than 90 degrees, the predicted direction vector is reversed. Keeping the endplate midpoint and length unchanged, the sign-aligned predicted direction vector and the original direction vector are weighted and fused using α=0.048 to obtain the final endplate direction vector. The four corner points of each vertebra are reconstructed based on the final endplate direction vector. The endplate direction of each vertebra is determined based on the reconstructed corner points, and the end vertebrae in the PT, MT, and TL regions are identified. The angle between the endplate directions of the end vertebrae is calculated to obtain three Cobb angles.

[0050] This application also provides a spinal key point detection system, including: The acquisition module is used to acquire the image of the spine to be tested, input the image of the spine to be tested into the key point detection network, and obtain the center point and four corner points of each vertebra. The processing module is used to determine the original direction vector of the endplate of each vertebra based on the four corner points of each vertebra, wherein the endplate is the line connecting the upper or lower bony boundary lines of each vertebra; input the four corner points of each vertebra into the direction prediction module to obtain the predicted direction vector of the endplate of each vertebra; align the predicted direction vector of the endplate with the corresponding original direction vector of the endplate, while keeping the center point of the vertebra and the length of the endplate unchanged; fuse the sign-aligned predicted direction vector of the endplate with the original direction vector of the endplate to obtain the final direction vector of the endplate; and reconstruct the four corner points of each vertebra based on the final direction vector of the endplate of each vertebra. The execution module calculates the Cobb angle based on the reconstructed four corner points.

[0051] This application also provides an electronic device, including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit performs the above-described method.

[0052] This application also provides a storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the above-described method.

[0053] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting key points in the spine, characterized in that, include: Acquire an image of the spine to be tested, input the image of the spine to be tested into a key point detection network, and obtain the center point and four corner points of each vertebra; Based on the four corner points of each vertebra, the original direction vector of the endplate of each vertebra is determined, wherein the endplate is the line connecting the upper or lower bony boundary lines of each vertebra. Input the four corner points of each vertebra into the direction prediction module to obtain the endplate prediction direction vector of each vertebra. The predicted direction vector of the endplate is aligned with the corresponding original direction vector of the endplate, while keeping the center point of the vertebral body and the length of the endplate unchanged. The predicted direction vector of the endplate after alignment is fused with the original direction vector of the endplate to obtain the final direction vector of the endplate. Reconstruct the four corner points of each vertebra based on the final direction vector of the endplate of each vertebra; Based on the reconstructed four corner points, the Cobb angle is calculated.

2. The method for detecting key points of the spine according to claim 1, characterized in that, The step of fusing the symbol-aligned predicted direction vector of the final board with the original direction vector of the final board to obtain the final direction vector of the final board includes: The symbol-aligned predicted direction vector of the final board is multiplied by a correction coefficient, and then added to the original direction vector of the final board multiplied by the complement of the correction coefficient. The result of the addition is then normalized to obtain the final direction vector of the final board. Wherein, the sum of the correction coefficient and the complement of the correction coefficient is equal to 1.

3. The method for detecting key points of the spine according to claim 2, characterized in that, The correction coefficient is less than 0.5 and greater than 0.

4. The method for detecting key points of the spine according to claim 1, characterized in that, The key point detection network includes a feature extraction network and a corner prediction head connected to the feature extraction network. The feature extraction network is used to extract features from the spinal image to be tested, and the corner prediction head is used to output the offset of the four corner points of each vertebra relative to its center point based on the features.

5. The method for detecting key points of the spine according to claim 2, characterized in that, The endplate includes an upper endplate and a lower endplate, and the direction prediction module outputs the upper endplate prediction direction vector and the lower endplate prediction direction vector for each vertebra.

6. The method for detecting key points of the spine according to claim 1, characterized in that, Also includes: When the angle between the predicted direction vector of the final board and the corresponding original direction vector of the final board is greater than 90 degrees, the predicted direction vector of the final board is reversed to achieve the symbol alignment.

7. The method for detecting key points of the spine according to claim 1, characterized in that, The calculation of the Cobb angle based on the reconstructed four corner points includes: Based on the reconstructed four corner points, the endplate orientation of each vertebra is determined; The end vertebrae of the target curvature region are determined based on the endplate direction, and the Cobb angle is obtained by calculating the endplate direction angle of the end vertebrae. The target curvature region includes at least one of the proximal thoracic curve, the main thoracic curve, and the thoracolumbar curve.

8. A spinal key point detection system, characterized in that, include: The acquisition module is used to acquire the image of the spine to be tested, input the image of the spine to be tested into the key point detection network, and obtain the center point and four corner points of each vertebra. The processing module is used to determine the original direction vector of the endplate of each vertebra based on the four corner points of each vertebra, wherein the endplate is the line connecting the upper or lower bony boundary lines of each vertebra; input the four corner points of each vertebra into the direction prediction module to obtain the predicted direction vector of the endplate of each vertebra; align the predicted direction vector of the endplate with the corresponding original direction vector of the endplate, while keeping the center point of the vertebra and the length of the endplate unchanged; fuse the sign-aligned predicted direction vector of the endplate with the original direction vector of the endplate to obtain the final direction vector of the endplate; and reconstruct the four corner points of each vertebra based on the final direction vector of the endplate of each vertebra. The execution module calculates the Cobb angle based on the reconstructed four corner points.

9. An electronic device, characterized in that, The method includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the method described in any one of claims 1 to 7.

10. A storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the method described in any one of claims 1 to 7.

Images