Spine X-ray film Cobb angle automatic measurement system based on deep learning

Abstract

The invention provides a spine X-ray film Cobb angle automatic measurement system based on deep learning, and relates to the technical field of medical image processing, and the system comprises a vertebral body end plate positioning module based on an end-to-end Keypoint R-CNN, and the vertebral body end plate positioning module carries out vertebral body positioning and feature extraction based on a clinical spine X-ray film; the multi-scale feature fusion and ROI processing module optimizes centrum positioning precision based on feature extraction; the multi-instance post-processing optimization module performs post-processing on the optimized cone positioning to obtain an ordered cone array; a self-adaptive Cobb angle calculation module based on geometrical morphology analysis is used for carrying out geometrical morphology analysis on the ordered cone array; a multi-type Cobb angle measurement module based on an end plate tilt angle calculates a Cobb angle based on a result of the geometric morphology analysis. The average centrum detection precision of the system provided by the invention reaches 93.2%, multi-stage error accumulation is avoided by adopting single network joint optimization, and the system exceeds a multi-model assembly line architecture by 15%-20%; the average positioning precision of the key points of the end plates reaches 97.0%, and the positioning error of the end points of the four end plates is smaller than + / -2 pixels.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and more specifically, to an automatic measurement system for the Cobb angle of spinal X-rays based on deep learning. Background Technology

[0002] Scoliosis is a common spinal disease, and its diagnosis and severity assessment mainly rely on the measurement of the Cobb angle. Traditional Cobb angle measurement methods have the following technical problems: Traditional methods rely on radiologists manually marking the vertebral endplates on X-ray films, drawing vertical lines, and measuring the angle; measurements from different doctors can vary by 5-10 degrees. Vertebral identification and endplate localization depend entirely on the doctor's experience, lacking objective standards, unified measurement standards, and quality control mechanisms; the same doctor may obtain inconsistent results when measuring the same X-ray film at different times. Measuring a single X-ray film takes 15-30 minutes, which cannot meet the needs of large-scale screening. To achieve automated Cobb angle measurement, existing technologies include: (1) Key point detection method: The key point detection method is essentially a combination of image recognition and direct calculation of Codd angle. First, the coordinates of the key points are identified, and then the Codd angle is calculated based on the identified key point coordinates. Existing technologies that adopt this scheme include: Chinese invention patent A method for measuring the Cobb angle of spinal X-ray images (publication number CN112734757A), Chinese invention patent Method, device and electronic equipment for measuring the Cobb angle of the spine (publication number CN119693362A). The problem with this type of scheme is that in order to directly identify the coordinates of the key points, the multi-stage pipeline architecture (including body position recognition-target detection-key point recognition) adopted by this type of scheme requires training and maintaining multiple independent models, resulting in high system complexity.

[0003] (2) Image reconstruction detection method: Compared with the key point detection method, the image reconstruction detection method does not directly calculate the Codd angle, but predicts and analyzes the Codd angle based on the reconstructed image. Existing technologies using this approach include: Chinese invention patent A method and system for measuring the Cobb angle of the spine (publication number CN113223072A) and Chinese invention patent A method for automatic evaluation of the Cobb angle of scoliosis based on deep learning (publication number CN114881957A). The problem with this type of approach is that scoliosis itself is not data that can be directly predicted based on neural networks. Due to the influence of each person's physiological morphology, there are substantial differences in the spinal morphology of each person. The reconstruction result of the neural network trained on the training set is substantially different from the spinal tilt angle distribution of the detection result. Therefore, this type of approach only helps to enhance the interpretability of the X-ray itself by processing image data, while the data obtained based on the prediction itself has no practical value.

[0004] (3) Methods to improve recognition ability: This type of scheme further improves the recognition ability of images based on the scheme of (1). It can be understood as image processing, image recognition and Cobb angle calculation. Typical examples of this type of method are: Chinese invention patent A system and method for calculating the Cobb angle of scoliosis based on full-length X-ray of the spine based on artificial intelligence-image recognition (publication number CN112381757A) and Chinese invention patent A method for automatic measurement of Cobb angle based on layered reconstruction of the spine (publication number CN113870098A, which also performs image reconstruction). The problem with this type of scheme is that in order to improve the image recognition ability, the image processing first goes through the image segmentation process. The segmented images are then used for merging calculation, which makes it difficult to balance the weights and confidence. Summary of the Invention

[0005] To address the aforementioned problems, this application employs a deep learning-based automatic Cobb angle measurement system for spinal X-rays, comprising: Vertebral endplate localization module based on end-to-end Keypoint R-CNN: used for vertebral body localization and feature extraction based on clinical spinal X-rays; Multi-scale feature fusion and ROI processing module: used to optimize vertebral body localization accuracy based on feature extraction; Multi-instance post-processing optimization module: used to post-process the optimized cone positioning to obtain an ordered cone array; Adaptive Cobb Angle Calculation Module Based on Geometric Morphology Analysis: Used to perform geometric morphology analysis on ordered cone arrays; Multi-type Cobb angle measurement module based on endplate tilt angle: used to calculate Cobb angle based on the results of geometric morphology analysis.

[0006] Optionally, the vertebral endplate localization module based on end-to-end Keypoint R-CNN includes using a single deep learning network to simultaneously complete vertebral target detection and endplate key point localization through an end-to-end Keypoint R-CNN architecture; Endplate key point localization includes detecting four anatomical key points for each vertebral body using a four-point endplate endpoint localization algorithm: the left endpoint of the superior endplate, the right endpoint of the superior endplate, the left endpoint of the inferior endplate, and the right endpoint of the inferior endplate, directly corresponding to the endplate position required for Cobb angle measurement; using the ResNet-50-FPN backbone feature extraction network: employing a residual network combined with a feature pyramid to extract multi-scale image features; and generating vertebral body candidate regions through an adaptive region generation network (RPN), automatically adapting to vertebral body size variations in patients of different body types through an anchor frame mechanism.

[0007] Optionally, the multi-scale feature fusion and ROI processing module includes: constructing a multi-level feature representation through a feature pyramid network to enhance small target detection capabilities; extracting candidate region features through an ROI alignment algorithm; classifying candidate regions and performing bounding box regression through an ROI detection head; and generating keypoint heatmaps and locating precise coordinates through a keypoint detection head.

[0008] Optionally, the multi-instance post-processing optimization module includes: Confidence Filter: Used to filter detection results with a confidence level below the 0.7 threshold; Non-maximum suppressor: Used to eliminate duplicate detections, with an IoU threshold of 0.5; Anatomical position sorter: used to sort vertebral bodies from top to bottom according to their Y-coordinate; Quantity controller: Used to limit the maximum number of vertebrae to be detected to 17.

[0009] Optionally, the adaptive Cobb angle calculation module based on geometric morphology analysis includes: calculating the midpoint coordinates of the upper and lower endplates of each vertebra using the geometric mean method based on the four detected endplate endpoints; forming a continuous spinal centerline by connecting the midpoints of the endplates of adjacent vertebrae through a spinal centerline builder; providing a spinal type determiner based on vector cross product, using a vector cross product geometric algorithm to analyze the change in the bending direction of the line connecting the midpoints, and determining C-type or S-type scoliosis by calculating the sign change of the cross product of three consecutive point vectors; and calculating the rate of change of curvature at each point on the spinal centerline through a curvature change rate analysis algorithm to locate the bending direction conversion point and the position of maximum scoliosis.

[0010] Optionally, the multi-type Cobb angle measurement module based on the endplate tilt angle includes a C-type single-bend Cobb angle calculator, an S-type double-bend Cobb angle calculator, an antitangent endplate angle measurement calculator, and a clinical segmentation angle output device: The C-type single-bend Cobb angle calculator is designed for unidirectional curved spines. It identifies the upper and lower vertebrae with the greatest tilt and calculates the difference in tilt angle between the endplates of the two vertebrae as the Cobb angle of the main curve. The S-shaped double-curve Cobb angle calculator is designed for bidirectional curved spines. It divides the spine into segments based on the point of change of curvature direction and calculates the Cobb angle of the upper and lower curved segments separately. The arctangent endplate angle measurement calculator uses the arctangent function to calculate the angle between the endplate tangent and the horizontal line, obtains the endplate tilt angle through arctan(Δy / Δx), and then calculates the difference in intervertebral angle to obtain the Cobb angle; The clinical segmented angle output device outputs three segmented angle values ​​for the main thoracic curve, upper thoracic curve, and thoracolumbar curve according to clinical diagnostic standards.

[0011] Optionally, it also includes an intelligent visualization and results management module, which includes an image renderer, angle annotator, report generator, and quality control module. The image renderer is used to draw the detection box, key points, and measurement guide lines on the raw X-ray; An angle annotator is used to annotate the angle values ​​and vertebral numbers of each curved segment on the image; The report generator is used to automatically generate standardized diagnostic reports that include measurement data and visualization results; The quality control module is used to assess the reliability of test results and output confidence levels and quality warnings.

[0012] Optionally, the Cobb angle calculation includes: Calculation of the midpoint of the final plate: Midpoint of upper endplate = (left endpoint of upper endplate + right endpoint of upper endplate) / 2; Midpoint of lower endplate = (left endpoint of lower endplate + right endpoint of lower endplate) / 2; Calculation of the endplate tangent angle: θ = arctan((right endpoint.y - left endpoint.y) / (right endpoint.x - left endpoint.x)); In the formula, right endpoint.x and right endpoint.y represent the x-axis coordinate and y-axis coordinate of the right endpoint, respectively, and left endpoint.x and left endpoint.y represent the x-axis coordinate and y-axis coordinate of the left endpoint, respectively.

[0013] Cobb angle calculation: Cobb angle =|θ 上椎体 -θ 下椎体 |, when the angle is >90 degrees, Cobb angle =180°-Cobb angle .

[0014] Optionally, the vector cross product geometric algorithm includes: Initialize the bending direction change counter (direction) changes =0; For three consecutive vertebral body center points C on the central line of the spine i-1 C i and C i+1 Construct direction vectors v1 and v2; Direction vector v1=C i -C i-1 The direction vector v1 represents the direction vector from the previous vertebra to the current vertebra; Direction vector v2=C i+1 -C i The direction vector v2 represents the direction vector from the current vertebra to the next vertebra; Calculate the cross product of vectors. product Determine the direction of bending: cross product =v1.x×v2.y-v1.y×v2.x; In the formula, v1.x and v1.y represent the x-component and y-component of vector v1, respectively, and v2.x and v2.y represent the x-component and y-component of vector v2, respectively.

[0015] When cross product A value greater than 0 indicates a bend to the left; when cross... product <0 indicates bending to the right; when cross product When ≈0, it represents a locally approximate straight line; compare the sign changes of two adjacent cross products: If it exists: cross i >0 and cross i-1 <0; Or it may exist: cross i <0 and cross i-1 >0; Then direction changes +=1; The total number of directional changes along the entire spinal centerline is counted to quantify the complexity of spinal curvature.

[0016] Optionally, intelligent spinal type determination includes: if direction changes ≥1 indicates S-shaped scoliosis; if direction changes =0, indicating C-type scoliosis.

[0017] The beneficial effects of the deep learning-based automatic measurement system for Cobb angle of spinal X-ray provided in this application are as follows: (1) The end-to-end architecture brings high precision performance. Under the condition of IoU=0.50, the average accuracy of vertebral body detection reaches 93.2%. The single network joint optimization avoids the accumulation of multi-stage errors and surpasses the multi-model pipeline architecture by 15-20 percentage points. Under the condition of IoU=0.50, the average accuracy of endplate key point positioning reaches 97.0%, and the positioning error of the four endplate endpoints is less than ±2 pixels, which directly corresponds to the anatomical structure required for Cobb angle measurement. The Cobb angle measurement accuracy reaches 91.03%, with a symmetric average absolute percentage error of 8.97%. Through precise positioning of the final plate and optimization of geometric algorithms, it is significantly better than the method based on the Cobb angle matrix. The consistency with manual measurement reaches ICC>0.95, the measurement error is less than ±1.2 degrees, and the end-to-end optimization mechanism ensures the global optimal solution.

[0018] (2) The simplified model structure leads to high-efficiency processing. The processing time for a single X-ray is less than 5 seconds (average 2.8 seconds). A single network can complete all tasks in one forward propagation. Compared with the multi-stage pipeline architecture, it reduces model call overhead and improves efficiency by 40-60%. It supports batch parallel processing and can process more than 1,000 X-rays per hour without the need for serial calls of multiple models. The GPU-accelerated end-to-end inference speed reaches 35 FPS, and the memory usage of a single model is only 8GB, which saves more than 60% of resource consumption compared with the multi-model architecture. The system response time is <3 seconds, and the entire process from image input to result output is <10 seconds, avoiding the delay of switching between multiple models.

[0019] (3) Strong robustness, adaptable to different shooting conditions, image quality and patient body shape X-rays, with a success rate of >95%; good tolerance to image quality problems such as noise, contrast changes and underexposure; supports X-rays with resolution range of 1000×2000 to 3000×6000 pixels; has adaptive processing capability for incomplete vertebrae (partial obstruction or truncation).

[0020] (4) High reproducibility and standardization: the results of multiple measurements of the same image are completely consistent (difference = 0 degrees), eliminating human subjective error; different operators use the system to obtain the same results, and the test-retest reliability reaches 100%; a unified measurement standard and quality control mechanism are established, which conforms to clinical diagnostic norms; the measurement process is traceable, and all intermediate results can be saved and audited.

[0021] (5) High level of intelligence, fully automated processing flow, no need for manual annotation or intervention, reducing operating costs by more than 90%; adaptive algorithm can handle various complex spinal morphologies such as C-type and S-type; intelligent abnormality detection function, automatically identifies image quality problems and abnormal anatomical structures. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0023] Figure 1 This is an overall flowchart of the deep learning-based automatic measurement system for Cobb angle of spinal X-ray provided in the embodiments of this application; Figure 2 This is a flowchart of the Cobb angle calculation provided in the embodiments of this application. Detailed Implementation

[0024] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0025] like Figure 1 As shown, the deep learning-based automatic measurement system for the Cobb angle of spinal X-rays provided in this application includes the following main computational modules: Vertebral endplate localization module based on end-to-end Keypoint R-CNN: End-to-end Keypoint R-CNN architecture: It uses a single deep learning network to simultaneously complete vertebral target detection and endplate keypoint localization, realizing joint optimization of detection and localization and avoiding error accumulation in multi-stage pipelines; Four-point endplate localization algorithm: For each vertebra, detect four key anatomical points: the left endplate of the superior endplate, the right endplate of the superior endplate, the left endplate of the inferior endplate, and the right endplate of the inferior endplate, which directly correspond to the endplate position required for Cobb angle measurement; ResNet-50-FPN backbone feature extraction network: It uses a residual network combined with a feature pyramid to extract multi-scale image features, thereby enhancing the ability to detect cones of different sizes. Adaptive Region Generation Network (RPN): Generates candidate vertebral bodies and automatically adapts to changes in vertebral body size in patients with different body types through an anchor frame mechanism.

[0026] Multi-scale feature fusion and ROI processing module: Feature Pyramid Network: Constructs multi-level feature representations to enhance small target detection capabilities; ROI alignment algorithm: Extracts candidate region features to avoid quantization errors; ROI detection head: classifies candidate regions and performs bounding box regression; Key point detection head: Generates key point heatmaps and locates precise coordinates.

[0027] The core operations of the vertebral endplate localization module based on end-to-end Keypoint R-CNN are feature extraction and region generation. This module is responsible for initially locating the vertebral body region from X-ray images, forming the basis for subsequent precise processing. Its components include: The image preprocessing unit, as the first submodule, includes a format conversion component, an arraying component, and a normalization layer. In this embodiment, it specifically consists of OpenCV / PIL (format conversion), NumPy (arraying), and a normalization layer (normalizing pixel values ​​to [0,1]). This is used to convert clinical DICOM / JPG format spinal X-ray images into tensors that the model can process.

[0028] The backbone feature extraction network ResNet50-FPN serves as the second sub-module of the vertebral endplate localization module based on end-to-end Keypoint R-CNN. It includes a residual network component and a feature pyramid component, specifically ResNet50 (a 50-layer residual network) and a feature pyramid network (FPN) in this embodiment. It then outputs feature maps at four scales (denoted as P2-P5, corresponding to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 downsampling of the original image) to cover vertebrae of different sizes. For example, the cervical vertebrae are smaller in scale and correspond to a high-resolution P2 feature map, while the lumbar vertebrae are larger in scale and correspond to a low-resolution P5 feature map.

[0029] The Region Generation Network (RPN), as the third sub-module of the vertebral endplate localization module based on end-to-end Keypoint R-CNN, includes a convolutional component and an anchor box generation component. In this embodiment, it specifically consists of a 1×1 convolutional layer (with two branches) and an anchor box generator. It is used to generate anchor boxes on multi-scale feature maps and filter out candidate vertebral regions.

[0030] The core operations of the multi-scale feature fusion and ROI processing module are feature selection and precise extraction. It is responsible for refining the candidate regions and outputting the final cone classification, bounding box, and key points. Its components include: The ROI alignment unit, as the first sub-module of the multi-scale feature fusion and ROI processing module, includes an ROI alignment layer to replace traditional ROI pooling, avoid quantization errors, and extract features of candidate regions from multi-scale feature maps.

[0031] The multi-scale feature fusion unit is the second sub-module of the multi-scale feature fusion and ROI processing module. Its components include convolutional layers and feature concatenation layers (as a feasible implementation, the feature concatenation layer can also be a weighted summation layer). It is used to fuse ROI features at different scales to enhance the feature representation of the small cone.

[0032] The dual detection head, consisting of the classification regression head and the keypoint detection head, is the third sub-module of the multi-scale feature fusion and ROI processing module. It includes a classification regression head component and a keypoint detection head component. The classification regression head component includes a fully connected layer and a Softmax layer, while the keypoint detection head component includes a convolutional layer and a Gaussian kernel activation layer. They output the cone and background classification results, bounding box correction values, and keypoint heatmaps, respectively.

[0033] The data processing flow after inputting X-ray images into the vertebral endplate localization module and the multi-scale feature fusion and ROI processing module based on end-to-end Keypoint R-CNN includes: After inputting a clinical spinal X-ray (DICOM format, 16-bit grayscale), PIL reads the image, OpenCV converts the image to an 8-bit grayscale image, and NumPy converts the image into an array ([1,1,2000,4000], corresponding to batch, channel, height, and width, respectively). The normalization layer scales the pixel values ​​to [0,1] and outputs a preprocessed tensor ([1,1,2000,4000]). Data preprocessing is performed in this process, completing the conversion from X-ray input to tensor output.

[0034] Then, the backbone network extracts multi-scale features. The input data is preprocessed to obtain tensors, which are then downsampled five times by ResNet-50 to obtain four basic feature maps (C2-C5, [1,256,500,1000], [1,512,250,500], [1,1024,125,250], and [1,2048,62,125], respectively). The FPN then upsamples and laterally connects C2-C5, outputting four unified-channel multi-scale feature maps (P2-P5). This process completes the transformation from tensor input to multi-scale feature map output.

[0035] The P2-P5 feature maps are input into the region generation network. First, anchor boxes are generated. For each pixel of P2-P5, the anchor box generator generates anchor boxes with three ratios (1:1, 1:2, 2:1) to correspond to different sizes of the vertebrae (e.g., the anchor box size of P2 is 32×32-128×128 pixels, which is suitable for the cervical spine; the anchor box size of P5 is 256×256-512×512 pixels, which is suitable for the lumbar spine). Then, anchor box classification and regression are performed. A 1×1 convolutional layer outputs classification and regression branches. The classification branch predicts the probability that each anchor box is a cone, and the regression branch predicts the coordinate correction value of the anchor box. Candidate ROIs are screened, and anchor boxes with a classification probability > 0.5 are retained. Non-maximum suppression (NMS, IoU=0.5) is applied to the retained anchor boxes to eliminate duplicate boxes. Finally, 2000 candidate regions (ROI, shape: [2000,4], where 4 corresponds to x1, y1, x2, y2 coordinates) are selected. The output of 2000 candidate ROIs is used as suspected cone regions. In the above process, anchor boxes and candidate ROIs are output based on multi-scale feature maps.

[0036] The candidate ROIs and P2-P5 feature maps output by the end-to-end Keypoint R-CNN-based vertebral endplate localization module are input into the multi-scale feature fusion and ROI processing module.

[0037] First, ROI alignment is performed. Based on the size of each ROI, a feature map of the corresponding scale is matched. Small ROIs are matched with high-resolution feature maps such as P2, and large ROIs are matched with low-resolution feature maps such as P5. The ROI regions are also cropped to a uniform size to avoid the quantization error of traditional ROI pooling. Then, multi-scale feature fusion is performed. The features of ROIs aligned at different scales are compressed and concatenated into fused features through convolutional layers, and the fused ROI features are output.

[0038] The fully connected layer flattens the fused ROI features into vectors. The Softmax layer of the classification branch outputs the vertebral-background classification result, retaining ROIs with a classification probability > 0.7. The regression branch outputs bounding box correction values, adjusting the candidate ROIs into more accurate vertebral bounding boxes. Finally, it outputs the vertebral region and the vertebral confidence score for each region.

[0039] The fusion features corresponding to the final vertebral region are input into the key point detection head. The convolutional layer upsamples the fusion features step by step and outputs four key point heatmaps, which correspond to the left, right, left and right endpoints of the upper endplate, the lower endplate, and the lower endplate, respectively. The maximum value position of each heatmap is taken as the initial pixel coordinates of the key point. By using bilinear interpolation to perform weighted calculations on the pixels around the maximum value position, more accurate sub-pixel coordinates are obtained, and finally the coordinates of the four anatomical key points of each vertebra are output.

[0040] Multi-instance post-processing optimization module: Confidence Filter: Filters detection results with a confidence level below the 0.7 threshold; Non-maximum suppressor: eliminates duplicate detections, sets the IoU threshold to 0.5; Anatomical position sorter: Arranged from top to bottom according to the vertebral body Y-coordinate, conforming to the anatomical order; Quantity controller: Limits the maximum number of vertebrae to be detected to 17.

[0041] Adaptive Cobb angle calculation module based on geometric morphology analysis: Midpoint calculation algorithm for endplates: Based on the four detected endplate endpoints, the geometric mean method is used to calculate the midpoint coordinates of the upper and lower endplates of each vertebra, eliminating the influence of endplate tilt on the centerline. Spinal centerline builder: Connecting the midpoints of the endplates of adjacent vertebrae forms a continuous spinal centerline, reflecting the overall scoliosis morphology of the spine; A spine type detector based on vector cross product: It uses a vector cross product geometric algorithm to analyze the bending direction change of the line connecting the midpoints, and intelligently determines C-type or S-type scoliosis by calculating the sign change of the cross product of three consecutive point vectors. Curvature change rate analysis algorithm: Calculates the curvature change rate at each point on the spinal centerline, locates the bending direction conversion point and the position of maximum lateral curvature, and provides a quantitative basis for the selection of Cobb angle vertebrae.

[0042] Multi-type Cobb angle measurement module based on endplate tilt angle: C-type single-bend Cobb angle calculator: For a unidirectional curved spine, it automatically identifies the upper and lower vertebrae with the greatest tilt and calculates the difference in the tilt angle of the endplates of the two vertebrae as the Cobb angle of the main curve. S-shaped double-curve Cobb angle calculator: For bidirectional curved spines, the spine is segmented based on the point of change of curvature direction, and the Cobb angle of the upper and lower curved segments are calculated separately to achieve independent quantification of multiple curves; The algorithm for measuring the arctangent endplate angle is as follows: the arctangent function is used to calculate the angle between the endplate tangent and the horizontal line, the endplate tilt angle is obtained by arctan(Δy / Δx), and the Cobb angle is obtained by calculating the difference in the intervertebral angles. Clinical segmented angle output device: Outputs three segmented angle values ​​according to clinical diagnostic criteria: main thoracic curve (PT), upper thoracic curve (MT), and thoracolumbar curve (TL), to comprehensively assess the degree of scoliosis.

[0043] Intelligent visualization and results management module: Image renderer: Draws detection boxes, key points, and measurement guides on the raw X-ray image; Angle labeler: Labels the angle values ​​and vertebral numbers of each curved segment on the image; Report Generator: Automatically generates standardized diagnostic reports that include measurement data and visualization results; Quality control module: assesses the reliability of test results, outputs confidence level and quality warnings.

[0044] In this embodiment, the hardware environment used is as follows: the computing server is equipped with an NVIDIA GPU RTX 3080; the operating system is Linux Ubuntu 18.04; the deep learning frameworks are PyTorch 1.8+ and CUDA 11.0+; and the image processing libraries include OpenCV 4.0+, PIL 8.0+, and NumPy 1.20+.

[0045] The pre-trained model is trained on the COCO dataset and has good basic capabilities in object detection and key point localization. Transfer learning improves training efficiency. Model parameters are set as follows: the number of classification categories is 2 (background and vertebral body), and the number of key points is fixed at 4 (corresponding to the left and right endpoints of the upper and lower endplates of each vertebra). This directly corresponds to the anatomical key locations required for Cobb angle measurement. A single network architecture is used to achieve end-to-end joint optimization of the three tasks of vertebral body detection, classification, and key point localization, avoiding error propagation and system complexity in a multi-model pipeline.

[0046] Modification of the sorting and detection head: Extract the input feature dimensions of the original model's classification layer; Replace with a custom Fast R-CNN predictor to adapt to the spinal vertebral body detection task; Output two types of detection results: background region and cone region, and predict the bounding box coordinates of the cone.

[0047] Special modification for the end-board critical point detection head: Extract the number of input channels from the original keypoint prediction layer and analyze the spatial resolution and semantic information of the feature map; Replace it with a keypoint predictor specifically designed for the vertebral endplate endpoints, which outputs the precise positions of four fixed keypoints (more structured than the coordinates of an indefinite number of corner points). Each key point corresponds to a specific anatomical endpoint of the vertebral endplate: the left endpoint of the superior endplate, the right endpoint of the superior endplate, the left endpoint of the inferior endplate, and the right endpoint of the inferior endplate. These four points directly define the endplate orientation required for Cobb angle measurement. The key point heatmap is represented by a Gaussian kernel, and accurate coordinate prediction of the endpoints of the final board is achieved through sub-pixel level positioning, with a positioning error of less than ±2 pixels.

[0048] Forward reasoning process: Input X-ray image data; in training mode, annotation information is also input for supervised learning; in inference mode, the detection results are output: vertebral body bounding box, classification confidence, and coordinates of 4 key points; all detection, classification, and key point localization are completed in a single forward propagation, achieving end-to-end processing.

[0049] The dataset used is SpineWeb Dataset 16, which contains 609 labeled spinal X-ray images. Data augmentation was performed on the dataset, and as examples of data augmentation, this included random rotation ±10°, brightness adjustment ±20%, and contrast adjustment ±15%.

[0050] The format of the annotation information is: 4 key points for each vertebra (left and right endpoints of the upper endplate and left and right endpoints of the lower endplate) and bounding box.

[0051] Training and validation set split: 80% training set, 20% validation set; In this embodiment, the model training parameters are as follows: Learning rate: Initially 0.001, decreasing by 0.1 every 50 epochs. Batch size: 4 (limited by GPU memory); Training epochs: 200; Optimizer: SGD with momentum=0.9, weight decay =0.0005; Post-processing algorithm implementation process: Step 1: Confidence screening The model prediction results extract three types of data: vertebral body bounding box, confidence score, and key point coordinates. A confidence threshold of 0.7 is set to filter out all detection results with a confidence score below this threshold. High-confidence detections are retained to ensure the accuracy and reliability of subsequent vertebral body detection. The bounding box, score, and key point data are updated synchronously to maintain the correspondence between the three.

[0052] Step 2: Nonmaximum suppression The NMS (Non-Maximum Suppression) algorithm is used to eliminate duplicate detections; the IoU (Intersection over Union) threshold is set to 0.5, and two detection boxes are considered duplicates when their overlap exceeds this value; the result with the highest confidence is retained in the duplicate detections, and other duplicates are deleted; each vertebra is detected only once to avoid redundant results.

[0053] Step 3: Assembling the anatomical locations Calculate the vertical center coordinates (center point of the Y-axis) of the bounding box of each vertebra; sort all detected vertebrae by Y coordinate from smallest to largest (from top to bottom); the sorting result conforms to the natural anatomical order of the spine, namely cervical vertebrae, thoracic vertebrae, and lumbar vertebrae; this provides the correct order basis for subsequent vertebral numbering and Cobb angle calculation.

[0054] Step 4: Quantity Control The maximum number of vertebral bodies to be detected is set to 17 (7 cervical vertebrae + 12 thoracic vertebrae + 5 lumbar vertebrae, with a maximum of 24 vertebral bodies in the human body). When the number of detection results exceeds this limit, the first 17 (i.e., the vertebral bodies with the highest confidence and the highest position) are retained. Redundant detection results are deleted to avoid false detections interfering with subsequent analysis. The final processed vertebral body detection results are output, including bounding boxes, confidence scores, and key point information.

[0055] like Figure 2 As shown, the Cobb angle calculation includes: 1. Calculation of the midpoint of the final plate: Midpoint of upper endplate = (left endpoint of upper endplate + right endpoint of upper endplate) / 2; Midpoint of lower endplate = (left endpoint of lower endplate + right endpoint of lower endplate) / 2; 2. Endplate tangent angle: θ = arctan((right endpoint.y - left endpoint.y) / (right endpoint.x - left endpoint.x)); 3. Cobb's angle: Cobb angle =|θ 上椎体 -θ 下椎体 |, when the angle is >90 degrees, Cobb angle =180°-Cobb angle ; Vertebral body data extraction and processing: Key point coordinate extraction: Traverse all detected vertebrae and extract the coordinates of four key points for each vertebra: left endpoint of the upper endplate (x1, y1), right endpoint of the upper endplate (x2, y2); left endpoint of the lower endplate (x3, y3), right endpoint of the lower endplate (x4, y4); all coordinates are based on the pixel coordinate system of the X-ray image.

[0056] Calculation of the final plate angle: Calculate the horizontal tilt angle of the upper end plate: calculate the slope by the coordinate difference between the left and right endpoints; use the arctangent function: angle = arctan(Δy / Δx), where Δx is the horizontal distance and Δy is the vertical distance; Convert the radian value to an angle value (multiply by 180 / π); calculate the tilt angle of the endplate in the same way. The endplate angle reflects the degree of tilt of the vertebra relative to the horizontal direction.

[0057] Calculation of the midpoint of the final plate: Midpoint of the upper end plate = (coordinates of the left endpoint + coordinates of the right endpoint) / 2; Midpoint of the lower end plate = (coordinates of the left endpoint + coordinates of the right endpoint) / 2; The midpoint coordinates are used to construct the spinal centerline and analyze the overall morphology of the spine; the angles of the superior and inferior endplates and the midpoint coordinates of each vertebra are stored for later use.

[0058] Intelligent spine type determination algorithm based on vector cross product: Construction of the spinal midline: For each vertebra, the midpoint M of the upper endplate is calculated based on the four detected endplate endpoints. upper and the midpoint M of the lower end plate lower ; Calculate the center point C of the vertebral body C=(M upper +M lower ) / 2, which serves as the representative point of this vertebral body on the central line of the spine; Connect the center points of all vertebrae in anatomical order to form a continuous spinal centerline: L={C1,C2,...,C...} n The centerline reflects the overall curvature and lateral bending direction of the spine, providing basic data for subsequent geometric analysis.

[0059] Vector cross product bending direction analysis algorithm: Initialize the bending direction change counter (direction) changes =0; for three consecutive vertebral center points (C) on the spinal centerline. i-1 C i C i+1 Construct two direction vectors: Vector v1=C i -C i-1 (From the previous vertebral body to the current vertebral body); Vector v2=C i+1 -C i (From the current vertebral body to the next vertebral body); Calculate the cross product of vectors to determine the direction of bending: cross product =v1.x×v2.y-v1.y×v2.x; When cross product A value greater than 0 indicates a bend to the left; when cross... product <0 indicates bending to the right; when cross product When ≈0, it represents a locally approximate straight line.

[0060] Compare the sign changes of two consecutive cross products: If (cross) i >0 and cross i-1 <0) or (cross) i <0 and cross i-1 If >0), then direction changes +=1.

[0061] The total number of directional changes along the entire spinal centerline is counted to quantify the complexity of spinal curvature.

[0062] Intelligent spinal type determination: Determination rules: If direction changes ≥1 indicates S-shaped scoliosis (bidirectional curvature); if direction changes =0 indicates C-type scoliosis (unidirectional curvature); S-type scoliosis presents as a bidirectional "S"-shaped curvature, with the curvature direction reversing at a certain vertebral position (usually near T12-L1), exhibiting obvious primary and compensatory curves; C-type scoliosis presents as a unidirectional "C"-shaped curvature, tilting to one side without a clear point of change in curvature direction. This geometric morphology analysis method is based on the mathematical principle of vector cross product, which is more intuitive, stable, and interpretable than statistical judgment methods based on Cobb angle matrices. It outputs a spine type identifier (C or S), a centerline coordinate sequence, and the location of the direction reversal point for subsequent adaptive Cobb angle measurements.

[0063] Calculation of Cobb angle (Type C): Identify the upper and lower vertebrae with the largest tilt angles in the entire spinal curvature; calculate the difference between the endplate angles of these two vertebrae as the Cobb angle of the principal curve; output the angle value of a single principal thoracic curve (PT).

[0064] S-shaped Cobb angle calculation: The spine is divided into two segments, upward and downward, based on the point of change of curvature. The Cobb angles of the upper thoracic curve (MT) and main thoracic curve (PT) are calculated for the upper curve segment, and the Cobb angle of the thoracolumbar curve (TL) is calculated for the lower curve segment. The apical and terminal vertebrae of each curve segment are identified, and the angle values ​​of each segment are calculated. Three angle values ​​are output: MT, PT, and TL, to comprehensively assess the scoliosis.

[0065] Angle correction: When the calculated angle is greater than 90 degrees, use the supplementary angle: Cobb angle = 180° - calculated angle; Ensure all Cobb angle values ​​are within the range of 0-90°, meeting clinical measurement standards; output the final Cobb angle measurement results with an accuracy to one decimal place.

[0066] Based on the geometric morphological characteristics of the line connecting the midpoints of the vertebrae, the spinal type is determined: the curvature change of the line connecting the midpoints is calculated, the angle change formed by three consecutive points is analyzed, the number of transitions in the bending direction is analyzed, and the bending direction is determined by the cross product of vectors. The S-type has obvious bidirectional curvature, and the transition point is usually near T12-L1; the C-type has unidirectional curvature and no obvious transition point.

[0067] System integration and deployment examples: Web service architecture: Backend FastAPI + PyTorch + OpenCV; Frontend: User interface built with React.js and Next.js; Database: PostgreSQL stores the test results and user data; Deployment: Docker containerized deployment, supporting Kubernetes cluster management.

[0068] Validation results on the SpineWeb test suite: The average accuracy of vertebral body detection at IoU=0.5 was 93.2%; The average accuracy of keypoint detection at IoU=0.5 is 97.0%; Cobb angle SMAPE: 8.97% (accuracy 91.03%); Processing speed: Average 2.8 seconds per image; In the clinical application process, doctors upload patients' X-rays to the system via a web interface. The system automatically detects the vertebral body and locates four key points. The algorithm calculates the Cobb angle and determines the type of scoliosis, generating a diagnostic report that includes visualization results and measurement data. Doctors review and analyze the results and confirm the final diagnosis.

[0069] The end-to-end Keypoint R-CNN-based vertebral endplate localization module and multi-scale feature fusion and ROI processing module output detection boxes and key points for visualization and result management, which are then used by the image renderer for drawing and the quality control module for evaluating confidence. The multi-instance post-processing optimization module outputs sorted vertebral numbers and confidence scores for the angle annotator to number, the quality control module to evaluate confidence, and the report generator to record data. The adaptive Cobb angle calculation module based on geometric morphology analysis outputs centerlines and transformation points for the image renderer to draw auxiliary lines and the quality control module to trigger morphological anomaly warnings. The multi-type Cobb angle measurement module based on endplate tilt angle outputs angle values ​​for the angle annotator to annotate and the report generator to record data.

[0070] This implementation method enables fully automated measurement of the Cobb angle on spinal X-rays, significantly improving diagnostic efficiency and accuracy, and providing reliable technical support for the clinical diagnosis of scoliosis.

[0071] The end-to-end endplate localization architecture innovatively applies the Keypoint R-CNN end-to-end architecture to the detection of four anatomical key points (left and right endpoints of the upper and lower endplates) of the spinal vertebral endplates, realizing the joint optimization of vertebral target detection and endplate key point localization, avoiding the error accumulation of multi-stage pipeline architecture (position recognition, target detection, key point recognition), and the average accuracy is 93.2% when vertebral detection IoU=0.5.

[0072] Based on the geometric morphology analysis algorithm of vector cross product, an intelligent algorithm for judging spinal type is proposed based on the geometric morphology of the line connecting the midpoints of the endplate and the vector cross product. By calculating the sign change of the cross product of three consecutive point vectors, it can automatically distinguish between C-type and S-type scoliosis. Compared with the method based on Cobb angle matrix, it is more intuitive and stable, and the measurement accuracy reaches 91.03%.

[0073] The multi-level feature pyramid fusion technology adopts the ResNet-50-FPN multi-scale feature fusion architecture, ROI alignment algorithm and feature pyramid network to establish multi-level feature representation, which significantly improves the detection capability of small target vertebrae (cervical vertebrae, lumbar vertebrae) and the accuracy of endplate endpoint localization.

[0074] A complete endplate angle calculation system was established based on the arctangent function. The endplate tilt angle is calculated using the arctangent function arctan(Δy / Δx), and the Cobb angle is calculated by the difference in endplate angles between the vertebrae. Compared with the method that only outputs the coordinates of the vertebral body corners, this method is more in line with the definition of the Cobb angle and clinical measurement standards.

[0075] The unified network high-efficiency processing framework realizes a single network end-to-end processing flow from X-ray input to Cobb angle result output, with a single image processing time of less than 5 seconds. Compared with multi-model pipeline architecture, it reduces model calling overhead and system complexity, and supports large-scale clinical screening applications.

[0076] This technical solution addresses the challenges of low accuracy, poor efficiency, high subjectivity, and poor reproducibility in traditional Cobb angle measurements, particularly overcoming the limitations of existing multi-stage pipeline architectures. Compared to schemes employing separate models for posture recognition, target detection, and keypoint recognition, this invention achieves joint optimization of vertebral body detection and endplate keypoint localization through an end-to-end Keypoint R-CNN single network architecture, avoiding the problem of multi-model error accumulation. It innovatively proposes a localization method based on four endplate endpoints and a geometric morphology analysis algorithm based on vector cross products, which is more intuitive and stable than the judgment method based on the Cobb angle matrix. Through the application of end-to-end deep learning technology, it achieves a technological leap from multi-model pipelines to single-network end-to-end optimization, providing a more efficient and accurate innovative technical solution for the intelligent diagnosis of scoliosis and making a significant contribution to the development of the medical image analysis field.

[0077] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A deep learning-based automatic measurement system for the Cobb angle of spinal X-rays, characterized in that, include: Vertebral endplate localization module based on end-to-end Keypoint R-CNN: used for vertebral body localization and feature extraction based on clinical spinal X-rays; Multi-scale feature fusion and ROI processing module: used to optimize vertebral body localization accuracy based on feature extraction; Multi-instance post-processing optimization module: used to post-process the optimized cone positioning to obtain an ordered cone array; An adaptive Cobb angle calculation module based on geometric morphology analysis is used to perform geometric morphology analysis on the ordered cone array. Multi-type Cobb angle measurement module based on endplate tilt angle: used to calculate Cobb angle based on the results of geometric morphology analysis.

2. The automatic measurement system for Cobb angle of spinal X-ray films based on deep learning according to claim 1, characterized in that: The vertebral endplate localization module based on end-to-end Keypoint R-CNN includes a single deep learning network that simultaneously performs vertebral target detection and endplate key point localization through an end-to-end Keypoint R-CNN architecture. The endplate key point localization includes detecting four anatomical key points for each vertebra: the left endpoint of the upper endplate, the right endpoint of the upper endplate, the left endpoint of the lower endplate, and the right endpoint of the lower endplate, using a four-point endplate endpoint localization algorithm. These points directly correspond to the endplate positions required for Cobb angle measurement. A ResNet-50-FPN backbone feature extraction network is used, employing a residual network combined with a feature pyramid to extract multi-scale image features. An adaptive region generation network (RPN) generates vertebral candidate regions, automatically adapting to vertebral size variations in patients of different body types via an anchor frame mechanism.

3. The deep learning-based automatic measurement system for Cobb angle of spinal X-rays according to claim 2, characterized in that: The multi-scale feature fusion and ROI processing module includes: constructing a multi-level feature representation through a feature pyramid network to enhance small target detection capabilities; extracting candidate region features through an ROI alignment algorithm; classifying candidate regions and performing bounding box regression through an ROI detection head; and generating a keypoint heatmap and locating precise coordinates through a keypoint detection head.

4. The automatic measurement system for Cobb angle of spinal X-ray films based on deep learning according to claim 1, characterized in that: The multi-instance post-processing optimization module includes: Confidence Filter: Used to filter detection results with a confidence level below the 0.7 threshold; Non-maximum suppressor: Used to eliminate duplicate detections, with an IoU threshold of 0.5; Anatomical position sorter: used to sort vertebral bodies from top to bottom according to their Y-coordinate; Quantity controller: Used to limit the maximum number of vertebrae to be detected to 17.

5. The automatic measurement system for Cobb angle of spinal X-ray films based on deep learning according to claim 4, characterized in that: The adaptive Cobb angle calculation module based on geometric morphology analysis includes: calculating the midpoint coordinates of the upper and lower endplates of each vertebra using the geometric mean method based on the four detected endplate endpoints; forming a continuous spinal centerline by connecting the midpoints of the endplates of adjacent vertebrae through a spinal centerline builder; providing a spinal type determiner based on vector cross product, using a vector cross product geometric algorithm to analyze the bending direction change of the line connecting the midpoints, and determining C-type or S-type scoliosis by calculating the sign change of the cross product of three consecutive point vectors; and calculating the rate of curvature change of each point on the spinal centerline through a curvature change rate analysis algorithm to locate the bending direction conversion point and the position of maximum scoliosis.

6. The automatic measurement system for Cobb angle of spinal X-rays based on deep learning according to claim 1, characterized in that: The multi-type Cobb angle measurement module based on endplate tilt angle includes a C-type single-bend Cobb angle calculator, an S-type double-bend Cobb angle calculator, an antitangent endplate angle measurement calculator, and a clinical segmentation angle output device. The C-type single-bend Cobb angle calculator is designed for a unidirectional curved spine, identifies the upper and lower vertebrae with the greatest tilt, and calculates the difference in the tilt angle of the endplates of the two vertebrae as the Cobb angle of the main curve. The S-shaped double-curve Cobb angle calculator is designed for bidirectional curved spines. It divides the spine into segments based on the point of change of curvature direction and calculates the Cobb angles of the upper and lower curved segments separately. The arctangent endplate angle measurement calculator uses the arctangent function to calculate the angle between the endplate tangent and the horizontal line, obtains the endplate tilt angle through arctan(Δy / Δx), and then calculates the intervertebral angle difference to obtain the Cobb angle; The clinical segmented angle output device outputs three segmented angle values ​​for the main thoracic curve, upper thoracic curve, and thoracolumbar curve according to clinical diagnostic standards.

7. The deep learning-based automatic measurement system for Cobb angle of spinal X-rays according to claim 6, characterized in that: It also includes an intelligent visualization and results management module, which comprises an image renderer, an angle annotator, a report generator, and a quality control module. The image renderer is used to draw detection boxes, key points, and measurement auxiliary lines on the original X-ray film; The angle annotator is used to annotate the angle values ​​and vertebral numbers of each curved segment on the image; The report generator is used to automatically generate standardized diagnostic reports that include measurement data and visualization results; The quality control module is used to assess the reliability of the test results and output confidence level and quality warning.

8. The automatic measurement system for Cobb angle of spinal X-ray films based on deep learning according to claim 6, characterized in that: The Cobb angle calculation includes: Calculation of the midpoint of the final plate: Midpoint of upper endplate = (left endpoint of upper endplate + right endpoint of upper endplate) / 2; Midpoint of lower endplate = (left endpoint of lower endplate + right endpoint of lower endplate) / 2; Calculation of the endplate tangent angle: θ = arctan((right endpoint.y - left endpoint.y) / (right endpoint.x - left endpoint.x)); Cobb angle calculation: Cobb angle =|θ 上椎体 -θ 下椎体 |, when the angle is >90 degrees, Cobb angle =180°-Cobb angle .

9. The deep learning-based automatic measurement system for the Cobb angle of spinal X-rays according to claim 5, characterized in that: The vector cross product geometric algorithm includes: Initialize the bending direction change counter (direction) changes =0; For three consecutive vertebral body center points C on the central line of the spine i-1 C i and C i+1 Construct direction vectors v1 and v2; Direction vector v1=C i -C i-1 The direction vector v1 represents the direction vector from the previous vertebra to the current vertebra; Direction vector v2=C i+1 -C i The direction vector v2 represents the direction vector from the current vertebra to the next vertebra; Calculate the cross product of vectors to determine the direction of bending: cross product =v1.x×v2.y-v1.y×v2.x; When cross product A value greater than 0 indicates a bend to the left; when cross... product <0 indicates bending to the right; when cross product When ≈0, it represents a locally approximate straight line; compare the sign changes of two adjacent cross products: If it exists: cross i > 0 and cross i-1 < 0; Or it may exist: cross i <0 and cross i-1 > 0; Then direction changes +=1; The total number of directional changes along the entire spinal centerline is counted to quantify the complexity of spinal curvature.

10. The automatic measurement system for Cobb angle of spinal X-ray films based on deep learning according to claim 1, characterized in that: The intelligent determination of spine type includes: if direction changes ≥1 indicates S-shaped scoliosis; if direction changes =0, indicating C-type scoliosis.

Images