Scoliosis comprehensive angle detection method, device, medium, program product and terminal

Abstract

The application provides a scoliosis comprehensive angle detection method, device, medium, program product and terminal. First, a spine orthotopic X-ray image is acquired, then a target detection algorithm is used to automatically identify and locate the spine, pelvis and chest cavity region, and corresponding images are generated. Next, a key point detection operation is performed to generate a plurality of vertebral angle point coordinates, thoracic key points and left and right iliac crest top points. Finally, the scoliosis comprehensive angle is measured by synthesizing the data. The problem that the prior art mainly focuses on the local inclination of the spine without fully considering the overall inclination relationship between the spine and the pelvis, thereby affecting the comprehensive assessment of the scoliosis condition, is effectively solved. The automation processing is realized, the measurement accuracy and consistency are improved, the human error is significantly reduced, the medical work efficiency is improved, and the burden of doctors is reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent medical care, and in particular to a method, device, medium, program product and terminal for detecting the comprehensive angle of scoliosis. Background Technology

[0002] Scoliosis is a common skeletal deformity, and its assessment and treatment often rely on accurate measurement of the spinal tilt angle. Currently, X-ray imaging is the primary clinical method for assessment, with doctors observing the curvature of the spine by taking X-rays at different angles. However, this method often focuses on localized spinal tilt and fails to adequately consider the overall tilt relationship between the spine and pelvis.

[0003] Existing measurement techniques have limitations in assessing the three-dimensional structure of the spine, failing to comprehensively reflect the overall dynamic changes from the spine to the pelvis. Furthermore, the measurement process does not fully utilize data from the thoracic cavity and pelvis, resulting in insufficient assessment of the overall function and structure of the spine. These limitations may affect the accurate diagnosis of scoliosis, thereby impacting treatment planning. Therefore, exploring more comprehensive and automated assessment techniques is of great significance for improving the diagnosis and treatment outcomes of scoliosis. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, medium, program product and terminal for detecting the comprehensive angle of scoliosis, in order to solve the problem that the prior art mainly focuses on the local tilt of the spine and fails to effectively consider the overall tilt relationship between the spine and the pelvis, thereby affecting the comprehensive assessment of scoliosis.

[0005] To achieve the above and other related objectives, a first aspect of this application provides a method for detecting the comprehensive angle of scoliosis. The method includes: acquiring an anteroposterior X-ray image of the spine; identifying and locating the spine, pelvis, and thoracic cavity regions in the anteroposterior X-ray image using a target detection algorithm to generate corresponding spine, pelvis, and thoracic cavity images; performing a first keypoint detection operation on the spine image to generate multiple vertebral angle coordinates; performing a second keypoint detection operation on the thoracic cavity image to generate multiple thoracic keypoints; performing a third keypoint detection operation on the pelvis image to generate the left and right iliac crest apexes; and measuring the comprehensive angle of scoliosis in the anteroposterior X-ray image based on the multiple vertebral angle coordinates, the multiple thoracic keypoints, and the left and right iliac crest apexes.

[0006] In some embodiments of the first aspect of this application, the process of performing a first keypoint detection operation on the spinal image to generate multiple vertebral corner coordinates includes: preprocessing the spinal image; inputting the preprocessed spinal image into a convolutional neural network to extract spinal feature data; generating a heatmap of the center point of the spinal vertebrae based on the spinal feature data; calculating the initial corner coordinates of the vertebrae and their corresponding confidence values ​​based on the heatmap of the center point of the spinal vertebrae; and filtering the generated initial corner coordinates of the vertebrae using the confidence values ​​to obtain multiple vertebral corner coordinates.

[0007] In some embodiments of the first aspect of this application, the process of generating a heatmap of the center point of the spinal vertebrae based on the spinal feature data includes: preprocessing the spinal feature data; performing network construction and output layer setting operations on the preprocessed spinal feature data to generate a heatmap model; training and optimizing the heatmap model based on a preset loss function to generate a trained heatmap model; and inputting the spinal features into the heatmap model to generate a heatmap of each center point of the spinal vertebrae.

[0008] In some embodiments of the first aspect of this application, the process of performing a second keypoint detection operation on the thoracic cavity image to generate multiple thoracic keypoints includes: segmenting the thoracic cavity image using an image segmentation algorithm to generate a thoracic segmentation result; calculating the center point coordinates of the corresponding vertebrae based on the coordinates of multiple vertebral corner points; generating one or more horizontal straight lines in the thoracic segmentation result according to the center point coordinates of each vertebra; and extracting the distal intersection points of the horizontal straight lines and the thoracic segmentation result to generate multiple thoracic keypoints.

[0009] In some embodiments of the first aspect of this application, the process of performing a third keypoint detection operation on the pelvic image to generate the left iliac crest apex and the right iliac crest apex includes: inputting the pelvic image into a neural network to generate multiple initial pelvic keypoints and their corresponding confidence values; and filtering the initial pelvic keypoints based on the confidence values ​​to generate optimized pelvic keypoints.

[0010] In some embodiments of the first aspect of this application, the process of performing a third keypoint detection operation on the pelvic image to generate the left iliac crest apex and the right iliac crest apex includes: determining whether the left iliac crest apex and / or the right iliac crest apex are missing in the optimized pelvic keypoints; if they are missing, performing an image segmentation operation on the pelvic image to generate a pelvic segmentation result; and traversing the optimized keypoints that fall into the pelvic segmentation result to generate the coordinates of the missing left iliac crest apex and / or right iliac crest apex.

[0011] To achieve the above and other related objectives, a second aspect of this application provides a scoliosis comprehensive angle detection device, comprising: an image acquisition module for acquiring an anteroposterior X-ray image of the spine; a region recognition module for identifying and locating the spine, pelvis, and thoracic cavity regions in the anteroposterior X-ray image of the spine using a target detection algorithm, to generate corresponding spine images, pelvis images, and thoracic cavity images; a key point detection module for performing a first key point detection operation on the spine image to generate multiple vertebral angle coordinates; performing a second key point detection operation on the thoracic cavity image to generate multiple thoracic key points; performing a third key point detection operation on the pelvis image to generate the left iliac crest apex and the right iliac crest apex; and a scoliosis assessment module for measuring the comprehensive angle of scoliosis in the anteroposterior X-ray image of the spine based on the multiple vertebral angle coordinates, the multiple thoracic key points, the left iliac crest apex, and the right iliac crest apex.

[0012] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the scoliosis comprehensive angle detection method.

[0013] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code, which, when executed on a computer, causes the computer to implement the scoliosis comprehensive angle detection method.

[0014] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the scoliosis comprehensive angle detection method.

[0015] As described above, the scoliosis comprehensive angle detection method, device, medium, program product, and terminal of this application have the following beneficial effects: By processing and analyzing spinal anteroposterior X-ray images through target detection algorithms, the spine, pelvis, and thoracic cavity regions can be efficiently and accurately identified, reducing the risk of misdiagnosis. Multiple coordinates are obtained through key point detection to support the assessment of spinal and thoracic structures. Simultaneously, the comprehensive scoliosis angle can be accurately measured, providing quantitative evidence for personalized treatment. Automated processing improves medical work efficiency, reduces manual intervention, and provides important data support for clinical research, ultimately improving the efficiency and accuracy of spinal X-ray image analysis, and significantly promoting clinical diagnosis and treatment. Attached Figure Description

[0016] Figure 1 The flowchart of an embodiment of the scoliosis comprehensive angle detection method of this application is shown.

[0017] Figure 2 The image shown is an anteroposterior X-ray image of the spine in one embodiment of the scoliosis comprehensive angle detection method of this application.

[0018] Figure 3 This paper presents a schematic diagram of the process for region identification in an anteroposterior X-ray image of the spine in one embodiment of the scoliosis comprehensive angle detection method of this application.

[0019] Figure 4 This illustration shows a schematic diagram of region identification in an anteroposterior X-ray image of the spine in one embodiment of the scoliosis comprehensive angle detection method of this application.

[0020] Figure 5 The results of key point identification in an anteroposterior X-ray image of the spine are shown in one embodiment of the scoliosis comprehensive angle detection method of this application.

[0021] Figure 6 The image shows the thoracic cavity segmentation results of one embodiment of the scoliosis comprehensive angle detection method of this application.

[0022] Figure 7 The image shows the pelvic segmentation results of the pelvic region in one embodiment of the scoliosis comprehensive angle detection method of this application.

[0023] Figure 8 The diagram shows a schematic of the upper endplate line and the lower endplate line in one embodiment of the scoliosis comprehensive angle detection method of this application.

[0024] Figure 9 This paper shows a schematic diagram of the vertebral angle of a scoliosis spine in one embodiment of the comprehensive angle detection method of this application.

[0025] Figure 10 A schematic diagram of an embodiment of the scoliosis comprehensive angle detection device of this application is shown.

[0026] Figure 11 A schematic diagram of the structure of an embodiment of the scoliosis comprehensive angle detection terminal of this application is shown. Detailed Implementation

[0027] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0028] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0029] <1> Axillary spinal X-ray: An anteroposterior spinal X-ray is a vertical view of the spine obtained through X-ray photography. It is commonly used to observe the overall structure and alignment of the spine and to help doctors assess the health of the spine.

[0030] <2> Scoliosis angle: The scoliosis angle refers to the angle by which the spine deviates from the normal midline in a side view. It is usually measured by the Cobb angle and is used to assess the degree of scoliosis.

[0031] <3> Vertebral angle points: Vertebral angle points are specific corner points of the vertebrae in a spinal X-ray image, used to measure the geometric features and alignment of the spine.

[0032] <4> Heatmap model: A heatmap model is a visualization technique that uses color changes to represent the intensity or density of data. It is often used in medical imaging to represent clusters of specific areas such as pain or abnormalities.

[0033] <5> Iliac crest apex: The iliac crest apex is an anatomical landmark in the hip region of the human body, usually referring to the uppermost protruding part of the ilium. This point can be used to measure the relationship between the spine and the pelvis.

[0034] <6> YOLOv5 model: The YOLOv5 model is a real-time object detection algorithm based on deep learning. It can quickly and accurately detect objects in images or videos and is often used for automatic analysis of medical images.

[0035] <7> Convolutional Block Attention Module: The convolutional block attention module is a module that introduces an attention mechanism into a convolutional neural network to emphasize important features in an image and improve the model's performance.

[0036] <8> Skeleton detection model: A skeleton detection model is a model used to detect and analyze human movement or posture by estimating the layout of bones by identifying key points (such as joints) in the body.

[0037] <9> Cobb angle: The Cobb angle is a measurement method used to quantify scoliosis. It is the angle formed by two straight lines connecting the apex and the base of the most curved vertebra.

[0038] <10> Superior endplate line: The superior endplate line is the boundary line formed on the upper surface of the vertebral body and serves as a structural reference for each vertebra of the spine.

[0039] <11> Inferior endplate line: The inferior endplate line is the boundary line formed on the lower surface of the vertebral body and serves as a structural reference for the lower part of each vertebral body in the spine.

[0040] <12> Upper thoracic curve: The upper thoracic curve refers to the normal curvature of the spine in the upper part of the chest, which is usually a physiological curve that protrudes backward.

[0041] <13> Thoracic curve: The thoracic curve refers to the natural curvature of the spine in the chest area, which is usually curved backward.

[0042] <14> Lumbar curvature: Lumbar curvature refers to the physiological curvature of the spine in the lumbar region, which is usually a forward curve to help support body weight and maintain stability.

[0043] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 A flowchart illustrating a method for detecting the comprehensive angle of scoliosis according to an embodiment of the present invention is shown. The method for detecting the comprehensive angle of scoliosis in this embodiment mainly includes the following steps:

[0044] Step S11: Obtain an anteroposterior X-ray image of the spine.

[0045] Figure 2 An anteroposterior (AP) X-ray image of the spine according to an embodiment of the present invention is shown. An X-ray image is a medical image generated using X-rays. X-rays are high-energy electromagnetic waves that can penetrate human tissue. Different types of tissues (such as bone, muscle, and fat) absorb X-rays to varying degrees, thus appearing with different grayscale levels and structures on the X-ray image. Specifically, an AP X-ray image of the spine refers to an image of the spine taken from the front (anteroposterior) of the patient to show the overall structure of the spine, including the vertebral bodies, intervertebral discs, and their relative positions. AP X-ray images of the spine are used to diagnose spinal diseases, injuries, or deformities.

[0046] Step S12: Using a target detection algorithm, the spine, pelvis, and thoracic cavity regions are identified and located in the anteroposterior X-ray image of the spine to generate corresponding spine images, pelvis images, and thoracic cavity images.

[0047] In one embodiment of the present invention, such as Figure 3 As shown, the spine, pelvis, and thoracic cavity regions are identified and located in the anteroposterior X-ray image of the spine using a target detection algorithm, which includes the YOLOv5 model. The identification and localization process is as follows: Figure 3 As shown, the process includes: inputting the anteroposterior X-ray image of the spine into a YOLOv5 model to extract features from the input anteroposterior X-ray image using a convolutional neural network (CNN). After feature extraction, multiple bounding boxes are generated, and a corresponding confidence score and category are predicted for each box. To optimize the model's detection performance, a loss function is used during the training phase to evaluate the difference between the predicted results and the actual labels.

[0048] Furthermore, the loss function includes classification loss and regression loss. The classification loss evaluates the model's prediction performance for the target class within each predicted bounding box. It is evaluated by calculating the difference between the probability of the predicted class and the true class within each bounding box, using the cross-entropy in Equation 1. Where L... CE The output value of the cross-entropy loss measures the difference between the probability distribution predicted by the model and the true distribution; N represents the total number of samples, and the batch size in batch training; C represents the total number of classes in the classification task. This represents the true label of sample i in category c. It is usually represented using one-hot encoding, where only the position that matches the true category is 1, and the rest are 0. This represents the probability value that the model predicts for sample i to belong to category c.

[0049]

[0050] Furthermore, the regression loss measures the difference between the predicted bounding box location and the true bounding box location. The regression loss is calculated by the error between the coordinates of the predicted bounding box and the true bounding box coordinates, and is typically defined using methods such as mean squared error (MSE) or smoothed L1 loss. The regression loss function is shown in Equation 2. i y represents the model's prediction for the i-th sample; i |x represents the target value of the i-th sample; i -y i | represents the absolute error between the predicted value and the true value of the i-th sample. When the absolute error is less than 1, using squared loss helps to more accurately penalize small errors, thus improving the model's sensitivity to small prediction errors. When the absolute error is greater than or equal to 1, linear loss is used to reduce the penalty for large errors, thereby preventing excessive influence from large errors and improving the model's robustness, avoiding being dominated by outliers.

[0051]

[0052] Subsequently, during training, the YOLOv5 model continuously adjusts its parameters using the backpropagation algorithm to minimize the sum of classification and regression losses, thereby improving the model's accuracy and robustness. Next, the model applies non-maximum suppression (NMS) to process bounding boxes with significant overlap, reducing redundant detection results and ultimately determining the accurate location and category of each region. The recognition results are output as bounding boxes, including the target's location information and corresponding category label. Figure 4 The results of region segmentation in one embodiment of the present invention are shown. Figure 5 The results of key point identification in various regions are shown in one embodiment of the present invention.

[0053] In one embodiment of the invention, the spinal image consists of vertebrae from different locations. These vertebrae include 7 cervical vertebrae (C1 to C7), 12 thoracic vertebrae (T1 to T12), 5 lumbar vertebrae (L1 to L5), 5 sacral vertebrae (S1 to S5), and 4 fused coccygeal vertebrae. The relative positions and alignment of the identified vertebrae can effectively detect scoliosis (abnormal curvature of the spine). The X-ray also shows the intervertebral discs and intervertebral foramina, which helps to assess other potential problems affecting the normal alignment of the vertebrae.

[0054] In one embodiment of the invention, an image of the thoracic region shows the 12 thoracic vertebrae and the connecting rib structures. The morphology and position of the thoracic vertebrae have a significant impact on scoliosis. Scoliosis can lead to changes in the morphology of the thoracic cavity, which in turn affects the alignment and development of the ribs.

[0055] In one embodiment of the invention, X-ray images of the pelvic region show the relationship between the sacrum and hip bones. The connection between the spine and pelvis is particularly important in scoliosis, which affects the tilt and position of the pelvis. Observing the relative positions of the sacrum and hip bones can directly provide information on the impact of scoliosis on the lower limbs and overall body posture.

[0056] Step S13: Perform a first keypoint detection operation on the spinal image to generate multiple vertebral angle coordinates; perform a second keypoint detection operation on the thoracic image to generate multiple thoracic keypoints; perform a third keypoint detection operation on the pelvic image to generate the left iliac crest apex and the right iliac crest apex.

[0057] In one embodiment of the present invention, the process of performing a first keypoint detection operation on the spinal image to generate multiple vertebral corner coordinates includes: preprocessing the spinal image; inputting the preprocessed spinal image into a convolutional neural network to extract spinal feature data; generating a heatmap of the center point of the spinal vertebrae based on the spinal feature data; calculating the initial corner coordinates of the vertebrae and their corresponding confidence values ​​based on the heatmap of the center point of the spinal vertebrae; and filtering the generated initial corner coordinates of the vertebrae using the confidence values ​​to obtain multiple vertebral corner coordinates.

[0058] In this embodiment, as Figure 6 As shown, the preprocessing of the spine image includes image denoising, contrast enhancement, edge detection, image scaling, and normalization. First, Gaussian filtering or median filtering is used to reduce image noise and improve the accuracy of keypoint detection. Next, histogram equalization is used to enhance image contrast, making spine features more clearly visible. Then, the Canny edge detection algorithm is applied to identify the spine contour, providing a more precise input. Finally, the image is scaled to a uniform size to meet the requirements of the convolutional neural network, and finally, normalization is applied to center each pixel value near 0.

[0059] In this embodiment, the preprocessed spine image is first input into a Convolutional Neural Network (CNN). Within the network, the image first passes through convolutional layers, where multiple filters are used to perform convolution operations, generating multiple sets of feature maps. Subsequently, the ReLU (Rectified Linear Unit) activation function is applied to these feature maps, introducing non-linearity to ensure the network can recognize complex patterns. Next, the feature maps pass through pooling layers, where they are downsampled using max pooling. This process reduces the dimensionality of the feature maps by selecting the maximum value in a local region, thus alleviating the computational burden.

[0060] Furthermore, the processed feature maps are fed into the Convolutional Block Attention Module (CBAM). The first stage is the channel attention module, which calculates the importance weights for each channel. This is obtained through global average pooling and max pooling; global features are used to adjust the feature map, strengthening the responses of important channels. The second stage is the spatial attention module, which generates an attention map by analyzing the spatial dimensions of the feature map, weighting key locations to further improve the quality of feature representation. Finally, the processed feature maps are input to the output layer, where fully connected layers transform these features into classification results or regression values.

[0061] In this embodiment, Focal Loss and L1 loss are used to optimize the training process of the spine keypoint detection model. The purpose of introducing Focal Loss is to address the class imbalance problem and enhance attention to hard-to-classify samples, as shown in Equation 3. Where p t This represents the probability of predicting a positive class, and γ is a modulating factor, typically set to 2. By introducing this modulating factor, the focus loss can reduce the loss contribution of easily classified samples, allowing the model to focus more on difficult-to-classify samples, thereby improving overall classification performance.

[0062] FL(p t )=-(1-p t )γlog(p t ) (Formula 3)

[0063] Simultaneously, the L1 loss function was also employed, as shown in Equation 4, where x represents the input sample and y represents the true label. f(x) iThe expression represents the model's predicted output. n represents the total number of samples. This embodiment combines two loss functions to effectively improve the model's performance in classification and regression tasks. The focus loss function enhances the ability to identify difficult-to-classify samples, while the L1 loss accurately measures the difference between the predicted results and the true values, thus improving the overall generalization ability of the model.

[0064]

[0065] In one embodiment of the present invention, the process of generating a heatmap of the center point of the spinal pyramid based on the spinal feature data includes: preprocessing the spinal feature data; performing network construction and output layer setting operations on the preprocessed spinal feature data to generate a heatmap model; training and optimizing the heatmap model based on a preset loss function to generate a trained heatmap model; and inputting the spinal features into the heatmap model to generate a heatmap of each center point of the spinal pyramid.

[0066] In this embodiment, the preprocessing of the spinal feature data includes: data cleaning, deleting duplicate data and outliers to ensure data integrity. Subsequently, based on the dimensions of the spinal feature data, an input layer of a convolutional neural network is constructed, and corresponding hidden and output layers are set. The corresponding output layer is then set based on preset data of the spinal vertebral body center points.

[0067] Furthermore, an appropriate loss function, such as mean squared error (MSE) or cross-entropy loss, is selected to evaluate the difference between the model's predictions and the true values. The preprocessed data is divided into training and validation sets, for example, in an 80 / 20 or 70 / 30 ratio. The heatmap model is trained using the training set, and backpropagation is used during optimization to adjust the model parameters (weights and biases). Based on the performance on the validation set, hyperparameters such as the learning rate, batch size, and number of training epochs are adjusted to enhance the model's generalization ability.

[0068] Finally, the new spinal feature data is input into the trained heatmap model for inference operations to obtain the output. Based on the model's output, a heatmap of the center point of each spinal vertebra is generated. The heatmap can be represented as a two-dimensional image, where different colors or intensities represent the probability or confidence level of the spinal vertebra center point.

[0069] In this embodiment, the process of calculating the initial corner coordinates of the vertebra and its corresponding confidence value based on the heat map of the center point of the spinal vertebra includes: identifying high-confidence regions and marking potential vertebral corners in the heat map of the center point of the spinal vertebra based on a preset threshold; locating the initial corner coordinates in the high-confidence regions using maximum suppression technology; extracting the confidence value corresponding to each corner; filtering valid corners based on the confidence value, and outputting the vertebral corner coordinates and their corresponding confidence values.

[0070] In one embodiment of the present invention, after generating multiple vertebral angle coordinates, the following trunk tilt calculation operation is performed. The coordinates of the apex vertebra in the thoracic curve are extracted from the multiple vertebral angle coordinates, and the center point is calculated based on the four corner points of the apex vertebra. The line connecting the center of the apex vertebra and key points of the thoracic cage is obtained, and the midpoint of the line connecting the key points of the thoracic cage is calculated. A plumb line is drawn downwards from this midpoint, and simultaneously, the midpoint of the line connecting the key points of the sacral endplate on the pelvis is obtained and a plumb line is drawn. The distance between these two plumb lines is measured; this distance is the trunk tilt, used to represent the degree of trunk tilt relative to the direction of gravity.

[0071] In one embodiment of the present invention, the process of performing a second keypoint detection operation on the thoracic cavity image to generate multiple thoracic keypoints includes: segmenting the thoracic cavity image using an image segmentation algorithm to generate a thoracic segmentation result; calculating the center point coordinates of the corresponding vertebrae based on the coordinates of multiple vertebral corner points; generating one or more horizontal straight lines in the thoracic segmentation result according to the center point coordinates of each vertebra; and extracting the distal intersection points of the horizontal straight lines and the thoracic segmentation result to generate multiple thoracic keypoints.

[0072] Figure 6 This illustration shows a schematic diagram of the second keypoint detection operation performed on the thoracic cavity image in this embodiment. In addition to segmenting the thoracic cavity, the position and posture of the vertebral body within the spine are located by calculating the straight line between the center point of the T6 vertebra and the center point of the vertebra itself. Furthermore, this calculation result can be used to assess the curvature of the spine and determine its overall health status, thus providing important data support for clinical diagnosis and treatment.

[0073] In one embodiment of the present invention, the process of performing a third keypoint detection operation on the pelvic image to generate the left iliac crest apex and the right iliac crest apex includes: inputting the pelvic image into a neural network to generate multiple initial pelvic keypoints and their corresponding confidence values; and filtering the initial pelvic keypoints based on the confidence values ​​to generate optimized pelvic keypoints.

[0074] In this embodiment, the process of inputting a pelvic image into a neural network to generate multiple initial pelvic keypoints and their corresponding confidence values ​​includes: using a convolutional neural network (CNN) or a skeleton detection model (such as OpenPose or PoseNet) to detect pelvic keypoints, adjusting the weights using a loss function (such as mean squared error) and an optimization algorithm (such as Adam), and monitoring with a validation set to prevent overfitting. New images are input into the network to obtain keypoint coordinates and confidence values, and non-maximum suppression (NMS) is applied to remove low-confidence keypoints to improve accuracy. The model performance is evaluated using evaluation metrics (such as mean squared error (MSE) or mean absolute error (MAE), and the model is optimized to improve detection performance.

[0075] In one embodiment of the present invention, the process of performing a third keypoint detection operation on the pelvic image to generate the left iliac crest vertex and the right iliac crest vertex includes: determining whether the left iliac crest vertex and / or the right iliac crest vertex in the optimized pelvic keypoints are missing; if they are missing, performing an image segmentation operation on the pelvic image to generate a pelvic segmentation result; and traversing the optimized keypoints that fall into the pelvic segmentation result to generate the coordinates of the missing left iliac crest vertex and / or the right iliac crest vertex.

[0076] In this embodiment, as Figure 7 As shown, the process of performing image segmentation on the pelvic image includes: taking a preprocessed 572x572 pixel pelvic image slice as input, then passing it through a convolutional layer (conv 3x3, ReLU) to generate a 570x570 feature map, followed by a 2x2 max pooling layer to halve the feature map size to 284x284 and increase the depth to 128. Subsequently, multiple convolutional and pooling layers are used to gradually extract features, finally obtaining a feature map with a depth of 512. After entering the upsampling stage, a 2x2 upconvolutional operation is used to gradually restore the image space, while copying and cropping are performed to fuse high-level features. Finally, the generated output segmentation map has a size of 392x392 pixels and a depth of 2, representing different segmentation regions.

[0077] Furthermore, to evaluate the segmentation performance of the model, this embodiment uses Mean Squared Error (MSE) as the loss function, as shown in Equation 5. Here, MSE represents the mean squared error, which is the average of the squared differences between the predicted and true values. n represents the total number of samples, i.e., the number of samples used to calculate the loss. f(x) represents the model's predicted output, i.e., the segmentation result obtained after model processing. y represents the ground truth label, i.e., the actual segmentation value of the corresponding sample. The MSE loss function effectively reflects the accuracy of the model's predictions by calculating the squared difference between each predicted and true value. A smaller MSE value means that the model's predictions are closer to the true values, thus indicating an improvement in segmentation performance. By continuously optimizing MSE, this algorithm can gradually improve the accuracy of left and right pelvic image segmentation.

[0078]

[0079] In this embodiment, the third keypoint detection operation on the pelvic image aims to generate the left and right iliac crest apexes to improve the completeness and accuracy of keypoint detection. First, the optimized pelvic keypoints are analyzed to confirm the presence of missing left and / or right iliac crest apexes, with the judgment primarily based on keypoint information output from the neural network. If missing keypoints are identified, image segmentation is performed on the pelvic image to generate detailed pelvic segmentation results. This segmentation accurately presents the pelvic structure and supports subsequent analysis. Next, the optimized keypoints in the segmentation results are traversed to ensure the generation of the missing iliac crest apex coordinates, achieving comprehensive keypoint recovery.

[0080] Simultaneously, through spinal key point detection, 24 sets of key points are output, each corresponding to the four corners of the vertebral body and its confidence level. These key points originate from the heatmap regression results of the neural network, ensuring that each set of key points has its corresponding confidence level for effective screening; key points with confidence levels below a set threshold are not included in the final results. This ensures the accuracy of the key point information. Through comprehensive detection of pelvic and spinal key points, the overall completeness and accuracy are improved, providing important support for subsequent scoliosis detection. This process is rigorously structured and logically clear, ensuring the efficiency and practicality of key point detection.

[0081] In one embodiment of the present invention, the following operation is also performed: calculating the pelvic tilt based on the apex of the left and right iliac crests. Specifically, the coordinates of the apex of the left and right iliac crests are obtained, then the line connecting the highest points of both iliac crests is calculated, and a horizontal line is defined. By measuring the angle between the connecting line and the horizontal line, the pelvic tilt is obtained, reflecting the degree of tilt of the pelvis relative to the ground. Pelvic tilt is an important indicator for assessing scoliosis. Scoliosis causes the pelvis to tilt in a certain direction, changing the body's center of gravity and affecting gait. The calculation of pelvic tilt provides a quantitative basis for the comprehensive angle of scoliosis, supporting subsequent diagnosis and intervention.

[0082] Step S14: Based on the coordinates of the multiple vertebral angle points, multiple key points of the thoracic cavity, the left iliac crest apex and the right iliac crest apex, measure the comprehensive angle of scoliosis in the anteroposterior X-ray image of the spine.

[0083] In one embodiment of the present invention, the process of measuring the comprehensive angle of scoliosis in the anteroposterior X-ray image of the spine includes: calculating the Cobb angle of the spine. The Cobb angle is the angle formed between the superior and inferior endplates of the upper and lower vertebral bodies in the X-ray image. This angle is formed by the highest point of the upper vertebral body and the lowest point of the lower vertebral body. Measuring the Cobb angle provides a quantitative basis for the diagnosis and treatment of scoliosis; the larger the angle, the more severe the scoliosis.

[0084] Furthermore, such as Figure 8 As shown, the calculation process for the Cobb angle includes: identifying the upper and lower vertebrae of the curved region using the coordinates of multiple vertebral angle points; these two vertebrae are the two vertebrae with the highest inclination in the curved region. Then, a straight line is drawn horizontally on the superior endplate of the upper vertebra and the inferior endplate of the lower vertebra. The superior endplate line should be parallel to the upper surface of the upper vertebra, while the inferior endplate line should be parallel to the lower surface of the lower vertebra. The Cobb angle is calculated using the angle between the superior and inferior endplate lines.

[0085] In this embodiment, as Figure 9 As shown, the Cobb angle can be used to assess the degree of scoliosis in multiple regions, including the upper thoracic curve, the main thoracic curve, and the lumbar curve. Specifically, for the upper thoracic curve, the angle between the superior endplate of the upper vertebra and the inferior endplate of the lower vertebra is measured; for the main thoracic curve, the angle between the superior endplate of the upper thoracic vertebra and the inferior endplate of the lower thoracic vertebra is measured; and for the lumbar curve, the angle between the superior endplate of the upper lumbar vertebra and the inferior endplate of the lower lumbar vertebra is measured.

[0086] It is important to note that measuring the Cobb angles of the upper thoracic curve, main thoracic curve, and lumbar curve separately has significant clinical value, helping to accurately assess the characteristics of scoliosis. The upper thoracic curve affects shoulder balance, while the main thoracic curve may affect chest morphology and lung function, and the lumbar curve has a significant impact on the biomechanical load on the lower limbs. By measuring the Cobb angles of each region individually, the degree of scoliosis in each segment can be more accurately identified and analyzed, thus providing a scientific basis for developing personalized treatment plans. Implementing this method helps to develop more targeted management and rehabilitation plans for patients.

[0087] In one embodiment of the present invention, the comprehensive scoliosis angle further includes pelvic tilt. The measurement process of the pelvic tilt includes: measuring the Cobb angles of different regions of the spine (such as the upper thoracic segment, main thoracic segment, and lumbar segment) to obtain multiple scoliosis angles; measuring the vertical height difference between the two iliac crest apexes in the image to generate the pelvic tilt; and weighted averaging the Cobb angles and the pelvic tilt angle to generate the comprehensive scoliosis angle. For example, the Cobb angle accounts for 60%, and the pelvic tilt angle accounts for 40%.

[0088] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first keypoint detection operation and the second keypoint detection operation are only used to distinguish different keypoint detection operations and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that terms such as "first" and "second" do not necessarily imply that they are different.

[0089] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0090] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0091] Figure 10 This is a schematic block diagram of the scoliosis comprehensive angle detection device 100 provided in the embodiments of this application. Figure 10 As shown, the device includes an image acquisition module 101, a region recognition module 102, a key point detection module 103, and a scoliosis assessment module 104.

[0092] Image acquisition module 101: Used to acquire anteroposterior X-ray images of the spine.

[0093] Region recognition module 102: Used to identify and locate the spine, pelvis and thoracic cavity regions in the anteroposterior X-ray image of the spine through a target detection algorithm, so as to generate corresponding spine image, pelvis image and thoracic cavity image.

[0094] Key point detection module 103: performs a first key point detection operation on the spinal image to generate multiple vertebral angle coordinates; performs a second key point detection operation on the thoracic image to generate multiple thoracic key points; and performs a third key point detection operation on the pelvic image to generate the left iliac crest apex and the right iliac crest apex.

[0095] Scoliosis assessment module 104: used to measure the comprehensive angle of scoliosis in the anteroposterior X-ray image of the spine based on the coordinates of the multiple vertebral angle points, multiple key points of the thoracic cage, the left iliac crest apex and the right iliac crest apex.

[0096] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0097] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0098] Figure 11 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 11 As shown, the electronic terminal includes at least one processor 111, a memory 112, at least one network interface 113, and a user interface 115. The various components in the device are coupled together via a bus system 114. It is understood that the bus system 114 is used to implement communication between these components. In addition to a data bus, the bus system 114 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 11 The general will label all buses as bus systems.

[0099] The user interface 115 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0100] It is understood that memory 112 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0101] In this embodiment of the invention, the memory 112 is used to store various types of data to support the operation of the electronic terminal 110. Examples of this data include: any executable program for operation on the electronic terminal 110, such as the operating system 1121 and application programs 1122; the operating system 1121 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 1122 may contain various applications, such as a media player, browser, etc., for implementing various application services. The scoliosis comprehensive angle detection method provided in this embodiment of the invention can be included in the application program 1122.

[0102] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 111. Processor 111 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 111 or by instructions in software form. The processor 111 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 111 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 111 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0103] In an exemplary embodiment, the electronic terminal 110 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0104] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute the scoliosis comprehensive angle detection method of any of the embodiments shown above.

[0105] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to execute the scoliosis comprehensive angle detection method of any of the embodiments shown above.

[0106] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0107] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0108] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0112] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state drives, SSDs, etc.).

[0113] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0115] In summary, this application provides a method, device, medium, program product, and terminal for detecting the comprehensive angle of scoliosis. First, an anteroposterior X-ray image of the spine is acquired. Then, a target detection algorithm automatically identifies and locates the spine, pelvis, and thoracic cavity regions, generating corresponding images. Next, a key point detection operation is performed, generating coordinates of multiple vertebral angle points, key points of the thoracic cavity, and the apexes of the left and right iliac crests. Finally, the comprehensive angle of scoliosis is measured using the integrated data. This effectively solves the problem that existing technologies mainly focus on local spinal tilt without fully considering the overall tilt relationship between the spine and pelvis, thus affecting the comprehensive assessment of scoliosis. It achieves automated processing, improves the accuracy and consistency of measurements, significantly reduces human error, and simultaneously improves medical work efficiency and reduces the burden on doctors. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0116] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for detecting the comprehensive angle of scoliosis, characterized in that, The method includes: Obtain an anteroposterior X-ray image of the spine; The spine, pelvis, and thoracic cavity regions are identified and located in the anteroposterior X-ray image of the spine using a target detection algorithm to generate corresponding spine, pelvis, and thoracic cavity images. A first keypoint detection operation is performed on the spinal image to generate multiple vertebral angle coordinates; a second keypoint detection operation is performed on the thoracic cavity image to generate multiple thoracic keypoints; and a third keypoint detection operation is performed on the pelvic image to generate the left and right iliac crest apexes. Based on the coordinates of multiple vertebral angle points, multiple key points of the thoracic cavity, the apex of the left iliac crest, and the apex of the right iliac crest, the comprehensive angle of scoliosis in the anteroposterior X-ray image of the spine is measured.

2. The method for detecting the comprehensive angle of scoliosis according to claim 1, characterized in that, The process of performing a first keypoint detection operation on the spinal image to generate multiple vertebral body corner coordinates includes: The spinal image is preprocessed; The preprocessed spine image is input into a convolutional neural network to extract spine feature data; Based on the spinal feature data, a heat map of the center point of the spinal vertebrae is generated; Based on the heat map of the center point of the spinal vertebra, calculate the initial corner coordinates of the vertebra and their corresponding confidence values; The initial corner coordinates of the generated cone are filtered using the confidence level to obtain multiple cone corner coordinates.

3. The method for detecting the comprehensive angle of scoliosis according to claim 2, characterized in that, The process of generating a heatmap of the center point of the spinal pyramids based on the spinal feature data includes: The spinal feature data is preprocessed; Network construction and output layer setup operations are performed on the preprocessed spinal feature data to generate a heatmap model; The heatmap model is trained and optimized based on a preset loss function to generate a trained heatmap model. The spinal features are input into the heatmap model to generate a heatmap of the center point of each spinal vertebra.

4. The method for detecting the comprehensive angle of scoliosis according to claim 2, characterized in that, The process of performing a second keypoint detection operation on the thoracic image to generate multiple thoracic keypoints includes: The thoracic cavity image is segmented using an image segmentation algorithm to generate a thoracic circumference segmentation result; Based on the coordinates of multiple vertebral corner points, calculate the coordinates of the corresponding vertebral center point; In the thoracic segmentation results, one or more horizontal straight lines are generated based on the coordinates of the center point of each cone segment; Extract the distal intersection point of the horizontal straight line and the thoracic segmentation result to generate multiple thoracic key points.

5. The method for detecting the comprehensive angle of scoliosis according to claim 1, characterized in that, The process of performing a third keypoint detection operation on the pelvic image to generate the left and right iliac crest apexes includes: The pelvic image is input into a neural network to generate multiple initial pelvic key points and their corresponding confidence values; The initial pelvic key points are filtered based on the confidence level to generate optimized pelvic key points.

6. The method for detecting the comprehensive angle of scoliosis according to claim 4, characterized in that, The process of performing a third keypoint detection operation on the pelvic image to generate the left and right iliac crest apexes includes: Determine whether the left iliac crest apex and / or right iliac crest apex are missing in the optimized pelvic key points. If they are missing, perform image segmentation on the pelvic image to generate pelvic segmentation results. The optimization key points falling into the pelvic segmentation results are traversed to generate the coordinates of the missing left iliac crest apex and / or right iliac crest apex.

7. A scoliosis comprehensive angle detection device, characterized in that, include: Image acquisition module: used to acquire anteroposterior X-ray images of the spine; Region recognition module: used to identify and locate the spine, pelvis and thoracic cavity regions in the anteroposterior X-ray image of the spine using a target detection algorithm, so as to generate corresponding spine image, pelvis image and thoracic cavity image; Key point detection module: used to perform a first key point detection operation on the spinal image to generate multiple vertebral corner coordinates; A second keypoint detection operation is performed on the chest cavity image to generate multiple chest cavity keypoints; A third keypoint detection operation is performed on the pelvic image to generate the left and right iliac crest apexes; Scoliosis assessment module: used to measure the comprehensive angle of scoliosis in the anteroposterior X-ray image of the spine based on the coordinates of multiple vertebral angle points, multiple key points of the thoracic cage, the left iliac crest apex and the right iliac crest apex.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the scoliosis comprehensive angle detection method according to any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, causes the computer to implement the scoliosis comprehensive angle detection method as described in any one of claims 1 to 6.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the scoliosis comprehensive angle detection method according to any one of claims 1 to 6.

Images