A scoliosis detection method, computer equipment and storage medium

Abstract

The application discloses a scoliosis detection method, computer equipment and a storage medium, wherein the method comprises the following steps: training a preset recognition network model by using a data training set to obtain a pre-training recognition network model, wherein the training adopts a loss function provided with a regularization penalty term based on an anatomical prior constraint; recognizing a spine image by using the pre-training recognition network model to obtain corner points of each vertebra in the spine; and calculating a Cobb angle of the spine according to the corner points. The technical scheme provided by the application can improve the accuracy of the obtained Cobb angle.

Description

Technical Field

[0001] This invention relates to the field of scoliosis detection technology, specifically to a method for detecting scoliosis, a computer device, and a storage medium. Background Technology

[0002] Scoliosis is a common spinal disorder that primarily occurs during adolescence. It is characterized by a "C" or "S" shaped spine, and the effectiveness of diagnosis and treatment depends on the severity of the curvature. Currently, the Cobb angle is commonly used as an indicator of spinal curvature. Existing methods for obtaining the Cobb angle involve acquiring X-ray images of the spine via coronal or thoracic endoscopy, and then measuring the Cobb angle of a tilted vertebra. However, X-ray images of the spine show low contrast between tissues and significant intravertebral variations, making accurate measurement of the Cobb angle a challenge.

[0003] To reduce the workload of staff, machine learning algorithms can be used to measure the Cobb angle of the spine in X-ray images. This method can directly measure the Cobb angle or perform preprocessing on the X-ray images, such as locating vertebral labels, separating vertebral bodies, or extracting spinal curves, to obtain a more accurate Cobb angle. For vertebral label localization, label regression is an intuitive method; however, the dense connections and loss of fine spatial details in fully connected layers can impair generalization performance. Currently, heatmap-based algorithms are widely used in label localization. These algorithms can indirectly extract labels from keypoint heatmaps, with each pixel value representing the probability of being labeled as a vertebra.

[0004] While heatmap estimation methods have wide applications in spinal label detection, most lack sufficient reliability in practice, primarily because they do not consider anatomical characteristics, which are crucial for producing anatomically meaningful results. Multichannel keypoint heatmaps represent different vertebrae in different channels, where each adjacent channel displays similar patterns. This leads to overlapping detected labels or significant gaps between adjacent labels, affecting the accuracy of spinal Cobb angle detection. Summary of the Invention

[0005] The purpose of this invention is to provide a method, computer device and storage medium for detecting scoliosis of the spine, so as to solve the problem of inaccurate Cobb angle detection of the spine in the prior art.

[0006] To achieve the above objectives, the present invention provides a method for detecting scoliosis, comprising:

[0007] The pre-trained recognition network model is trained using a data training set to obtain a pre-trained recognition network model, wherein the training uses a loss function with a regularization penalty term based on anatomical prior constraints.

[0008] The pre-trained recognition network model is used to identify the spinal images to obtain the corner points of each vertebra in the spine;

[0009] The Cobb angle of the spine is calculated based on each of the aforementioned corner points.

[0010] According to one embodiment of the present invention, the step of using the pre-trained recognition network model to recognize the spinal image to obtain the corner points of each vertebra in the spine includes:

[0011] Feature extraction is performed on the spinal image to obtain the corresponding feature map;

[0012] Based on the feature map, the center point of each vertebra is identified in a preset order;

[0013] Based on the center points, the corner points are obtained.

[0014] According to one embodiment of the present invention, before performing feature extraction on the spine image, the method further includes:

[0015] The spine image is downsampled using a first preset number of convolutional layers.

[0016] According to one embodiment of the present invention, the loss function includes a focus loss term, a residual loss term, an angular offset loss term, and the regularization penalty term.

[0017] According to one embodiment of the present invention, the regularization penalty term includes the loss of the center points of three adjacent vertebrae calculated by elastic network regularization.

[0018] According to one embodiment of the present invention, the preset recognition network model is a neural network model based on HRNet.

[0019] According to one embodiment of the present invention, the HRNet-based neural network model includes a third preset number of parallel subnetworks, and each of the subnetworks has a different resolution.

[0020] According to one embodiment of the present invention, the depth direction of the HRNet-based neural network model is divided into four consecutive stages, wherein the first stage includes a fourth preset number of bottleneck residual blocks and a convolutional unit, and the other stages respectively include a corresponding number of exchange modules.

[0021] On the other hand, the present invention also provides a machine-readable storage medium having a machine-executable program stored thereon, which, when executed by a processor, implements the scoliosis detection method according to any of the above embodiments.

[0022] In another aspect, the present invention also provides a computer device including a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor executes the machine-executable program to implement the scoliosis detection method according to any of the above embodiments.

[0023] The technical solution provided by this invention has the following advantages:

[0024] The technical solution provided by this invention includes a regularization penalty term based on anatomical prior constraints in the loss function when training a preset recognition network model to obtain a pre-trained recognition network model. This can improve the accuracy of the obtained pre-trained recognition network model. Therefore, when using the pre-trained recognition network model to recognize spinal images, the corner points of each vertebra in the spine can be accurately obtained. Furthermore, when calculating the Cobb angle of the spine based on the corner points of each vertebra, the accuracy of the obtained Cobb angle can be improved.

[0025] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0026] Figure 1 This is a schematic flowchart of a method for detecting scoliosis according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of a spine image according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic flowchart illustrating the use of a pre-trained recognition network model to recognize a spine image according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the location of the center point of the vertebra in a probability heatmap according to an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of a neural network model based on HRNet according to an embodiment of the present invention;

[0031] Figure 6This is an overall schematic diagram of identifying spinal corner points according to an embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram illustrating a qualitative comparison between the predicted values ​​of labels and the true values ​​of annotations from multiple training algorithms according to an embodiment of the present invention.

[0033] Figure 8 This is a schematic diagram of the loss of anatomical prior constraints used in a multi-channel scheme according to an embodiment of the present invention;

[0034] Figure 9 This is a schematic diagram of anatomical-assisted reasoning used in a multi-channel scheme according to an embodiment of the present invention. Detailed Implementation

[0035] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art should understand that the embodiments described below are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] The preferred embodiments of the present invention will now be described in detail with reference to examples. It should be understood that the following examples are given for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make various modifications and substitutions to the present invention without departing from its spirit and intent, and all such modifications and substitutions fall within the scope of protection claimed in the present invention.

[0037] Unless otherwise specified, the experimental methods used in the following examples are conventional methods. Unless otherwise specified, all materials and reagents used in the following examples are commercially available.

[0038] The method for detecting scoliosis provided in this embodiment is as follows: Figure 1 As shown, it includes the following steps:

[0039] Step S1: Train the preset recognition network model using the data training set to obtain the pre-trained recognition network model, and the loss function used in the training process includes a regularization penalty term based on anatomical prior constraints;

[0040] Step S2: Use a pre-trained recognition network model to identify the spinal image to obtain the corner points of each vertebra in the spine;

[0041] Step S3: Calculate the Cobb angle of the spine based on the angle points of each vertebra.

[0042] In step S1 above, the preset recognition network model can be a neural network model, and the training data set can be multiple preset spinal images obtained from a designated hospital (such as Peking University First Hospital) or a designated platform. When training the preset recognition network model, the corner points of each vertebra in each preset spinal image are first labeled. Then, the labeled preset spinal images are used to train the preset recognition network model to solidify its parameters, resulting in a pre-trained recognition network model. Furthermore, a loss function with a regularization penalty term based on anatomical prior constraints is used to calculate the loss during the training process, thereby improving the accuracy of the pre-trained recognition network model.

[0043] In this embodiment, the anatomical prior constraint refers to the distribution of vertebrae in a preset order, and the distance between adjacent vertebrae within a preset range. Figure 2 Taking the spinal image shown as an example, let the horizontal direction be the x-axis and the vertical direction be the y-axis. Then the anatomical prior constraints can include: the distance between the center points of each vertebra is within a preset range, and the center points of each vertebra are distributed sequentially in the y-axis direction.

[0044] For example, let Figure 2 The center points of the (i+1)th vertebra and the center point of the ith vertebra, distributed from top to bottom, are respectively and The distance between the center points of each vertebra within the preset range can be expressed as:

[0045]

[0046] Where a and b are the first and second thresholds, respectively, and can be determined based on the size of the heatmap. For example, for a heatmap with a size of 128×64, the value of a can be 1.42, and the value of b can be 11.50. The distribution of the center points of each vertebra along the y-axis can be represented as follows:

[0047]

[0048] in and Here, y-coordinates are the center points of the (i+1)th and ith vertebrae, respectively. This positional relationship ensures that the center point of the (i+1)th vertebra is located below the center point of the ith vertebra. Because distance constraints are added during the vertebral identification process, errors in vertebral identification are prevented, improving the accuracy of vertebral identification.

[0049] In step S2 above, the spinal image is an X-ray image of the spine acquired through X-ray imaging technology. The resulting corner points of each vertebra in the spine include the four corners of each vertebra. Taking the human spine as an example, since the human spine has 17 vertebrae, and each vertebra has four corner points, a total of 68 corner points can be obtained.

[0050] In step S3 above, the Cobb angle of the spine can be calculated based on the corner points of each vertebra, using existing methods. For example, the position and orientation of each vertebra can be determined based on the four corner points of each vertebra to obtain the overall shape of the spine, and then the Cobb angle of the spine can be calculated.

[0051] In this embodiment, the technical solution introduces a regularization penalty term based on anatomical prior constraints into the loss function used to train the preset recognition network model to obtain the pre-trained recognition network model. This can improve the accuracy of the obtained pre-trained recognition network model. As a result, when using the pre-trained recognition network model to recognize spinal images, the corner points of each vertebra in the spine can be accurately obtained. Furthermore, when calculating the Cobb angle of the spine based on the corner points of each vertebra, the accuracy of the obtained Cobb angle can be improved.

[0052] In one embodiment of the present invention, the process of using a pre-trained recognition network model to recognize the spine image in step S2 is as follows: Figure 3 As shown, it includes:

[0053] Step S21: Extract features from the spine image to obtain the corresponding feature map;

[0054] Step S22: Identify the center points of each vertebra in the above feature map in a preset order;

[0055] Step S23: Obtain the corner points of each vertebra based on its center point.

[0056] In step S21 above, the preset neural network model can extract features from the spine image to obtain a feature map containing low-level spatial detail information and high-level semantic information of the spine image.

[0057] In step S22 above, a preset activation function (such as the sigmoid activation function) can be used to activate the obtained feature map to obtain a probability heatmap with multiple channels. Unlike the existing technology that uses a single channel (that is, treating each vertebra as a category), in this embodiment, the number of channels is the same as the number of vertebrae. Since there are 17 vertebrae in the human body, a probability heatmap with 17 channels is obtained, so that each vertebra can be separated from the vertebrae in a preset order along the y-axis. This process is also called sequential reasoning, and the sequential reasoning based on anatomical prior constraints must meet the following conditions:

[0058]

[0059]

[0060] Where Δy min Δy max and Δx max It is a hyperparameter, and Δy min Δy is the minimum threshold for the distance between the center points of two adjacent vertebrae on the y-axis. max Δx is the maximum threshold for the distance between the center points of two adjacent vertebrae on the y-axis. max The maximum threshold for the distance between the center points of two adjacent vertebrae on the x-axis.

[0061] After obtaining probability heatmaps from multiple channels, the probability that each pixel in each probability heatmap can be used as the center point of a vertebra can be obtained. Pixels with a probability greater than a set probability are then used as the center points of the vertebrae, thus enabling the prediction of the center point of each vertebra. Furthermore, during the recognition process, the center points of the vertebrae in each probability heatmap are identified sequentially along the y-axis in a preset order, thereby achieving sequential recognition of the center points of each vertebra.

[0062] In step S23 above, the corner points of each vertebra can be obtained based on the distance between the center point and each corner point of each vertebra. For example, for the center point of a vertebra, the nearest corner points to its upper left, upper right, lower left, and lower right can be obtained as the four corner points of that vertebra.

[0063] By using the configuration method of this embodiment, the center point of each vertebra can be accurately identified in a preset order, and then the corner point of each vertebra can be accurately obtained based on the center point of each vertebra. Therefore, the accuracy of the corner point identification of each vertebra can be improved, so as to accurately calculate the Cobb angle of the spine based on the corner point.

[0064] In one embodiment of the present invention, when using a pre-trained recognition network model to recognize the spinal image in step S2, before using a preset neural network model to extract features from the spinal image in step S21, the method further includes: using a first preset number of convolutional layers to perform downsampling processing on the spinal image.

[0065] In this embodiment, the first preset number is 4, and the size of the convolutional layer is 3×3. That is, the spinal image is downsampled 4 times in a continuous manner using 3×3 to increase the receptive field of the spinal image and facilitate the extraction of features from the spinal image.

[0066] In one embodiment of the present invention, the loss function used in step S1 above includes a focus loss term, a residual loss term, an angular offset loss term, and a regularization penalty term based on anatomical prior constraints.

[0067] In this embodiment, during the training of the preset recognition network model, a real heatmap Y is used to supervise the probabilistic heatmap. The extraction of the center point of the middle vertebra is performed, where the real heatmap Y contains the real labels of each vertebra. For the real heatmap Y, the proportionally scaled-down real center point of each vertebra in each channel is also included. A non-normalized two-dimensional Gaussian ellipse following the 3σ verification rule is placed around the vertebra. The span of this ellipse is the standard deviation of the Gaussian distribution, which is determined by the area of ​​the vertebra. Therefore, σ may occur. x ≠σ y The weight of the pixel at the center point of the i-th vertebra in the above two-dimensional Gaussian ellipse is:

[0068]

[0069] Where σ x σ is the standard deviation along the x-axis. y The standard deviation is in the y-axis direction. This represents the true center point of the i-th vertebra. and Let x and y be the x-coordinates of the true center point of the i-th vertebra on the x-axis and y-axis, respectively. i and y i Let x and y be the x-coordinates of the predicted center point of the i-th vertebra on the x-axis and y-axis, respectively. The focus loss was adjusted to reduce the penalty for floating points inside the Gaussian ellipse, which are also candidate points for the vertebral center point.

[0070] In this embodiment, the focus loss term is obtained using the following calculation formula:

[0071]

[0072] in For probability heatmap The weight value of the center point pixel of the i-th vertebra is given by α, which is the first hyperparameter and can take the value 2; β is the second hyperparameter and can take the value 4; and N is the number of vertebrae.

[0073] When performing four downsampling operations on the spinal image, the center point of the i-th vertebra will be scaled to... This will cause rounding errors. Where P i This represents the center point of the i-th vertebra in the spinal image before downsampling. In this embodiment, 2-channel feature mapping... The predicted center point of each vertebra is extracted. Let the predicted center point of the i-th vertebra be denoted as . To compensate for the error caused by the downsampling, this error can be added as an offset to the scaled center point to obtain the final center point of each vertebra, where the final center point of the i-th vertebra is...

[0074]

[0075] Therefore, the residual loss term can be obtained as follows:

[0076]

[0077] To obtain the corner points of each vertebra, it is necessary to estimate the relative coordinates of the four corner points of each vertebra. These corner points are represented by a vector of length 8, which is derived from the corresponding center point. From the feature map F c Extract from the center point. Add the angle offset to the center point to generate the corresponding corner point.

[0078] For example, the upper left corner of the i-th vertebra is:

[0079]

[0080] The vertebral angular offset loss term L can then be obtained using the following formula. c :

[0081]

[0082] Where d i This represents an angular offset vector of length 8, and the predicted offset is determined by... generate.

[0083] The loss function obtained through this embodiment is:

[0084] L = L h +L a +L c +λL tri

[0085] Where L tri λ is the regularization penalty term based on anatomical prior constraints, and λ is the weight of this regularization penalty term.

[0086] By using the configuration method in this embodiment, the loss in the positioning of the spine's center point, the loss caused by downsampling, and the loss in the positioning of each corner point can be reduced during training, thereby improving the accuracy of the obtained pre-trained recognition network model.

[0087] In one embodiment of the invention, the regularization penalty term based on anatomical priors includes the loss of the center points of three adjacent vertebrae calculated by elastic network regularization.

[0088] In order to take into account both position and distance constraints in the anatomical prior constraints during training, this embodiment uses the Soft-argmax function to estimate the peak position of the probability heatmap to infer the position of the center point. This involves a regularization penalty term related to anatomical prior knowledge, which is added to the loss function.

[0089] Normally, the operation of extracting pixel peak positions is non-differentiable. Therefore, gradients cannot be backpropagated through the argmax function during training. Instead, this embodiment uses the soft-argmax function, i.e.

[0090]

[0091] Because the argmax function operation is infinitesimally variable, during training, the probability heatmap can be used to... Extract the peak position from the i-th channel, such as Figure 4 As shown, and based on this, the regularization penalty term is calculated in a two-dimensional window W. i The spatial argmax function is used to calculate the pixel position, with the actual peak position. Let be the center point of the i-th vertebra. In the above formula:

[0092]

[0093] In the above formula, t is a learnable parameter used to adjust the differences between pixels.

[0094] In this embodiment, the second preset quantity is 3. This means that, considering the anatomical relationship between vertebrae, three adjacent vertebrae are treated as a single unit. The label at the center point of the middle vertebra is placed near the center points of two adjacent labels, indicating the actual peak position. Substitution It can be obtained Predicted values:

[0095]

[0096] Furthermore, the regularization penalty term based on anatomical prior constraints includes the loss of the center points of three adjacent vertebrae calculated through elastic network regularization, and the calculation formula for the regularization penalty term is as follows:

[0097]

[0098] By using the configuration method in this embodiment, a regularization penalty term based on anatomical prior constraints can be added to the loss function, and the accuracy of the pre-trained recognition network model can be improved by training the pre-trained recognition network model through this loss function.

[0099] In one embodiment of the present invention, the preset neural network model is a neural network model based on HRNet.

[0100] Since the HRNet-based neural network model can perform frequent fusion and interaction between multiple resolutions, thereby preserving multi-scale information, the setup in this embodiment can improve the accuracy of corner point identification for each vertebra.

[0101] In one embodiment of the present invention, the HRNet-based neural network model includes a third preset number of subnetworks, each subnetwork being set in parallel and having different resolutions.

[0102] In this embodiment, the third preset number can be 4. The HRNet consists of four parallel high-to-low resolution subnets, each maintaining its resolution through forward propagation. The resolution of the four subnets is halved sequentially, while their widths gradually double.

[0103] In one embodiment of the present invention, the depth direction of the HRNet network can be divided into a fourth preset number of stages. Figure 5 As shown, the first stage includes four bottleneck residual blocks and one 3×3 convolutional unit, providing the highest resolution. The following three stages contain one, four, and three swapping modules, respectively, with the multi-resolution subnets progressively swapping and fusing features using multiple downsampling and upsampling steps. The final output of HRNet is extracted from the highest-resolution subnet in the fourth stage. Furthermore, HRNet can employ an HRNet-48 variant, where the width of the highest-resolution subnet is set to 48. Correspondingly, the number of channels for the other three subnets will be 96, 192, and 384, respectively.

[0104] The configuration method in this embodiment allows for the fusion and interaction between multiple resolutions during the feature map extraction process, enabling the feature map to retain multi-scale information of the spinal image and thereby improving the accuracy of vertebral corner point recognition.

[0105] In summary, the overall technical solution provided by this invention is as follows: Figure 6 As shown, the spinal image can first be downsampled using a 3×3 convolutional layer, and then a pre-trained recognition network model based on HRNet can be used to process the downsampled spinal image to identify the corner points of each vertebra. Furthermore, when training the pre-trained recognition network model, the loss function used includes a focus loss term L. hResidual loss item L a Angular offset loss term L c and regularization penalty term L based on anatomical prior constraints tri This is to improve the accuracy of the pre-trained recognition network model, thereby improving the accuracy of the Cobb angle calculation for the spine.

[0106] The technical effectiveness of the proposed solution will be verified below through specific experiments.

[0107] The cleaned spinal X-ray image dataset used in this experiment includes 819 self-annotated images collected from Peking University First Hospital and 591 images selected from dataset 16 on SpinalWeb, totaling 1410 X-ray-based spinal images at different scales. Each spinal image contains 68 manually annotated corner labels for 17 vertebrae. Furthermore, the center point of each vertebra was calculated by averaging the coordinates of the true center point of each vertebra. All annotations passed a sans-seriousness check, ensuring that the four nearest corner points cluster together to represent a vertebra, adjacent vertebrae satisfy anatomical prior constraints, and vertebral labels are ordered from top to bottom along the y-axis according to the vertebral center point.

[0108] The metrics used in this experiment included basic metrics and anatomically relevant metrics. The basic metrics included localization error and mean absolute percentage error (MASE). Localization error, calculated by the average L2 distance between the actual and predicted positions of the tag, was used to assess the accuracy of tag detection. The localization error E for the center or corner points of the vertebra can be calculated using the following formula. l k oc :

[0109]

[0110] In the above formula, k represents either corner or center. When k is corner, d represents the localization error of the corner points of each vertebra in the spinal image. i and g i Let be the detected position and the true position of the i-th corner point at the original resolution of the spine image, respectively, and M represent the total number of labels for the corner points of the vertebrae in the test dataset. When k is center, d represents the positioning error of the center point of each vertebra in the spinal image. i and g i Let be the detected position and the true position of the center point of the i-th vertebra at the original resolution of the spinal image, respectively, and M represent the total number of labels for the center points of the vertebrae in the test dataset.

[0111] In addition to positioning error, this experiment also uses symmetrical mean absolute percentage error, i.e.

[0112]

[0113] The Cobb angle was evaluated based on the solution from the AASCE MICCAI 2019 challenge. Using the algorithm provided by the AASCE challenge, N... I This represents the total number of test images. This represents the predicted value of the j-th Cobb angle type of the spine in the i-th test image. The true value of the j-th type of Cobb angle of the spine in the i-th test image. In this experiment, there are three types of Cobb angles, referring to the proximal thoracic vertebrae (PT), main thoracic vertebrae (MT), and thoracolumbar vertebrae (TL) of the spine. The SMAPE index corresponding to each location is named SMAPE. PT SMAPE MT SMAPE TL .

[0114] Anatomy-related metrics: In addition to the basic metrics, we also propose anatomy-related metrics to quantitatively measure how label predictions conform to anatomical priors, rather than just visual quality observations.

[0115] In this embodiment, the interval Defined as the interval from the initial prediction result of the i-th vertebra. Defined as starting from the center point of the i-th vertebra. Pointing to the next center point The vector, that is:

[0116]

[0117] There are 16 intervals between the 17 vertebrae in each spinal image; this experiment will... or The center point is taken as the center point whose distance does not conform to the prior knowledge of anatomy, and N is used. fmin and N fmax They represent Number of center points and The number of center points.

[0118] For center points whose order does not conform to anatomical priors, i.e., the y-axis coordinate of the center point or Among them The number of center points is N eq , The number of center points is denoted as N. neg Furthermore, for the center points of intervals with negative y-axis values, we calculate... The average value, and denoted as Let the minimum value be min(y) neg ).

[0119] Because an arrangement operation is used on a single-channel heatmap, adjacent vertebrae will always satisfy the sequence constraint, i.e. This leads to a loss of effectiveness for order-related indicators. Therefore, we primarily use anatomically relevant indicators to evaluate anatomical relationships on the multi-channel heatmap-based algorithm.

[0120] We conduct a 5x cross-validation experiment to scan the entire dataset. The dataset will be split into train:val:test = 3:1:1, where train represents the number of training images, val represents the number of validation images, and test represents the number of test images. We resize all input images to a fixed resolution of 512×256, resulting in an output heatmap scale of 128×64. Common data validation techniques such as horizontal flipping, channel shuffling, contrast shifting, and brightness shifting are also employed. We use the Adam network for optimization with an initial learning rate of 10. -4 If the localization error fails to decrease after more than 20 training epochs, the learning rate will be reduced by 10. Once the learning rate reaches 10... -9 The training process will terminate if this condition is met. The maximum number of training epochs is limited to [200, 400]. All experiments follow the same training settings.

[0121] The weight λ of the regularization penalty term based on anatomical constraints is set to 0.15. A uniform window of size 40 is used in the soft-argmax function, and t is assigned a value of 0.005. For inference, we select the model with the smallest localization error on the validation dataset across all training epochs. For the output heatmap of size 128×64, the anatomical hyperparameters will be set as follows:

[0122] Δy min =2, Δy max =10, Δx max =7

[0123] The algorithm used in this experiment was mainly compared with a baseline model based on heatmaps. The obtained data are shown in Tables 1, 2, 3, and 4. Table 1 contains basic indicators, Table 2 contains anatomical-related indicators, Table 3 contains basic indicators for recognition models using different main architecture networks, and Table 4 contains anatomical-related indicators for different spines. In Tables 1, 2, 3, and 4, Heatmap-based is a preset recognition network model based on heatmaps, and Heatmap-based+L... triIt is a pre-defined recognition network model based on heatmaps with regularized penalty terms based on anatomical prior constraints. tri +Sequentual Inference is a pre-defined recognition network model that incorporates a regularization penalty term based on anatomical prior constraints and a heatmap for sequential reasoning. In this experiment, the heatmap refers to the image obtained by extracting features from a spinal image and then activating those features using an activation function.

[0124] Table 1

[0125]

[0126] Table 2

[0127]

[0128] Table 3

[0129]

[0130] Table 4

[0131]

[0132]

[0133] The visualization results of the basic and anatomical-related indicators obtained in this experiment are as follows: Figure 7 , Figure 8 and Figure 9 As shown. In Figure 7 This diagram illustrates a qualitative comparison between the predicted and annotated values ​​of labels from multiple training algorithms. Part a shows the detection results obtained by the keypoint detection algorithm based on the heatmap; part b shows the detection results obtained by a pre-defined recognition network model based on a single-channel heatmap and the HRNet network; part c shows the detection results obtained by a pre-defined recognition network model based on a multi-channel heatmap and the HRNet network; part d shows the detection results obtained by a pre-defined recognition network model based on a multi-channel heatmap, the HRNet network, and a regularization penalty term incorporating anatomical prior constraints; and part e shows the detection results obtained by a pre-defined recognition network model based on a multi-channel heatmap, the HRNet network, a regularization penalty term incorporating anatomical prior constraints, and sequential reasoning. The input resolution is set to 512×256 for all algorithms, including the baseline model.

[0134] Figure 8The diagram shows the loss of the anatomical prior constraints used in the multi-channel scheme. Part a is a diagram obtained by a preset recognition network model based on the multi-channel heatmap and HRNet network, and part b is a diagram obtained by introducing a regularization penalty term based on the anatomical prior constraints based on the multi-channel heatmap and HRNet network.

[0135] Figure 9 The diagram shows the anatomical-assisted reasoning used in the multi-channel scheme. Part a is a diagram based on the multi-channel heatmap and HRNet network, with the introduction of a regularization penalty term based on anatomical prior constraints. Part b is a diagram based on the multi-channel heatmap and HRNet network, with the introduction of a regularization penalty term based on anatomical prior constraints and sequential reasoning.

[0136] To ensure a fair comparison with baseline models on different new datasets, instead of simply listing the results, we use code that retrains the baseline model on our mixed dataset, keeping the input resolution and training settings the same as our algorithm.

[0137] Compared to the baseline model with an input resolution of 512×256, our algorithm, after adding all design components, achieves the lowest detection error for both low-variance corner and center labels. Furthermore, the SMAPE values ​​also show varying degrees of reduction. Our proposed algorithm demonstrates better accuracy and stability than the baseline model in both label localization and Cobb angle estimation.

[0138] 1) Ablation on HRNet: After switching from a U-Net-like backbone network to HRNet-48, the label detection error significantly decreased from 39.67 / 36.61 to 35.89 / 32.89; at the same time, the accuracy of Cobb angle estimation was also improved. HRNet has become the best-performing backbone network in label detection tasks because frequent fusion and interaction between multiple resolutions preserves multi-scale information.

[0139] 2) Ablation on multi-channel thermal maps: When all 17 vertebrae are inferred from a single-channel thermal map by selecting the highest points along the longitudinal axis, obvious outliers may appear on clear images of the spine. When a multi-channel approach is applied, unreasonable predictions are reduced due to individual supervision of different vertebrae.

[0140] By simply changing the number of channels, the quantitative detection error decreased from 39.67 / 36.61 to 36.88 / 33.81, as shown in Table 3. When the HRNet-48 backbone adopted a multi-channel scheme (35.89 / 32.89→34.97 / 31.88), the detection error showed a consistent downward trend.

[0141] 3) Ablation of Anatomical Constraint Losses: For algorithms using multi-channel heatmaps, there are a considerable number of outlier threshold predictions that do not conform to anatomical priors, as shown in Tables 2 and 4. Based on extensive observation of the output images, one or two anomalous vertebrae may appear on anatomically unreliable images. Figure 8 As shown in section a, the predicted vertebrae exhibit severe disorder and uneven vertebral distribution. When the designed anatomical constraint loss is incorporated into the training objective, the milestone predictions are constrained to conform to... Figure 8 The anatomical prior is shown in b in Table 2. According to the self-designed indicators in Table 2, the number of outliers decreased relatively significantly (more than 1 / 3, even close to 1 / 2), while y... neg and min(y neg The error rate also decreased. In addition, with the regularization of anatomical constraints, the detection error also decreased.

[0142] To validate the effectiveness of our designed penalty on different backbones, we incorporated the proposed penalty into a ResNet34-based baseline model. The anatomical constraint loss yielded significant improvements in anatomical-related metrics while maintaining the accuracy of labels and Cobb angles in Tables 3 and 4, demonstrating that the proposed penalty is a general and flexible regularization term compatible with different feature extractors.

[0143] 4) Ablation of Anatomical-Assisted Reasoning: For specimens that still present anatomical problems after regularization of anatomical-related training penalties, such as... Figure 7 part d in Figure 9 Part a; We calibrate the prediction based on anatomical priors during inference time, such as Figure 7 part e in Figure 9 Part b of the paper discusses how the proposed post-processing steps further enhance anatomical performance by utilizing anatomical constraint losses. The number of abnormal intervertebral spaces is reduced by nearly 30–40% while maintaining label positioning accuracy. Furthermore, the versatility of the designed adjustment procedure is demonstrated across different trunks, as shown in Tables 2 and 4.

[0144] This embodiment also provides a machine-readable storage medium and a computer device, wherein the computer device may include a memory, a processor, and a machine-executable program stored on the memory and running on the processor, and the processor executes the machine-executable program to implement the scoliosis detection method of any of the above embodiments.

[0145] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any machine-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-based system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0146] For the purposes of this embodiment, a machine-readable storage medium can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, a computer-readable medium can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0147] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0148] Computer devices can be, for example, servers, desktop computers, laptop computers, tablet computers, or smartphones. In some examples, a computer device can be a cloud computing node. A computer device can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, a program module can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer devices can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0149] Computer devices may include a processor adapted to execute stored instructions and memory that provides temporary storage for the operation of said instructions during operation. The processor may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0150] The processor can be connected via a system interconnect (such as PCI, PCI-Express, etc.) to an I / O interface (input / output interface) suitable for connecting computer devices to one or more I / O devices (input / output devices). I / O devices may include, for example, a keyboard and indicating devices, where indicating devices may include a touchpad or touchscreen, etc. I / O devices can be built into the computer device or can be external devices connected to the computing device.

[0151] The processor can also be linked via a system interconnect to a display interface suitable for connecting computer devices to display devices. The display device may include a screen that is a built-in component of the computer device. The display device may also include an external computer monitor, television, or projector connected to the computer device. Furthermore, a network interface controller (NIC) may be adapted to connect computer devices to a network via the system interconnect. In some embodiments, the NIC may use any suitable interface or protocol (such as an Internet Minicomputer System Interface) to transmit data. The network may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. Remote devices can connect to the computing device via the network.

[0152] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0153] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for detecting scoliosis, characterized in that, include: The pre-set recognition network model is trained using a data training set to obtain a pre-trained recognition network model, wherein the training uses a loss function with a regularization penalty term based on anatomical prior constraints. The pre-trained recognition network model is used to identify the spinal images to obtain the corner points of each vertebra in the spine; Calculate the Cobb angle of the spine based on each of the aforementioned corner points; The preset recognition network model is a neural network model based on HRNet; The regularization penalty term includes the loss of the center points of three adjacent vertebrae calculated through elastic network regularization. The calculation formula for the regularization penalty term is as follows: Among them, L tri The regularization penalty term is based on anatomical prior constraints, where N is the number of vertebrae; Probability heatmap The weight value of the center point pixel of the i-th vertebra, where t is a learnable parameter used to adjust the differences between pixels, W i Represents a two-dimensional window. Represents 2-channel feature mapping Indicates the 2-channel feature map The error of the predicted value of the center point extracted above.

2. The method for detecting scoliosis according to claim 1, characterized in that, The step of using the pre-trained recognition network model to identify the spinal image to obtain the corner points of each vertebra in the spine includes: Feature extraction is performed on the spinal image to obtain the corresponding feature map; Based on the feature map, the center point of each vertebra is identified in a preset order; Based on the center points, the corner points are obtained.

3. The method for detecting scoliosis according to claim 2, characterized in that, Before performing feature extraction on the spine image, the method further includes: The spine image is downsampled using a first preset number of convolutional layers.

4. The method for detecting scoliosis according to claim 3, characterized in that, The loss function includes a focus loss term, a residual loss term, an angle offset loss term, and a regularization penalty term.

5. The method for detecting scoliosis according to claim 1, characterized in that, The HRNet-based neural network model includes a second preset number of parallel sub-networks, and each sub-network has a different resolution.

6. The method for detecting scoliosis according to claim 1, characterized in that, The depth direction of the HRNet-based neural network model is divided into four consecutive stages. The first stage includes a third preset number of bottleneck residual blocks and a convolutional unit, while the other stages each contain a corresponding number of swapping modules.

7. A machine-readable storage medium having a machine-executable program stored thereon, the machine-executable program, when executed by a processor, implementing the scoliosis detection method according to any one of claims 1 to 6.

8. A computer device comprising a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor, when executing the machine-executable program, implements the scoliosis detection method according to any one of claims 1 to 6.

Images