An automatic scoring method for ankylosing spondylitis in sacroiliac joint MRI images
By combining deep learning methods with multi-scale feature fusion and knowledge of the sacroiliac joint space, an automatic SPARCC scoring of sacroiliac joint MRI images was achieved, which solves the problem of low automation in existing technologies and improves scoring efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- EAST CHINA NORMAL UNIV
- Filing Date
- 2024-02-19
- Publication Date
- 2026-07-31
AI Technical Summary
In the current technology, the SPARCC score of sacroiliac joint MRI images mainly relies on manual classification by professional doctors. The degree of automation is low, resulting in a large workload and low efficiency, and it is impossible to achieve accurate and quantitative scoring.
A sacral and iliac bone recognition and segmentation model based on multi-scale feature fusion and a multi-head scoring prediction model are adopted. The SPARCC score is automatically predicted by deep learning methods, including sacral and iliac bone segmentation, target quadrant mapping and multi-head scoring prediction. Combined with knowledge of the sacroiliac joint space, the automated SPARCC scoring is achieved.
It improves the automation of the SPARCC score for the sacroiliac joint, reduces the workload of doctors, provides accurate quantitative scoring results, and improves scoring efficiency.
Smart Images
Figure CN118037660B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing, and in particular relates to an automatic scoring method for ankylosing spondylitis in sacroiliac joint MRI images. Background Technology
[0002] Ankylosing spondylitis (AS) is a general term for rheumatic diseases affecting the sacroiliac joints, sacrum, iliac bones, intervertebral discs, and facet joints. It is a common spinal joint disease in adults, primarily manifested as pain and stiffness in the sacroiliac joints. Currently, commonly used imaging methods in clinical practice include X-rays, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, and radionuclide bone scintigraphy. In the initial diagnosis, CT images of the sacroiliac joints can be used to observe for bright signals indicating inflammation and edema, and the affected area can be graded on a five-level scale. Further examination relies on MRI images of the sacroiliac joints to evaluate and provide quantitative assessment indicators: by observing abnormal signals in structures such as the sacrum, iliac bones, and sacroiliac joints on MRI images, the inflammation status is quantitatively assessed according to the Spondyloarthritis Research Consortium of Canada (SPARCC) scoring system to track inflammatory activity and develop subsequent treatment plans. Currently, the SPARCC scoring mainly relies on professional doctors to observe the MRI images of the sacroiliac joint layer by layer, manually divide the eight quadrants of the sacroiliac joint that the SPARCC scoring focuses on, and subjectively analyze and score them. This is time-consuming, labor-intensive, and has a low degree of automation.
[0003] With the development of computer-aided diagnostic technology, deep learning-based image analysis methods have been widely researched and applied, assisting doctors in diagnosis and reducing their workload in clinical applications. Shantou University Medical College has published a classification and rating method for sacroiliac joint CT images. This method inputs sacroiliac joint CT images into a 3D-UNet network for segmentation, and the segmentation results are then input into a hybrid neural network for classification and rating. The Air Force Medical University of the Chinese People's Liberation Army has proposed a task-based deep learning method for automated grading of four levels of sacroiliac joint X-ray diseases. However, existing methods can only address coarse grading in the initial diagnosis and do not analyze the quantitative indicator, the SPARCC score, of sacroiliac joint MRI images, thus failing to provide more detailed and accurate SPARCC score predictions. Therefore, there is an urgent need for an automated scoring method for ankylosing spondylitis using sacroiliac joint MRI images to address the shortcomings of existing technologies. Summary of the Invention
[0004] The purpose of this invention is to propose an automatic scoring method for ankylosing spondylitis in sacroiliac joint MRI images. By combining deep learning methods, the method can automatically predict the SPARCC score of sacroiliac joint MRI images, providing doctors with accurate and quantitative assistance in diagnosis.
[0005] To achieve the above objectives, the present invention provides an automatic scoring method for ankylosing spondylitis on sacroiliac joint MRI images, specifically including the following steps:
[0006] Obtain MRI images of the sacroiliac joint to be scored;
[0007] The MRI image of the sacroiliac joint to be scored is input into the sacrum and iliac bone recognition and segmentation model based on multi-scale feature fusion, and the sacrum and iliac bone segmentation map is output. The sacrum and iliac bone recognition and segmentation model based on multi-scale feature fusion is obtained by training the first training set, which is the MRI image of the sacroiliac joint with sacrum and iliac bone segmentation mask annotation.
[0008] Based on the knowledge of the sacroiliac joint space and the sacral-iliac bone segmentation diagram, a target quadrant mapping diagram is obtained;
[0009] The target quadrant mapping is input into the multi-head rating prediction model, and the multi-head rating prediction network results are output. The multi-head rating prediction network results are summed to obtain the final SPARCC rating prediction result. The multi-head rating prediction model is trained on a second training set, which is a dataset including quadrant accumulation dimension, edema signal brightness dimension, and edema signal depth dimension.
[0010] Optionally, obtaining the first training set includes:
[0011] The first training set was obtained by annotating the sacroiliac joint MRI image dataset using 3D slicer annotation software;
[0012] The sacroiliac joint MRI image dataset consists of oblique coronal weighted image sequences.
[0013] Optionally, the sacral-iliac bone segmentation network model based on multi-scale feature fusion consists of a feature extraction model and a feature pyramid pooling model;
[0014] The feature extraction model takes a single-layer image from the first training set as input and uses a convolutional neural network ResNet50 to extract image features.
[0015] The feature pyramid pooling model uses PSPNet, a segmentation network that integrates multi-scale features.
[0016] Optionally, the sacral and iliac bone recognition and segmentation model based on multi-scale feature fusion, trained from the first training set, includes:
[0017] Obtain the first loss function;
[0018] Set hyperparameters, input the first training set into the sacral and iliac bone segmentation model to be trained based on multi-scale feature fusion, use the SGD optimizer to perform gradient backpropagation on the first loss function, update the parameters of the sacral and iliac bone segmentation network model to be trained based on multi-scale feature fusion, and train the sacral and iliac bone segmentation network model to be trained until the first loss function converges, and obtain the sacral and iliac bone recognition and segmentation network model based on multi-scale feature fusion.
[0019] Optionally, the first loss function is a weighted sum of the segmentation loss function and the classification auxiliary loss function:
[0020] L=λ1L seg +λ2L aux
[0021]
[0022] L aux =CrossEntropy(y,aux(f) θ (x)))
[0023] Where L is the loss function chosen for model training, L seg For the segmentation loss function, L aux Let λ1 and λ2 be the weights of the segmentation loss and classification auxiliary loss, respectively, and CrossEntropy(·) be the cross-entropy function, where y is the true label of the sample. Let f be the predicted label of the model, aux(·) be the classification auxiliary loss function, and f be the predicted label of the model. θ (·) represents the classification model parameters.
[0024] Optionally, based on the sacroiliac joint space knowledge and the sacral-iliac bone segmentation map, the target quadrant mapping map can be obtained by:
[0025] Based on the knowledge of the sacroiliac joint space, locate the sacroiliac joint space and determine its starting and ending points;
[0026] Calculate the perpendicular bisector of the sacroiliac joint space using the starting and ending points of the sacroiliac joint space;
[0027] Based on the sacroiliac joint space and the perpendicular bisector of the sacroiliac joint space, the sacral iliac bone segmentation diagram is divided to obtain the target quadrant mapping diagram.
[0028] Optionally, the multi-head scoring prediction model includes a feature extraction sub-model and a multi-head prediction sub-model;
[0029] The feature extraction sub-model takes as input the image patch and mask patch of a single-dimensional feature dataset and uses a convolutional neural network ResNet50 to extract image features.
[0030] The multi-head prediction sub-model consists of several sets of prediction heads, each prediction head consisting of a fully connected layer (FC), an activation function (RELU), and a fully connected layer (FC).
[0031] Optionally, training the multi-head rating prediction model based on the second training set includes:
[0032] Obtain the second loss function, which is the classification cross-entropy loss function;
[0033] Set hyperparameters, use a progressive training method to input the second training set into the multi-head rating prediction model, use the SGD optimizer to perform gradient backpropagation on the second loss function, update the parameters of the multi-head rating prediction model, train the multi-head rating prediction model, and obtain the trained multi-head rating prediction model.
[0034] Optionally, the SPARCC scoring criteria focus on several dimensions of edema information, including quadrant cumulative edema, and the brightness and depth of sacroiliac joint edema.
[0035] The present invention has the following beneficial effects: The automatic prediction method for SPARCC scores of sacroiliac joint MRI images provided by the present invention can give accurate SPARCC score prediction results through a two-stage deep learning network, which improves the efficiency of sacroiliac joint SPARCC scoring, improves the automation level of ankylosing spondylitis SPARCC scoring, solves the problem of low automation level of sacroiliac joint SPARCC scoring in the prior art, and reduces the workload of doctors. Attached Figure Description
[0036] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0037] Figure 1 This is a flowchart of an automatic scoring method for ankylosing spondylitis using MRI images of the sacroiliac joint, according to an embodiment of the present invention.
[0038] Figure 2 The images shown are MRI images of the sacroiliac joint proposed in this embodiment of the invention, wherein (a) is a single-layer image of the oblique coronal T1 sequence, (b) is a single-layer image of the oblique coronal T2 sequence, and (c) is a schematic diagram of the eight quadrant division criteria of the SPARCC score.
[0039] Figure 3 This is a schematic diagram of the sacral and iliac bone segmentation network based on multi-scale feature fusion proposed in an embodiment of the present invention;
[0040] Figure 4 The following is a diagram of the sacroiliac joint proposed in an embodiment of the present invention: (a) is the original diagram of the sacroiliac joint; (b) is a segmentation and labeling mask diagram of the sacrum (light-colored highlighted area) and ilium (dark-colored highlighted area); (c) is the prediction result diagram of the segmentation network; and (d) is the location result diagram of the SPARCC score attention quadrant of the sacroiliac joint.
[0041] Figure 5 This is a diagram of the multi-head scoring prediction network structure proposed in an embodiment of the present invention;
[0042] Figure 6 The above is a visualization of the sacroiliac joint SPARCC score prediction results proposed in this embodiment of the invention. (a) is a single image of the T2-weighted sequence of oblique coronal MRI image, (b) is the eight-quadrant division result image, (c) is the eight-quadrant score prediction image, (d) is the brightness score prediction image, and (e) is the depth score prediction result image. Detailed Implementation
[0043] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments.
[0044] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0045] This invention proposes an automatic scoring method for ankylosing spondylitis in sacroiliac joint MRI images. This method follows the principle of "localization first, prediction later". First, a segmentation network that integrates multi-scale features is used to segment the sacrum and ilium. Prior knowledge is used to finely locate the sacroiliac joint space and the eight quadrants. After the fine localization results are fused with the original image, a multi-head prediction network is used to score the edema accumulation range, edema signal intensity and depth respectively. The scores and prediction results are summed to give the final SPARCC prediction score.
[0046] The specific implementation standard for the SPARCC score is to obtain a T2-weighted STIR sequence of the sacroiliac joint in an oblique coronal position. This sequence has 12 slices, and 6 consecutive slices with intact sacroiliac joint structure are selected for scoring (slices 4-9 are usually selected). Scoring is based on the presence or absence of bone marrow edema, and the intensity and depth of bone marrow edema signal. The scoring rules are as follows: Each sacroiliac joint on each side of each layer is divided into 4 quadrants, for a total of 8 quadrants bilaterally; Cumulative range scoring: 1 point is awarded for high signal bone marrow edema in each quadrant of the STIR sequence, and 0 points are awarded for none, with a total score of 48 points for bilateral sacroiliac joints across 6 layers; Edema signal intensity scoring: 1 point is awarded for lesions on each side of each layer if they have a bright signal, i.e., the signal intensity reaches or exceeds that of the presacral venous plexus at the same layer, and 0 points are awarded otherwise, with a total score of 12 points for bilateral sacroiliac joints across 6 layers; Edema depth scoring: 1 point is awarded for the depth of bone marrow edema lesions on each side of each layer if they extend more than 1 cm from the cartilage surface, and 0 points are awarded otherwise, with a total score of 12 points for bilateral sacroiliac joints across 6 layers; The scores of all layers are summed to obtain the SPARCC score, with a maximum total score of 72 points.
[0047] like Figure 1 As shown, this embodiment provides an automatic scoring method for ankylosing spondylitis using MRI images of the sacroiliac joint. The method includes: acquiring a dataset of sacroiliac joint MRI images; constructing a first-stage sacral and iliac bone segmentation network based on multi-scale feature fusion using the sacroiliac joint MRI dataset; locating the sacroiliac joint space using prior knowledge of the sacral and iliac bone positions obtained from the segmentation network, thereby obtaining the vertical line of the sacroiliac joint space and eight target quadrants for SPARCC scoring; training a second-stage multi-head scoring prediction network according to the SPARCC scoring criteria to obtain quadrant edema, brightness, and depth scores; summing the results of the multi-head scoring prediction network to obtain the final SPARCC score prediction result and providing visualization of the predicted score region. The specific steps are as follows:
[0048] S1. Construct a sacroiliac joint MRI image SPARCC score dataset for ankylosing spondylitis. Since there is currently no publicly available sacroiliac joint MRI image SPARCC score dataset, this embodiment uses data provided by Shanghai Changzheng Hospital to create the dataset.
[0049] The sacroiliac joint MRI image data includes axial T1-weighted image sequences, axial T2-weighted image sequences, oblique coronal T1-weighted image sequences, and oblique coronal T2-weighted image sequences. Based on the Spine Research Consortium of Canada (SPARCC) scoring criteria, the oblique coronal image sequences were selected to construct the dataset. Figure 2 (a) and Figure 2 As shown in (b). Figure 2 (a) is a single-frame image of the oblique coronal T1-weighted sequence. Figure 2(b) is a single-frame image of the oblique coronal T2-weighted sequence.
[0050] This dataset contains sacroiliac joint MRI images and SPARCC scoring data for ankylosing spondylitis. The data was annotated by professional physicians using 3D slicer software. Figure 2 (c) The diagram shows the eight quadrant division criteria for the SPARCC score. The annotations include the positions of the sacrum and iliac bone on the T1-weighted sequence image and the positions of the sacrum and iliac bone on the T2-weighted sequence. According to the SPARCC score criteria, the SPARCC score results of the sequence are added to the score table, as shown in Table 1.
[0051] Table 1
[0052]
[0053]
[0054] After the annotation is completed, the annotated data will be randomly divided into training, validation and test sets at the case level in a ratio of 6:2:2.
[0055] S2. Train a sacral-iliac bone segmentation network model based on multi-scale feature fusion;
[0056] S201. In the sacroiliac joint MRI image ankylosing spondylitis SPARCC score dataset, select the layers required for the SPARCC score to construct a sacral-iliac bone segmentation dataset. That is, establish a sacral-iliac bone segmentation dataset based on the sacroiliac joint SPARCC score dataset. This data consists of pairs of MRI single-layer grayscale images x and corresponding sacral and iliac bone segmentation masks y, D={x i ∈R 1×h×w ,y i ∈R 1×h×w}, i = 1, ..., n, where 0 represents the background, 1 represents the sacral region, and 2 represents the iliac region on the sacral-iliac segmentation mask.
[0057] S202. Construct a sacral-iliac bone segmentation network model based on multi-scale feature fusion. This segmentation network model consists of a feature extraction network and a feature pyramid pooling module. The input to the feature extraction network is a single-level image from the labeled SPARCC scoring dataset, such as... Figure 3 As shown.
[0058] In this embodiment, the feature extraction network selected is the ResNet50 residual network, i.e., a convolutional neural network ResNet50 is used to extract image features. To match the channels of the input image and extract features that focus more on small-scale information, the first convolutional layer of this network is modified to have 1 input channel, 64 output channels, and a 3×3 kernel size, and is randomly initialized. The unmodified network layer parameters are loaded from parameters pre-trained on the large-scale image dataset ImageNet.
[0059] The feature pyramid pooling module, referencing the feature scale feature fusion module of the PSPNet segmentation network, inputs the output features of the feature extractor into the pooling layer, and then passes them through four convolutional layers of different scales, with sizes of 1×1, 2×2, 3×3, and 6×6, to extract information at different scales. The features of the four scales are upsampled to the original image size, the features are concatenated, and then passed through the convolutional layer to finally output the segmentation prediction result.
[0060] The loss function L chosen for model training is the segmentation loss function L. seg With classification auxiliary loss function L aux The weighted sum is calculated using the following formula:
[0061] L=λ1L seg +λ1L aux (1)
[0062]
[0063] L aux =CrossEntropy(y,aux(f) θ (x))) (3)
[0064] Where λ1 and λ2 are the weights of the segmentation loss and classification auxiliary loss, respectively, CrossEntropy(·) is the cross-entropy function, and y is the true label of the sample. Let f be the predicted label of the model, aux(·) be the classification auxiliary loss function, and f be the predicted label of the model. θ (·) represents the classification model parameters, and the classification auxiliary loss L is... aux Calculate the loss between the output feature upsampling prediction result and the true value of the fourth residual module of the ResNet50 feature extraction network.
[0065] S203. Set hyperparameters and train a sacral-iliac bone segmentation network model based on multi-scale feature fusion;
[0066] The optimizer selected was the SGD optimization algorithm, with an initial learning rate of 0.01, an update momentum of 0.9, and a weight decay coefficient of 0.005. An exponential transformation learning rate adjustment strategy was adopted, with λ1 and λ2 set to 1.0 and 0.4, respectively. Data augmentation operations included resizing the images to 480 pixels, setting a random flip probability of 0.5, and normalizing the dataset. The dataset created in S201 was input into the segmentation network model. The SGD optimizer was used to perform gradient backpropagation on the loss function values to update the model parameters. The segmentation network was trained until the loss function converged, yielding the optimal network model.
[0067] S204. Performance evaluation of the segmentation network: The training set partitioned in S201 is fed into the saved optimal model to predict the positions of the sacrum and ilium. Evaluation metrics for the segmentation network are calculated; here, IoU and Dice are chosen as the evaluation metrics for the sacrum and ilium segmentation network.
[0068]
[0069]
[0070] Where T represents the ground truth mask of the sacrum and iliac bone locations in the dataset, P represents the prediction result of the multi-scale feature fusion network, and IoU and Dice metrics measure the similarity between the segmentation network's prediction and the ground truth.
[0071] The segmentation prediction results of the network on the test set are shown in Table 2. Compared with the UNet segmentation network, the multi-scale feature fusion network has higher segmentation accuracy. The labeled masks and prediction results of the sacrum and iliac bones of the multi-scale feature fusion network are visualized, as shown below. Figure 4 (a) Figure 4 (b) and Figure 4 As shown in (c).
[0072] Table 2
[0073]
[0074] S3. Based on prior knowledge of sacroiliac joint anatomy, locate the sacroiliac joint space and the eight quadrants of interest for the SPARCC score;
[0075] Prior anatomical knowledge of the sacroiliac joint includes the relative positions of the sacrum and ilium, such as the coordinates of the upper left corner S of the sacrum. L up and the S in the bottom right corner R dowm The coordinates are the points with the minimum and maximum sum of pixel coordinates of the sacrum, respectively. L up For distance S L upSimilarly, by calculating the coordinates of other points such as the nearest iliac bone pixel, the starting and ending points of the sacroiliac joint space can be determined. From the starting and ending points of the sacroiliac joint space, the expression for its perpendicular bisector can be calculated. Using the sacroiliac joint space and its perpendicular bisector to divide the sacrum and iliac bone, a segmentation map is obtained, showing the positions of eight quadrant regions on the quadrant mapping map. Details are as follows:
[0076] S301. In the anatomical structure of the sacroiliac joint, the sacroiliac joint space is the gap where the sacrum and ilium meet. On the segmented image of the sacrum and ilium, first locate the adjacent points of the sacrum and ilium. First consider the adjacent points of the left sacroiliac joint. Denote the set of pixel coordinates belonging to the sacrum as S = {(x,y)|Y(x,y) = 1}, and the set of pixel coordinates belonging to the left ilium as I = {(x,y)|Y(x,y) = 2}. First consider the adjacent points of the left sacroiliac joint; the starting point of the joint space on the image is S. L up with I L up The midpoint and the endpoint are S. L down with I L down The midpoint.
[0077] Among them, S L up I is the point with the smallest sum of pixel coordinates of the sacrum. L up For distance S L up Recent iliac bone pixels:
[0078] S L up =(x s ,y s = min(x+y) for (x,y)∈S (6)
[0079] I L up =(x i ,y i ) = min(||xx s || 2 +||yy s || 2 for(x,y)∈I (7)
[0080] S L down I L down The calculation method is the same.
[0081] Let A and B be the starting and ending points of the sacroiliac joint space, respectively. The coordinates of point B are similar.
[0082] S302. Based on the starting and ending points of the sacroiliac joint space obtained in S301, the slope of the approximate straight line of the sacroiliac joint space is obtained:
[0083]
[0084] The slope of the vertical bisector of the sacroiliac joint is The midpoint coordinates are
[0085] The calculation of the right sacroiliac joint space and the perpendicular bisector of the sacroiliac joint space is similar to that above.
[0086] S303. Based on the sacroiliac joint space obtained in S301 and the perpendicular bisector of the sacroiliac joint space obtained in S302, divide the sacrum and ilium into segmentation maps (output of S2 or segmentation masks annotated in the dataset), obtaining eight quadrant region pixels, denoted as quadrant mapping map Map∈R. 1×h×w In the Map, 1-8 represent the eight quadrants. For example... Figure 4 As shown in (d), the division results of the eight quadrants are marked with different colors.
[0087] S4. Train a multi-head rating prediction network model that can focus on quadrant accumulation, brightness and depth respectively;
[0088] The rating prediction network consists of a ResNet18 convolutional neural network feature extraction network and a multi-head prediction module. The multi-head prediction module comprises three prediction heads, each consisting of a fully connected layer (FC), an activation function (RELU), and another fully connected layer (FC). The three branches focus on quadrant cumulative edema signal, edema signal brightness, and edema signal depth, respectively. The three branches are trained progressively, freezing the parameters of the feature extraction network and updating the quadrant cumulative prediction head, edema signal brightness prediction head, and edema signal depth prediction head sequentially. The loss function is classification cross-entropy loss. Pairs of images and masks from the single-dimensional rating dataset are used as input to predict the doctor's labeled rating.
[0089] The single-dimensional scoring dataset is generated by cropping the original image and quadrant mapping map, resulting in datasets including quadrant cumulative dimension, edema signal brightness dimension, and edema signal depth dimension; each dimension dataset is in the format of image patch, mask patch, and score. The specific process is as follows:
[0090] S401. Create a dataset for the multi-head scoring prediction network. Based on the specific implementation standards of the SPARCC scoring system, which primarily focus on three dimensions of edema information—quadrant cumulative edema, and the brightness and depth of sacroiliac joint edema—a dataset D containing information for each of these three dimensions needs to be created. quadrant D bright and D depth .
[0091] Quadrant Cumulative Dataset D quadrantBased on the attention quadrants of the SPARCC scores output by S3, the original images and quadrant shape masks for each of the eight quadrants are cropped out. The score values s and D for each quadrant are obtained from the doctor's annotation table in the S1 dataset. quadrant ={concat(I q M q ), score|I q M q ∈I[Map={1,2,…,8}], M[Map]={1,2,…,8}}.
[0092] Sacroiliac joint edema brightness dataset D bright :D bright ={concat(I b M b ), score|I b M b ∈I[Map={{1,…,4},{5,…,8}],M[Map={{1,…,4},{5,…,8}]}.
[0093] Sacroiliac joint edema depth dataset D depth :D depth ={concat(I d M d ), score|I d M d ∈I[Map={{1,…,4},{5,…,8}],M[Map={{1,…,4},{5,…,8}]}.
[0094] Where I represents a single-slice image of the sacroiliac joint MRI, M represents a segmentation mask of the sacrum and iliac bone, and the score ∈ {0,1}.
[0095] S402, Construct a multi-head rating prediction network, such as Figure 5 As shown, the multi-head scoring prediction network consists of a feature extraction module and a multi-head prediction module.
[0096] In this embodiment, ResNet18 residual network is selected as the feature extraction network. The network input consists of image patch I_patch and mask patch M of the single-dimensional feature dataset created in S401. patch The splicing, i.e., conat(I patch M patch )∈R 2 ×ph×pw In this embodiment, the input layer of the ResNet18 network is changed to have 2 input channels, 64 output channels, and a 3×3 convolutional kernel size, while the remaining layers remain unchanged. The network uses pre-trained parameters from the large-scale image dataset ImageNet. Let f be the feature extraction network.θ The output of this feature extraction network is z∈R 1000 .
[0097] The multi-head prediction module consists of three classification heads, MltiH={H quadrant H bright H depth The system assigns three classification heads to the input datasets of quadrant accumulation, edema region depth, and edema region brightness, and performs score predictions on fixed attention dimensions. Classification head H i It consists of a fully connected layer (FC), an activation function (ReLU), and another fully connected layer (FC), with parameters initialized using the Kaiming method.
[0098] S403. Set hyperparameters and train the multi-head scoring prediction network. Since the feature extraction network is shared, this embodiment adopts a progressive training method, freezing the parameters of the feature extraction network that have loaded pre-trained weights, and only training the parameters of the input layer features and the multi-head prediction module.
[0099] First load D quanrant Data set and training of the quadrant cumulative branch H of the multi-head network quadrant The hyperparameters are set as follows: SGD optimizer is used, with a learning rate of 0.0001, update momentum of 0.99, weight decay of 0.005, and loss function is the cross-entropy loss between predicted and true ratings. Training is performed for 100 epochs, and the H-value that performs best on the validation set is selected. quadrant Parameter combination. The SGD optimizer is used to perform gradient backpropagation on the loss function to update the model parameters. The multi-head rating prediction network is then trained until the loss function converges, yielding the optimal parameter model.
[0100] Load D in the same way bright With D depth Dataset and training H bright With H depth The network parameters are selected, and the optimal parameter model is saved.
[0101] Accuracy evaluation of S404 multi-head score prediction network.
[0102] Load the test set of the three-dimensional information dataset obtained from S401 and the multi-head rating prediction network parameters obtained from S403, and calculate the accuracy of the three networks predicting ratings on the test set. The results are shown in Table 3, and the accuracy is significantly improved compared with the traditional method.
[0103] Table 3
[0104] SVM 0.663325 0.782846 0.7098540 DecisionTree 0.886560 0.8175182 0.76459 Ours 0.88560 0.88843 0.8883
[0105] S5. The summation of the scoring results yields the final SPARCC score prediction result, which is then visualized.
[0106] The final SPARCC score prediction result for a case is obtained by summing the predicted scores across all levels. This prediction result is then saved in a table and output. A highlighted area with a SPARCC score of 1 indicates the presence of edema in that area, and the visualized SPARCC score prediction result is output as follows:
[0107] On the test set segmented by pathological levels in S1, the sacral-iliac bone network obtained through multi-scale feature fusion in S2 yielded segmentation masks for the sacral and iliac bones. The top six levels were selected as the focus areas for SPARCC scoring, sorted in descending order by the number of sacral and iliac bone pixels on the masks. These six segmentation masks were then processed through S3 to locate the sacroiliac joint space and eight quadrants, yielding corresponding maps. Following the SPARCC scoring standards, mask-image pairs were extracted, similar to S401, and fed into the corresponding multi-head scoring prediction network to obtain predicted scores for the corresponding regions. The predicted scores were thresholded to 0 and 1, saved as a scoring table, completing the automatic prediction of the sacroiliac joint SPARCC score. In areas where the SPARCC score is 1, a highlighted area indicates the presence of edema signals. The scoring results are visualized as follows: Figure 6 As shown, Figure 6 (a) is a single image from a T2-weighted sequence of an oblique coronal MRI image. Figure 6 (b) shows the results of the eight quadrant division. Figure 6 (c) shows the score prediction map for the eight quadrants, with the highlighted areas indicating a predicted score of 1. Figure 6 (d) is the brightness score prediction map. Figure 6 (e) is a graph showing the depth rating prediction results.
[0108] To measure the accuracy of SPARCC score prediction at the patient level, the Spearmen correlation coefficient was selected as the evaluation index. The Spearmen correlation coefficient was 0.732 (p>0.005), indicating that there is a high correlation between the prediction results and the actual SPARCC score, and the prediction method can accurately provide the SPARCC score.
[0109] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method of automatic scoring of ankylosing spondylitis from sacroiliac joint MRI images, characterized in that, Specifically, the following steps are included: Obtain MRI images of the sacroiliac joint to be scored; The MRI image of the sacroiliac joint to be scored is input into a sacral and iliac bone recognition and segmentation model based on multi-scale feature fusion, which outputs a sacral and iliac bone segmentation map. The sacral and iliac bone recognition and segmentation model based on multi-scale feature fusion is trained on a first training set, which consists of MRI images of the sacroiliac joint with sacral and iliac bone segmentation mask annotations. The sacral and iliac bone segmentation network model based on multi-scale feature fusion consists of a feature extraction model and a feature pyramid pooling model. The feature extraction model is input to a single-layer image from the first training set, and uses a ResNet50 convolutional neural network to extract image features. The feature pyramid pooling model uses a segmentation network PSPNet that fuses multi-scale features. Based on the knowledge of the sacroiliac joint space and the sacral iliac bone segmentation diagram, a target quadrant mapping diagram is obtained: According to the knowledge of the sacroiliac joint space, the sacroiliac joint space is located, and the starting point and ending point of the sacroiliac joint space are determined. Through the starting point and ending point of the sacroiliac joint space, the vertical bisector of the sacroiliac joint space is calculated. Based on the sacroiliac joint space and the vertical bisector of the sacroiliac joint space, the sacral iliac bone segmentation diagram is divided, and the target quadrant mapping diagram is obtained. The target quadrant mapping is input into the multi-head scoring prediction model, which outputs the multi-head scoring prediction network results. The multi-head scoring prediction network results are summed to obtain the final SPARCC scoring prediction result. The multi-head scoring prediction model is trained on a second training set, which is a dataset including quadrant accumulation dimension, edema signal brightness dimension, and edema signal depth dimension. The multi-head scoring prediction model includes a feature extraction sub-model and a multi-head prediction sub-model. The input of the feature extraction sub-model is the concatenation of image patches and mask patches from a single-dimensional feature dataset, and a ResNet50 convolutional neural network is used to extract image features. The multi-head prediction sub-model consists of several prediction heads, each prediction head consisting of a fully connected layer (FC), an activation function (RELU), and a fully connected layer (FC).
2. The automatic scoring method for ankylosing spondylitis on sacroiliac joint MRI images as described in claim 1, characterized in that, Obtaining the first training set includes: The first training set was obtained by annotating the sacroiliac joint MRI image dataset using 3D slicer annotation software; The sacroiliac joint MRI image dataset consists of oblique coronal weighted image sequences.
3. The automatic scoring method for ankylosing spondylitis on sacroiliac joint MRI images as described in claim 1, characterized in that, The sacral and iliac bone recognition and segmentation model based on multi-scale feature fusion is obtained by training on the first training set and includes: Obtain the first loss function; Set hyperparameters, input the first training set into the sacral and iliac bone segmentation model to be trained based on multi-scale feature fusion, use the SGD optimizer to perform gradient backpropagation on the first loss function, update the parameters of the sacral and iliac bone segmentation network model to be trained based on multi-scale feature fusion, and train the sacral and iliac bone segmentation network model to be trained until the first loss function converges, and obtain the sacral and iliac bone recognition and segmentation network model based on multi-scale feature fusion.
4. The automatic scoring method for ankylosing spondylitis on sacroiliac joint MRI images as described in claim 3, characterized in that, The first loss function is a weighted sum of the segmentation loss function and the classification auxiliary loss function: in, The loss function chosen for model training. For the segmentation loss function, For classification auxiliary loss function, , These are the weights of the segmentation loss and the classification auxiliary loss, respectively. The cross-entropy function, For the true label of the sample, The predicted labels for the model, For classification auxiliary loss function, These are the parameters for the classification model.
5. The automatic scoring method for ankylosing spondylitis on sacroiliac joint MRI images as described in claim 1, characterized in that, Training the multi-head scoring prediction model based on the second training set includes: Obtain the second loss function, which is the classification cross-entropy loss function; Set hyperparameters, use a progressive training method to input the second training set into the multi-head rating prediction model, use the SGD optimizer to perform gradient backpropagation on the second loss function, update the parameters of the multi-head rating prediction model, train the multi-head rating prediction model, and obtain the trained multi-head rating prediction model.
6. The automatic scoring method for ankylosing spondylitis on sacroiliac joint MRI images as described in claim 5, characterized in that, The SPARCC scoring criteria focus on several dimensions of edema information, including quadrant cumulative edema, and the brightness and depth of sacroiliac joint edema.