Sacroiliac joint bone marrow edema prediction method based on deep learning

Abstract

The invention belongs to the field of medical image mode recognition, provides a sacroiliac joint bone marrow edema prediction method based on deep learning, and aims to accurately predict existence of sacroiliac joint bone marrow edema. The specific process comprises the following steps: firstly, collecting and sorting two groups of distinct magnetic resonance image data: one group is used as a healthy control group without bone marrow edema signs; and the other group clearly shows the pathological characteristics of sacroiliac joint bone marrow edema. Subsequently, we establish a region of interest focused on analysis through a precise image processing technique. And it is ensured that image data in all regions of interest reach consistent standards in the aspects of brightness, contrast and the like. Gaussian random noise is superposed in the standardized region of interest, image interference in a real scene is simulated, and model learning is promoted to be more robust feature representation. A visual and reliable reference basis is provided for clinicians, and more accurate illness state assessment is facilitated.

Description

Technical Field

[0001] The present invention belongs to the field of medical image pattern recognition, and proposes a deep learning-based sacroiliac joint bone marrow edema prediction method for patients with axial spondyloarthritis. Background Art

[0002] Sacroiliac joint bone marrow edema plays an important role in axial spondyloarthritis. In magnetic resonance imaging, sacroiliac joint bone marrow edema appears as a high signal, and its signal intensity is similar to that of blood flow and spinal cord. According to the 2009 classification criteria for axial spondyloarthritis, multiple bone marrow edema on a single level in sacroiliac joint magnetic resonance imaging, or a single bone marrow edema involving multiple levels, is used as one of the imaging prediction criteria. Bone marrow edema is often accompanied by other inflammatory changes, such as synovitis and tendinitis, which together constitute the active manifestation of the disease. The inflammatory stimulation caused by bone marrow edema is one of the main causes of local pain in patients. The pain may manifest as a fixed pain point and is often accompanied by limited joint movement. Due to inflammatory stimulation and articular cartilage damage, patients may experience symptoms such as joint stiffness and limited movement. These symptoms are particularly obvious when getting up in the morning or after a long period of inactivity, and gradually alleviate with the increase in activity. The presence of bone marrow edema may indicate further progression of the disease. In axial spondyloarthritis, persistent bone marrow edema may lead to further destruction of the joint structure, such as bone erosion and bone sclerosis, and then form serious complications such as bone ankylosis. By monitoring changes in bone marrow edema, the patient's response to treatment can be evaluated. If bone marrow edema is significantly reduced or disappears after treatment, it usually means that the treatment is effective; otherwise, the treatment plan may need to be adjusted. When formulating a treatment plan, doctors need to comprehensively consider factors such as the patient's clinical symptoms, imaging manifestations, and disease activity. As one of the important signs of disease activity, bone marrow edema is of great significance for formulating targeted treatment plans. During the treatment process, if the patient's clinical symptoms or imaging manifestations change (such as worsening or relief of bone marrow edema), the doctor needs to adjust the treatment plan in time according to these changes to ensure maximum treatment effect.

[0003] Therefore, the present invention proposes a deep learning-based prediction method for sacroiliac joint bone marrow edema, which has a high accuracy, sensitivity, and low computational cost multi-classification recognition method for ECG abnormalities. The automatic detection method of this method is based on algorithms and big data analysis, which can reduce manual errors, improve the accuracy and consistency of predictions, and avoid missed diagnoses or misdiagnoses caused by doctors' lack of experience or lack of concentration. It is an effective supplementary way for clinicians to predict sacroiliac joint bone marrow edema in patients with axial spondyloarthritis, and is expected to provide a new means for clinical intelligent medical care. Summary of the invention

[0004] The present invention aims to solve the problems described in the background technology and provide a method for predicting sacroiliac joint bone marrow edema based on deep learning.

[0005] A deep learning-based method for predicting sacroiliac joint bone marrow edema comprises the following steps:

[0006] Step 1: Strictly screen MRI image data from multiple medical centers or databases. The MRI image data includes each patient's age, gender, medical history information, and specific parameters of the MRI scan. Check and eliminate low-quality images caused by motion artifacts and interference from metal implants. Obtain magnetic resonance images of a group of healthy controls and a group of patients with sacroiliac joint bone marrow edema.

[0007] Step 2: Use image segmentation algorithms to automatically identify the general outlines of the sacrum and ilium, followed by manual fine-tuning by doctors. Use the multi-planar reconstruction function of MRI to observe the ROI from multiple angles, establish regions of interest, and exclude areas other than the sacrum and ilium.

[0008] Step 3: adjust the grayscale values ​​of all images to the same range, and use a standard template or a statistically based spatial normalization method to adjust the ROI to a uniform spatial position and size;

[0009] Step 4: Generate Gaussian random noise of appropriate intensity according to the noise level of the image, and superimpose the generated Gaussian noise on the ROI in a certain proportion in the region of interest;

[0010] Step 5: Mark the images of patients with bone marrow edema as 1 and the images of healthy subjects as 0;

[0011] Step 6: Divide the data sets of all subjects into training sets and test sets at a ratio of 7:3, and divide the data sets into 5 subsets. Select 4 subsets as training sets each time, and the remaining 1 subset as validation set. Repeat this process 5 times, selecting a different subset as validation set each time to comprehensively evaluate the performance of the model.

[0012] Step 7: Use the ResNet network to perform grid search to optimize the model’s learning rate, batch size, and number of iterations to improve the model’s training efficiency and performance; use the training set data to train the model while monitoring the performance indicators on the validation set;

[0013] Step 8: After the model training is completed, use Grad-CAM technology to generate a heat map to intuitively display the areas that the model pays most attention to in the classification decision-making process; perform clinical interpretation on the highlighted areas in the heat map, analyze their potential connection with bone marrow edema, and use gradient-weighted class activation mapping to analyze the areas that contribute most to the classification.

[0014] Preferably, in step 1, a short TI inversion recovery sequence scanning method is used for collecting the subject data.

[0015] Preferably, the standardization method is as follows:

[0016]

[0017] Where M represents the signal of voxel point i of the magnetic resonance image, min represents the minimum value, α% represents the first α percent of the signal, and N represents the number of all voxel points. is the signal mean representing the fat area, is the mean signal value of the spine area.

[0018] Based on the content of the present invention, at least the following technical effects are included:

[0019] The deep learning model can automatically learn and identify complex features in sacroiliac joint MRI images through training, especially the characteristic imaging manifestation of bone marrow edema. These features may be difficult to detect or there may be subjective differences in traditional manual reading, while the deep learning model can provide a more objective and accurate assessment.

[0020] The deep learning-based prediction method has high sensitivity and specificity in detecting sacroiliac joint bone marrow edema. For example, in an external test set, the sensitivity and specificity for inflammatory changes in a study reached 88% and 71%, respectively, and the sensitivity and specificity for structural changes were 85% and 78%, respectively. This shows that the method can effectively distinguish cases with bone marrow edema from normal or other lesions.

[0021] Deep learning models can automatically process and analyze MRI images, greatly reducing the time and labor intensity of manual reading. For clinicians, this means that they can obtain diagnosis results faster and formulate treatment plans earlier.

[0022] MRI is an important tool for early diagnosis of axSpA, and deep learning models can more accurately identify early inflammatory changes, such as bone marrow edema. This helps to intervene and treat the disease in the early stages, thereby improving patient outcomes.

[0023] The prediction results of the deep learning model can provide clinicians with objective and reliable diagnostic basis. Through in-depth analysis of patients' MRI images, the deep learning model can also provide doctors with more information about the severity, activity and progression trend of the disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0025] Figure 1 A flow chart of a method for predicting sacroiliac joint bone marrow edema of the present invention;

[0026] Figure 2 Schematic diagram of the sacroiliac joint used in the present invention. (A) is a schematic diagram of the entire joint, (B) is a schematic diagram of the sacrum (green) and ilium (red) and bone marrow edema (yellow); (C) is a schematic diagram of the region of interest, where the image signal of the blue area on both sides of the subject is extracted; (D) is a schematic diagram of the region of interest after standardization and with random noise added;

[0027] Figure 3 Schematic diagram of the intelligent model proposed in the present invention; (A) is a schematic diagram of image normalization and adding Gaussian noise; (B) is a schematic diagram of the deep learning prediction model for bone marrow edema;

[0028] Figure 4 This is a schematic diagram of the ResNet network module;

[0029] Figure 5 is the receiver operating curve using 5-fold cross validation on the example data;

[0030] Figure 6 The contribution of gradient-weighted class activation mapping to classification. DETAILED DESCRIPTION

[0031] In order to make the purpose and technical implementation of the present invention clearer, embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0032] In the description of the present invention, unless otherwise specified and limited, it should be noted that terms such as "signal", "image", "framework", "weight", "connection", "model", "training set", "validation set" and "test set" should be understood in a broad sense, and those of ordinary skill in the art can understand the specific meanings of the above terms according to specific circumstances.

[0033] like Figure 1-4 As shown, the present invention adopts the following technical solutions:

[0034] Step 1: Obtain magnetic resonance images of sacroiliac joint bone marrow edema in a group of healthy controls (HC) and a group of patients with axial spondyloarthritis; for the data collection method of the subjects, a 3T scanner is required for data collection, and T1 and T2 weighted images are collected, with TR of 544ms, TE of 11ms, flip angle of 140°, bandwidth of 326Hz, field of view of 250×250mm2, layer thickness of 2mm, layer spacing of 0.2mm, scanning matrix of 448×448, number of layers of 37, and fat suppression scanning method.

[0035] Step 2: Establish the region of interest and exclude the areas other than the sacrum and ilium. When establishing the region of interest, three experienced radiologists performed image segmentation on all subjects, retaining the areas where the sacrum, ilium and joints are located, and took the union of the three doctors' segmentation on the entire data set.

[0036] Step 3: Standardize the region of interest. Standardization includes setting the region of interest in each layer of each subject and extracting the signal of the region of interest. After the signal of each pixel in the region of interest is extracted, the logarithm is taken and multiplied by 10. 4 To improve the effectiveness of calculation. In addition, in order to remove the influence of signal acquisition between different subjects, the invention proposes a new magnetic resonance signal standardization method, the specific standardization method is as follows:

[0037]

[0038] Where M represents the signal of voxel point i of the magnetic resonance image, min represents the minimum value, α% represents the first percent of the signal, which is 50% here, and N represents the number of all voxel points. is the signal mean representing the fat area, is the mean signal value of the spine area.

[0039] Step 4: Superimpose Gaussian random noise on the region of interest. The noise follows a Gaussian distribution with a mean of 0 and a variance of 0.3.

[0040] Step 5: Mark the images of patients with bone marrow edema as 1 and the images of healthy subjects as 0;

[0041] Step 6: Divide the data sets of all subjects into training set and test set according to the ratio of 7:3, and use 5-fold cross validation to evaluate the performance of the model;

[0042] Step 7: Use the ResNet network to train the model. The model loss function uses the cross entropy loss function. ResNet contains five convolution stages. The first stage contains 64 7×7 convolution kernels, and the second to fifth convolution stages contain 3×3 convolution blocks. The model structure is shown in the attached figure. Figure 5 As shown. The fully connected layer consists of a 512-dimensional vector, and the output layer is determined by the Sigmoid function. Specifically, the Sigmoid function is determined by the following probabilities,

[0043]

[0044] Among them, x is the 512-dimensional feature of each subject input, and y is the label to be predicted. When the probability is greater than 0.5, the predicted label is 1, and when the probability is less than 0.5, the predicted label is 0. w and b are the parameters for linearizing the features, which can be obtained using the maximum likelihood probability.

[0045]

[0046] That is, the greater the probability that each sample belongs to its true label, the better. According to convex optimization theory, using the gradient descent method, w and b can be solved.

[0047] Let γ=(w;b), Then T x+b can be abbreviated as

[0048] Re-order

[0049]

[0050] but

[0051]

[0052] is equivalent to,

[0053]

[0054] In order to obtain γ,

[0055]

[0056] The accuracy, precision, recall, specificity and score A were used to evaluate the model classification effect; the specific calculation method is as follows:

[0057]

[0058] Step 8: Use gradient-weighted class activation mapping to explore the areas that contribute most to the classification.

[0059] The present application provides a deep learning-based prediction method for sacroiliac joint bone marrow edema, and is tested on a 815-layer dataset consisting of sacroiliac joints from 79 subjects (60 patients with axial spondyloarthritis and sacroiliac joint bone marrow edema, and 19 healthy controls);

[0060] The test on the data includes the following steps:

[0061] Step 1: Magnetic resonance images of sacroiliac joint bone marrow edema were obtained from 19 healthy controls (HC) and a group of patients with axial spondyloarthritis.

[0062] Step 2: Establish the region of interest and exclude the area other than the sacrum and ilium;

[0063] Step 3: Standardize the region of interest. In this test, the number of image layers including the sacrum and ilium of the subjects ranges from 18 to 25, so 23 layers are selected as the final input image for each subject in this test.

[0064] Step 4: Superimpose Gaussian random noise on the region of interest; the noise follows a Gaussian distribution with a mean of 0 and a variance of 0.3.

[0065] Step 5: Mark the images of patients with bone marrow edema as 1 and the images of healthy subjects as 0;

[0066] Step 6: Divide the data sets of all subjects into training set and test set according to the ratio of 7:3, and use 5-fold cross validation to evaluate the performance of the model;

[0067] Step 7: Use the ResNet network to train the model; use accuracy, precision, recall, specificity, score A and ROC curve to evaluate the performance of the model. The classification results for each layer are as follows:

[0068]

[0069] The results of five-fold cross validation are as follows Figure 5 shown.

[0070] The classification results for each subject are as follows:

[0071]

[0072] Step 8: Use gradient weighted class activation mapping to explore the area that contributes most to classification. The result is shown in the figure below. Figure 6 As shown, the redder the color, the greater the contribution to the classification.

Claims

1. A deep learning-based method for predicting sacroiliac joint bone marrow edema, characterized in that: The steps include: Step 1: Strictly screen MRI image data from multiple medical centers or databases. The MRI image data includes each patient's age, gender, medical history information, and specific parameters of the MRI scan. Check and eliminate low-quality images caused by motion artifacts and interference from metal implants. Obtain magnetic resonance images of a group of healthy controls and a group of patients with sacroiliac joint bone marrow edema. Step 2: Use image segmentation algorithms to automatically identify the general outlines of the sacrum and ilium, followed by manual fine-tuning by doctors. Use the multi-planar reconstruction function of MRI to observe the ROI from multiple angles, establish the region of interest, and exclude areas other than the sacrum and ilium. Step 3: adjust the grayscale values ​​of all images to the same range, and use a standard template or a statistically based spatial normalization method to adjust the ROI to a uniform spatial position and size; Step 4: Generate Gaussian random noise of appropriate intensity according to the noise level of the image, and superimpose the generated Gaussian noise on the ROI in a certain proportion in the region of interest; Step 5: Mark the images of patients with bone marrow edema as 1 and the images of healthy subjects as 0; Step 6: Divide the data sets of all subjects into training sets and test sets at a ratio of 7:3, and divide the data sets into 5 subsets. Select 4 subsets as training sets each time, and the remaining 1 subset as validation set. Repeat this process 5 times, selecting a different subset as validation set each time to comprehensively evaluate the performance of the model. Step 7: Use the ResNet network to perform grid search to optimize the model’s learning rate, batch size, and number of iterations to improve the model’s training efficiency and performance; use the training set data to train the model while monitoring the performance indicators on the validation set; Step 8: After the model training is completed, use Grad-CAM technology to generate a heat map to intuitively display the areas that the model pays most attention to in the classification decision-making process; perform clinical interpretation on the highlighted areas in the heat map, analyze their potential connection with bone marrow edema, and use gradient-weighted class activation mapping to analyze the areas that contribute most to the classification.

2. The deep learning-based sacroiliac joint bone marrow edema prediction method according to claim 1, characterized in that: In step 1, a short TI inversion recovery sequence scanning method is used for collecting the subject data.

3. The deep learning-based sacroiliac joint bone marrow edema prediction method according to claim 1, characterized in that: The standardization method is as follows: Where M represents the signal of voxel point i of the magnetic resonance image, min represents the minimum value, α% represents the first α percent of the signal, and N represents the number of all voxel points. is the signal mean representing the fat area, is the mean signal value of the spine area.

Images