Intelligent reading system and method for sacroiliac joint magnetic resonance imaging

Abstract

The present invention discloses a method for intelligent interpretation of sacroiliac joint magnetic resonance images, characterized by comprising the following steps: step 1, obtaining a medical image dataset; step 2, constructing a ROI automatic segmentation model, wherein the ROI automatic segmentation model is a UNet3D network; step 3, using the medical image as input, automatically extracting the ROI using the ROI automatic segmentation model, and extracting radiomic features on the ROI; step 4, using the radiomic features as input, obtaining new radiomic features using a feature selection algorithm based on harmony search; step 5, inputting the new radiomic features into a classifier E for forward reasoning to obtain a structural classification result. This method utilizes deep learning technology to automatically extract ROIs from MRI images, extracts radiomic features based on the ROIs, and selects the optimal radiomic features using a heuristic algorithm. Finally, the classifier classifies the structural lesions.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to an intelligent reading system and method for sacroiliac joint magnetic resonance imaging. Background Art

[0002] Axial spondyloarthritis (axSpA) is a chronic autoimmune disease that primarily affects the sacroiliac joints and spine. Studies have shown that compared with patients diagnosed early, patients with delayed diagnosis of axSpA have poorer disease specificity, reduced efficacy of medications, and increased rates of work disability, which imposes multiple burdens on patients and society. Therefore, it is crucial to treat axSpA patients before irreversible changes occur. Studies have shown that structural lesions have a guiding significance in clinical diagnosis. Therefore, it is possible to attempt to detect structural lesions in the SIJ on MRI.

[0003] Commonly used imaging examinations for axSpA mainly include computed tomography (CT) and magnetic resonance imaging (MRI). CT has a lower resolution, and MRI is more sensitive to the evaluation of early inflammation and soft tissue structure. However, in the diagnosis of structural lesions, CT is superior to MRI to a certain extent, and structural lesions can be used as an important supplementary condition to determine whether a patient is diagnosed with axSpA. However, due to CT radiation, especially the sacroiliac joint is located in the pelvis, there may be potential risks of radiation exposure to the reproductive system, especially for people preparing for pregnancy. On the other hand, whether it is CT or MRI, the evaluation of images is a time-consuming and laborious process. Especially when evaluating a large number of image sequences, doctors need to invest considerable time and energy. In addition, to correctly diagnose and interpret images, experienced doctors are required, which places high demands on the professional quality of doctors.

[0004] Region of Interest (ROI) is widely used in medical image analysis. ROIs designate specific areas within medical images, which may include disease lesions, organ structures, anatomical features, and more. However, manual ROI delineation is time-consuming and labor-intensive, and suffers from low accuracy, hindering clinical implementation to some extent.

[0005] A classifier is a model in machine learning used to solve classification problems. In supervised learning, classification involves predicting the corresponding category or label given a set of input features. The classifier's task is to learn the mapping from input features to output categories and to make accurate classification predictions for new, unseen samples. The goal of a classifier is to classify samples as accurately as possible into one of the known categories, thereby automatically classifying new data. However, since most features may contribute little to the classification, using a classifier alone can result in poor classification results.

[0006] Therefore, there is an urgent need for a computer-assisted classification tool that can diagnose structural lesions in the sacroiliac joint using only MRI to meet radiation safety requirements; implement technical means of automatic segmentation of the sacroiliac joint to improve the accuracy of ROI delineation and reduce time consumption; and use a deeply verified feature selection algorithm to solve the problem of how to find the key features to identify the current structural lesions, ultimately helping doctors to diagnose and evaluate the patient's disease more accurately, reduce workload, and reduce the risk of missed diagnosis due to fatigue. Summary of the Invention

[0007] In response to the shortcomings of the existing technology, the present invention proposes an intelligent reading system and method for sacroiliac joint magnetic resonance imaging, which uses deep learning technology to automatically extract ROIs from MRI images, extracts radiomic features based on the ROIs, and screens the optimal radiomic features through a heuristic algorithm. Finally, a classifier classifies structural lesions.

[0008] In order to solve the above technical problems, the technical solution of the present invention is:

[0009] A method for intelligent interpretation of sacroiliac joint magnetic resonance images comprises the following steps:

[0010] Step 1: Obtain a medical image dataset;

[0011] Step 2: Build an ROI automatic segmentation model, where the ROI automatic segmentation model is a UNet3D network;

[0012] Step 3: Take the medical image as input, automatically extract the ROI through the ROI automatic segmentation model, and extract the radiomics features x on the ROI n ;

[0013] Step 4: Transform the radiomics features x n As input, a new radiomics feature h is obtained through a feature selection algorithm based on harmony search. n ;

[0014] Step 5: h n Input the classifier E for forward reasoning and get h nThe structural classification result y n .

[0015] Preferably, in step 3, the method for automatically extracting ROI by using the ROI automatic segmentation model is:

[0016] Step 3.1: Read a single medical image I from the medical image dataset m ;

[0017] Step 3.2: Extract the feature map through the encoder S and downsample the feature map to obtain a downsampled feature map;

[0018] Step 3.3, the downsampled feature map is subjected to a transposed convolution operation to obtain an upsampled feature map, and the upsampled feature map is concatenated with the feature map extracted by the encoder to obtain a fused feature map;

[0019] Step 3.4: Repeat steps 3.2 and 3.3 with the fused feature map as input, and use the pyradiomics library to extract multiple radiomics features x on the final feature map. n .

[0020] Preferably, the specific method of step 3.2 is:

[0021] Step 3.2.1, I m The input encoder S is subjected to the i-th convolution operation to obtain the feature map C i ;

[0022] Step 3.2.2, C i After the i-th layer pooling operation, the size is halved to obtain the downsampled feature map P i ;

[0023] Step 3.2.3: The downsampled feature map undergoes the i+1th layer of convolution to obtain a new feature map C. i+1 ;

[0024] Step 3.2.4: The new feature map is then pooled in the i+1th layer, and its size is halved to obtain a smaller downsampled feature map P. i+1 ;

[0025] Step 3.2.5, repeat steps 3.2.3 and 3.2.4 to get the deepest feature map C deep And the smallest downsampled feature map P min .

[0026] Preferably, the specific method of step 3.3 is:

[0027] Step 3.3.1, P min After the i-th layer transposed convolution operation, the size is restored and the upsampled feature map UP is obtained.i ;

[0028] Step 3.3.2, UP i and the feature map C of the corresponding encoder layer deep Perform splicing to obtain the fused feature map Merge1.

[0029] Step 3.3.3, Merge1 undergoes the mth layer of convolution operation to obtain a new feature map C m ;

[0030] Step 3.3.4, C m After the mth layer of transposed convolution operation, the size is restored again to obtain a larger upsampled feature map UP m ;

[0031] Step 3.3.5, UP m and the feature map C of the corresponding encoder layer deep-1 Splice and get the fusion feature map Merge m ;

[0032] Step 3.3.6, repeat steps 3.3.3, 3.3.4, 3.3.5 to get the final fusion feature map Merge end And the final feature map C end .

[0033] Preferably, the specific method of step 4 is:

[0034] Step 4.1. Initialize the parameters HMS, HMCR, PAR, K, and N, where the number of decision variables N represents the number of features in the original feature set in the feature selection problem.

[0035] Step 4.2: Initialize the harmony memory library in the following format:

[0036]

[0037] Step 4.3: Use the random number to determine whether to search the current optimal harmony vector from the memory bank or randomly generate a variable with a value of 0 or 1. If the search is successful, continue to use the random number to determine whether to mutate until a complete harmony vector is generated.

[0038] Step 4.4: Determine whether the fitness of the new harmony vector is better than the worst harmony vector in the current memory bank according to the fitness function; if so, replace the worst harmony vector in the memory bank with the new harmony vector;

[0039] Step 4.5: Repeat steps 4.3 and 4.4 until the iteration conditions are met and output the final optimal solution set h n , that is, new imaging omics features.

[0040] Preferably, the fitness function expression in step 4.4 is as follows:

[0041] fitness=α×err+(1-AUC)×β+n fprob ×γ

[0042] Where err represents the error; AUC represents the area under the ROC curve, which is used to measure the performance of the binary classification model; n fprob Represents the probability of feature selection; α is used to control the weight of error err in the fitness function; β is used to control the weight of AUC in the fitness function; γ is used to control the probability of feature selection n fprob Weight in the fitness function.

[0043] The present invention also discloses an intelligent sacroiliac joint magnetic resonance imaging reading system, which includes an ROI automatic segmentation model module and a feature selection classification module.

[0044] Preferably, the ROI automatic segmentation model module includes an encoder S and a decoder D.

[0045] The encoder S is composed of multiple double convolutional layers and maximum pooling layers, and its input is a single sequence medical image I m , the output is the corresponding high-dimensional feature map;

[0046] The decoder D consists of multiple deconvolution layers and double convolution layers, and its input is the radiomics feature data x n , the output is a new radiomics feature h restored to the same size as the original input image n .

[0047] Preferably, the feature selection classification module adopts a classifier E, and the input of the classifier E is the new radiomics feature h n , and get h n The structural classification result y n .

[0048] The present invention has the following characteristics and beneficial effects:

[0049] 1) This automated diagnostic tool for structural sacroiliac joint lesions, developed in this study, detects multiple structural sacroiliac joint lesions, such as bone erosion and joint space lesions, solely on SIJ MRI. This tool meets radiation safety requirements, making it particularly useful for pregnant women. This tool can directly identify structural lesions on MRI, rather than requiring doctors to refer to CT scans.

[0050] 2) In the present invention, the use of automated computer-aided diagnosis tools can optimize the diagnosis process to a certain extent, improve diagnosis efficiency, and reduce diagnosis time.

[0051] 3) In the present invention, the constructed framework shows great potential in the diagnosis of structural lesions at the sacroiliac joint. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a functional block diagram of an embodiment of an intelligent reading system and method for sacroiliac joint magnetic resonance imaging according to the present invention; DETAILED DESCRIPTION

[0054] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0055] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0056] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0057] The present invention provides a method for intelligent interpretation of sacroiliac joint magnetic resonance imaging. Figure 1 As shown, it includes the following steps:

[0058] Step 1: Construct a UNet3D network with a moderate number of parameters and a widely verified network structure to perform the automatic segmentation task of the sacroiliac joint. In this example, the model is built based on Python (version 3.10.6) and the deep learning library PyTorch (version 2.0.1), and the learning rate used during training is 1×10 -5 , Adam optimizer is used, and weight decay is set to 0.01, random number seed is set to 2024, batch size is set to 4, total epoch number is set to 800, and the hardware parameters used for model training and inference are CPU Intel Core i7-13700K and GPU NVIDIA GeForce RTX 4090;

[0059] Step 2: Apply the UNet3D network as the ROI automatic segmentation model to automatically extract the ROI, and further extract multiple radiomics features x on the ROI n , the specific steps are as follows:

[0060] Step 2.1: Read a single medical image I m ;

[0061] Step 2.2: I m The input encoder S is subjected to the i-th convolution operation to obtain the feature map C i In this example, the medical image I m The MRI image data of patients who underwent 3.0T SIJ-MRI examination, I m The number is 638;

[0062] Step 2.3: C i After the i-th layer pooling operation, the size is halved to obtain the downsampled feature map P i ;

[0063] Step 2.4: The downsampled feature map undergoes the i+1th layer of convolution operation to obtain a new feature map C i+1 ;

[0064] Step 2.5: The new feature map is then pooled in layer i+1, and its size is halved to obtain a smaller downsampled feature map P. i+1 ;

[0065] Step 2.6: Repeat steps 2.4 and 2.5 to get the deepest feature map C deep And the smallest downsampled feature map P min ; In this example, deep is 4, min is 4, and the initial value of i is 1;

[0066] Step 2.7: P minAfter the i-th layer transposed convolution operation, the size is restored and the upsampled feature map UP is obtained. i ;

[0067] Step 2.8: UP i and the feature map C of the corresponding encoder layer deep Perform splicing to obtain the fused feature map Merge1.

[0068] Step 2.9: Merge1 undergoes the mth convolution operation to obtain a new feature map C m ;

[0069] Step 2.10: C m After the mth layer of transposed convolution operation, the size is restored again to obtain a larger upsampled feature map UP m ;

[0070] Step 2.11: UP m and the feature map C of the corresponding encoder layer deep-1 Splice and get the fusion feature map Merge m ;

[0071] Step 2.12: Repeat steps 2.9-2.11 to get the final fusion feature map Merge end And the final feature map C end .

[0072] In this example, end is 9 and the initial value of m is 1;

[0073] Step 2.13: Using the pyradiomics library in C end Extract multiple radiomics features x n

[0074] In this example, n is 960.

[0075] Step 3: Input radiomics features x n , and use a feature selection algorithm based on harmony search to select the new radiomics feature h that is most suitable for the classifier E through a heuristic global search algorithm n , the specific steps are as follows:

[0076] Step 3.1: Initialize the parameters HMS, HMCR, PAR, K, and N, where HMS represents the size of the harmony memory, HMCR represents the minimum fitness limit of the harmony memory, PAR represents the mutation rate, K represents the number of random components used to generate the harmony vector, and the number of decision variables N represents the number of features in the original feature set in the feature selection problem.

[0077] Step 3.2: Initialize the harmony memory library in the following format:

[0078]

[0079] Step 3.3: Use the random number to determine whether to search the current optimal harmony vector from the memory bank or randomly generate a variable with a value of 0 or 1. If the search is successful, continue to use the random number to determine whether to mutate until a complete harmony vector is generated;

[0080] Specifically, the random number is used to determine whether to search the current optimal harmony vector from the memory bank or randomly generate a variable with a value of 0 or 1. If a new harmony vector is generated, which comes from the harmony memory bank or is randomly generated and meets the constraints of the problem, it means that the search is successful. Then the random number is continued to be used to determine whether to mutate. Mutation means changing the values ​​of one or more dimensions of the generated harmony vector until a complete harmony vector is generated.

[0081] Step 3.4: Determine whether the fitness of the new harmony vector is better than the worst harmony vector in the current memory bank according to the fitness function. If so, replace the worst harmony vector in the memory bank with the new harmony vector. The fitness function is:

[0082] fitness=α×err+(1-AUC)×β+n fprob ×γ;

[0083] In this example, α is 0.8, β is 0.15, and γ is 0.05;

[0084] Step 3.5: Repeat steps 3.3 and 3.4 until the iteration conditions are met and output the final optimal solution set h n , that is, new imaging omics features.

[0085] Step 4: Place h n Input the classifier E for forward reasoning and get h n Structural lesion prediction results n ; In this example, the learners E are: MLP Classifier (MLP), Ada Boost Classifier (Ada), RandomForest (RF), Logistic Regression (LR) and GradientBoostingClassifier (GBDT).

[0086] This embodiment also provides an intelligent reading system for sacroiliac joint magnetic resonance imaging, which mainly includes an ROI automatic segmentation model module and a feature selection and classification module:

[0087] (1) The ROI automatic segmentation model module includes an encoder S and a decoder D.

[0088] (2) The encoder S consists of 4 double convolutional layers and 3 maximum pooling layers, and its input is a single sequence medical image I m , the output is the corresponding high-dimensional feature map;

[0089] (3) The decoder D consists of 4 transposed convolution layers and 4 double convolution layers, and its input is the radiomics feature data x n , the output is a new radiomics feature h restored to the same size as the original input image n The encoder and decoder are connected in series by multiple double convolutional layers - max pooling layers.

[0090] (4) The feature selection classification module includes a classifier E and uses five independent classifiers E1 to E5;

[0091] The input of the classifier E is the new radiomics feature h n , and get h n The structural classification result y n .

[0092] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions, and variations of these embodiments, including components, without departing from the principles and spirit of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for intelligent interpretation of sacroiliac joint magnetic resonance imaging, characterized in that: The steps include: Step 1: Obtain a medical image dataset; Step 2: Build an ROI automatic segmentation model, where the ROI automatic segmentation model is a UNet3D network; Step 3: Take the medical image as input, automatically extract the ROI through the ROI automatic segmentation model, and extract the radiomics features x on the ROI n ; In step 3, the method for automatically extracting ROI through the ROI automatic segmentation model is: Step 3.1: Read a single medical image I from the medical image dataset m ; Step 3.2: Extract the feature map through the encoder S and downsample the feature map to obtain a downsampled feature map; Step 3.2.1, I m The input encoder S is subjected to the i-th convolution operation to obtain the feature map C i ; Step 3.2.2, C i After the i-th layer pooling operation, the size is halved to obtain the downsampled feature map P i ; Step 3.2.3: The downsampled feature map undergoes the i+1th layer of convolution to obtain a new feature map C. i+1 ; Step 3.2.4: The new feature map is then pooled in the i+1th layer, and its size is halved to obtain a smaller downsampled feature map P. i+1 ; Step 3.2.5, repeat steps 3.2.3 and 3.2.4 to get the deepest feature map C deep And the smallest downsampled feature map P min ; Step 3.3, the downsampled feature map is subjected to a transposed convolution operation to obtain an upsampled feature map, and the upsampled feature map is concatenated with the feature map extracted by the encoder to obtain a fused feature map; Step 3.3.1, P min After the i-th layer transposed convolution operation, the size is restored and the upsampled feature map UP is obtained. i ; Step 3.3.2, UP i and the feature map C of the corresponding encoder layer deep Splice and get the fusion feature map Merge 1； Step 3.3.3, Merge1 undergoes the mth layer of convolution operation to obtain a new feature map C m ; Step 3.3.4, C m After the mth layer of transposed convolution operation, the size is restored again to obtain a larger upsampled feature map UP m ; Step 3.3.5, UP m and the feature map C of the corresponding encoder layer deep-1 Splice and get the fusion feature map Merge m ; Step 3.3.6, repeat steps 3.3.3-3.3.5 to get the final fusion feature map Merge end And the final feature map C end ; Step 3.4: Repeat steps 3.2 and 3.3 with the fused feature map as input, and extract multiple radiomics features x on the final feature map. n ; Step 4: Transform the radiomics features x n As input, a new radiomics feature h is obtained through a feature selection algorithm based on harmony search. n ; Step 5: h n Input the classifier E for forward reasoning and get h n The structural classification result y n .

2. The intelligent interpretation method of sacroiliac joint magnetic resonance imaging according to claim 1, characterized in that: The specific method of step 4 is: Step 4.

1. Initialize the parameters HMS, HMCR, PAR, K, and N, where the number of decision variables N represents the number of features in the original feature set in the feature selection problem. Step 4.2: Initialize the harmony memory library in the following format: Step 4.3: Use the random number to determine whether to search the current optimal harmony vector from the memory bank or randomly generate a variable with a value of 0 or 1. If the search is successful, continue to use the random number to determine whether to mutate until a complete harmony vector is generated. Step 4.4: Determine whether the fitness of the new harmony vector is better than the worst harmony vector in the current memory bank according to the fitness function; if so, replace the worst harmony vector in the memory bank with the new harmony vector; Step 4.5: Repeat steps 4.3 and 4.4 until the iteration conditions are met and output the final optimal solution set h n , that is, new imaging omics features.

3. The intelligent interpretation method of sacroiliac joint magnetic resonance imaging according to claim 2, characterized in that: The fitness function expression in step 4.4 is as follows: fitness=α×err+(1-AUC)×β+n fprob ×γ Where err represents the error; AUC represents the area under the ROC curve, which is used to measure the performance of the binary classification model; n fprob Represents the probability of feature selection; α is used to control the weight of error err in the fitness function; β is used to control the weight of AUC in the fitness function; γ is used to control the probability of feature selection n fprob Weight in the fitness function.

4. An intelligent reading system for sacroiliac joint magnetic resonance imaging, characterized in that: It comprises an ROI automatic segmentation model module and a feature selection classification module, and is used to execute the intelligent reading method of sacroiliac joint magnetic resonance images according to any one of claims 1 to 3.

5. The intelligent sacroiliac joint magnetic resonance imaging system according to claim 4, characterized in that: The ROI automatic segmentation model module includes an encoder S and a decoder D. The encoder S is composed of multiple double convolutional layers and maximum pooling layers, and its input is a single sequence medical image I m , the output is the corresponding high-dimensional feature map; The decoder D consists of multiple deconvolution layers and double convolution layers, and its input is the radiomics feature data x n , the output is a new radiomics feature h restored to the same size as the original input image n .

6. The intelligent sacroiliac joint magnetic resonance imaging system according to claim 5, characterized in that: The feature selection classification module uses a classifier E, and the input of the classifier E is the new radiomics feature h n , and get h n The structural classification result y n .

Images