Automatic grading method of lumbar disc degeneration based on regional block feature enhancement and suppression

Abstract

The present invention discloses an automatic grading method for lumbar disc degeneration based on regional block feature enhancement and suppression, which mainly solves the problem that the existing method will miss some features that are not obvious but still have distinguishing characteristics when extracting lumbar disc image features, resulting in low grading accuracy. The scheme is: obtain lumbar magnetic resonance imaging, generate a positioning data set after preprocessing it; use the existing DETR network to locate the lumbar disc region in the positioning data set to generate a grading data set; construct an automatic grading network for lumbar disc degeneration based on regional block feature enhancement and suppression; train the network with a grading training set and select the best trained model with a grading validation set; input the grading test set into the selected trained model to obtain the grading result. The present invention enables the network to mine some features that are not obvious but still have distinguishing characteristics, effectively improves the effect of grading lumbar disc degeneration, and can be used for the detection and classification of lesions in magnetic resonance imaging.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for grading human lumbar intervertebral disc degeneration, which can be used for detecting and classifying lesions in nuclear magnetic resonance images. Background Art

[0002] At present, low back pain has become a social public health problem, and lumbar disc degeneration is the main cause of low back pain. Accurate diagnosis of the degeneration grade is crucial for clinical diagnosis and rational treatment of patients with low back pain. Magnetic resonance imaging (MRI) is one of the most commonly used imaging examination methods for evaluating lumbar disc degeneration, mainly based on the morphology and signal characteristics of the lumbar disc in sagittal T2-weighted sequence images. The Pfirrmann standard divides lumbar disc degeneration into five gradually increasing grades: I, II, III, IV, and V, based on the structure of the nucleus pulposus, the boundary between the nucleus pulposus and the annulus fibrosus, the signal intensity of the nucleus pulposus, and the height of the intervertebral disc. It is recognized as the gold standard for judging the degree of lumbar disc degeneration.

[0003] However, the existing technology for diagnosing lumbar disc degeneration is mainly based on the judgment of physicians based on magnetic resonance imaging. This process is cumbersome and time-consuming, and may lead to misjudgment due to lack of experience, and has a certain degree of subjectivity. This subjectivity is reflected in the fact that the same physician will have different judgments on the same lumbar disc at different times, and different physicians will have different judgments on the same lumbar disc at the same time, resulting in under-diagnosis or over-diagnosis of lumbar disc degeneration. Therefore, it is urgent to establish a computer-aided diagnosis method that automatically identifies the grade of lumbar disc degeneration, so as to provide a basis for radiologists to objectively evaluate lumbar disc degeneration. With the development of computer-aided diagnosis in clinical medicine, a large number of researchers have invested in the computer analysis of magnetic resonance spinal images. Existing computer-aided diagnosis and grading methods for lumbar disc degeneration can be divided into traditional methods and deep learning methods, among which:

[0004] The traditional method is to classify lumbar disc degeneration based on a semi-automatic process. Da Silva Barreiro et al. described a semi-automatic lumbar disc degeneration classification method based on image processing technology in "Semiautomatic classification of intervertebral disc degeneration inmagnetic resonance images of the spine". This method first extracts the region of interest by manually segmenting the lumbar disc and performs normalization operations, and then separates each lumbar disc for feature extraction, attribute selection, and pattern classification. Ruiz- et al. proposed a semi-automatic computer-aided classification of degenerative lumbar spinal degeneration in magnetic resonance imaging. This method first artificially places a landmark in the lumbar disc, then uses the gradient vector flow algorithm to extract the shape features of the lumbar disc and detect contour abnormalities, and uses the signal intensity method to segment and detect spinal stenosis. Castro-Mateos et al. proposed in Intervertebral disc classification by its degree of degeneration from T2-weighted magnetic resonance images: first, the center point of the lumbar disc is selected for manual intervention, then the active contour model is used to delineate the boundary of the lumbar disc, and eight designed features including shape and intensity information are extracted, and finally different classifiers are used for comparison. These traditional methods require manual intervention, are inefficient, and require manual design of features closely related to lumbar disc degeneration, which requires very high experience from researchers and is not easy to achieve.

[0005] Deep learning methods do not require manual intervention and can automatically extract features, showing great advantages in the automatic grading of lumbar disc degeneration. He et al. proposed a supervised distance metric learning method in "Automated grading of lumbar disc degeneration via supervised distance metric learning" to solve the problem of overlapping adjacent degeneration grades of lumbar discs. This method divides lumbar disc degeneration into four different grades: normal, mild, obvious, and severe. By learning distance metrics from labeled instances, images of the same grade are closer and images of different grades are farther away. Jamaludin et al. proposed a multi-task framework in "Automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist" that can locate lesions and generate radiological grading of lumbar spine images. This method obtains the lumbar disc area from the middle of two adjacent vertebrae by detecting the vertebrae, and then uses a convolutional neural network for grading. Niemeyer et al. proposed a grading method for lumbar disc degeneration based on magnetic resonance imaging in "A Deep Learning Model for the Accurate and Reliable Classification of DiscDegeneration Based on MRI Data". This method uses a convolutional neural network similar to the VGG16 architecture and compares several loss functions, including cross entropy loss, soft kappa loss, ordinal cross entropy loss, and regression loss. Although these deep learning methods have achieved good results, they use convolutional neural networks and can only extract the most significant features in the feature map, which will miss some insignificant but still discriminative features in the lumbar disc images. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies of the prior art and propose an automatic grading method for lumbar disc degeneration based on regional block feature enhancement and suppression, so as to reduce human intervention and improve efficiency, and automatically extract distinctive features in lumbar disc images to improve the effect of lumbar disc degeneration grading.

[0007] To achieve the above object, the technical solution of the present invention includes the following:

[0008] 1. A method for automatic grading of lumbar disc degeneration based on regional block feature enhancement and suppression, characterized by comprising:

[0009] (1) Obtaining the lumbar spine magnetic resonance imaging dataset:

[0010] (1a) Sagittal T2-weighted sequence images were selected from the acquired 500 lumbar magnetic resonance imaging datasets to form a localization dataset for lumbar disc area localization;

[0011] (1b) For all lumbar MRI images in the positioning dataset, the labels and position coordinate information of the lumbar intervertebral discs are marked, and an XML format file is generated, which is then made into a COCO format dataset;

[0012] (1c) Using the random selection method, the COCO format positioning dataset is divided into 300 data as the positioning training set, 100 data as the positioning verification set, and 100 data as the positioning test set in a ratio of 3:1:1;

[0013] (2) The positioning training set is input into the existing DETR target detection network and iterative training is performed using the back propagation method. In each iteration, the positioning verification set is input into the trained model to obtain the detection accuracy. When the number of iterations reaches 50, the training is stopped and the model with the highest detection accuracy is selected as the final trained lumbar disc detection model.

[0014] (3) Obtain the data set of the lumbar intervertebral disc area:

[0015] (3a) According to the position coordinate information marked in (1b), the lumbar vertebra images in the positioning training set are cropped to obtain lumbar intervertebral disc images, and the degeneration grade of the lumbar intervertebral disc images is determined according to the Pfirrmann grading diagnostic standard, and marked as one of grade I, grade II, grade III, grade IV, and grade V, and the grade V lumbar intervertebral disc images are rotated and translated to obtain a graded training set;

[0016] (3b) According to the position coordinate information marked in (1b), the lumbar vertebra images in the positioning verification set are cropped to obtain lumbar intervertebral disc images, and the degeneration grade of all lumbar intervertebral disc images is determined according to the Pfirrmann grading diagnostic standard, and marked as one of grade I, grade II, grade III, grade IV, and grade V to obtain a graded verification set;

[0017] (3c) Input the positioning test set into the lumbar intervertebral disc detection model trained in (2) to obtain the target detection result, i.e., the position coordinate information of the lumbar intervertebral disc, crop the lumbar vertebrae images in the positioning test set according to the position coordinate information to obtain the lumbar intervertebral disc images, and judge the degeneration grade of all lumbar intervertebral disc images according to the Pfirrmann grading diagnostic standard, mark them as one of grade I, grade II, grade III, grade IV, and grade V, and obtain the graded test set;

[0018] (4) Constructing an automatic lumbar disc degeneration classification network W based on regional block feature enhancement and suppression:

[0019] (4a) Design a region-based feature enhancement and suppression module RFBS consisting of a feature enhancement branch and a feature suppression branch connected in parallel;

[0020] (4b) The regional block-based feature enhancement and suppression module is sequentially connected to the third and fourth layers of the existing CMT network, and the existing feature diversity module FDM is connected to the back of the two RFBS modules and the fifth layer of the CMT network to form an automatic lumbar disc degeneration classification network based on regional block feature enhancement and suppression;

[0021] (5) Inputting the classification training set into the constructed lumbar disc degeneration automatic classification network and performing iterative training using the back propagation method. In each iteration, the classification verification set is input into the trained model to obtain the classification accuracy. When the number of iterations reaches 50, the training is stopped and the model with the highest classification accuracy is selected as the final trained lumbar disc degeneration automatic classification model.

[0022] (6) The classification test set is input into the trained lumbar disc degeneration automatic classification model to obtain the classification results of the classification test set.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] First, the present invention automatically extracts lumbar image features through a two-stage neural network structure of first positioning and then grading, without the need for human intervention, and can thus fully automatically grade the degree of lumbar disc degeneration.

[0025] Second, the present invention uses a feature enhancement and suppression module based on regional blocks, uses a feature enhancement branch to enhance the significant features extracted by the network, and uses a feature suppression branch to suppress the significant features, thereby forcing the network to continue to explore the insignificant but still distinguishing features in the lumbar disc images, thereby improving the effect of lumbar disc degeneration grading. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of the implementation process of the present invention;

[0027] Figure 2 It is a hierarchical network framework diagram in the present invention;

[0028] Figure 3 for Figure 2 Schematic diagram of the region-based feature enhancement and suppression module RFBS in the hierarchical network;

[0029] Figure 4 for Figure 2 Schematic diagram of the feature diversity module FDM in a hierarchical network;

[0030] Figure 5 This is an attention map for grading the degree of lumbar disc degeneration before and after using the RFBS module in the present invention. DETAILED DESCRIPTION

[0031] The embodiments and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0032] See also Figure 1 The specific implementation steps of this example include the following:

[0033] Step 1: Obtain a lumbar spine magnetic resonance imaging dataset.

[0034] 1.1) Obtain 500 lumbar magnetic resonance imaging data from the hospital's imaging department, and select sagittal T2-weighted sequence images to form a data set for lumbar disc area localization;

[0035] 1.2) For all lumbar MRI images in the positioning data set, the labels and position coordinate information of the lumbar intervertebral disc are marked, and an XML format file is generated, which is then made into a COCO format data set;

[0036] 1.3) The random selection method is used to divide the COCO format positioning data set into 300 data as the positioning training set, 100 data as the positioning verification set, and 100 data as the positioning test set in a ratio of 3:1:1.

[0037] Step 2: Use the positioning data set to train the existing DETR target detection network to obtain a trained lumbar disc detection model.

[0038] 2.1) Set the total number of iterations to 50;

[0039] 2.2) Set the Hungarian loss function as the loss function of the network It is expressed as follows:

[0040]

[0041] Among them, y is the true label, is the prediction result, N is 100, c i is the category label of the i-th target, It is the predicted The category of the target is c i The probability of is a Boolean function, when 1 if yes, 0 otherwise. is the loss function between two target bounding boxes, expressed as:

[0042] In the formula, λ iou is the weight coefficient of IoU loss, λ L1 is the weight coefficient of L1 loss, b i and are the true bounding box coordinates and predicted bounding box coordinates of the i-th target, respectively. is the IoU loss function, is the enclosing polygon area of ​​the true bounding box and predicted bounding box of the i-th target;

[0043] 2.3) Input the positioning training set in 1.3) into the existing DETR target detection network to obtain the detection result;

[0044] 2.4) Substitute the true label of the positioning training set and the predicted label of the detection result into the loss function The loss value of the network is calculated in

[0045] 2.5) Back-propagate the loss value obtained in 2.4) until the loss function converges to obtain the initially trained network;

[0046] 2.6) The network after preliminary training is cyclically executed from 2.3) to 2.5). In each iteration, the positioning verification set in 1.3) is input into the network to obtain the detection accuracy. When the number of iterations reaches 50, the training is stopped and the network with the highest detection accuracy is selected as the final trained lumbar disc detection model.

[0047] Step 3: Obtain a data set of the lumbar intervertebral disc area.

[0048] 3.1) According to the position coordinate information marked in 1.2), the lumbar vertebra images in the positioning training set are cropped to obtain lumbar intervertebral disc images, and the degeneration grade of the lumbar intervertebral disc images is determined according to the Pfirrmann grading diagnostic standard, and marked as one of grade I, grade II, grade III, grade IV, and grade V, and the grade V lumbar intervertebral disc images are rotated and translated to obtain a graded training set;

[0049] 3.2) According to the position coordinate information marked in 1.2), the lumbar vertebra images in the positioning verification set are cropped to obtain lumbar intervertebral disc images, and the degeneration grade of all lumbar intervertebral disc images is determined according to the Pfirrmann grading diagnostic standard, and marked as one of grade I, grade II, grade III, grade IV, and grade V to obtain a graded verification set;

[0050] 3.3) Input the positioning test set into the lumbar disc detection model trained in step 2 to obtain the target detection result, i.e., the position coordinate information of the lumbar disc. According to the position coordinate information, the lumbar vertebrae images in the positioning test set are cropped to obtain the lumbar disc images. According to the Pfirrmann grading diagnostic standard, the degeneration grade of all lumbar disc images is determined and marked as one of grade I, grade II, grade III, grade IV, and grade V to obtain the graded test set.

[0051] Step 4: Construct an automatic lumbar disc degeneration classification network W based on regional block feature enhancement and suppression.

[0052] 4.1) Design a feature enhancement and suppression module RFBS based on regional blocks:

[0053] See also Figure 3 , the specific implementation of this step is as follows:

[0054] 4.1.1) Let the length and width be g respectively w , g h The two-dimensional area block in the feature map A∈R C×W×H Slide along the width W and height H of the feature map, dividing the feature map into n feature area blocks, and the i-th feature area block is represented as in,

[0055] 4.1.2) For each feature region block A (i) Perform 1×1 convolution, BatchNorm batch normalization, ReLU activation, and adaptive average pooling in sequence to obtain the importance score of each feature area block.

[0056] 4.1.3) Importance score for each feature region block Apply the softmax operation to normalize and get the normalized importance score S = (s (1) ,…,s (i) ,…,s (n) ) T , where s (i) represents the normalized importance score of the i-th feature region block, i∈1,…,n;

[0057] 4.1.4) According to the normalized importance scores S of all feature region blocks and the feature map A, obtain the output A of the feature enhancement branch b ,in α represents the feature enhancement factor;

[0058] 4.1.5) Calculate the median value s for the importance score S mid , select the importance score s (i) Greater than the median value mid The importance score of the region block is set to β to suppress it, and the importance score of the remaining region block is set to 1, and the importance score after feature suppression is obtained as M = (m (1) , ..., m (i) ,... ， m (n) ) T ,in, β represents the characteristic inhibitor;

[0059] 4.1.6) According to the suppressed importance score M and feature map A, obtain the output of the feature suppression branch

[0060] 4.1.7) The feature enhancement branch and the feature suppression branch are connected in parallel to form the RFBS module.

[0061] 4.2) Choose CMT network:

[0062] The full name of the CMT network is Convolutional Neural Networks Meet Vision Transformers, which includes a CMT Stem module, four feature extraction modules of different sizes, and i ,i∈1,2,3,4, a classification layer, where:

[0063] The CMT Stem module consists of a 3x3 convolution kernel with a stride of 2 and two 3x3 convolution kernels with a stride of 1 connected in series;

[0064] The four feature extraction modules of different scales are each composed of a Patch Aggregation layer and different numbers of CMT Blocks connected in series. The structures of the four modules are as follows:

[0065] The first feature extraction module M1, the second feature extraction module M2, and the fourth feature extraction module M4 are all composed of a Patch Aggregation layer and three CMT Blocks connected in series.

[0066] The third feature extraction module M3 consists of a Patch Aggregation layer and sixteen CMT Blocks connected in series. The Patch Aggregation layer includes a 2x2 convolution kernel with a stride of 2 and a LayerNorm layer. The CMT Block includes a local perception unit, a lightweight multi-head self-attention module, and a reverse residual feedforward network.

[0067] The classification layer is composed of an AdaptiveAvgPool average pooling layer, a Linear layer, a ReLU activation layer, and a Linear layer which are connected in series.

[0068] 4.3) Select feature diversity module FDM:

[0069] like Figure 4 As shown, the FDM module consists of three sub-branches connected in parallel, each of which includes two convolutional blocks of different sizes and a paired complementary module, where;

[0070] The two convolution blocks of different sizes are composed of a convolution layer, a BatchNorm batch normalization layer, and a ReLU activation layer connected in series, where:

[0071] The convolution kernel in the first convolution layer has a stride of 1 and a size of 1x1.

[0072] The convolution kernel in the second convolution layer has a stride of 1 and a size of 3x3;

[0073] The paired complementary module is composed of two parallel-connected view layers, two parallel-connected normalize layers, a BMM layer, two parallel-connected softmax layers, and two parallel-connected BMM layers, which are connected in series in sequence.

[0074] 4.4) The RFBS module designed in 4.1) is connected to the third and fourth layers of the CMT network selected in 4.2) in sequence, and the FDM module selected in 4.3) is connected to the back of the two RFBS modules and the fifth layer of the CMT network, and then the three classification layers are connected to form an automatic classification network W for lumbar disc degeneration based on regional block feature enhancement and suppression, as shown in FIG. Figure 2 shown.

[0075] Step 5: Use the classification data set to train the lumbar disc degeneration automatic classification network W based on regional block feature enhancement and suppression.

[0076] 5.1) Set the total number of iterations to 50, the batch size to 32, the initial learning rate to 0.01, warm up the learning rate from 0 to 0.01 in the first 5 rounds, and adjust the learning rate using linear decay in the remaining 45 rounds. The optimizer is stochastic gradient descent SGD, the momentum is 0.9, and the weight decay is 0.001;

[0077] 5.2) Set the cross entropy loss function as the loss function of the hierarchical network Where: N represents the total number of samples, i represents the number of samples, M represents the number of levels, c represents the specific level, y ic If the true level of sample i is equal to c, it takes 1, otherwise it takes 0, p ic represents the predicted probability that sample i belongs to class c;

[0078] 5.3) Inputting the classification training set obtained in 3.1) into the lumbar disc degeneration automatic classification network W to obtain the classification results;

[0079] 5.4) Substitute the real labels of the classification training set and the predicted labels of the classification results into the loss function L to calculate the loss value of the network;

[0080] 5.5) Back-propagate the loss value obtained in 5.4) until the loss function converges to obtain the network W after preliminary training. i , i is the number of training iterations, i = 1, 2, ..., 50;

[0081] 5.6) After the initial training, the network W i The process of 5.3)-5.5) is executed cyclically, and the graded validation set in 3.2) is input into the network W in each iteration. i The classification accuracy is obtained until the number of iterations reaches 50, then the training is stopped and the network with the highest classification accuracy is selected as the final trained lumbar disc degeneration automatic classification network W′.

[0082] Step 6: perform lumbar disc degeneration classification on the classification test set.

[0083] The classification test set in 3.3) is input into the trained lumbar disc degeneration automatic classification network W′ to obtain the classification result of the classification test set.

[0084] The technical effects of the present invention are further described below in conjunction with simulation experiment results:

[0085] 1. Simulation conditions:

[0086] The simulation experiment platform of the present invention is a Win10 operating system, configured with a 3.20GHz Intel Core i7-6900K CPU and an NVIDIA GeForce GTX 1080Ti GPU, using the PyTorch deep learning framework, and the development language is Python.

[0087] The simulation experiment data of the present invention comes from the MRI imaging data of 500 patients with lumbar disc degeneration in the hospital. The thickness of the MRI slices in the data set ranges from 4 to 5 mm, and the spacing between slices ranges from 4.4 to 6.25 mm. The location information and Pfirrmann degeneration grade of the lumbar disc in the data set are annotated by a radiologist and reviewed by two orthopedic surgeons, where the location information is annotated by a rectangular bounding box and the Pfirrmann degeneration grade is represented by I, II, III, IV, and V.

[0088] 2. Simulation content and result analysis:

[0089] Simulation 1, using the existing lumbar disc degeneration grading method and the method of the present invention to grade the above lumbar disc grading data set, the results are shown in Table 1:

[0090] Table 1 Comparison of different quantitative indicators of the grading effect of different methods on the degree of degeneration in patients with lumbar disc degeneration

[0091]

[0092] In Table 1, Accuracy represents accuracy, Average Specificity represents average specificity, Average Precision represents average precision, Average Sensitivity represents average sensitivity, Average F1-score represents average F1 score, Cohen's κ represents Kappa correlation coefficient, MCC represents Matthews correlation coefficient, Average Jaccardsimilarity represents average Jaccard similarity coefficient, and Pearson's r represents Pearson correlation coefficient. The range of these indicators is 0 to 1. The closer the indicator is to 1, the better the grading effect is.

[0093] As can be seen from Table 1, the accuracy, average specificity, average precision, average sensitivity, average F1 score, Kappa correlation coefficient, Matthews correlation coefficient, average Jaccard similarity coefficient, and Pearson correlation coefficient of the method of the present invention for grading the degree of degeneration of patients with lumbar disc degeneration are all improved compared with the existing methods, which proves that the automatic grading method for lumbar disc degeneration based on regional block feature enhancement and suppression proposed in the present invention can improve the performance of lumbar disc degeneration grading.

[0094] Simulation 2, an ablation experiment is conducted on the feature enhancement and suppression module RFBS based on regional blocks proposed in the present invention to verify the effectiveness of the RFBS module, that is, the existing CMT network is first used to perform a grading experiment on the above-mentioned lumbar intervertebral disc grading dataset, and then the RFBS module is sequentially connected to the third and fourth layers in the CMT network, and the network is used to perform a grading experiment on the above-mentioned lumbar intervertebral disc grading dataset. The results are shown in Table 2:

[0095] Table 2 Effect of RFBS module on grading of lumbar disc degeneration

[0096]

[0097] It can be seen from Table 2 that after adding the RFBS module, the accuracy, average specificity, average precision, average sensitivity, average F1 score, Kappa correlation coefficient, Matthews correlation coefficient, average Jaccard similarity coefficient and Pearson correlation coefficient are increased by 0.019, 0.005, 0.02, 0.019, 0.019, 0.024, 0.025, 0.031 and 0.006 respectively, which proves the effectiveness of the regional block-based feature enhancement and suppression module RFBS proposed in the present invention.

[0098] Simulation 3, for the network before and after adding the RFBS module in the ablation experiment, the class activation heat map of the network is visualized for two different patients. The results are as follows Figure 5 As shown in the figure, 5(a) represents patient 1, 5(b) represents patient 2. In each patient, the first column represents the original image, the ellipse mark in the original image represents the degenerative feature area in the lumbar disc image, the first row of the second column represents the class activation heat map of the third layer of the network before adding the RFBS module, the first row of the third column represents the class activation heat map of the fourth layer of the network before adding the RFBS module, and the first row of the fourth column represents the class activation heat map of the fifth layer of the network before adding the RFBS module; the second row of the second column represents the class activation heat map of the fourth layer of the network after adding the RFBS module, the second row of the third column represents the class activation heat map of the sixth layer of the network after adding the RFBS module, and the second row of the fourth column represents the class activation heat map of the seventh layer of the network after adding the RFBS module.

[0099] Depend on Figure 5It can be seen that when the RFBS module is not added and only the CMT network is used to grade the degree of lumbar disc degeneration, the network only pays attention to some degeneration features in the lumbar disc image and misses some inconspicuous degeneration features. For example, the obvious signal area in the class activation heat map of the network in the first row of the third column and the first row of the fourth column for patient 1 is paid attention to, while the inconspicuous small shadow area on the left side of the lumbar disc image is not paid attention to; the highlighted signal area in the class activation heat map of the network in the first row of the second column, the first row of the third column, and the first row of the fourth column for patient 2 is paid attention to, while the slight tear signal feature on the left side of the lumbar disc image is not paid attention to.

[0100] The method of the present invention can focus on these inconspicuous features and improve the effect of lumbar disc degeneration grading, such as the inconspicuous small shadow area in the class activation heat map of the network in the second row of the fourth column for patient 1 and the slight tearing signal feature in the class activation heat map of the network in the second row of the third column for patient 2.

[0101] In summary, in order to address the problem that existing methods miss some insignificant but still distinguishing features when extracting features from lumbar disc images, the feature enhancement and suppression module based on regional blocks proposed in the present invention can mine out some insignificant but still distinguishing features, thereby improving the effect of lumbar disc degeneration grading.

Claims

1. An automatic grading method for lumbar disc degeneration based on regional block feature enhancement and suppression, characterized in that: include: (1) Obtaining the lumbar spine magnetic resonance imaging dataset: (1a) Sagittal T2-weighted sequence images were selected from the acquired 500 lumbar magnetic resonance imaging datasets to form a localization dataset for lumbar disc area localization; (1b) For all lumbar MRI images in the positioning dataset, the labels and position coordinate information of the lumbar intervertebral discs are marked, and an XML format file is generated, which is then made into a COCO format dataset; (1c) Using the random selection method, the COCO format positioning dataset is divided into 300 data as the positioning training set, 100 data as the positioning verification set, and 100 data as the positioning test set in a ratio of 3:1:1; (2) The positioning training set is input into the existing DETR target detection network and iterative training is performed using the back propagation method. In each iteration, the positioning verification set is input into the trained model to obtain the detection accuracy. When the number of iterations reaches 50, the training is stopped and the model with the highest detection accuracy is selected as the final trained lumbar disc detection model. (3) Obtain the data set of the lumbar intervertebral disc area: (3a) According to the position coordinate information marked in (1b), the lumbar vertebra images in the positioning training set are cropped to obtain lumbar intervertebral disc images, and the degeneration grade of the lumbar intervertebral disc images is determined according to the Pfirrmann grading diagnostic standard, and marked as one of grade I, grade II, grade III, grade IV, and grade V, and the grade V lumbar intervertebral disc images are rotated and translated to obtain a graded training set; (3b) According to the position coordinate information marked in (1b), the lumbar vertebra images in the positioning verification set are cropped to obtain lumbar intervertebral disc images, and the degeneration grade of all lumbar intervertebral disc images is determined according to the Pfirrmann grading diagnostic standard, and marked as one of grade I, grade II, grade III, grade IV, and grade V to obtain a graded verification set; (3c) Input the positioning test set into the lumbar intervertebral disc detection model trained in (2) to obtain the target detection result, i.e., the position coordinate information of the lumbar intervertebral disc, crop the lumbar vertebrae images in the positioning test set according to the position coordinate information to obtain the lumbar intervertebral disc images, and judge the degeneration grade of all lumbar intervertebral disc images according to the Pfirrmann grading diagnostic standard, mark them as one of grade I, grade II, grade III, grade IV, and grade V, and obtain the graded test set; (4) Constructing an automatic lumbar disc degeneration classification network W based on regional block feature enhancement and suppression: (4a) Design a region-based feature enhancement and suppression module RFBS consisting of a feature enhancement branch and a feature suppression branch connected in parallel; its implementation is as follows: (4a1) with length and width respectively being g w , g h The two-dimensional area block in the feature map A∈R C×W×H Slide along the width W and height H of the feature map, dividing the feature map into n feature area blocks, and the i-th feature area block is represented as in, (4a2) For each feature region block A (i) Perform 1×1 convolution, BatchNorm batch normalization, ReLU activation, and adaptive average pooling in sequence to obtain the importance score of each feature area block. (4a3) Importance score for each feature region block Apply the softmax operation to normalize and get the normalized importance score S = (s (1) ,…,s (i) ,…,s (n) ) T , where s (i) represents the normalized importance score of the i-th feature region block, i∈1,…,n; (4a4) According to the normalized importance scores S of all feature region blocks and the feature map A, the output A of the feature enhancement branch is obtained b ,in α represents the feature enhancement factor; (4a5) Calculate the median value s for the importance score S mid , select the importance score s (i) Greater than the median value mid The importance score of the region block is set to β to suppress it, and the importance score of the remaining region block is set to 1, and the importance score after feature suppression is obtained as M = (m (1) ,…,m (i) ,…,m (n) ) T ,in, β represents the characteristic suppressor; (4a6) According to the suppressed importance score M and feature map A, obtain the output of the feature suppression branch (4a7) The feature enhancement branch and the feature suppression branch are connected in parallel to form the RFBS module; (4b) The regional block-based feature enhancement and suppression module is sequentially connected to the third and fourth layers of the existing CMT network, and the existing feature diversity module FDM is connected to the back of the two RFBS modules and the fifth layer of the CMT network to form an automatic lumbar disc degeneration classification network based on regional block feature enhancement and suppression; (5) Inputting the classification training set into the constructed lumbar disc degeneration automatic classification network and performing iterative training using the back propagation method. In each iteration, the classification verification set is input into the trained model to obtain the classification accuracy. When the number of iterations reaches 50, the training is stopped and the model with the highest classification accuracy is selected as the final trained lumbar disc degeneration automatic classification model. (6) The classification test set is input into the trained lumbar disc degeneration automatic classification model to obtain the classification results of the classification test set.

2. The method according to claim 1, characterized in that The CMT network in (4b) consists of a CMT Stem module, four feature extraction modules M of different sizes, and i ,i∈1,2,3,4, a classification layer is connected in series; The CMT Stem module is composed of a 3x3 convolution kernel with a stride of 2 and two 3x3 convolution kernels with a stride of 1 connected in series; The four feature extraction modules M of different sizes i , are composed of a Patch Aggregation layer and different numbers of CMT Blocks connected in series, where: The first feature extraction module M1, the second feature extraction module M2, and the fourth feature extraction module M4 are all composed of a Patch Aggregation layer and three CMT Blocks connected in series. The third feature extraction module M3 consists of a Patch Aggregation layer and sixteen CMT Blocks connected in series. The Patch Aggregation layer includes a 2x2 convolution kernel with a stride of 2 and a LayerNorm layer. The CMT Block includes a local perception unit, a lightweight multi-head self-attention module, and a reverse residual feedforward network. The classification layer is composed of an AdaptiveAvgPool average pooling layer, a Linear layer, a ReLU activation layer, and a Linear layer which are connected in series.

3. The method according to claim 1, characterized in that The feature diversity module FDM in (4b) consists of three sub-branches connected in parallel, each of which consists of two convolutional blocks of different sizes and a pairwise complementary module connected in series; The two convolution blocks of different sizes are composed of a convolution layer, a BatchNorm batch normalization layer, and a ReLU activation layer connected in series, where: The convolution kernel in the first convolution layer has a stride of 1 and a size of 1x1. The convolution kernel in the second convolution layer has a stride of 1 and a size of 3x3; The paired complementary module is composed of two parallel-connected view layers, two parallel-connected normalize layers, a BMM layer, two parallel-connected softmax layers, and two parallel-connected BMM layers, which are connected in series in sequence.

4. The method according to claim 1, characterized in that: (5) The automatic classification network of lumbar disc degeneration based on regional block feature enhancement and suppression is trained as follows: (5a) Set the total number of iterations to 50, the batch size to 32, the initial learning rate to 0.01, warm up the learning rate from 0 to 0.01 in the first 5 rounds, and adjust the learning rate using linear decay in the remaining 45 rounds. The optimizer is stochastic gradient descent SGD, the momentum is 0.9, and the weight decay is 0.001; (5b) Set the cross entropy loss function as the loss function of the hierarchical network Where: N represents the total number of samples, i represents the number of samples, M represents the number of levels, c represents the specific level, y ic If the true level of sample i is equal to c, it takes 1, otherwise it takes 0, p ic represents the predicted probability that sample i belongs to class c; (5c) inputting the classification training set obtained in (3a) into the lumbar disc degeneration automatic classification network to obtain the classification result; (5d) Calculate the loss value based on the classification result obtained in (5c) and the classification label of the classification training set; (5e) Back-propagate the loss value of (5d) until the loss function converges, and obtain the network W after preliminary training i , i is the number of training iterations, i = 1, 2, ..., 50; (5f) After initial training, the network W i The process (5c)-(5e) is executed cyclically, and the graded validation set in (3b) is input into the network W in each iteration. i The classification accuracy is obtained until the number of iterations reaches 50, then the training is stopped and the network with the highest classification accuracy is selected as the final trained network W ′ .

Images