Automatic diagnosis model and construction method for lumbar disc herniation based on semi-supervised learning

Abstract

The application provides a construction method of an automatic diagnosis model of lumbar disc herniation based on semi-supervised learning, comprising the following steps: S10, constructing a training data set S and preprocessing MRI images in the training data set S; S20, constructing a first neural network; S30, training the first neural network to generate a feature extractor; S40, constructing a second neural network; S50, extracting a corresponding semantic feature vector for each labeled MRI image sample; and S60, training the second neural network by using the semantic feature vector generated in the step S50 to generate a feature classifier. The construction method of the automatic diagnosis model uses two-stage semi-supervised learning to construct a model, effectively reduces the number of labeled MRI image samples, visualizes the judgment basis of the feature classifier, reduces the burden of an imaging doctor, and improves the diagnosis efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image analysis, in particular to a lumbar disc herniation automatic diagnosis model based on semi-supervised learning and a construction method thereof. BACKGROUND

[0002] Lumbar disc herniation is one of the common diseases causing low back pain. Due to the high cost of manual analysis and diagnosis, combined with the problems of shortage and uneven distribution of medical resources, an automatic analysis and diagnosis system based on lumbar MRI (Magnetic Resonance Imaging) images has become an urgent need. The commonly used method in the field of artificial intelligence is to use a supervised method to train a target detection model, which has two difficulties in practice. First, the cost of labeled data is high, and the target detection model needs to label a large number of target selection boxes and corresponding disc shape categories, and generally at least thousands of labeled samples are needed to train a more useful model. Second, the end-to-end model is not interpretable. For an uninterpretable model, when used as a computer-aided diagnosis model, it will consume more effort for the doctor to review the model results, and it is also difficult for the model researchers to judge the rationality of the model results. SUMMARY

[0003] To solve the above problems, the present application provides a construction method of a lumbar disc herniation automatic diagnosis model based on semi-supervised learning, which uses two-stage semi-supervised learning to construct the model, effectively reduces the number of labeled MRI image samples used, visualizes the judgment basis of the feature classifier, reduces the burden of radiologists, and improves the diagnosis efficiency.

[0004] In order to achieve the above purpose, the present application mainly adopts the following technical solutions:

[0005] The construction method of the lumbar disc herniation automatic diagnosis model based on semi-supervised learning comprises the following steps:

[0006] Step S10: Collecting unlabeled lumbar disc MRI image samples, constructing a training data set S, and preprocessing the MRI images in the training data set S;

[0007] Step S20: Constructing a first neural network;

[0008] Step S30: Training the first neural network using the MRI images in the training data set S to generate a feature extractor for extracting semantic feature vectors of the MRI images;

[0009] Step S40: Constructing a second neural network;

[0010] Step S50: Input the labeled lumbar intervertebral disc MRI image samples into the feature extractor in sequence, and extract the corresponding semantic feature vector for each MRI image sample;

[0011] Step S60: The second neural network is trained using the semantic feature vector generated in step S50 to generate a feature classifier, which classifies the semantic feature vector and outputs the prediction result.

[0012] In some embodiments, the construction method further includes the following steps:

[0013] Step S70: When the prediction result of the feature classifier is positive, a heat map is generated for the positive sample, and the key information that causes the positive result in the positive sample image is marked.

[0014] In some embodiments, in step S20, the first neural network includes a ResNet50 network and multiple fully connected layers, wherein the ResNet50 network and the multiple fully connected layers are connected in series.

[0015] In some embodiments, step S10, preprocessing the MRI image includes the following steps:

[0016] Step S11: When performing each training step, first randomly select n MRI images from the training dataset S to construct a training data subset P;

[0017] Step S12: Perform two random transformations on each MRI image in the training data subset P to obtain 2n transformed MRI images.

[0018] In some embodiments, step S30 includes the following steps:

[0019] Step S31: Input the 2n MRI transformation images into the first neural network in pairs, and the first neural network maps a pair of semantic feature vectors for each pair of input MRI transformation images;

[0020] Step S32: Calculate the predicted probability value of each pair of semantic feature vectors mapped by the first neural network expressing similar semantics based on the softmax function.

[0021] Where, x i (1≤i≤2n) represents the semantic feature vector obtained by mapping the i-th transformed MRI image through the first neural network, x jrepresents the semantic feature vector of the jth transformed MRI image mapped by the first neural network, and the ith and jth transformed MRI images are a group, which are simultaneously input into the first neural network, and τ is a commonly used hyperparameter in the softmax function;

[0022] Step S33: According to the optimization training objective function of the first neural network

[0023] Calculate the loss value of the training data subset P, wherein f E is the first neural network, represents the expectation; represents an indicator function, and Φ is a proposition. When Φ is true, When Φ is false, represents the upward rounding;

[0024] Step S34: Training and minimizing the loss value of each training data subset P by using the gradient descent algorithm, and finally generating a feature extractor.

[0025] In some embodiments, in the step S50, each labeled lumbar disc MRI image sample includes multiple transverse axial MRI images of the same lumbar disc.

[0026] In some embodiments, the step S60 includes the following steps:

[0027] Step S61: Obtain the semantic feature vector of each labeled lumbar disc MRI image sample, and the second neural network is trained for feature classification of the semantic feature vector;

[0028] Step S62: Calculate the probability value of the current trained lumbar disc MRI image sample being a positive sample using the Sigmoid function;

[0029] Step S63: Train the second neural network using multiple labeled lumbar disc MRI image samples by using the method of steps S61 to S62, and generate a feature classifier.

[0030] In some embodiments, in the step S70, the method for generating a heat map for a positive sample includes the following steps:

[0031] Step S71: Obtain the feature map set A output by the feature extractor for each labeled lumbar disc MRI image sample;

[0032] Step S72: According to the formula Calculate the weight value of each feature map, wherein u and v are the height and width of the deepest layer feature map, respectively, Let be the gradient of k feature maps, 1≤k≤n;

[0033] Step S73: According to the formula Normalize each weight value, where σ(·) represents the standard deviation;

[0034] Step S74: Calculate the weight values ​​of each feature map, expressed as follows:

[0035] Where 'a' represents the number of axial MRI images contained in each labeled lumbar disc MRI image sample.

[0036] Step S75: Sum the n feature maps of each image with their corresponding weight values ​​to obtain a heatmap;

[0037] Step S76: Standardize each heatmap:

[0038]

[0039]

[0040] Among them, H i This represents the heatmap corresponding to the i-th cross-section of the sample. The maxσ(·) function represents finding the maximum element value in the tensor. A (k) Let be the k-th feature map in the feature map set A.

[0041] In some embodiments, a random transformation function f is used to perform random transformation processing on MRI images. T : This includes random rotation, random cropping, random flipping, random changes in brightness or contrast, and random Gaussian blur. Two random transformations of the same MRI image express similar semantic features, while random transformations of different MRI images express different semantic features.

[0042] This invention also provides an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning. The automatic diagnostic model is obtained using the construction method for the automatic diagnostic model of lumbar disc herniation based on semi-supervised learning provided by this invention. The automatic diagnostic model includes:

[0043] Feature extractor, used to extract semantic feature vectors from MRI images;

[0044] A feature classifier is used to classify semantic feature vectors and output prediction results;

[0045] The heatmap generation module generates a heatmap when the prediction result is positive, marking the key information that causes the positive result in the positive MRI image.

[0046] The automatic diagnostic model for lumbar disc herniation provided in this invention uses a two-stage semi-supervised learning approach. With a smaller number of labels, the resulting classifier maintains stable performance, achieving a level comparable to classic supervised models using only about one-third of the labeled training subset. Furthermore, this invention improves and adapts the Grad-CAM interpretability analysis algorithm, visualizing the judgment criteria of the feature classifier using a heatmap. The automatic diagnostic model for lumbar disc herniation constructed in this invention yields reasonable results, can assist in diagnosis to some extent, reduce the burden on radiologists, improve diagnostic efficiency, and provide a reference for future research. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0048] Figure 1 A structural block diagram of an automatic diagnostic model for lumbar disc herniation;

[0049] Figure 2 This is a schematic diagram illustrating the principle of training an automatic diagnostic model for lumbar disc herniation using a two-stage semi-supervised learning approach.

[0050] Figure 3 A flowchart illustrating the construction method of an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning;

[0051] Figure 4 This is a schematic diagram of the unsupervised training phase A.

[0052] Figure 5 Flowchart of the method for training the first neural network;

[0053] Figure 6 It has a fully connected layer structure;

[0054] Figure 7 This is a schematic diagram of the supervised training phase B.

[0055] Figure 8 Flowchart of the method for training the second neural network;

[0056] Figure 9 A flowchart illustrating the method for generating heatmaps;

[0057] Figure 10 ROC curves of classifiers obtained by feature extractors under different regularization levels;

[0058] Figure 11 Performance comparison of classifiers with different numbers of labeled samples and classic supervised classifiers;

[0059] Figure 12 This is a heatmap of some positive samples.

[0060] Explanation of reference numerals in the attached figures:

[0061] 100 - Automatic diagnostic model for lumbar disc herniation, 110 - Feature extractor, 120 - Feature classifier, 130 - Heatmap generation module, 140 - Heatmap;

[0062] 200 - Automatic diagnostic model for lumbar disc herniation, 210 - First neural network, 220 - Second neural network, 230 - Heat map generation module, 240 - Heat map;

[0063] 410 - First neural network, 411 - ResNet50 network, 412 - Fully connected layer, 413 - Feature extractor;

[0064] 710 - Feature extractor, 720 - Second neural network. Detailed Implementation

[0065] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0066] This invention provides an automatic diagnostic model for lumbar disc herniation, which uses multi-slice lumbar disc sectional MRI images as input, automatically analyzes and diagnoses the condition, and outputs prediction results. Figure 1 As shown, the automatic diagnostic model 100 for lumbar disc herniation includes a feature extractor 110 and a feature classifier 120. The feature extractor 110 extracts semantic feature vectors from the input MRI image; the feature classifier 120 classifies the input semantic feature vectors, that is, it classifies and judges the lumbar disc morphology of the input multi-slice lumbar disc MRI image and outputs a prediction result, which is divided into positive and negative types. The morphology of the lumbar disc is divided into normal and abnormal categories. Abnormal conditions include disc herniation, disc bulging, disc extrusion, and annulus fibrosus tear. When the prediction result is negative, it indicates that the lumbar disc morphology is normal; when the prediction result is positive, it indicates that the lumbar disc morphology is one or more of the abnormal conditions.

[0067] like Figure 1As shown, the automatic diagnosis model 100 for lumbar disc herniation provided by the present invention also includes a heat map generation module 130. When the feature classifier 120 outputs a positive prediction result, the heat map generation module 130 generates a heat map 140 based on the feature map output by the feature extractor 110, marking the key information that causes the positive result in the positive MRI image, so that the judgment basis of the prediction result can be visualized.

[0068] This invention employs a two-stage semi-supervised learning approach to construct an automatic diagnostic model for lumbar disc herniation, such as... Figure 2 As shown, the construction of the automatic diagnosis model 200 for lumbar disc herniation includes an unsupervised training phase A and a supervised training phase B. In the unsupervised training phase A, the first neural network 210 is trained using an unlabeled MRI image set S to generate a feature extractor that extracts semantic feature vectors from the input MRI images. In the supervised training phase B, each sample in the labeled MRI image set M is first processed by the trained feature extractor to extract the semantic feature vector corresponding to each MRI image sample. Then, the semantic feature vector corresponding to each MRI image sample is used to train the second neural network 220 to generate a feature classifier, which is used to classify the semantic feature vectors and output prediction results. Therefore, in the process of constructing the automatic diagnosis model for lumbar disc herniation, the unsupervised training phase A should be implemented first, followed by the supervised training phase B; that is, the feature extractor is trained first, and then supervised training is performed using the generated feature extractor to generate the feature classifier.

[0069] The unsupervised training phase A and supervised training phase B of constructing an automatic diagnostic model for lumbar disc herniation will be described below.

[0070] First, we introduce the unsupervised training phase A. Unsupervised training phase A requires training the feature extraction network to ultimately generate a feature extractor that extracts semantic feature vectors from the input MRI image. To facilitate later classification, the extracted semantic feature vectors should reflect the semantic differences between samples as much as possible—such as the morphology of the intervertebral disc and its relative position to the dural sac—while ignoring differences in the representation of the same semantic meaning—such as image contrast and camera angle. Based on this goal, this invention employs a contrastive self-supervised method to achieve feature extraction, such as... Figure 3 part (a) and Figure 4 As shown, the method for training and generating a feature extractor mainly includes the following steps:

[0071] Step S10: Collect unlabeled lumbar intervertebral disc MRI image samples, construct training dataset S, and preprocess the MRI images in training dataset S;

[0072] Step S20: Construct the first neural network;

[0073] Step S30: Train the first neural network using MRI images from the training dataset S to generate a feature extractor for extracting semantic feature vectors from the MRI images.

[0074] In step S10, the samples in the training dataset S consist of axial MRI images of the intervertebral disc, including T1-weighted and T2-weighted images, with each MRI image serving as a sample. Before training, the MRI images in the training dataset S need to be preprocessed. The preprocessing process mainly includes the following steps:

[0075] Step S11: When performing each training step, first randomly select n MRI images from the training dataset S to construct a training data subset P;

[0076] Step S12: Perform two random transformations on each MRI image in the training data subset P to obtain 2n transformed MRI images.

[0077] First, define a random transformation function f. T : random transformation function f T It can vary depending on the specific circumstances, f T The design needs to maximize the magnitude and randomness of the transformation while retaining most of the key information. In this invention, the random transformation function f T Random rotation, random cropping, random flipping, random changes in brightness or contrast, and random Gaussian blur are employed. To better train the model, the degree of random transformation generally needs to be limited to a certain range. If the degree of transformation is too drastic, it will damage the original semantic information, and if the degree of transformation is too small, the model will not be able to be trained effectively. Based on the training objective of contrastive self-supervised training, two random transformations of the same MRI image express similar semantic features, while random transformations of different MRI images express different semantic features. For example, in this embodiment, random rotation and random cropping are two coupled operations, which require the following constraints: (1) In two random transformations of the same image, the cropping area must have an overlapping part; (2) The overlapping area should have a high probability of being located in the center of the original image; (3) In two random transformations of the same image, the rotation center point of the random rotation operation is consistent, and each random cropping area must contain the center point of random rotation; (4) During the rotation process, "0" pixel filling areas may be generated around the image, and random cropping should avoid these "0" pixels. (5) The aspect ratio of the randomly cropped region in the original image must be within [3 / 4, 4 / 3]; (6) The proportion of the randomly cropped region in the original image must be greater than a specific value, denoted as p. cropSince the training objective of contrastive self-supervised training is to make two transformations of the same image express similar semantics, it is obvious that the two transformations should contain common information, and the information contained in each transformation should not be too little. Therefore, constraints (1), (5), and (6) restrict the cropping region. On the other hand, rotation operations may interfere with the implementation of the constraint "ensuring that the two cropping regions overlap". If the center points of the two rotations are different, the location of the region containing common information will also be different. Therefore, constraint (3) is added to eliminate this interference. In addition, since the key information in most axial MRI images of intervertebral discs is located in the central region of the image, cropping to the edge region will cause the neural network to focus too much on some less important information, but it cannot completely discard this information. Therefore, constraint (2) is to increase the probability of key information being extracted. Finally, rotation operations may introduce some random noise information, that is, "0" pixels are generated around the image. Rotation of a rectangular image will cause a part of the region to move outside the rectangle. The missing part inside the rectangle will be filled with "0" pixels. In order to eliminate this noise, constraint (4) is introduced.

[0078] like Figure 4 As shown in part (a), each image is input into the random transformation function f. T The MRI images in the training data subset P are subjected to two random transformations according to a specified rule, resulting in two transformed images. Thus, the n MRI images in the training data subset P are transformed twice by the random transformation function f. T The transformation generates 2n transformed MRI images. For example: m i (1≤i≤2n) represents the i-th transformed image, m 2h-1 and m 2h These are two transformations of the h-th original input image.

[0079] In step S20, the present invention uses a deep neural network to construct a first neural network, such as... Figure 4 As shown, the first neural network 410 consists of a ResNet50 network 411 connected to four fully connected layers 412. It should be noted that the construction of the first neural network is not limited to... Figure 4 The ResNet50 network shown in the diagram can be replaced by other neural networks, and the number of fully connected layers can also be changed, as long as it can extract high-level semantic features of the samples.

[0080] Figure 5 As shown, in step S30, the process of training the first neural network 410 using unlabeled MRI images includes the following steps:

[0081] Step S31: The first neural network 410 maps a pair of semantic feature vectors to the two input MRI transformed images.

[0082] In step S12, each MRI image in the training data subset P undergoes two random transformations, resulting in 2n transformed MRI images. These 2n transformed MRI images are then paired and input into the first neural network 410 for training. The first neural network 410 maps each MRI image to a d-dimensional semantic feature vector; therefore, each input pair produces two semantic feature vectors. Figure 4 As shown in part (b), let m i (1≤i≤2n) represents the i-th transformed image, m j (1≤j≤2n) represents the j-th transformed image, where m i and m j A set of inputs is fed into the first neural network 410 for training, mapping out a pair of semantic feature vectors x i and x j , where x i m i The semantic feature vector x after mapping by the first neural network 410 j m j The semantic feature vector after being mapped by the first neural network 410.

[0083] Step S32: Calculate the semantic feature vector x using the softmax function. i and x j Probability prediction of expressing similar semantics

[0084] Here, τ is a commonly used hyperparameter in the softmax function, often referred to as temperature, used to adjust the absolute values ​​of the exponential components, preventing the softmax function from becoming unusable due to the exponent being too large or too small. For example... Figure 4 As shown in part (c), the semantic feature vector x is calculated using the softmax function. i and x j Probability prediction of expressing similar semantics

[0085] Step S33: Calculate the loss value of the training data subset P.

[0086] Based on the optimization training objective function: Calculate the loss value for each subset P of training data, where f E For the first neural network, Expressing expectations; This indicates an indicator function, where Φ is a proposition that states, when Φ is true, When Φ is false This indicates rounding up, such as Figure 4 As shown in section (c) of the document.

[0087] This invention uses the cross-entropy loss function as the objective function for optimal training, which can achieve the training objective of "ignoring the differences in the representation of the same semantics and reflecting the semantic differences between samples as much as possible", which facilitates the classification of semantic feature vectors and is beneficial for supervised tuning training in the later stage.

[0088] Step S34: Train the first neural network 410 using the gradient descent algorithm, minimize the loss value of each training data subset P, and finally generate the feature extractor.

[0089] In the unsupervised training phase A, a large amount of unlabeled MRI image data is required for training. That is, the training dataset S contains a large number of unlabeled MRI images. The first neural network 410 is trained extensively until the loss value of the training dataset basically stops decreasing.

[0090] It should be noted that when using the first trained neural network as a feature extractor, it is not always necessary to use the entire neural network. The output of a single layer can be chosen as the feature extractor's output. This is because the outputs of each layer in a trained neural network actually contain semantic features, although the complexity of the features expressed by different layers varies. For example... Figure 4 As shown, in this invention, the first neural network 410 used during training consists of a ResNet50 network 411 and four fully connected layers 412. In the subsequent supervised training phase B, the feature extractor 413 discards the last three fully connected layers 412; that is, given the same input, the output of the first fully connected layer of the first neural network 410 is used as the output of the feature extractor 413. Furthermore, the structure of the fully connected layer designed in this invention is as follows: Figure 6 As shown, Figure 6 (a) shows the structure of the first to third fully connected layers. Figure 6 (b) shows the structure of the fourth fully connected layer. The only difference between the fourth fully connected layer and the first to third fully connected layers is that the ReLU activation function is not used. The retention probability of the random deactivation layer (Dropout) is dr. The output dimension of the four fully connected blocks is d, and d = 256, that is, the output of each layer is a 256-dimensional vector.

[0091] This invention, while training the first neural network, simultaneously uses a small number of unlabeled lumbar disc MRI images to verify the real-time training results. The accuracy of the predicted results regarding whether two semantic feature vectors express similar semantics compared to the actual results is used as the verification metric. When this accuracy metric no longer improves after a period of training, the unsupervised training ladder A can be terminated.

[0092] Next, we will introduce supervised training phase B. Figure 7 The diagram shows the principle of supervised training phase B. The feature extractor 710 generated in unsupervised training phase A is used to train the second neural network 720, generating a feature classifier to classify lumbar intervertebral discs. Figure 3 As shown in part (b), the process of training the second neural network 720 to generate a feature classifier mainly includes the following steps:

[0093] Step S40: Construct the second neural network 720;

[0094] The second neural network 720 designed in this invention includes only one simple fully connected layer. Since the output of the feature extractor 710 is a 256-dimensional vector, and in the supervised training phase B, each time 3 MRI images are input into the feature extractor 710, 3 256-dimensional vectors will be generated. Therefore, the input of the fully connected layer is designed to be 3x256, and no other layers such as random deactivation layers or batch normalization layers are needed before and after the fully connected layer.

[0095] Step S50: Input the labeled lumbar intervertebral disc MRI image samples into the feature extractor in sequence, and extract the corresponding semantic feature vector for each MRI image sample;

[0096] Step S60: Train the second neural network 720 to generate a feature classifier;

[0097] Step S70: When the prediction result of the feature classifier is positive, generate a heatmap for the positive sample.

[0098] In step S50, T2-weighted axial MRI image samples are used to construct a classification dataset, with each MRI image sample corresponding to an intervertebral disc. In this embodiment, each MRI image sample contains 3 axial MRI images of the same lumbar intervertebral disc, and each sample has a label indicating whether the morphology of the intervertebral disc is abnormal. It should be noted that the number of axial MRI images contained in the MRI image sample of the same intervertebral disc can also be 1, 2, 4, 5, 6, 7... The specific number is designed according to actual needs. Ideally, these 3 cross sections should be located in the central position of the vertical direction of the intervertebral disc. In order to obtain a model with good generalization ability as much as possible, the following strategy is used when constructing the intervertebral disc classification dataset: (1) For an intervertebral disc with 5 cross sections, the central 3 layers, namely the 2nd, 3rd, and 4th layers, are taken as one intervertebral disc sample; for an intervertebral disc with 4 cross sections, the first 3 layers are taken as one intervertebral disc sample, and then the last 3 layers are taken as another intervertebral disc sample. That is, for such an intervertebral disc, two intervertebral disc samples will be generated. (2) Only lumbar intervertebral discs are selected as samples, and all data from thoracic and sacral intervertebral discs are discarded. In addition, there may be duplicate data in the original data, and deduplication should be taken into account when constructing the intervertebral disc classification dataset.

[0099] During supervised training, each MRI image sample in the classification dataset, namely three axial MRI images, is sequentially input into the feature extractor 710 to extract three semantic feature vectors.

[0100] It should be noted that during supervised training, simple data augmentation strategies are applied to the input image, such as performing some random transformations on the input image. However, these transformations do not need to be too complex. The main purpose is only to remove a portion of the area around the input image, without losing the content of the main part of the image.

[0101] like Figure 7 and Figure 8 As shown, in step S60, the method for training the second neural network 720 to generate a feature classifier includes the following steps:

[0102] Step S61: After obtaining the semantic feature vector of each labeled lumbar intervertebral disc MRI image sample, input it into the second neural network 720 for feature classification training;

[0103] Step S62: Calculate the probability value of the current training lumbar intervertebral disc MRI image sample being a positive sample;

[0104] Step S63: Using the methods of steps S61 to S62, the second neural network is trained with multiple labeled lumbar intervertebral disc MRI image samples to generate a feature classifier.

[0105] In step S61, the feature extractor 710 generates three semantic feature vectors for each labeled MRI image sample. These three semantic feature vectors are concatenated to obtain the semantic feature vector of the MRI image sample. This semantic feature vector is then input into the second neural network 720 for feature classification training. The sigmoid function is then used to map the output value of the second neural network 720 to a value between [0, 1], representing the probability that the current training sample is a positive sample. Based on this probability, it is possible to predict whether the training sample is a positive or negative sample. The second neural network is trained extensively using labeled MRI image samples from the classification dataset, ultimately generating a second neural network 720 that meets the requirements, i.e., the feature extractor. This feature classifier can output a positive or negative prediction result. In the supervised training phase B, all parameters in the feature extractor 710 remain fixed, maintaining the state at the end of the unsupervised training phase A. Only the parameters in the fully connected layers of the second neural network 720 change during training.

[0106] In step S70, when the second neural network 220 outputs a positive prediction result, the heatmap generation module 230 generates a heatmap 240, marking the key information causing the positive result in the positive MRI image. In this embodiment, an improved Grad-CAM analysis algorithm is used to generate the heatmap, such as... Figure 9 As shown, the process of generating heatmap 240 mainly includes the following steps:

[0107] Step S71: Obtain the feature map set A output by the feature extractor for each MRI image sample;

[0108] When constructing the feature classifier, MRI scan images of three intervertebral disc sections are used as input; therefore, three heatmaps should be generated. The set of feature maps output by the deepest convolutional layer in the ResNet50 network used to construct the feature extractor is denoted as A. A contains all the feature maps from the three input MRI images. Let each input MRI image correspond to n feature maps, and let the k-th feature map be Ak. (k) .

[0109] Step S72: According to the formula Calculate the weight values ​​for each feature map;

[0110] Where u and v are the height and width of the deepest feature map, respectively. Let be the gradient of k feature maps, 1≤k≤n;

[0111] Step S73: According to the formula Normalize each weight value, where σ(·) represents the standard deviation;

[0112] Step S74: Use the softmax function to obtain the weight values ​​of each feature map. The expression is:

[0113]

[0114] Step S75: Sum the n feature maps of each image with their corresponding weight values ​​to obtain a heatmap;

[0115] Step S76: Normalize each heatmap and limit it to the numerical range of [0,1].

[0116] The standardized calculation process is as follows:

[0117]

[0118]

[0119] Among them, H i This represents the heatmap corresponding to the i-th cross-section of the sample. The max(·) function finds the maximum element value in the tensor. A (k) Let be the k-th feature map in the feature map set A.

[0120] The above provides a detailed description of the method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning, as provided by this invention.

[0121] The following experiments further verify the practical effect of the semi-supervised learning-based automatic diagnosis model for lumbar disc herniation provided in this invention. In the unsupervised training phase, 16,707 unlabeled axial MRI images of lumbar discs were collected to construct a training dataset S, including T1-weighted and T2-weighted images. An additional 512 MRI images were collected as a validation set to test the accuracy of the predicted results compared to the actual results. In the supervised training phase, 3,676 samples were collected to construct a classification dataset, including 1,706 positive samples and 1,970 negative samples. Each sample corresponds to one disc and is labeled. After randomly shuffling the classification dataset, 256 positive and 256 negative samples were randomly selected to form the test set, and 64 positive and 64 negative samples were randomly selected to form the validation set. The experimental settings for hyperparameters in the unsupervised training phase are shown in Table 1, and the experimental settings for hyperparameters in the supervised training phase are shown in Table 2.

[0122] Table 1. Hyperparameter settings for the unsupervised training phase.

[0123]

[0124] Table 2. Hyperparameter settings for the supervised training phase

[0125] Hyperparameters lr n w Values 1 x 10 -4 ]] 32 0

[0126] In the table above, lr represents the learning rate, n is the number of samples in batch training, w is the regularization weight coefficient, dr is the retention probability of the random deactivation layer (experiments have shown that a retention probability of 0 for the random deactivation layer yields better results), θ is the range of random rotations during the random transformation, and p... crop The percentage of randomly cropped regions is used, and both are only effective during unsupervised training. Training continues until the accuracy on the validation set essentially stops improving and the model's loss function value essentially stops decreasing.

[0127] In the unsupervised training phase, we adopted L2 regularization and random deactivation regularization strategies. The weight parameter w for L2 regularization and the retention probability dr for random deactivation regularization reflect the strength of the regularization level. We set four different sets of regularization parameter values, keeping other hyperparameters constant, and trained four different feature extractors. For each feature extractor, supervised training was performed using the same settings for 200 epochs. The resulting classifiers were tested on the test set, and their performance was compared and analyzed. The results are shown in Table 3, and the ROC curves for the four different cases are shown in Table 3. Figure 10 As shown.

[0128] Table 3 shows the performance of the feature classifiers obtained by the feature extractors under different regularization levels.

[0129]

[0130] In the table above, Accuracy (ACC) is the proportion of samples whose predicted value matches the actual value. Sensitivity, also known as the True Positive Rate (TPR), is the proportion of true positive samples out of the actual positive samples. Specificity, also known as the True Negative Rate (TNR), is the proportion of true negative samples out of the actual negative samples. AUC (Area Under the ROC Curve) represents the area under the ROC curve.

[0131] exist Figure 10 In (a), w = 0, dr = 0; Figure 10 In (b), w = 1 × 10 -4 dr = 0; Figure 10 In (c), w = 1 × 10 -4 dr = 0.25; Figure 10 In (d), w = 1 × 10 -4dr = 0.5. The experimental results show that as the regularization level increases, both accuracy and AUC initially rise and then fall. When w = 0 and dr = 0, i.e., no regularization is applied, neither accuracy nor AUC reaches its optimal level. w = 1 × 10⁻⁵ -4 dr=0 and w=1×10 -4 In both cases where dr = 0.25, the feature classifiers produce almost identical classification results on the test set. The complete uniformity of accuracy, sensitivity, and specificity is somewhat accidental, but in reality, their classification performance is indeed very close. However, the latter's AUC shows a significant decrease. When a stronger regularization level is used, such as w = 1 × 10⁻⁶ in the experiment... -4 When dr = 0.5, the performance of the feature classifier will significantly degrade. Therefore, selecting an appropriate regularization parameter is crucial for training the feature extractor. In practice, due to the differences between different datasets, there is no universally applicable optimal setting for the regularization parameter; it needs to be fine-tuned according to different datasets. Generally, smaller L2 regularization weights and random deactivation parameters can achieve good results. In the automatic diagnostic model constructed in this invention, w = 1 × 10⁻⁶ is selected. -4 dr=0 is used as the regularization parameter setting.

[0132] This invention investigates the impact of different label counts on the performance of feature classifiers. First, we train multiple different feature classifiers by varying the number of labeled samples used in the supervised training phase. All feature classifiers use the exact same feature extractor, meaning the feature extractor obtained in the unsupervised training phase remains unchanged. During the experiment, 1024, 1536, 2016, and 2528 samples were randomly selected from the original classification dataset as training subsets, respectively. Training lasted for 200 epochs. The original complete intervertebral disc classification dataset contained 3036 training samples. The four feature classifiers using the training subsets are compared with the feature classifiers obtained using the complete training set. Furthermore, to facilitate comparison between the model built using semi-supervised learning and classic supervised learning models, we use the ResNet50 model for supervised training on the complete intervertebral disc classification dataset with a learning rate of 1×10⁻⁶. -4 After training for 370 epochs, another feature classifier was obtained.

[0133] The performance comparison of the feature classifiers with different training sample numbers and the classic supervised feature classifier is as follows: Figure 11 As shown, Figure 11 (a) represents the accuracy of each model. Figure 11(b) shows the AUC of each model. The horizontal axis represents the number of labels, indicating the amount of labeled data used. The vertical axis represents the accuracy and AUC values, respectively. Supervised models using ResNet50 are indicated by "supv" on the horizontal axis. Figure 11 As can be seen, the accuracy and AUC of the feature classifier both maintain an upward trend as the number of labels increases. It can also be seen that when the change in the number of labels is small, the performance change of the feature classifier under a certain metric may also be small, such as the accuracy performance of the two classifiers when the number of labels is 2528 and 3036, and the AUC performance of the two classifiers when the number of labels is 1024 and 1536. However, when the number of labels increases significantly, a significant improvement in model performance can generally be observed. This also shows that the performance of the two-stage semi-supervised learning model has a certain stability when using training subsets.

[0134] On the other hand, the supervised model used the complete original training set with 3036 labels, while the semi-supervised learning model provided in this invention only requires 1024 labels to achieve a similar level. When trained on the complete original training set, the semi-supervised learning model outperforms classic supervised learning in overall performance. This demonstrates that the two-stage semi-supervised learning paradigm can extract effective semantic information from a large amount of unlabeled intervertebral disc MRI data, thereby strengthening supervised classification learning and enabling the feature classifier to achieve better results.

[0135] In this invention, heatmaps are used to visualize the data that are identified as positive samples by the feature classifier. Some samples, for example... Figure 12 As shown, for positive intervertebral disc samples, the heatmap can identify the cross-sectional locations with more severe degenerative changes, displaying them with higher heat values. The feature classifier's classification is primarily based on the intervertebral disc region in the image, indicating that the feature classifier can extract effective semantic information from the image. For herniated or bulging discs, the heatmap clearly shows the location of the herniation or bulge, providing valuable reference information.

[0136] The present invention provides a method for constructing an automatic diagnostic model for lumbar disc herniation. This method employs a two-stage semi-supervised learning approach, using multi-layered cross-sectional scan images of the intervertebral disc as input. It extracts comprehensive semantic information to determine the disc morphology. By training the model using training subsets with varying numbers of labels, it is observed that the two-stage semi-supervised training model achieves good performance in intervertebral disc MRI image classification tasks. Even with a smaller number of labels, the classifier's performance remains stable, and it achieves a level comparable to classic supervised models using only about one-third of the labeled training subset. Furthermore, this invention improves and adapts the Grad-CAM interpretability analysis algorithm to the model constructed in this invention, visualizing the judgment criteria of the feature classifier in the form of a heatmap. The automatic diagnostic model for lumbar disc herniation constructed in this invention yields reasonable results, can assist in diagnosis to some extent, reduces the burden on radiologists, improves diagnostic efficiency, and provides a reference for subsequent research.

[0137] It is understood that the above specific description of the present invention is only for illustrating the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention to achieve the same technical effect; as long as the use needs are met, they are all within the protection scope of the present invention.

Claims

1. A method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning, characterized in that, The construction method includes the following steps: Step S10: Collect unlabeled lumbar intervertebral disc MRI image samples, construct a training dataset S, and preprocess the MRI images in the training dataset S; in step S10, the MRI image preprocessing includes the following steps: Step S11: When performing each training step, first randomly select n MRI images from the training dataset S to construct a training data subset P; Step S12: Perform two random transformations on each MRI image in the training data subset P to obtain 2n transformed MRI images; the specific steps include defining a random transformation function. And the random transformation function f T Random rotation, random cropping, random flipping, random changes in brightness or contrast, and random Gaussian blur are employed. Random rotation and random cropping are two coupled operations and require the following constraints: (1) In two random transformations of the same image, the cropping regions must overlap; (2) The overlapping regions should have a high probability of being located in the center of the original image; (3) In two random transformations of the same image, the rotation center point of the random rotation operation is consistent, and each random cropping region must contain the center point of the random rotation; (4) During the rotation process, "0" pixel filling areas may be generated around the image, and random cropping should avoid these "0" pixels; (5) The aspect ratio of the random cropping region in the original image needs to be within [3 / 4, 4 / 3]; (6) The proportion of the random cropping region in the original image needs to be greater than a specific value, denoted as p. crop ; Step S20: Construct a first neural network; wherein in step S20, the first neural network includes a ResNet50 network and multiple fully connected layers, wherein the ResNet50 network and the multiple fully connected layers are connected in series; Step S30: Train the first neural network using MRI images from the training dataset S to generate a feature extractor for extracting semantic feature vectors from the MRI images; Step S30 includes the following steps: Step S31: Input the 2n MRI transformation images into the first neural network in pairs, and the first neural network maps a pair of semantic feature vectors for each pair of input MRI transformation images; Step S32: Calculate the predicted probability value of each pair of semantic feature vectors mapped by the first neural network expressing similar semantics based on the softmax function. Where, x i (1≤i≤2n) represents the semantic feature vector obtained by mapping the i-th transformed MRI image through the first neural network, x j (1≤j≤2n) represents the semantic feature vector obtained by mapping the j-th transformed MRI image through the first neural network. The i-th and j-th transformed MRI images are a group and are simultaneously input into the first neural network. τ is a commonly used hyperparameter in the softmax function. Step S33: Optimize the training objective function of the first neural network. Calculate the loss value of the training data subset P, where f E For the first neural network, Expressing expectations; This indicates an indicator function, where Φ is a proposition that states, when Φ is true, When Φ is false Indicates rounding up; Step S34: Train the first neural network using the gradient descent algorithm, minimize the loss value of each training data subset P, and finally generate the feature extractor; Step S40: Construct the second neural network; Step S50: Input the labeled lumbar intervertebral disc MRI image samples into the feature extractor in sequence, and extract the corresponding semantic feature vector for each MRI image sample; Step S60: The second neural network is trained using the semantic feature vector generated in step S50 to generate a feature classifier, which classifies the semantic feature vector and outputs the prediction result.

2. The method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning according to claim 1, characterized in that, The construction method further includes the following steps: Step S70: When the prediction result of the feature classifier is positive, a heat map is generated for the positive sample, and the key information that causes the positive result in the positive sample image is marked.

3. The method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning according to claim 2, characterized in that, In step S50, each labeled lumbar disc MRI image sample includes multiple axial MRI images of the same lumbar disc.

4. The method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning according to claim 3, characterized in that, Step S60 includes the following steps: Step S61: Obtain the semantic feature vector of each labeled lumbar intervertebral disc MRI image sample, and train the second neural network to perform feature classification on the semantic feature vector; Step S62: Use the Sigmoid function to calculate the probability value that the currently trained lumbar intervertebral disc MRI image sample is a positive sample; Step S63: Using the methods of steps S61 to S62, train the second neural network with multiple labeled lumbar intervertebral disc MRI image samples to generate a feature classifier.

5. The method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning according to claim 3, characterized in that, In step S70, the method for generating a heatmap for a positive sample includes the following steps: Step S71: Obtain the feature map set A output by the feature extractor for each labeled lumbar intervertebral disc MRI image sample; Step S72: According to the formula Calculate the weight values ​​for each feature map, where u and v are the height and width of the deepest feature map, respectively. Let be the gradient of k feature maps, 1≤k≤n; Step S73: According to the formula Normalize each weight value, where σ(·) represents the standard deviation; Step S74: Calculate the weight values ​​of each feature map, expressed as follows: Where 'a' represents the number of axial MRI images contained in each labeled lumbar disc MRI image sample. Step S75: Sum the n feature maps of each image with their corresponding weight values ​​to obtain a heatmap; Step S76: Standardize each heatmap: Among them, H i This represents the heatmap corresponding to the i-th cross-section of the sample. The max(·) function finds the maximum element value in the tensor. A (k) Let be the k-th feature map in the feature map set A.

6. The method for constructing an automatic diagnostic model for lumbar disc herniation based on semi-supervised learning according to claim 1, characterized in that, Random transformation function for random transformation processing of MRI images This includes random rotation, random cropping, random flipping, random changes in brightness or contrast, and random Gaussian blur. Two random transformations of the same MRI image express similar semantic features, while random transformations of different MRI images express different semantic features.

7. An automatic diagnostic model for lumbar disc herniation based on semi-supervised learning, wherein the automatic diagnostic model automatically outputs analysis and prediction results based on input lumbar disc MRI images, characterized in that, The automatic diagnostic model is obtained using the construction method described in any one of claims 1 to 6, and the automatic diagnostic model includes: Feature extractor, used to extract semantic feature vectors from MRI images; A feature classifier is used to classify semantic feature vectors and output prediction results; The heatmap generation module generates a heatmap when the prediction result is positive, marking the key information that causes the positive result in the positive MRI image.

Images