A medical image processing method based on semi-supervised neural network

Abstract

This invention discloses a medical image processing method based on a semi-supervised neural network. S1: Acquire an image dataset; S2: Construct and train a DFCPS model. The DFCPS model includes a data augmentation strategy and a neural network. The neural network includes a backbone network, an ASPP module, an upsampling module, and a softmax function. The backbone network uses ResNet-50 and introduces residual connections to construct a deep network. The ASPP module includes four convolutional layers with dilated convolution rates of 1, 12, 24, and 36, respectively. S3: Use the trained neural network for image segmentation. This method trains the model and achieves ideal results by rationally utilizing unlabeled data and a small amount of labeled data, as well as strong and weak data augmentation techniques.

Description

Technical Field

[0001] The present invention relates to the technical fields of data enhancement and semantic segmentation, and specifically to a medical image processing method based on a semi-supervised neural network. Background Art

[0002] In medicine, in order to conduct a comprehensive diagnosis of the patient's condition, determine whether there are organ lesions, and propose corresponding treatment plans, the methods commonly used on patients include: DR scan, CT scan or MRI (magnetic resonance imaging) scan. After obtaining the corresponding images, the doctor will use the naked eye to observe and determine the lesions of the patient's internal organs. Before the rise of deep learning, this process was often completed by experienced doctors through direct observation.

[0003] While doctors have a low misdiagnosis rate and high accuracy, training doctors who meet these requirements requires significant time and financial investment. Furthermore, doctors, as human beings, are subject to emotional fluctuations, lack of energy due to long hours working, and other factors, leading to instability in their judgment accuracy. Therefore, to reduce the misdiagnosis rate, medical image segmentation has emerged to assist doctors in diagnosis.

[0004] Neural networks, as an important tool in semi-supervised learning, have achieved remarkable results in medical image segmentation. They are capable of learning complex feature representations from large-scale image data and optimizing them through an end-to-end training process. However, due to the limitations of labeled data and the complexity of the models, neural networks trained solely on limited labeled data struggle to achieve satisfactory performance in medical image segmentation. Therefore, the need for neural networks that address these challenges is urgent.

[0005] Self-training for medical images is a semi-supervised learning method that leverages unlabeled data to improve model performance through continuous iterative training, thereby reducing the amount of labeled data required. Self-training can be subject to high pseudo-label noise, and the performance of self-training methods depends on the quality and reliability of the pseudo-labels. Therefore, the choice of pseudo-labels requires careful consideration of confidence and noise. Due to the similarities in medical anatomical structures, labeled images provide strong reference points for matching and information transfer to unlabeled images.

[0006] The core idea of ​​consistency learning is to perturb the original sample to varying degrees, such as with Gaussian noise, color changes, and random rotations, while maintaining a model's output similar to the original sample. Through consistency learning, the model can gain additional training signal from unlabeled data and learn more robust and generalizable feature representations. However, if the model is not well optimized and is provided with incorrect supervision, the risk of conflict between soft labels and true labels becomes high.

[0007] If the above two problems can be solved, it may be an effective breakthrough in improving the accuracy of medical image segmentation and recognition. Summary of the Invention

[0008] Aiming at the shortcomings of existing technologies, a medical image processing method based on semi-supervised neural networks is proposed. By rationally utilizing unlabeled data and a small amount of labeled data, as well as strong and weak enhancement data augmentation techniques, the model is trained and ideal results are achieved.

[0009] In order to solve the above technical problems, the technical solution of the present invention is:

[0010] A medical image processing method based on a semi-supervised neural network comprises the following steps:

[0011] S1. Obtain image dataset;

[0012] S2. Build and train a DFCPS model, wherein the DFCPS model includes a data enhancement strategy and a neural network, wherein the neural network includes a backbone network, an ASPP module, an upsampling module, and a Softmax function;

[0013] The backbone network adopts ResNet-50, and introduces residual connections to construct a deep network. The ASPP module includes four convolutional layers, and the dilated convolution rates of the four convolutional layers are 1, 12, 24, and 36 respectively;

[0014] The DFCPS model training method is as follows: First, the original sample X is augmented twice with different degrees of strength and weakness through a data augmentation strategy, generating two groups of enhanced samples: one group is strongly enhanced and the other is weakly enhanced. The strong and weak enhanced samples are grouped and combined. The combined strong and weak enhanced samples are fed into four neural networks for training, and the loss value is minimized by continuously optimizing the model parameters.

[0015] During the training process, the model continuously optimizes the parameters of the model to minimize the loss value, prevent overfitting and underfitting, and improve the performance and accuracy of the model. The loss design of the present invention involves two key loss functions in the entire neural network training process: supervision loss L and s and the cross pseudo-supervision loss L cps .

[0016] In summary, the loss function in the entire training process mainly includes supervision loss and cross-pseudo-supervision loss. Supervision loss guides the network's learning by comparing the pseudo-labels of strongly enhanced samples with those of weakly enhanced samples. Cross-pseudo-supervision loss encourages the network to learn more consistent segmentation results overall by comparing the pseudo-segmentation maps generated between different groups. Such a training strategy helps improve the performance of the model and enables the network to better adapt to data and labels from different groups. This design borrows the idea of ​​Fixmatch in the strong and weak enhancement methods and expands it accordingly to be applicable to medical image label segmentation tasks. However, special attention should be paid to the reliability, accuracy, and cleanliness of the pseudo-labels, because the quality of the pseudo-labels directly affects the performance and generalization ability of the model.

[0017] In order to ensure that the quality of the pseudo-label is high enough, the present invention introduces the concept of a confidence threshold, which is defined as μ in the present invention. By setting the confidence threshold, the model can screen out pseudo-labels with higher quality, thereby avoiding negative impact on model training. Specifically, the confidence of the pseudo-label is compared with the confidence threshold. If the confidence of the pseudo-label is close to the confidence threshold, that is, around 0.5, it can be ignored, because such a pseudo-label may not be reliable enough, and the part of the pseudo-segmentation map that is greater than the confidence threshold μ is included in the loss calculation. For strongly amplified samples, the output prediction result and the corresponding pseudo-segmentation map obtained for the target weakly labeled sample that exceeds the confidence threshold are also subjected to cross entropy loss.

[0018] S3. Use the trained neural network for image segmentation

[0019] S3-1. Take the preprocessed dataset as input and obtain the feature map through the backbone network;

[0020] S3-2, taking the acquired feature map as input, the ASPP module uses multiple parallel branches to implement multi-scale comprehensive feature extraction;

[0021] S3-3. The obtained comprehensive feature representation is represented by increasing data details through upsampling technology, and the output of the model is mapped to the probability of each category through the Softmax function. Then, the most likely category is selected as the prediction result based on the probability to generate a prediction sample with high credibility.

[0022] Preferably, in step S1, the image data is preprocessed using a data enhancement strategy.

[0023] Preferably, the data enhancement strategy includes random rotation, random brightness, contrast adjustment, and random translation.

[0024] Preferably, in step S2, during training, each group of neural networks for strong and weak enhancements share parameters and weights, and the pseudo labels generated by the prediction results output by the neural network after weak enhancement in each group serve as the target for prediction of each strongly enhanced sample.

[0025] As an advantage, in step S2, during the training process, the supervision loss L is introduced. s , which uses the information of strong and weak enhancement samples to guide the learning process of the DFCPS model. Specifically, the difference between the prediction results of the strong enhancement samples generated by the neural network and the pseudo labels of the corresponding weak enhancement samples is used to calculate the supervision loss, and then iterate the model parameters.

[0026] In the above technical solution, the supervision loss L s This is achieved by using the pseudo-labels generated from weakly augmented samples as targets for strongly augmented samples. Specifically, the supervised loss is calculated using the difference between the predictions generated by the neural network for strongly augmented samples and the pseudo-labels for the corresponding weakly augmented samples. This aims to allow the network to learn the mapping from weakly augmented samples to strongly augmented samples, thereby improving the model's prediction accuracy on strongly augmented samples. By minimizing the supervised loss, the network is encouraged to learn the correct segmentation targets.

[0027] As an advantage, in step S2, the training process design introduces a cross pseudo-supervision loss L cps , so that the pseudo labels generated by different groups of weakly supervised versions of samples will constrain each other.

[0028] In the above technical solution, the cross pseudo-supervision loss L cps This is used to constrain the differences between pseudo-segmentation maps generated by different groups. This design treats the pseudo-labels generated by each group of weakly supervised examples as the target for training the weakly supervised examples of other groups, and uses a cross-entropy loss function to calculate the loss between the pseudo-labels. This cross-pseudo-supervision allows the pseudo-labels of different groups to influence and correct each other, improving overall segmentation consistency.

[0029] Preferably, the supervision loss L s Determined by the cross entropy loss function, normal supervised learning for labeled samples is expressed as follows:

[0030]

[0031] Among them, D u : represents the set of unlabeled samples, D l Represents a set of samples with existing labels, S is defined as the area of ​​the input image, calculated by height*width, p i is the corresponding confidence vector, y j is the ground truth, Lce is the cross entropy loss function.

[0032] Preferably, the cross pseudo-supervision loss is defined as and in It represents the supervision loss of labeled data. It represents the supervision loss of unlabeled data and is expressed as follows:

[0033]

[0034] Therefore, the cross pseudo-supervision loss can be defined as:

[0035]

[0036] The total loss can then be defined as:

[0037] Loss = L s +ωL cps (5)

[0038] Where ω is the weight.

[0039] In this technical solution, an initial model is first trained using labeled samples with true labels. This initial model is then used to make predictions for unlabeled samples, and the predictions are added to these samples as pseudo-labels. These pseudo-labeled unlabeled samples are then combined with labeled samples with true labels to form an expanded training set. Finally, the model is retrained using the expanded training set. Through repeated iterations, the model gradually leverages information from the unlabeled data to improve performance.

[0040] Preferably, the specific method of step S3-1 is: using the trained DFCPS model, in the backbone network ResNet-50, the preprocessed data set is used as input, and the pooling layer and the fully connected layer are used for feature extraction and classification. A deep network is constructed by introducing residual connection. Residual connection allows information to skip some layers directly, so that the network can learn the identity mapping more easily, avoiding the problem of deep network degradation. The network structure of ResNet-50 is relatively deep, containing 50 convolutional layers, including multiple residual blocks. Each residual block consists of two convolutional layers, one of which is used to reduce the size of the feature map, and the other is used to keep the size of the feature map unchanged.

[0041] Preferably, the specific method of step S3-2 is as follows: after obtaining the corresponding output, the sample enters the ASPP module, and multiple parallel branches are used to realize multi-scale feature extraction. By using hole convolution to increase the receptive field, the ASPP module can make full use of the contextual information of the input feature map, which can help the model better understand the semantic associations of different positions in the image, and can be flexibly embedded in different network architectures. Each branch adopts a different sampling rate (also called hole rate or expansion rate), which determines the size of the receptive field of the branch on the input feature map. The sampling rate is positively correlated with the receptive field. By using strong and weak enhanced samples and the ASPP module, the neural network designed by the present invention can obtain global and local contextual information from different scales to improve the performance of the image segmentation task.

[0042] Preferably, the specific method of step S3-3 is as follows: the final classification or prediction task provides a more discriminative feature representation. After convolution, batch normalization, and ReLU function activation, the results of all dilated convolution branches and spatial pyramid pooling branches are connected in parallel to form a comprehensive feature representation. A 1x1 convolution layer is then applied to the input features to reduce the dimensionality, thereby reducing the number of channels. This helps reduce the computational burden by reducing the number of channels of the input features to a smaller dimension, thereby reducing computational complexity and extracting more abstract features. Next, the obtained comprehensive feature representation is upsampled to increase data detail, thereby improving image quality. Upsampling is a processing technique used to increase low-resolution or low-frequency data to a higher resolution or frequency. The upsampled comprehensive features are connected with the underlying features that have undergone convolution to form a more comprehensive feature representation. After upsampling and the final convolution, the input features are nonlinearly transformed and feature extracted through the convolution layer operation to produce a higher-level feature representation. These operations help capture more complex patterns and structures in the image and provide more discriminative feature representations for the final classification or prediction task. The output of the model is mapped to the probability of each category through the Softmax function. Then, the most likely category can be selected as the prediction result based on the probability to generate a highly credible prediction sample.

[0043] The present invention has the following characteristics and beneficial effects:

[0044] The present invention trains the model and achieves ideal results by rationally utilizing unlabeled data and a small amount of labeled data, as well as strong and weak data enhancement techniques. First, the original sample X is subjected to two different degrees of strong and weak data enhancement to generate two groups of enhanced samples: one group is strongly enhanced samples and the other group is weakly enhanced samples. The strong and weak enhanced samples are grouped and combined, and the combined strong and weak enhanced samples enter four different neural networks F(θ n), using the information of strong and weak enhancement samples to guide the learning process of the model and improve the performance of the model.

[0045] The backbone network used in the present invention is ResNet-50, which mainly solves the problems of gradient vanishing and gradient exploding in the training process of deep neural networks, and constructs a deep network by introducing residual connections. Residual connections allow information to skip some layers directly, so that the network can learn the identity mapping more easily, avoiding the problem of deep network degradation. The network structure of ResNet-50 is relatively deep, containing 50 convolutional layers, including multiple residual blocks. Each residual block consists of two convolutional layers, one of which is used to reduce the size of the feature map, and the other is used to keep the size of the feature map unchanged. In addition, ResNet-50 also uses pooling layers and fully connected layers for feature extraction and classification. The network structure of ResNet-50 is very suitable for processing large-scale image classification tasks, and has strong expressive power and generalization capabilities. It has been widely used in various tasks in the field of computer vision, such as image classification, target detection, and semantic segmentation, and has achieved remarkable results in multiple computer vision tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a network architecture diagram of the DFCPS model in an embodiment of the present invention.

[0048] Figure 2 This is the neural network structure diagram in the DFCPS model.

[0049] Figure 3 Design a graph for the designed loss function.

[0050] Figure 4 This is a Loss trend diagram of an embodiment of the present invention.

[0051] Figure 5 This is a mIOU curve diagram of an embodiment of the present invention.

[0052] Figure 6 2 is a comparison diagram of the segmentation structure of an embodiment of the present invention.

[0053] Figure 7This is a mIOU comparison curve diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0054] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0055] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0056] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0057] The present invention provides a medical image processing method based on a semi-supervised neural network, comprising the following steps:

[0058] S1. Obtain image dataset;

[0059] This embodiment provides two datasets, namely the PASCAL VOC 2012 dataset and the Kvasir-SEG dataset.

[0060] PASCAL VOC 2012 dataset: PASCAL VOC 2012 is a standard object-centric semantic segmentation dataset consisting of more than 13,000 images, including 20 object classes and 1 background class. The standard training set, validation set, and test set consist of 1,464, 1,449, and 1,456 images respectively. [3]. It covers a wide range of real scenes and object categories, such as people, vehicles, animals, furniture, etc. The images in the dataset come from various sources, including Internet images, annotated images, and contributions from professional photographers. Each image has a corresponding pixel-level annotation that is used to mark the location and category of the object in the image. For the target detection task, each object has a bounding box annotation; for the semantic segmentation task, each pixel is labeled as to which category it belongs to. This design will use this dataset as a pre-training set to obtain pre-training weights.

[0061] The Kvasir-SEG dataset is based on the previous Kvasir dataset, which is the first multi-class dataset for gastrointestinal (GI) disease detection and classification. It contains a series of endoscopic images, including gastroscopy and colonoscopy images. These images cover a variety of common gastrointestinal diseases, such as polyps, ulcers, and cancers. Each image is equipped with a pixel-level segmentation label that marks the boundaries of different lesion areas in the image. The original Kvasir [6] The dataset contains 8,000 GI tract images from 8 categories, each category consists of 1,000 images. [6] The segmented polyp dataset replaces 13 polyp images with new ones to improve the quality of the dataset. This experiment uses a dataset of 1,000 intestinal polyp images, and the model will be used to identify specific lesion areas of intestinal polyps.

[0062] According to the CPC model's partitioning protocol, the Kvasir-SEG dataset was randomly divided into two groups: one containing 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the labeled data, and the other containing the remaining unlabeled data, referred to as the unlabeled group, to simulate label scarcity. During the evaluation process, the baseline model and the designed model were trained and tested using the same backbone network to examine the model's performance in the absence of labeled data. This setup closely simulates the performance differences of the evaluation models in the medical image field when labeled data is scarce.

[0063] S2. Construct and train a DFCPS model. The DFCPS model is as follows: Figure 1 As shown, including data enhancement strategies and neural networks.

[0064] Further, such as Figure 2 As shown, the neural network includes a backbone network, an ASPP module, an upsampling module and a Softmax function;

[0065] The backbone network adopts ResNet-50, and introduces residual connections to construct a deep network. The ASPP module includes four convolutional layers, and the dilated convolution rates of the four convolutional layers are 1, 12, 24, and 36 respectively;

[0066] The DFCPS model training method is as follows: First, the original sample X is augmented twice with different degrees of strength and weakness through a data augmentation strategy, generating two groups of enhanced samples: one group is strongly enhanced and the other is weakly enhanced. The strong and weak enhanced samples are grouped and combined. The combined strong and weak enhanced samples are fed into four neural networks for training, and the loss value is minimized by continuously optimizing the model parameters.

[0067] During the training process, the model continuously optimizes the parameters of the model to minimize the loss value, prevent overfitting and underfitting, and improve the performance and accuracy of the model. The loss design of the present invention involves two key loss functions in the entire neural network training process: supervision loss L and s and the cross pseudo-supervision loss L cps .

[0068] Specifically, in this embodiment, pre-training will first be completed in the PASCAL VOC 2012 data set. By pre-training on this rich data set to obtain corresponding pre-training weights, the neural network can learn rich visual features, thereby improving its performance on various image segmentation tasks. The purpose of pre-training is to enable the network to have a better understanding and representation of the characteristics of the image, and to provide a good initial state for subsequent fine-tuning. Fine-tuning is then performed based on the specific tasks of the Kvasir-SEG data set, and optimization is performed for this specific task so that the network can better adapt to the characteristics and segmentation requirements of medical images. Fully consider the data characteristics, task requirements and model limitations, as well as the processing and verification of pseudo-labels. As data enhancement, random cropping, rotation, Gaussian noise and random jitter with additional color flipping will be used to achieve the design goals. Specifically, the data enhancement parameters are shown in Table 1:

[0069]

[0070] Table 1: Data augmentation methods

[0071] We trained the PASCAL VOC 2012 dataset for 60 epochs with a base learning rate of 0.01. We then transferred the pre-trained model weights to the Kvasir-SEG dataset for 100 epochs. We then determined the current batch_size and adaptively adjusted the learning rate. The default batch_size was set to 12, the maximum learning rate to 1e-4, and the minimum learning rate to 0.01 times the maximum learning rate.

[0072] Furthermore, training is divided into two phases: freezing and thawing. In the freezing phase, some or all parameters of the pre-trained model are fixed, and only the newly added layers are trained. In the thawing phase, after a certain number of training cycles, the restrictions on the pre-trained model parameters are removed, allowing it to participate in the training of the entire model.

[0073] The main advantage of freezing training is that it speeds up training. The following are the specific steps for freezing training:

[0074] 1. Create a base model: First, create a base model, i.e., the backbone network, on which to perform frozen training. This base model can be a trained model or a new, untrained model.

[0075] 2. Freeze the layers of the model: Determine which layers will be frozen. Typically, the bottom layers (i.e., the layers around the input layer) and the top layers (the layers around the output layer) of the model will be retained because these layers contain the core functions of the model, while the middle layers will be frozen.

[0076] 3. Adding new layers: In order to adapt to specific problems, it is necessary to add new layers to the frozen model or replace some intermediate layers to increase the depth or width of the model.

[0077] 4. Freeze the weights of the layers: Set the weights of the frozen layers to non-trainable and lock their weights. This means that these layers will not be updated during training.

[0078] 5. Compile the model: Set the model's parameters such as loss function, optimizer, and evaluation metrics, and combine them with the model. This is a key step in the training process.

[0079] 6. Train the model: After setting the loss function, optimizer, and evaluation metrics, begin training the model. Typically, the model is trained using batches of data as input. Only the added layers are trained, while the weights of the frozen layers remain unchanged.

[0080] 7. Unlocking Frozen Layers: If the model does not produce satisfactory results, you can fine-tune it by unlocking the previously frozen layers. This allows the weights of these layers to be further trained and adjusted to further improve the performance of the model.

[0081] 8. Evaluate the model: After training, perform a final evaluation of the model using an independent test dataset to evaluate the performance and accuracy of the model.

[0082] Generally speaking, the first few layers of a model learn low-level features, while later layers learn higher-level features. However, in some cases, the parameters of the first few layers already capture the characteristics of the input data well enough and do not require further updates, while later layers require more training to improve performance. By training only the newly added layers, the training computational load can be significantly reduced, thereby improving training efficiency. Furthermore, freezing training helps prevent overfitting. In the early stages of training, model parameters may overfit the training data, resulting in poor generalization performance. Freezing some parameters reduces the model's learning capacity, effectively reducing the risk of overfitting and improving generalization. Because pre-trained models have been trained on large datasets, they have good generalization capabilities. Freezing their parameters can prevent overfitting on small sample datasets, improving the model's generalization performance. Furthermore, freezing training preserves the pre-trained model's weights. In some cases, the pre-trained model's parameters may be of high precision and quality. Freezing the pre-trained model's parameters can prevent them from being corrupted or overwritten. Furthermore, freezing training allows training to continue even when machine performance is insufficient. However, as training progresses, the model gradually learns higher-level features, and fixed parameters may become a bottleneck that limits model performance. To fully utilize the model's expressive power, unfreezing training gradually unfixes the parameters, allowing the entire model's parameters to be updated.

[0083] The main advantage of unfreezing training is to fine-tune the pre-trained model. The following are the specific steps of unfreezing training:

[0084] 1. Freeze some model layers: After completing the training of the backbone model, freeze some model layers layer by layer from the output layer forward, usually selecting layers far away from the output layer for freezing.

[0085] 2. Train some model layers: Train the remaining unfrozen model layers, generally using a smaller learning rate.

[0086] 3. Unfreeze more model layers: As the model continues to improve during training, continue to unfreeze more model layers, gradually making the model more complex.

[0087] 4. Repeat steps 2 and 3: Repeat steps 2 and 3 until all model layers are trained.

[0088] 5. Overall fine-tuning: Finally, the entire model is fine-tuned to further improve performance and accuracy.

[0089] By removing restrictions on the pre-trained model's parameters and incorporating them into overall model training, model performance can be further optimized. The pre-trained model already possesses strong feature extraction capabilities, and unfreezing and fine-tuning can further adapt it to the target task. Unfreezing training can improve the model's performance on the target task and further enhance its generalization. In summary, the combination of frozen and unfreezing training leverages the advantages of the pre-trained model, accelerating training and improving the model's generalization.

[0090] After model training is complete, appropriate evaluation metrics are selected to evaluate and compare the results of different models on the training or validation sets. By calculating the values ​​of various metrics, the model with the best performance is selected as the final choice. It is then applied to the dataset segmentation task and its segmentation results are observed. The clarity, accuracy, and consistency of the segmentation boundaries, as well as the degree of compliance with professional knowledge and clinical applications, are evaluated. Through these evaluations, the model's performance in the dataset segmentation task is ultimately determined, and its performance is comprehensively evaluated.

[0091] Further, such as Figure 3 As shown in the figure, in summary, the loss function in the entire training process mainly includes supervision loss and cross-pseudo-supervision loss. Supervision loss guides the learning of the network by comparing the pseudo labels of strongly enhanced samples with those of weakly enhanced samples. Cross-pseudo-supervision loss encourages the network to learn more consistent segmentation results as a whole by comparing the pseudo segmentation maps generated between different groups. Such a training strategy helps to improve the performance of the model and enables the network to better adapt to data and labels from different groups. This design borrows the idea of ​​Fixmatch from the strong and weak enhancement methods and expands it accordingly to suit the task of medical image label segmentation. However, special attention should be paid to the reliability, accuracy, and cleanliness of the pseudo labels, because the quality of the pseudo labels directly affects the performance and generalization ability of the model.

[0092] The supervision loss L s Determined by the cross entropy loss function, normal supervised learning for labeled samples is expressed as follows:

[0093]

[0094] Among them, D u : represents the set of unlabeled samples, D l Represents a set of samples with existing labels, S is defined as the area of ​​the input image, calculated by height*width, p i is the corresponding confidence vector, y j is the ground truth, l ce is the cross entropy loss function.

[0095] The cross-pseudo-supervision loss is defined as and in It represents the supervision loss of labeled data. It represents the supervision loss of unlabeled data and is expressed as follows:

[0096]

[0097] Therefore, the cross pseudo-supervision loss can be defined as

[0098]

[0099] The total loss can be defined accordingly as, where ω is the weight

[0100] Loss = L s +ωL cps .

[0101] In order to ensure that the quality of the pseudo-label is high enough, the present invention introduces the concept of a confidence threshold, which is defined as μ in the present invention. By setting the confidence threshold, the model can screen out pseudo-labels with higher quality, thereby avoiding negative impact on model training. Specifically, the confidence of the pseudo-label is compared with the confidence threshold. If the confidence of the pseudo-label is close to the confidence threshold, that is, around 0.5, it can be ignored, because such a pseudo-label may not be reliable enough, and the part of the pseudo-segmentation map that is greater than the confidence threshold μ is included in the loss calculation. For strongly amplified samples, the output prediction result and the corresponding pseudo-segmentation map obtained for the target weakly labeled sample that exceeds the confidence threshold are also subjected to cross entropy loss.

[0102] After training, in this embodiment, mIOU (Mean Intersection over Union) is further used as an indicator to evaluate model performance. By calculating the intersection over union ratio of the model's predicted segmentation results and the true labels for each category and taking the average, a comprehensive performance evaluation indicator can be obtained. It measures the degree of overlap between the model's predicted segmentation results and the true labels. The higher the mIOU, the higher the model's segmentation accuracy for different categories. The model is evaluated every 10 iterations.

[0103] However, simply observing the mIOU value does not necessarily mean higher is better. A higher mIOU value can reflect better generalization performance of the model. It is also necessary to observe the size of the loss value to determine whether the model has converged, but more importantly, observe its trend, especially the trend of the validation set loss. When the validation set loss continues to decrease, it can be considered that the model is converging. If the validation set loss remains basically unchanged, then the model has basically converged. When the model converges, the loss value usually shows a downward trend, especially the loss on the validation set. This means that the model is continuously optimizing during training and gradually approaching the optimal solution. If the validation set loss remains basically unchanged, it means that the model has stabilized and further optimization may bring smaller benefits.

[0104] To evaluate model performance, we evaluate the model every 10 iterations during training. This allows for timely monitoring of model performance and allows for adjustments and optimization based on the evaluation results. During the evaluation process, we use image samples from the test set for inference and compare them with the corresponding true labels to calculate the mIOU metric.

[0105] During the evaluation process, the model that performs best on the training set is selected and applied to the test set to verify the model's generalization ability on unseen data. By applying the best model to the test set, a more accurate and reliable evaluation result is obtained, further verifying the model's performance. This design ensures the model's stability and generalization ability across different datasets.

[0106] The CPC and CPS models were used as baselines for comparison with the designed model to evaluate model performance. This comparative evaluation with the CPC and CPS models provides a comprehensive understanding of the performance of the neural network model designed in this paper in semi-supervised learning tasks. This comparative analysis helps validate the advantages and innovations of the designed model and provides guidance for further improvement and optimization.

[0107] When the constructed DFCPS model meets the mIOU indicator, the segmentation model trained with DFCPS is used for simulation experiments:

[0108] 1. Take the preprocessed dataset as input and obtain the feature map through the backbone network;

[0109] 2. Take the acquired feature map as input and use the ASPP module to implement multi-scale comprehensive feature extraction through multiple parallel branches;

[0110] 3. The obtained comprehensive feature representation is represented by increasing data details through upsampling technology, and the output of the model is mapped to the probability of each category through the Softmax function. Then, the most likely category is selected as the prediction result based on the probability to generate a highly credible prediction sample.

[0111] 4. Use the mIOU metric to compare the performance of model prediction results across different experimental settings.

[0112] In this embodiment, given the significant impact of data augmentation strategies on the FixMatch model, we designed two sets of ablation experiments to verify the impact of different levels of data augmentation strategies on the DFCPS model. The original data augmentation strategy for this model was to perform different levels of strong and weak augmentation on the same group of samples, combining the strong and weak augmentation strategies into a group before entering the network.

[0113] In the current ablation experiment setup, this design sets two custom data augmentation strategies for the experiment. One data augmentation strategy is to perform two strong augmentations of varying degrees on the same set of image samples. These are combined, leaving all other conditions unchanged, and then fed into the network to obtain the results for observing model performance. The other data augmentation strategy is to perform two weak augmentations of varying degrees on the same set of original image samples. These are combined, leaving all other conditions unchanged, and then fed into the network to observe model performance. In addition, the original samples (without any data augmentation strategy processing) are directly fed into the network for training and evaluating model performance. By comparing the aforementioned control data augmentation strategy with the data augmentation strategy used by DFCPS and analyzing the impact of different degrees of data augmentation strategies on the DFCPS model, we can better understand the role of data augmentation in improving model performance and provide guidance for further model optimization.

[0114] In this example, the experimental server is configured with six Nvidia GTX 2080Ti graphics cards with a memory size of 64GB and runs on an Ubuntu 20.04.1 system. The version of opencv_python is 4.1.2.30

[0115] After 100 epochs of training, we visualized the loss and mIOU. By observing the loss trend and mIOU curve, we can see that the loss gradually stabilized, while the mIOU value reached over 80%, which is a very ideal result.

[0116] In the visualization of Loss trend, such as Figure 4 As shown in , it is observed that the Loss gradually decreases and stabilizes during the training process, which indicates that the training of the model has converged. In the visualization of the mIOU curve, such as Figure 5As shown, the mIOU value gradually increases as training progresses, eventually exceeding 80%, and remains relatively accurate in the later stages of training. This demonstrates that the model performs well for segmentation tasks, accurately capturing object boundaries in images and performing segmentation, with good performance and generalization capabilities. Therefore, the overall design proposed in this paper achieves the expected performance level, and these results further verify the effectiveness and feasibility of the proposed method.

[0117] Through this neural network, we can obtain the segmentation image of intestinal polyps, such as Figure 6 As shown in Figure 1, in addition to numerical indicators, visual evaluation is also an important method for evaluating medical image segmentation results. By comparing the segmentation results with the true labels, we can visually check their boundary clarity, contour consistency and other characteristics. When observing the segmentation results with the naked eye, we can make the following observations:

[0118] First, the segmentation boundaries are clear, and the edge contours of intestinal polyps can be clearly distinguished without any blur or ambiguity. Secondly, the segmentation results are basically consistent with the contours of the true labels, that is, the segmented areas match the actual locations of intestinal polyps. In addition, the segmentation results do not miss or mis-segment, and no intestinal polyp areas are missed or normal areas are mistakenly labeled as intestinal polyps. In addition to boundary clarity and contour consistency, the segmentation results are also consistent with medical expertise and clinical application requirements. This means that the model can accurately mark the lesion area, and the segmented area contains the anatomical structure of interest, which can basically complete the designed task. The performance advantages of this model are further verified by comparative evaluation with the baseline model. Compared with the baseline model, this model performs better in contour segmentation and clarity. Its pixel accuracy is slightly higher than the baseline model, indicating that this model can perform pixel-level segmentation more accurately.

[0119] In summary, visual evaluation shows that this model performs well on the intestinal polyp segmentation task. The segmentation results have clear boundaries, are generally consistent with the outlines of the ground truth, and have no missed or mis-segmented instances. Furthermore, the segmentation results meet medical expertise and clinical application requirements.

[0120] Table 2 shows the mIOU values ​​of different methods under different label dataset ratios under the RESNET-50 model. Compared with the CPC and CPS baseline groups, the proposed method performs better. In particular, when the label coverage is 1 / 2, DFCPS achieves the best performance, which is significantly higher than CPC. [2] and CPS [4]The DFCPS method designed in this embodiment improves the mIOU values ​​by 2.21% and 1.65% respectively. These results show that the method of the present invention has certain advantages in medical image segmentation tasks under the condition of lack of annotation and shows ideal performance.

[0121]

[0122] Table 2mIOU values

[0123] By comparing the line chart, such as Figure 7 As shown in the figure, it can be clearly seen that the method of the present invention achieves better performance under different label dataset ratios. Compared with the baseline group, the DFCPS model achieves higher mIOU values ​​under the conditions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 label coverage, showing a significant advantage. This further demonstrates its stability and robustness under different dataset partitioning conditions, indicating that the model is more competent for medical image annotation tasks when annotations are scarce. It also shows that the method of the present invention can more accurately annotate medical images and improve the model's ability to detect and locate targets. As can be seen from the visualization diagram, the semi-supervised learning neural network designed by the present invention shows better performance than the baseline group in all the partitioned datasets. These experimental results further support the feasibility of the semi-supervised learning neural network designed by the present invention in solving the medical image annotation problem. Compared with the baseline group, the method of the present invention fully utilizes the limited labeled data and uses unlabeled data for model training and optimization through the concept of semi-supervised learning. This method not only improves training efficiency but also significantly improves model performance, providing an effective solution to the challenge of data scarcity in medical image annotation.

[0124] In ablation experiments, we can observe the impact of different augmentation strategies on model performance. The results show (Table 3) that the weak-weak augmentation combination outperforms the strong-strong augmentation combination, while directly using the original samples without any data augmentation strategy has the worst performance. In addition, the DFCPS model designed in this paper performs best when using the strong-weak augmentation combination data augmentation strategy, which shows that the data augmentation strategy adopted in this work is reasonable.

[0125]

[0126] Table 3: mIOU values ​​under different enhancement strategies

[0127] In the experimental environment of six 2080ti cards, the training time is compared with the baseline model (Table 4), DFCPS is compared with CPS [4] and CPC [2]The model has a longer training time. In DFCPS, the increase in training time mainly comes from two aspects. First, DFCPS needs to use additional neural network models to extract features and calculate the consistency loss of features. This involves training and optimizing multiple models, which will increase the overall training time. Second, DFCPS also needs to update the policy network and feature extraction network through backpropagation. This process usually requires multiple iterations and training cycles in order to achieve better performance. In contrast, CPS [4] and CPC [2] Models may omit this feature consistency loss and backpropagation step, so their training time may be relatively short. After DFCPS completes training, the time consumption for inference using the learned policy is usually similar. Since DFCPS performs better in terms of performance, we believe this trade-off is worthwhile.

[0128]

[0129] Table 4: Time comparison

[0130] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions, and variations of these embodiments, including components, without departing from the principles and spirit of the present invention are still within the scope of protection of the present invention.

Claims

1. A medical image processing method based on a semi-supervised neural network, characterized in that: The steps include: S1. Obtain image dataset; S2. Build and train a DFCPS model, wherein the DFCPS model includes a data enhancement strategy and a neural network, wherein the neural network includes a backbone network, an ASPP module, an upsampling module, and a Softmax function; The backbone network adopts ResNet-50, and introduces residual connections to construct a deep network. The ASPP module includes four convolutional layers, and the dilated convolution rates of the four convolutional layers are 1, 12, 24, and 36 respectively; The DFCPS model training method is as follows: First, the original sample X is augmented twice with different degrees of strength and weakness through a data augmentation strategy, generating two groups of enhanced samples: one group is strongly enhanced and the other is weakly enhanced. The strong and weak enhanced samples are grouped and combined. The combined strong and weak enhanced samples are fed into four neural networks for training, and the loss value is minimized by continuously optimizing the model parameters. During training, the neural networks for each group of strong and weak augmented samples share parameters and weights. The pseudo labels generated by the prediction results of the weakly augmented samples in each group through the neural network serve as the prediction targets for each strongly augmented sample. During the training process, the supervision loss is introduced as L s , which uses the information of strong and weak enhancement samples to guide the learning process of the DFCPS model; The supervision loss L s Determined by the cross entropy loss function, normal supervised learning for labeled samples is expressed as follows: Among them, D l Represents a set of samples with existing labels, S is defined as the area of ​​the input image, calculated by height*width, p i is the corresponding confidence vector, y j is the ground truth, l ce is the cross entropy loss function; The training process design introduces the cross pseudo-supervision loss L cps , so that the pseudo labels generated by different groups of weakly supervised samples will constrain each other; The cross-pseudo-supervision loss is defined as and in It represents the supervision loss of labeled data. It represents the supervision loss of unlabeled data and is expressed as follows: Among them, D u represents the set of unlabeled samples; Therefore, the cross pseudo-supervision loss is defined as: The total loss can then be defined as: Loss=L s +ωL cps (5) Where ω is the weight; S3. Use the trained neural network for image segmentation S3-1. Take the preprocessed dataset as input and obtain the feature map through the backbone network; S3-2, taking the acquired feature map as input, extracting features through the ASPP module and fusing them to obtain comprehensive features; S3-3. The obtained comprehensive features are used to increase data details through upsampling technology, and the output of the model is mapped to the probability of each category through the Softmax function. Then, the most likely category is selected as the prediction result based on the probability to generate a prediction sample with high credibility.

2. A medical image processing method based on a semi-supervised neural network according to claim 1, characterized in that: The data enhancement strategy is used to preprocess image data.

3. The medical image processing method based on a semi-supervised neural network according to claim 2, characterized in that: The data augmentation strategies include random rotation, random brightness, contrast adjustment, and random translation.

4. The medical image processing method based on a semi-supervised neural network according to claim 1, characterized in that: The specific method of step S3-1 is as follows: in the backbone network ResNet-50, the preprocessed data set is used as input, the pooling layer and the fully connected layer in ResNet-50 are used for feature extraction and classification, and the residual connection is used to allow information to directly skip some layers, thereby outputting a feature map.

5. The medical image processing method based on a semi-supervised neural network according to claim 4, characterized in that: The specific method of step S3-2 is: using the output of the neural network as the input of the ASPP module, using multiple parallel branches in the ASPP module to perform multi-scale feature extraction, and enhancing the receptive field through void convolution, and finally using the context information of the input feature map to perform semantic association at different positions, and then output the extracted features.

Images