A semi-supervised image segmentation method based on knowledge distillation

Abstract

The present invention provides a semi-supervised image segmentation method based on knowledge distillation, comprising the following steps: obtaining a dataset of raw images to be processed; preprocessing the raw image dataset to obtain labeled data and unlabeled data; constructing a teacher-student network model, using the teacher network to extract and learn features from a small amount of labeled data to generate initial labeled information, which is used as training guidance for the student network; employing a feature distillation method to guide the student network in learning and correcting target feature information using the feature information provided by the teacher network; adaptively training the teacher-student network model using unlabeled data based on a semi-supervised learning strategy to obtain a trained teacher-student network model; and inputting the image to be segmented into the trained teacher-student network model to achieve image segmentation. The present invention is applicable to a variety of image segmentation tasks and provides reliable technical support for fields such as medical image analysis, autonomous driving, and video surveillance.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and relates to a semi-supervised image segmentation method based on knowledge distillation. Background Art

[0002] Image segmentation plays a key role in the field of computer vision and is widely used in tasks such as autonomous driving, video surveillance, and object detection. In these application scenarios, accurately segmenting the target area not only improves the overall recognition performance of the algorithm but also significantly enhances the practicality and stability of the system. For example, autonomous driving systems require accurate segmentation of elements such as pedestrians, vehicles, and roads to ensure driving safety. In video surveillance, precise segmentation of specific target areas enables more effective event detection and behavior recognition. In brain tumor diagnosis, accurately segmenting the tumor area in MRI or CT images can help doctors better understand the size, shape, and location of the tumor, providing important reference for formulating surgical or radiotherapy plans.

[0003] In recent years, with the development of deep learning technology, convolutional neural networks (CNNs), with their powerful feature extraction capabilities, have achieved remarkable results in the field of image segmentation. However, CNNs are limited in their ability to capture global contextual information, making it difficult to meet the global information requirements of certain application scenarios. To address this shortcoming, attention mechanisms, due to their advantage in capturing global information, have been gradually introduced into image segmentation tasks, complementing the local feature extraction capabilities of CNNs.

[0004] While these methods have improved segmentation performance to a certain extent, they typically rely on large amounts of precisely labeled data for training. Accurately labeling data often requires significant time and labor costs, making it difficult to obtain sufficient high-quality labeled data, especially when labeling complex objects or scenes that are difficult to accurately label. Therefore, insufficient data has become a bottleneck restricting further improvements in image segmentation performance. Furthermore, existing methods still have limitations in utilizing unlabeled or coarsely labeled data, resulting in insufficient extraction of target region features, affecting segmentation accuracy and stability.

[0005] To address these issues, semi-supervised learning and knowledge distillation techniques have gradually gained attention. Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data, enabling effective learning even with limited data resources. Knowledge distillation, on the other hand, utilizes a high-performance teacher model to guide the training of a student model, thereby improving the student model's learning and generalization capabilities. However, existing semi-supervised learning and knowledge distillation methods fail to fully integrate the advantages of different architectures in image segmentation tasks and lack targeted feature learning strategies. This results in segmentation accuracy and robustness that fall short of expectations. Therefore, innovative methods are needed to improve segmentation accuracy and robustness. Summary of the Invention

[0006] In order to solve the above problems, the technical solution adopted by the present invention is: a semi-supervised image segmentation method based on knowledge distillation, comprising the following steps:

[0007] Obtain raw image data for image segmentation tasks;

[0008] Preprocess the original image dataset, check and delete duplicate images in the dataset, eliminate damaged or unreadable image files, and annotate some images to obtain a set of images with coarse-grained annotations, a set of images with fine-grained annotations, and a set of unannotated images;

[0009] Constructing a teacher-student network model using a semi-supervised learning strategy, wherein the teacher-student network model constructed using the semi-supervised learning strategy includes a teacher network and a student network;

[0010] The teacher network learns basic segmentation knowledge by training on coarse-grained labeled data, and this knowledge is transferred to the student network through knowledge distillation;

[0011] The feature distillation method and the teacher network generate annotation information of unlabeled images to guide the student network to learn features and segment objects in unlabeled images, thereby improving the overall segmentation accuracy.

[0012] Based on a set of fine-grained annotated images, the segmentation performance of the teacher-student network is verified by comparing the image annotations generated by the teacher-student network with the real image annotations.

[0013] The image to be segmented is input into the trained teacher-student network model to achieve image segmentation.

[0014] Further: The preprocessed image dataset is divided into three parts: a coarse-grained labeled image set for training the teacher network, including image data X t and its corresponding coarse-grained annotation image SL t ;

[0015] Unlabeled image dataset X for training the student network s ,

[0016] A fine-grained annotated image set for evaluating the teacher-student network model, including image data X test and the corresponding fine-grained annotation image FL test .

[0017] Furthermore, the process of adaptively training the teacher-student network model based on the semi-supervised learning strategy to obtain the trained teacher-student network model is as follows:

[0018] Using image data X t and its corresponding coarse-grained annotation image SL t to train the teacher network;

[0019] Using the trained teacher network, for the image data X s Generate annotated image PL s , labeled image PL s With image data X s Together they form the image data for training the student network;

[0020] Use image data X respectively s and the labeled image PL s To jointly train the student network; in this way, the teacher-student network model can not only integrate the teacher network from the labeled image PL s The information learned in s Learn more target-related features;

[0021] Generate image data X using teacher network and student network test The annotated image and its corresponding fine-grained annotated image FL test Comparison is made to verify the segmentation performance of the teacher and student network models.

[0022] Furthermore: the teacher network uses a deep learning segmentation network, ResUNet, AttUNet, SwinUNet or UTransformer network to annotate the image.

[0023] Furthermore: the student network adopts a hybrid convolutional neural network-attention mechanism architecture and is designed based on an encoder-decoder framework;

[0024] The encoder consists of three convolutional layers, each of which includes a normalization layer and an activation function. The normalization layer is used to accelerate the training process and stabilize the network structure, while the activation function enables the network to learn more complex feature representations. Through the three consecutive convolutional operations, the encoder gradually extracts local features of the image while gradually reducing the spatial resolution of the feature map to capture higher-level semantic information.

[0025] After the feature map generated by the encoder completes local feature extraction, it is input into the attention mechanism module for further processing. The attention mechanism module adopts a sliding window strategy. First, the feature map is divided into multiple regular non-overlapping windows, and self-attention calculation is performed in each window. This helps to capture the dependency relationship within the local area.

[0026] The window is then shifted to the right by half its width to generate new overlapping windows, and self-attention calculations are performed again within these overlapping windows. This sliding window approach effectively captures global information across windows and enhances the continuity of feature representation. During each self-attention calculation, the two spatial dimensions of the feature map are merged into one dimension to form an input sequence for the attention mechanism to process. In addition, a learnable relative position offset is introduced to enhance the model's ability to model spatial relationships. The attention mechanism module can perform detailed analysis and processing of feature maps at different scales and positions, thereby improving the performance of the entire model in image understanding tasks.

[0027] The decoder also consists of three layers, each of which includes upsampling, normalization, and activation functions. The upsampling operation gradually restores the spatial resolution of the feature map so that the final output can reach the same size as the input image.

[0028] The normalization layer is used to accelerate the training process and stabilize the network, ensuring the consistency of features across different layers. The activation function introduces nonlinearity, enabling the network to learn and express more complex feature representations.

[0029] Through the continuous operation of these three layers, the decoder not only restores the detailed information of the image, but also combines the features of the corresponding layers in the encoder to ultimately achieve high-resolution segmentation results. This design enables the model to capture local details and global semantics to achieve the final image segmentation goal.

[0030] Furthermore: the attention mechanism introduces a sliding window operation. The attention mechanism consists of 12 layers, and each layer performs an attention calculation. The attention calculation is performed in a regular non-overlapping window, and then the window is shifted to the right by half the width to generate a new window for calculation. In each layer of the attention mechanism, the self-attention calculation of each window is performed through a specific equation, and the two dimensions H and W of the feature map are merged into the same dimension to obtain the input sequence of the attention mechanism. Where N = HW / 16 is the number of elements in the input sequence, and the self-attention calculation of each window is calculated using the following formula:

[0031] [Q; K; V] = [W q ;W k ;W v ]·x i (1)

[0032]

[0033] in, and is the parameter matrix, d is the feature in each window These matrices are multiplied with the feature vector to produce three attention vectors: and Self-attention calculation via a learnable relative position offset Introduce location information to enhance the expression of spatial relationships.

[0034] A semi-supervised image segmentation device based on knowledge distillation, comprising:

[0035] Acquisition module: used to obtain the original image data set to be processed;

[0036] Preprocessing module: used to preprocess the original image dataset, check and delete duplicate images in the dataset, eliminate damaged or unreadable image files, and annotate some images to obtain the preprocessed image dataset, a set of images with coarse-grained annotations, a set of images with fine-grained annotations, and a set of unannotated images;

[0037] Construction module: used to construct a teacher-student network model using a semi-supervised learning strategy, wherein the teacher-student network model includes a teacher network and a student network;

[0038] Training module: The teacher network learns basic segmentation knowledge by training on coarse-grained labeled data. This knowledge is then transferred to the student network through knowledge distillation.

[0039] The feature distillation method and the teacher network generate annotation information of unlabeled images to guide the student network to learn features and segment objects in unlabeled images, thereby improving the overall segmentation accuracy.

[0040] Based on a set of fine-grained annotated images, the segmentation performance of the teacher-student network is verified by comparing the image annotations generated by the teacher-student network with the real image annotations.

[0041] Implementation module: used to input the image to be segmented into the trained teacher-student network model to achieve image segmentation

[0042] A computer device comprises: a processor and a memory, wherein the memory stores a program module, and wherein the program module runs on the processor to implement any one of the methods described above.

[0043] The present invention provides a semi-supervised image segmentation method based on knowledge distillation. This method adopts a hybrid architecture design of convolutional neural networks and attention mechanisms and a feature distillation strategy. It can not only effectively utilize unlabeled data when labeled data is insufficient, but also improve the model's segmentation accuracy for the target area, meeting the requirements for image segmentation accuracy in different application scenarios. The beneficial effects of the present invention are:

[0044] The present invention can solve the limitations of traditional methods in image segmentation tasks. First, by combining the advantages of convolutional neural networks and attention mechanisms, a hybrid architecture design is adopted, which not only retains fine-grained spatial details but also captures global contextual information, thereby achieving a more accurate segmentation effect, which is suitable for complex image segmentation tasks. At the same time, with the help of a semi-supervised learning strategy, unlabeled data is effectively utilized, the dependence on large-scale labeled data is reduced, and the cost of data labeling is significantly reduced. In addition, the present invention introduces knowledge distillation technology, which guides the training of the student model through the teacher model, so that the student model can learn more accurate feature information from the coarsely labeled data, thereby improving the generalization ability of the model. The sliding window operation of the attention mechanism further reduces the computational complexity of the self-attention mechanism, making the model suitable for practical applications while maintaining high performance. The method of the present invention is applicable to a variety of image segmentation tasks, and provides reliable technical support for fields such as medical image analysis, autonomous driving, and video surveillance. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] Figure 1 Schematic diagram of teacher-student framework training.

[0047] Figure 2 A schematic diagram of the overall structure of the model.

[0048] Figure 3 Schematic diagram of the original CT images of lung nodules in the LUNA16 dataset, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C, (d) CT scan image of individual D, and (e) CT scan image of individual E.

[0049] Figure 4 Schematic diagram of the comparison before and after sliding the LUNA16 dataset window, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C.

[0050] Figure 5 Schematic diagram of cropping for the LUNA16 dataset.

[0051] Figure 6Schematic diagram of the original CT images of lung nodules in the LC183 dataset, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C, (d) CT scan image of individual D, and (e) CT scan image of individual E.

[0052] Figure 7 Schematic diagram of the comparison before and after sliding the LC183 dataset window, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C, (d) CT scan image of individual D, and (e) CT scan image of individual E.

[0053] Figure 8 Schematic diagram of lung nodule segmentation effects under different teacher networks in the LUNA16 dataset.

[0054] Figure 9 Schematic diagram of the lung nodule segmentation effect under different teacher networks in the LC183 dataset.

[0055] Figure 10 Schematic diagram of rough lung nodule annotations generated for different teacher models.

[0056] Figure 11 Schematic diagram of the lung nodule segmentation effect under different student segmentations of the LUNA16 dataset.

[0057] Figure 12 Schematic diagram of the lung nodule segmentation effect under different student segmentations of the LC183 dataset. DETAILED DESCRIPTION

[0058] It should be noted that, unless there is any conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] A semi-supervised image segmentation method based on knowledge distillation, characterized by comprising the following steps:

[0061] S1: Obtain the original image dataset to be processed, collect and summarize the original image data for the image segmentation task;

[0062] S2: Preprocess the original image dataset, check and delete duplicate images in the dataset, eliminate damaged or unreadable image files, and annotate some images to obtain a set of images with coarse-grained annotations (accounting for 30% of the total data), a set of images with fine-grained annotations (accounting for 20% of the total data), and a set of unannotated images (accounting for 50% of the total data);

[0063] S3: A teacher-student network model is constructed based on a semi-supervised learning strategy. This model significantly improves the accuracy and efficiency of image segmentation by effectively combining labeled and unlabeled data.

[0064] S4: The teacher network learns basic segmentation knowledge by training on coarse-grained labeled data, and this knowledge is transferred to the student network through knowledge distillation;

[0065] S5: Generate annotation information of unlabeled images through feature distillation method and teacher network to guide student network to learn features and segment objects in unlabeled images, thereby improving the overall segmentation accuracy.

[0066] S6: Based on the fine-grained annotated image set, the segmentation performance of the teacher-student network is verified by comparing the image annotation results generated by the teacher-student network with the real image annotations;

[0067] S7: Input the image to be segmented into the trained teacher-student network model to achieve image segmentation.

[0068] Steps S1 / S2 / S3 / S4 / S5 / S6 / S7 are executed in sequence, as detailed below;

[0069] Furthermore, the teacher-student network model includes a teacher network and a student network. The model significantly improves the accuracy and efficiency of image segmentation by effectively combining labeled data and unlabeled data.

[0070] Furthermore, the preprocessed image dataset is divided into three parts: a coarse-grained annotated image set for training the teacher network, including image data X t and its corresponding coarse-grained annotation image SL t ;

[0071] Unlabeled image dataset X for training the student network s ,

[0072] A fine-grained annotated image set for evaluating the teacher-student network model, including image data X testand the corresponding fine-grained annotation image FL test .

[0073] Furthermore, based on semi-supervised learning and knowledge distillation strategies, the process of training and validating the teacher-student network model is as follows:

[0074] Using image data X t and its corresponding coarse-grained annotation image SL t To train the teacher network to learn image annotation, in this stage, a model with good performance in the field of image segmentation, simple structure and easy implementation is selected as the teacher network.

[0075] Using the trained teacher network, we can s Generate annotated image PL s , labeled image PL s With image data X s Together they constitute the data for training the student network; in this way, the student network can not only learn from the annotation information PL provided by the teacher network, but also s Learning features can also be done through X s It learns more features related to the segmentation target.

[0076] Generate image data X using the teacher-student network model test The annotation results and the corresponding fine-grained annotation image FL test Comparison is made to verify the segmentation performance of the teacher-student network model.

[0077] Figure 1 A diagram for teacher-student framework training;

[0078] Figure 2 It is a schematic diagram of the overall structure of the model;

[0079] After setting up the training process, we first select a deep learning-based segmentation network as the teacher network. Previous research has demonstrated the effectiveness of UNet, so we chose it as the base teacher network. We then select a network such as ResUNet, AttUNet, SwinUNet, or UTransformer to annotate the images.

[0080] Furthermore, the student network utilizes a hybrid convolutional neural network-attention mechanism architecture. Designed based on an encoder-decoder framework, this framework is designed to efficiently extract and reconstruct image features for accurate image segmentation. This architecture combines the high-resolution spatial information extraction capabilities of convolutional neural networks with the global context capture capabilities of attention mechanisms to improve segmentation performance and localization accuracy. By introducing a hybrid design, this architecture effectively addresses the feature resolution loss that can occur during segmentation using attention mechanisms, ensuring that global features in the image are accurately captured while preserving fine-grained spatial details.

[0081] The encoder consists of three convolutional layers, each of which includes a normalization layer and an activation function. The normalization layer accelerates training and stabilizes the network structure, while the activation function enables the network to learn more complex feature representations. Through the three consecutive convolutional layers, the encoder gradually extracts local features of the image while gradually reducing the spatial resolution of the feature maps to capture higher-level semantic information.

[0082] After completing local feature extraction, the feature map generated by the encoder is input into the attention mechanism module for further processing.

[0083] The attention mechanism consists of two parts: channel attention and spatial attention. Channel attention includes global pooling, which performs global average pooling on feature maps to generate channel descriptors; nonlinear transformation, which uses a multi-layer perceptron to perform nonlinear transformations on descriptors to generate attention weights for each channel; and weighted processing, which applies the generated attention weights to the original feature map to enhance the feature representation of important channels and suppress irrelevant or redundant features. Spatial attention includes convolution, which applies a two-dimensional convolution operation to the feature map processed by channel attention to generate a spatial attention map; and region highlighting, which uses the generated spatial attention map to highlight key areas in the image and suppress background noise, thereby improving the segmentation accuracy of the target area. By combining channel attention and spatial attention, the attention mechanism module can dynamically adjust the important information in the feature map, allowing the student network to focus more on key features and regions, thereby improving overall segmentation performance.

[0084] Furthermore, the attention mechanism introduces a sliding window operation. The attention mechanism consists of 12 layers, each of which performs an attention calculation. The attention calculation is performed within a regular non-overlapping window, and then the window is shifted to the right by half its width to generate a new window for calculation. This sliding window operation significantly reduces the computational burden of the model, making it more suitable for image segmentation tasks. In each layer of the attention mechanism, the self-attention calculation of each window is performed through a specific equation, merging the two dimensions H and W of the feature map into the same dimension to obtain the input sequence of the attention mechanism. Where N = HW / 16 is the number of elements in the input sequence, and the self-attention calculation of each window is calculated using the following formula:

[0085] [Q; K; V] = [W q ;W k ;W v ]·x i (1)

[0086]

[0087] in, and is the parameter matrix, d is the feature in each window These matrices are multiplied with the feature vector to produce three attention vectors: and Self-attention calculation via a learnable relative position offset Position information is introduced to enhance the expression of spatial relationships. After the above operations, the shape of the feature map remains consistent with the output of the encoder.

[0088] The decoder also consists of three layers, each of which includes upsampling, normalization, and activation functions. The upsampling operation gradually restores the spatial resolution of the feature map, allowing the final output to reach the same size as the input image. The normalization layer is used to accelerate the training process and stabilize the network, ensuring the consistency of features across different layers. The activation function introduces nonlinearity, enabling the network to learn and express more complex feature representations. Through the continuous operation of these three layers, the decoder not only restores the image's detailed information but also combines the features of the corresponding layers in the encoder, ultimately achieving high-resolution segmentation results. This design enables the model to simultaneously capture local details and global semantics, achieving the ultimate goal of image segmentation.

[0089] A semi-supervised image segmentation device based on knowledge distillation, comprising:

[0090] Acquisition module: used to obtain the original image data set to be processed;

[0091] Preprocessing module: used to preprocess the original image dataset, check and delete duplicate images in the dataset, eliminate damaged or unreadable image files, and annotate some images to obtain the preprocessed image dataset, a set of images with coarse-grained annotations, a set of images with fine-grained annotations, and a set of unannotated images;

[0092] Construction module: used to construct a teacher-student network model using a semi-supervised learning strategy, wherein the teacher-student network model includes a teacher network and a student network;

[0093] Training module: The teacher network learns basic segmentation knowledge by training on coarse-grained labeled data. This knowledge is then transferred to the student network through knowledge distillation.

[0094] The feature distillation method and the teacher network generate annotation information of unlabeled images to guide the student network to learn features and segment objects in unlabeled images, thereby improving the overall segmentation accuracy.

[0095] Based on a set of fine-grained annotated images, the segmentation performance of the teacher-student network is verified by comparing the image annotations generated by the teacher-student network with the real image annotations.

[0096] Implementation module: Used to input the image to be segmented into the trained teacher-student network model to achieve image segmentation. Example 1: This paper designs a semi-supervised image segmentation method based on knowledge distillation. To more comprehensively measure the performance of the model, the performance of the teacher-student model is tested using open source datasets and constructed private datasets. A series of preprocessing operations are performed on both datasets to enhance their usability in model training. The following is a detailed description of the datasets and preprocessing operations:

[0097] (1)LUNA16: Figure 3 As shown, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C, (d) CT scan image of individual D, and (e) CT scan image of individual E.

[0098] The LUNA16 dataset is a subset of the largest public lung nodule dataset, LIDC-IDRI, with the main purpose of achieving automatic detection and segmentation of lung cancer. The LUNA16 dataset contains 888 lung CT scan images, each containing 1-4 nodules, for a total of 1,186 nodules. The image size ranges from 512×512×95 to 512×512×733, with a voxel size of 0.78×0.78×1.25mm. 2 The dataset’s annotations are generated using an automatic segmentation algorithm and include independent labels for the left and right lungs and the trachea. LUNA16’s annotation information includes the location of the nodules (x, y, and z coordinates), and the original image size is 512×512.

[0099] The patient lung cancer images in the LUNA16 dataset are computed tomography (CT) images. CT imaging uses precisely guided X-rays to scan a specific thickness cross-section of the human body and reconstruct a three-dimensional image of the human body part based on the received X-ray signals. Compared with traditional X-rays, it provides higher spatial resolution and clearer three-dimensional lesion images. The LUNA16 dataset contains multiple CT images containing lung nodules, such as Figure 3The information stored in the original CT file includes images of all organs in the scan area (including interference areas such as air and blood vessels). To enhance the model's ability to segment lung nodules, the CT image is first windowed to highlight the lung area.

[0100] Figure 4 Schematic diagram of the comparison before and after sliding the LUNA16 dataset window, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C.

[0101] Figure 4 The image shows a comparison of several CT images from the LUNA16 dataset before and after the sliding window. The CT images in the LUNA16 dataset were then resampled using a linear interpolation algorithm to ensure that the spacing between scan planes for each patient's CT images was 1 mm, thus resolving the spatial scale inconsistency caused by the varying spacing between scan planes.

[0102] The LUNA16 dataset includes the coordinate information of lung nodules in CT images. Previous studies have generated fine-grained annotations of lung nodules based on nodule coordinates. In order to explore the learning ability of the model proposed in this paper under coarse-grained annotation and its effect in identifying lung nodules, the nodule coordinate information is used to generate both fine-grained and coarse-grained annotations. Figure 5 As shown in the figure, the original 512×512 CT image is first cropped near the location of the lung nodule, generating a 96×96 image. Subsequently, the lung nodules are annotated in the CT image using the LUNA16 nodule coordinate information. Fine-grained annotations are formed as irregular shapes that precisely cover the nodule area, while coarse-grained annotations are formed as rectangular boxes that encompass the nodule area and its surroundings. Based on these fine-grained and coarse-grained annotations, detailed and coarse annotation maps are generated for model training and validation.

[0103] The preprocessed LUNA16 dataset is randomly divided into three non-overlapping subsets with a ratio of 3:5:2. The first subset contains lung nodule image data X t and its corresponding coarse-grained annotation image SL t , used to train the teacher network. The second group contains lung nodule image data X s , and use the teacher network to generate X s Pulmonary nodules marked with PL s .X s and PL s Together they are used to train the student network. The third group X test and its corresponding fine-grained annotation image FL test Used to test model performance.

[0104] Figure 6Schematic diagram of the original CT images of lung nodules in the LC183 dataset, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C, (d) CT scan image of individual D, and (e) CT scan image of individual E.

[0105] (2)LC183: Figure 6 As shown in the figure, CT data of 183 lung cancer patients were collected from the hospital. This dataset is called LC183. The data collection has obtained informed consent from all participants and complies with the ethical guidelines of the hospital ethics committee. Each patient's image contains at least 1 lung nodule and a maximum of 3 lung nodules. The image size ranges from 512×512×49 to 512×512×368, and the voxel size ranges from 0.50×0.50×0.50mm. 2 to 1.25×1.25×5mm 2 The pulmonary nodule annotation file format of the LC183 dataset is nii.gz, which stores the mask image of the pulmonary nodule.

[0106] In this paper, we invited several clinicians with extensive experience in diagnosing lung nodules to annotate the patient CT images in the LC183 dataset. The annotation process was the same as that of the LUNA16 dataset.

[0107] Figure 7 Schematic diagram of the comparison before and after sliding the LC183 dataset window, (a) CT scan image of individual A, (b) CT scan image of individual B, (c) CT scan image of individual C, (d) CT scan image of individual D, and (e) CT scan image of individual E.

[0108] Figure 7 An image of a lung nodule from the LC183 dataset is shown, and the tool used by clinicians to annotate it is Slicer5.2.1.

[0109] In combination with the solution of the present invention, the experimental analysis is carried out as follows:

[0110] (1) Experimental details

[0111] The network was implemented using Pytorch 1.10 and trained for 150 epochs on an NVIDIA GeForce RTX 3090. The Adam optimizer was used for network optimization, and the learning rate was set to 10 -5 If the validation loss does not improve within 10 consecutive epochs, the training is terminated early to avoid overfitting of the network. During the testing process, accuracy, Dice coefficient, sensitivity, specificity, and Hausdorff distance are used to evaluate the network performance.

[0112] (2) Evaluation indicators

[0113] After training, the segmentation model needs to be tested and evaluated. To evaluate the proposed method, the following metrics are used to evaluate the performance of the model: TP: True Positive, FP: False Positive, TN: True Negative, and FN: False Negative.

[0114] Accuracy (Acc): determines the performance in correctly classifying the classes in the test SSSSSSSSSSSSSSS set, as shown in Equation 3.

[0115]

[0116] Dice coefficient (Dice): used to evaluate the overlap between the prediction result and the ground truth. Dice ranges from 0 to 1. The better the prediction result, the larger the Dice value, as shown in Formula 4.

[0117]

[0118] Sensitivity (SENS): measures the proportion of actual positives that are correctly identified, as shown in Equation 5.

[0119]

[0120] Specificity (SPE): The ability to correctly exclude irrelevant samples, that is, the ability to accurately identify negative samples. The higher the specificity, the higher the probability of diagnosis, as shown in Formula 6.

[0121]

[0122] Hough distance (Hausdorff Distance, HF95): The distance between the boundary of the prediction result and the ground truth. The better the prediction result, the smaller the HF95 value. Where d(x, y) represents the Euclidean distance between x and y, as shown in Formula 7.

[0123] HF95=maxd(y,y′),d(y′,y) (7)

[0124] (3) Analysis of experimental results

[0125] In the experiments, we first conducted ablation experiments to examine the impact of different parameter configurations on model performance. We then studied the effect of replacing the teacher network on the student network's performance in the pulmonary nodule segmentation task. Finally, we explored the performance differences between the proposed method and other competing models in the segmentation field and analyzed the reasons for these differences.

[0126] 1) Ablation Experiments: To demonstrate the effectiveness of the datasets and understand the importance of each component in this invention, ablation experiments were conducted on the datasets. The results are shown in Tables 1 and 2. The ablation experiment results in Table 1 on the LUNA16 dataset and in Table 2 on the LC183 dataset show that both removing the attention mechanism layer from the model and not using the teacher-student training strategy during model training lead to a decline in model performance, validating the effectiveness of the proposed training strategy.

[0127] Table 1 Ablation experiment results of LUNA16 dataset

[0128] method Acc Dice SENS SPE HF95 Removing the attention mechanism 0.9836 0.4554 0.7563 0.9850 18.94 Removal of teacher-student training 0.9780 0.4829 0.7887 0.9810 20.05 This method 0.9850 0.5253 0.7603 0.9931 8.50

[0129] Note ①: The results in bold represent the best results among the relevant indicators.

[0130] Table 2 Ablation experiment results of LC183 dataset

[0131] method Acc Dice SENS SPE HF95 Removing the attention mechanism 0.9838 0.6245 0.7601 0.9852 78.90 Removal of teacher-student training 0.9785 0.6524 0.7703 0.9940 20.00 This method 0.9892 0.6837 0.8493 0.9813 8.03

[0132] Note ①: The results in bold represent the best results among the relevant indicators.

[0133] 2) Impact of Different Teacher Networks: In the real world, teachers often need to possess a broader knowledge base than students or provide guidance during the learning process. The UNet model has been shown to be highly effective and efficient in image segmentation. Therefore, we selected the UNet model as the base teacher network and combined it with its variants such as ResUNet, AttUNet, SwinUNet, and UTransformer to further investigate and compare the impact of different teacher network models on the student network segmentation performance.

[0134] Table 3 shows the segmentation performance of this method on the LUNA16 dataset under different teacher network conditions. As can be seen from Table 3, the performance is better when the teacher network is a UNet model, a ResUNet model, or a UTransformer. In particular, when the teacher network is a UNet model, most indicators show high performance. However, when the teacher network is an AttUNet model or a SwinUNet model, the performance on various indicators is relatively poor. Analysis suggests that this result may be related to the increase in the number of parameters in the teacher model. Model parameters are generally related to representation and learning capabilities. More parameters mean that more detailed features and patterns can be learned, thereby better fitting the training data. In some cases, more parameters can enhance performance by adapting to complex data distributions. However, when the amount of training data for the teacher network is limited, too many parameters may lead to insufficient model training. This leads to insufficient data learning, resulting in inaccurate generated annotations, which in turn affects learning and ultimately leads to poor performance.

[0135] Table 4 shows the performance of this method under different teacher network conditions on the LC183 dataset. As can be seen from Table 4, the best performance is achieved when the teacher network is a UNet model, while performance degrades significantly when the teacher network is a SwinUNet or AttUNet model. This performance difference may be due to differences in the architecture and complexity of the teacher models. The teacher model based on the attention mechanism architecture has a relatively large number of parameters, making it difficult to achieve sufficient training results on the experimental dataset. As a result, the encoder acting as the teacher is not accurate enough in identifying lung nodules, resulting in misleading guidance during subsequent training.

[0136] Figure 8 The lung nodule segmentation effect of this method under different teacher networks on the LUNA16 dataset is demonstrated. Figure 9 The results of lung nodule segmentation under different teacher networks on the LC183 dataset are shown. Figure 10 Rough lung nodule annotations generated by different teacher models are shown in .

[0137] Table 3 Performance of different teacher annotators on the LUNA16 dataset

[0138] teacher Acc Dice SENS SPE HF95 SwinUNet 0.9701 0.3153 0.4642 0.9851 26.71 AttUNet 0.9686 0.3446 0.6248 0.9783 20.11 UTransformer 0.9806 0.7520 0.7304 0.9905 9.98 ResUNet 0.9736 0.4612 0.8127 0.9833 20.13 UNet 0.9850 0.5253 0.7603 0.9931 8.50

[0139] Note ①: The results in bold represent the best results among the relevant indicators.

[0140] Table 4 Performance of different teacher annotators on the LC183 dataset

[0141]

[0142]

[0143] Note ①: The results in bold represent the best results among the relevant indicators.

[0144] 3) Performance of different student networks: In order to explore the competitiveness and advantages of this segmentation method, a horizontal comparison was made with the performance of other models (such as UNet, ResUNet and ttUNet) in the lung nodule segmentation task while keeping the teacher network consistent. Table 5 shows the segmentation performance of the LUNA16 dataset of this method under different student network conditions. Table 6 shows the segmentation performance of the LC183 dataset of this method under different student network conditions. It can be seen from Tables 5 and 6 that when this method is trained using the lung nodule annotations generated by the teacher network, it shows stronger learning and error correction capabilities than AttUNet, ResUNet, SwinUNet and UTransformer. It can not only learn the lung nodule features provided by the teacher network, but also has the ability to correct the inaccurate information provided by the teacher network to a certain extent. Figure 11and Figure 12 The segmentation results of lung nodules by this method and other models under the same teacher network on the LUNA16 and LC183 datasets are shown respectively.

[0145] Table 5 Performance of different student networks on the LUNA16 dataset

[0146] student Acc Dice SENS SPE HF95 UTransformer 0.9665 0.3920 0.5445 0.9750 27.45 AttUNet 0.9705 0.3783 0.6122 0.9868 12.40 ResUNet 0.9932 0.4573 0.7389 0.9850 16.20 SwinUNet 0.9806 0.7623 0.7401 0.9855 11.78 UNet 0.9850 0.5253 0.7603 0.9931 8.50

[0147] Note ①: The results in bold represent the best results among the relevant indicators.

[0148] Table 6 Performance of different student networks on the LC183 dataset

[0149]

[0150]

[0151] Note ①: The results in bold represent the best results among the relevant indicators.

[0152] In the present invention, a new image segmentation method is designed, which is based on a hybrid architecture of convolutional neural networks and attention mechanisms, aiming to achieve accurate segmentation of target areas to meet the needs of different application scenarios. In order to make full use of a large amount of unlabeled data, a training process based on a teacher-student network is designed, including training the teacher network to generate initial labeled data, and using these labeled data to train the student network to improve segmentation accuracy. The experiment verified the superior performance of this method in the image segmentation task. In addition, the experimental results also demonstrated the effectiveness of the training process proposed in the present invention. In future research, it is planned to further explore how to effectively incorporate a large amount of unlabeled data into the training process of the segmentation network, and to strive to design a better segmentation network to improve the segmentation effect in different task scenarios.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-supervised image segmentation method based on knowledge distillation, characterized by: The following steps are involved: Obtain raw image data for image segmentation tasks; Preprocess the original image dataset, check and delete duplicate images in the dataset, eliminate damaged or unreadable image files, and annotate some images to obtain the preprocessed image dataset, a set of images with coarse-grained annotations, a set of images with fine-grained annotations, and a set of unannotated images; Constructing a teacher-student network model based on a semi-supervised learning strategy, wherein the teacher-student network model constructed by the semi-supervised learning strategy includes a teacher network and a student network; The teacher network learns basic segmentation knowledge by training on coarse-grained labeled data, and this knowledge is transferred to the student network through knowledge distillation; The feature distillation method and the teacher network generate annotation information of unlabeled images to guide the student network to learn features and segment objects in unlabeled images, thereby improving the overall segmentation accuracy. Based on a set of fine-grained annotated images, the segmentation performance of the teacher-student network is verified by comparing the image annotations generated by the teacher-student network with the real image annotations. Input the image to be segmented into the trained teacher-student network model to achieve image segmentation; The preprocessed image dataset is divided into three parts: a coarse-grained labeled image set for training the teacher network, including image data X t and its corresponding coarse-grained annotation image SL t ; Unlabeled image dataset X for training the student network s , A fine-grained annotated image set for evaluating the teacher-student network model, including image data X test and the corresponding fine-grained annotation image FL test ; The teacher-student network model is adaptively trained based on the semi-supervised learning strategy to obtain a trained teacher-student network model. The process is as follows: Using image data X t and its corresponding coarse-grained annotation image SL t to train the teacher network; Using the trained teacher network, for the image data X s Generate annotated image PL s , labeled image PL s With image data X s Together they form the image data for training the student network; Use image data X respectively s and the labeled image PL s to jointly train student networks; Generate image data X using teacher network and student network test The annotated image and its corresponding fine-grained annotated image FL test Comparison is made to verify the segmentation performance of the teacher and student network models.

2. The semi-supervised image segmentation method based on knowledge distillation according to claim 1, characterized in that: The teacher network uses a deep learning segmentation network: ResUNet, AttUNet, SwinUNet or UTransformer network to annotate the image.

3. The semi-supervised image segmentation method based on knowledge distillation according to claim 1, characterized in that: The student network adopts a hybrid convolutional neural network-attention mechanism architecture and is designed based on an encoder-decoder framework; The encoder consists of three convolutional layers, each of which includes a normalization layer and an activation function. The normalization layer is used to accelerate the training process and stabilize the network structure, while the activation function enables the network to learn more complex feature representations. Through the three consecutive convolutional operations, the encoder gradually extracts local features of the image while gradually reducing the spatial resolution of the feature map to capture higher-level semantic information. After the feature map generated by the encoder completes local feature extraction, it is input into the attention mechanism module for further processing. The attention mechanism module adopts a sliding window strategy. First, the feature map is divided into multiple regular non-overlapping windows, and self-attention calculation is performed in each window. This helps to capture the dependency relationship within the local area. The window is then shifted to the right by half its width to generate new overlapping windows, and self-attention calculations are performed again within these overlapping windows. This sliding window approach effectively captures global information across windows and enhances the continuity of feature representations. During each self-attention calculation, the two spatial dimensions of the feature map are merged into one dimension to form an input sequence for the attention mechanism to process. In addition, a learnable relative position offset is introduced to enhance the model's ability to model spatial relationships. The attention mechanism module can perform detailed analysis and processing of feature maps at different scales and positions, thereby improving the performance of the entire model in image understanding tasks; The decoder also consists of three layers, each of which contains upsampling, normalization and activation functions; The upsampling operation gradually restores the spatial resolution of the feature map so that the final output can reach the same size as the input image; The normalization layer is used to accelerate the training process and stabilize the network, ensuring the consistency of features across different layers. The activation function introduces nonlinearity, enabling the network to learn and express more complex feature representations. Through the continuous operation of these three layers, the decoder not only restores the detailed information of the image, but also combines the features of the corresponding layers in the encoder to ultimately achieve high-resolution segmentation results. This design enables the model to capture local details and global semantics to achieve the final image segmentation goal.

4. The semi-supervised image segmentation method based on knowledge distillation according to claim 3, characterized in that: The attention mechanism introduces a sliding window operation. The attention mechanism consists of 12 layers, and each layer performs an attention calculation. The attention calculation is performed in a regular non-overlapping window, and then the window is shifted to the right by half the width to generate a new window for calculation. In each layer of the attention mechanism, the self-attention calculation of each window is performed through a specific equation, and the two dimensions H and W of the feature map are merged into the same dimension to obtain the input sequence of the attention mechanism. Where N = HW / 16 is the number of elements in the input sequence, and the self-attention calculation of each window is calculated using the following formula: [Q;K;V]=[W q ;W k ;W v ]·x i (1) in, and is the parameter matrix, d is the feature in each window These matrices are multiplied with the feature vector to produce three attention vectors: and Self-attention calculation via a learnable relative position offset Introduce location information to enhance the expression of spatial relationships.

5. A semi-supervised image segmentation device based on knowledge distillation, characterized by: include: Acquisition module: used to obtain the original image data set to be processed; Preprocessing module: used to preprocess the original image dataset, check and delete duplicate images in the dataset, eliminate damaged or unreadable image files, and annotate some images to obtain a preprocessed image dataset, a set of images with coarse-grained annotations, a set of images with fine-grained annotations, and a set of unannotated images; the preprocessed image dataset is divided into three parts: a coarse-grained annotated image set for training the teacher network, including image data X t and its corresponding coarse-grained annotation image SL t ; Unlabeled image dataset X for training the student network s , A fine-grained annotated image set for evaluating the teacher-student network model, including image data X test and the corresponding fine-grained annotation image FL test ; Construction module: used to construct a teacher-student network model using a semi-supervised learning strategy, wherein the teacher-student network model includes a teacher network and a student network; Training module: The teacher network learns basic segmentation knowledge by training on coarse-grained labeled data. This knowledge is then transferred to the student network through knowledge distillation. The feature distillation method and the teacher network generate annotation information of unlabeled images to guide the student network to learn features and segment objects in unlabeled images, thereby improving the overall segmentation accuracy. Based on a set of fine-grained annotated images, the segmentation performance of the teacher-student network is verified by comparing the image annotations generated by the teacher-student network with the real image annotations. The teacher-student network model is adaptively trained based on the semi-supervised learning strategy to obtain a trained teacher-student network model. The process is as follows: Using image data X t and its corresponding coarse-grained annotation image SL t to train the teacher network; Using the trained teacher network, for the image data X s Generate annotated image PL s , labeled image PL s With image data X s Together they form the image data for training the student network; Use image data X respectively s and the labeled image PL s to jointly train student networks; Generate image data X using teacher network and student network test The annotated image and its corresponding fine-grained annotated image FL test Compare and verify the segmentation performance of the teacher and student network models; Implementation module: used to input the image to be segmented into the trained teacher-student network model to achieve image segmentation.

6. A computer device comprising: A processor and a memory, wherein the memory stores a program module, wherein the program module runs on the processor to implement the method according to any one of claims 1 to 4.

Images