Domain-adaptive small sample CT medical picture semantic segmentation system based on variational auto-encoder

Abstract

The invention discloses a domain-adaptive small-sample CT medical picture semantic segmentation system based on a variational auto-encoder, relates to the technical field of image semantic segmentation, and solves the technical problem of low segmentation precision caused by the fact that an existing semantic segmentation method is susceptible to noise and modal difference interference. Comprising a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module and a loss calculation module. The prototype segmentation module is used for calculating a segmentation result of the small sample segmentation task set by using cosine similarity; the domain alignment module is used for calculating segmentation loss by using the segmentation result, performing domain alignment by using a variational auto-encoder, and calculating reconstruction loss and alignment loss; weighting based on the reconstruction loss and the alignment loss to obtain total reconstruction loss; the loss calculation module is used for calculating weighted loss and optimizing the model; and the structure can be accurately identified during segmentation, so that the segmentation precision is higher.

Description

Technical Field

[0001] This application belongs to the field of image semantic segmentation technology, specifically a domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder. Background Technology

[0002] Image semantic segmentation is a crucial task in computer vision, aiming to assign each pixel in an input image to its corresponding semantic category. With the development of deep learning techniques, image semantic segmentation methods based on deep convolutional neural networks have made significant progress. These methods can automatically learn feature representations in images and utilize convolution and upsampling operations to segment images into different semantic regions.

[0003] Existing technology (invention patent announcement number CN117745736B) discloses a cross-domain few-sample CT image semantic segmentation system and method based on meta-learning. The system includes: a data processing module for sampling from labeled source domain image data and combining them into a few-sample segmentation task set, and sampling unlabeled images from the target domain as training data; a feature extraction module for obtaining mid-level and prototype features of the source domain image data using a convolutional neural network, and obtaining mid-level features of images in the target domain using a convolutional neural network; a segmentation prediction module for calculating the segmentation result of the few-sample segmentation task using cosine similarity; and a loss calculation module for calculating the weighted loss and optimizing the model. This solves the problem that the number of labeled CT medical images in the source domain is insufficient to support meta-training, and can provide CT image semantic segmentation data in the medical field to quickly locate target regions, supporting more accurate disease diagnosis and treatment decisions.

[0004] The aforementioned semantic segmentation method calculates the loss function using the maximum mean difference algorithm to achieve cross-domain data alignment. However, this cross-domain alignment method only mimics the "surface values" of cross-domain features through statistical tools, and is easily interfered with by noise and modal differences during segmentation, resulting in low final segmentation accuracy. Therefore, a domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoders is needed. Summary of the Invention

[0005] This application provides a domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder, which solves the technical problem that existing semantic segmentation methods are susceptible to noise and modal differences, resulting in low segmentation accuracy.

[0006] To achieve the above objectives, this application adopts the following technical solution: In the first aspect, a domain-adaptive semantic segmentation system for small-sample CT medical images based on variational autoencoders is provided, comprising: a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module, and a loss calculation module; The data processing module is used to process source domain data to obtain a small sample segmentation task set, and to process target domain data to obtain training data. The feature extraction module is used to obtain mid-level features of the source domain image in the source domain data using a convolutional neural network, and to obtain mid-level features of the target domain image in the target domain data using a convolutional neural network; the mid-level features are obtained by concatenating multiple layer features; The prototype segmentation module is used to calculate the segmentation results of a small sample segmentation task set using cosine similarity. The domain alignment module is used to calculate the segmentation loss using the segmentation results, perform domain alignment using a variational autoencoder, and calculate the reconstruction loss and alignment loss; and to obtain the total reconstruction loss by weighting the reconstruction loss and alignment loss. The loss calculation module calculates the weighted loss and optimizes the model; the weighted loss is calculated through segmentation loss, reconstruction loss, and alignment loss.

[0007] Based on the above technical solution, in the domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder provided in this application, a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module, and a loss calculation module are set up. The data processing module is used to process source domain data to obtain a small-sample segmentation task set and to process target domain data to obtain training data. The feature extraction module is used to use a convolutional neural network to obtain mid-level features of the source domain image in the source domain data and to use a convolutional neural network to obtain mid-level features of the target domain image in the target domain data; the mid-level features are obtained by concatenating multiple layer features; the original... The model segmentation module is used to calculate the segmentation results of a small sample segmentation task set using cosine similarity. The domain alignment module is used to calculate the segmentation loss using the segmentation results, and to perform domain alignment using a variational autoencoder, and to calculate the reconstruction loss and alignment loss; and to obtain the total reconstruction loss by weighting the reconstruction loss and alignment loss. The loss calculation module calculates the weighted loss and optimizes the model; the weighted loss is calculated from the segmentation loss, reconstruction loss and alignment loss; through generative modeling, the "generation rules" of cross-domain features are learned, such as "how to generate lesion boundaries in CT from general boundary features", which can accurately identify structures during segmentation, resulting in higher segmentation accuracy.

[0008] Variational autoencoders (VAEs) fundamentally address the accuracy constraints in CT segmentation through a dual design of generative probabilistic modeling and feature reconstruction. They achieve deep semantic alignment, avoiding spurious alignment: VAEs extract the mean and variance of the latent probability space; they strip away modality-specific features, such as color in natural images and grayscale in CT, retaining only cross-domain core semantics, such as the "high density" and "boundary continuity" of lesions. Even with significant differences in statistical distribution between the source and CT domains, core features can still overlap in the latent space, ensuring that structure is recognized during segmentation, rather than merely matching surfaces.

[0009] This paper addresses the shortcomings of meta-learning methods in semantic segmentation of few-sample CT medical images. It first utilizes meta-learning to complete the few-sample segmentation task in the source domain, then uses a variational autoencoder to align the source domain with the CT medical image domain, ultimately achieving semantic segmentation of few-sample images in the CT medical image domain. A feature extraction module learns the latent embedding space of prototypes; a domain alignment module reconstructs cross-domain features multiple times using a variational autoencoder model to achieve domain alignment; and a parameterless prototype segmentation module utilizes prototypes to segment query images.

[0010] In conjunction with the first aspect above, in one possible implementation, the middle-layer feature is obtained by concatenating multiple layer features, including: Obtain the layer features of the second and third feature extraction layers of the feature extraction module, and concatenate the layer features of the second and third feature extraction layers to obtain the middle layer features: ; ; ; in, , and All are mid-level features, and , , ; This represents the convolution operation. This indicates a splicing operation at the channel level; , and This refers to the layer features of the second layer; , and This refers to the layer features of the third layer; , , ; , , ; The convolutional structure of the feature extraction module includes four feature extraction layers.

[0011] In conjunction with the first aspect above, in one possible implementation, the step of using a variational autoencoder for domain alignment and calculating the reconstruction loss and alignment loss includes: Obtain the mid-level features corresponding to the source domain data and the target domain data; perform global average pooling on the mid-level features corresponding to the source domain data and the target domain data to obtain the prototype feature vector and the target feature vector respectively; A variational autoencoder is defined by taking the prototype feature vector and the target feature vector as input feature vectors in turn. and decoder Based on the input feature vector, a variational autoencoder is used. and decoder Reconstruct the data; obtain the probability distribution and the restored feature vector. The alignment loss corresponding to the input feature vector is calculated based on the probability distribution. The reconstruction loss corresponding to the input feature vector is calculated based on the restored feature vector and the input feature vector. The alignment loss and reconstruction loss of the mid-level features corresponding to the source domain data, and the alignment loss and reconstruction loss of the mid-level features corresponding to the target domain data are calculated sequentially.

[0012] In conjunction with the first aspect above, in one possible implementation, global average pooling is performed on the mid-level features corresponding to the source domain data and the target domain data to obtain the prototype feature vector and the target feature vector, respectively; including: The mid-level features of the source domain data are used as input to the global average pooling function, and the corresponding output feature vector is denoted as the prototype feature vector. The mid-level features of the target domain data are used as input to the global average pooling function, and the corresponding output feature vector is denoted as the target feature vector. One expression for the global average pooling function is as follows: ; in, The feature vector output by the global average pooling function, and ; Input the mid-layer features of the global average pooling function; c is the number of channels; These represent the height and width of the feature map, respectively.

[0013] In conjunction with the first aspect described above, in one possible implementation, a variational autoencoder is used based on the input feature vector. and decoder Reconstruction is performed; the probability distribution and reconstructed feature vectors are obtained, including: The input feature vector is input into the variational autoencoder. Output a latent probability distribution in a cross-domain unified embedding space: ; in, The input feature vector is passed through a variational autoencoder The probability distribution of the output, and , It is the mean vector. It is the variance vector; The input feature vector; Variational autoencoder using reparameterization techniques The latent variables are sampled from the output probability distribution: ; in, These are the latent variables obtained through sampling; Let represent a random variable sampled from a normal distribution, and ; via decoder Mapping the latent variables back to their distribution in the original input space yields the reconstructed feature vectors: ; in, To restore the feature vector; and .

[0014] In conjunction with the first aspect above, in one possible implementation, the alignment loss corresponding to the input feature vector is calculated based on the probability distribution, including: Alignment loss is calculated using KL divergence based on probability distribution: ; in, This is the alignment loss corresponding to the input feature vector.

[0015] In conjunction with the first aspect above, in one possible implementation, the reconstruction loss corresponding to the input feature vector is calculated based on the restored feature vector and the input feature vector, including: The reconstruction loss is calculated using the mean squared error based on the prototype feature vector and the restored feature vector. ; in, This is the reconstruction loss corresponding to the input feature vector.

[0016] In conjunction with the first aspect above, in one possible implementation, the weighted average of the reconstruction loss and alignment loss to obtain the total reconstruction loss includes: Obtain the alignment loss and reconstruction loss corresponding to the source domain data, and perform a weighted summation based on the alignment loss and reconstruction loss to obtain the total reconstruction loss of the reconstructed source domain image; Obtain the alignment loss and reconstruction loss corresponding to the target domain data, and perform a weighted summation based on the alignment loss and reconstruction loss to obtain the total reconstruction loss of the reconstructed target domain image.

[0017] In conjunction with the first aspect above, in one possible implementation, the calculation of the segmentation loss using the segmentation results includes: The segmentation loss is calculated using the cross-entropy loss function on the segmentation results: ; in, To divide the loss, For the segmentation results, This represents the similarity value.

[0018] In conjunction with the first aspect above, in one possible implementation, the calculation of the weighted loss includes: Obtain the total reconstruction loss of the source domain image. and the total reconstruction loss of the target domain image ; and segmentation loss Total reconstruction loss based on the reconstructed source domain image and the total reconstruction loss of the target domain image and segmentation loss The total loss is obtained by weighting. : ; in, This is the loss weight.

[0019] This application provides a domain-adaptive few-sample CT medical image semantic segmentation system based on variational autoencoders. Based on the aforementioned technical solution, this system includes a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module, and a loss calculation module. The data processing module processes source domain data to obtain a few-sample segmentation task set and processes target domain data to obtain training data. The feature extraction module uses convolutional neural networks to obtain mid-level features of source domain images from the source domain data and to obtain mid-level features of target domain images from the target domain data. The mid-layer features are obtained by concatenating features from multiple layers; the prototype segmentation module is used to calculate the segmentation results of a small sample segmentation task set using cosine similarity; the domain alignment module is used to calculate the segmentation loss using the segmentation results, and to perform domain alignment using a variational autoencoder, and to calculate the reconstruction loss and alignment loss; and to obtain the total reconstruction loss by weighting the reconstruction loss and alignment loss; the loss calculation module calculates the weighted loss and optimizes the model; the weighted loss is calculated through the segmentation loss, reconstruction loss and alignment loss; through generative modeling, the "generation rules" of cross-domain features are learned; such as "how to generate lesion boundaries in CT from general boundary features", which can accurately identify structures during segmentation, resulting in higher segmentation accuracy.

[0020] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1This is a schematic diagram of the module connections of the domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder in this application. Figure 2 This is a schematic diagram of the training process of the domain-adaptive few-sample CT medical image semantic segmentation system based on variational autoencoder in this application. Figure 3 This is a reference example diagram showing sampling from a CT medical image dataset in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0024] Few-shot image semantic segmentation, also known as few-shot segmentation, refers to pixel-level semantic analysis and labeling of images to segment objects or regions of different categories when data is scarce or annotations are limited. Typically, the amount of image data available for training is very limited in a specific task, possibly due to high data acquisition costs, labeling difficulties, or insufficient data volume. Few-shot image semantic segmentation currently employs metric-based meta-learning methods to address this issue.

[0025] Variational Autoencoders (VAEs) are generative models that combine the ideas of autoencoders and probabilistic inference. They consist of an encoder network and a decoder network. The encoder network maps input data to probability distribution parameters in the latent space, while the decoder network reconstructs the original input data from samples in the latent space. Unlike traditional autoencoders, VAEs introduce randomness into the latent space, allowing the model to learn the probability distribution of latent variables while learning the data distribution, thus enabling the generation of diverse data samples. During training, VAEs learn model parameters by maximizing the lower bound between the reconstruction error and the prior distribution of the latent space, typically using methods such as stochastic gradient descent for optimization. VAEs have wide applications in image and text generation, and are also used in unsupervised learning, data compression, and data denoising tasks.

[0026] Domain adaptation refers to the process of transferring knowledge or models learned in one domain to another related but different domain. In machine learning and deep learning, a common problem is data distribution mismatch, where the data distributions of the source and target domains differ, leading to a performance degradation of the model trained in the source domain on the target domain. The goal of domain adaptation is to adjust the model or data representation so that knowledge learned in the source domain can generalize to the target domain, thereby improving the model's performance in the target domain. Domain adaptation methods typically include techniques such as feature alignment, inter-domain loss functions, and adversarial training to reduce the distributional differences between the source and target domains, thus improving the model's generalization ability and transfer performance. Domain adaptation has significant applications in object recognition, object detection, and semantic segmentation, effectively addressing data imbalance and domain discrepancies in real-world scenarios.

[0027] Please see Figures 1-2 The first aspect of this application provides a domain-adaptive semantic segmentation system for small-sample CT medical images based on a variational autoencoder. A domain-adaptive semantic segmentation system for small-sample CT medical images based on variational autoencoders consists of five parts: a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module, and a loss calculation module. The data processing module is used to process source domain data to obtain a small sample segmentation task set, and to process target domain data to obtain training data. The feature extraction module is used to obtain mid-level features of the source domain image in the source domain data using a convolutional neural network, and to obtain mid-level features of the target domain image in the target domain data using a convolutional neural network; the mid-level features are obtained by concatenating multiple layer features; The prototype segmentation module is used to calculate the segmentation results of a small sample segmentation task set using cosine similarity. The domain alignment module is used to calculate the segmentation loss using the segmentation results, perform domain alignment using a variational autoencoder, and calculate the reconstruction loss and alignment loss; and to obtain the total reconstruction loss by weighting the reconstruction loss and alignment loss. The loss calculation module calculates the weighted loss and optimizes the model; the weighted loss is calculated through segmentation loss, reconstruction loss, and alignment loss.

[0028] Based on the above technical solution, in the domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder provided in this application, a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module, and a loss calculation module are set up. The data processing module is used to process source domain data to obtain a small-sample segmentation task set and to process target domain data to obtain training data. The feature extraction module is used to use a convolutional neural network to obtain mid-level features of the source domain image in the source domain data and to use a convolutional neural network to obtain the target domain image in the target domain data. Mid-level features; these mid-level features are obtained by concatenating features from multiple layers; Prototype segmentation module: used to calculate the segmentation result of a small sample segmentation task set using cosine similarity; Domain alignment module: used to calculate the segmentation loss using the segmentation result, and to perform domain alignment using a variational autoencoder, and to calculate the reconstruction loss and alignment loss; and to obtain the total reconstruction loss by weighting the reconstruction loss and alignment loss; Loss calculation module: calculates the weighted loss and optimizes the model; the weighted loss is calculated through the segmentation loss, reconstruction loss and alignment loss; through generative modeling, the "generation rules" of cross-domain features are learned; such as "how to generate lesion boundaries in CT from general boundary features", which can accurately identify structures during segmentation, resulting in higher segmentation accuracy.

[0029] Variational autoencoders (VAEs) fundamentally address the accuracy constraints in CT segmentation through a dual design of generative probabilistic modeling and feature reconstruction. They achieve deep semantic alignment, avoiding spurious alignment: VAEs extract the mean and variance of the latent probability space; they strip away modality-specific features, such as color in natural images and grayscale in CT, retaining only cross-domain core semantics, such as the "high density" and "boundary continuity" of lesions. Even with significant differences in statistical distribution between the source and CT domains, core features can still overlap in the latent space, ensuring that structure is recognized during segmentation, rather than merely matching surfaces.

[0030] This paper addresses the shortcomings of meta-learning methods in semantic segmentation of few-sample CT medical images. It first utilizes meta-learning to complete the few-sample segmentation task in the source domain, then uses a variational autoencoder to align the source domain with the CT medical image domain, ultimately achieving semantic segmentation of few-sample images in the CT medical image domain. A feature extraction module learns the latent embedding space of prototypes; a domain alignment module reconstructs cross-domain features multiple times using a variational autoencoder model to achieve domain alignment; and a parameterless prototype segmentation module utilizes prototypes to segment query images.

[0031] Specifically, the data processing module mainly performs two functions: source domain data processing and target domain data processing. The source domain data processing involves sampling from labeled source domain data and combining it into a set of few-shot segmentation tasks. This set serves as input to the model during meta-training, enabling it to perform few-shot segmentation tasks in the source domain. This includes processing the source domain data to obtain the set of few-shot segmentation tasks. Specifically, based on the N-way-K-shot specification, the few-shot segmentation tasks used for training should have the following form: ; Where C represents the set of few-sample segmentation tasks, N represents the number of few-sample segmentation tasks sampled, S represents the support set, and Q represents the query set. Furthermore, the support set S should also include K labeled samples M, i.e. The query set Q should include one query image. and labels used only to calculate loss during training ; The target domain data processing is as follows: a small amount of unlabeled image data is sampled from the target domain and added to each mini-sample task. This unlabeled data is used for data domain alignment; that is: ; in, Indicates adding to the first In a small sample segmentation task, unlabeled image data is used, and the target domain image is denoted as... .

[0032] Specifically, the prototype segmentation module is used to complete segmentation after the feature extraction module can map the image to a specific latent embedding space. It uses a two-dimensional prototype as a segmentation guide, and the entire segmentation process is based on metric operations between the two-dimensional prototype and the three-dimensional query features, ensuring the robustness of the segmentation process. This includes: firstly, calculating the prototype of the category to be segmented using supporting features and supporting labels. ; Where, P represents the prototype, and (x, y) represents spatial coordinates, and the function This is an indicator function that takes the value 1 when the parameter is true and 0 otherwise. It is a tag The resolution size, in the formula The resolution has been adjusted using bilinear interpolation. ; Use parameterless metrics, such as cosine similarity, to measure prototype and query features: ; in, This represents the similarity value obtained from the measurement, and , For multipliers, in the formula The resolution has been adjusted to [value] using bilinear interpolation. .

[0033] The segmentation result obtained by the argmax function is as follows: ; in, For the segmentation result, and .

[0034] In one possible implementation, the intermediate-layer features are obtained by concatenating multiple layer features, including: obtaining the layer features of the second and third feature extraction layers of the feature extraction module, and concatenating the layer features of the second and third feature extraction layers to obtain the intermediate-layer features. ; ; ; in, , and All are mid-level features. To support the meaning of image features; To query image features; Features of the target image; and , , ; This represents the convolution operation, specifically... All use convolutions with a kernel of 3, a stride of 1, and padding of 1; This indicates a splicing operation at the channel level; , and This refers to the layer features of the second layer; , and This refers to the layer features of the third layer; it is understandable that... , , ; , , ; The convolutional structure of the feature extraction module includes four feature extraction layers.

[0035] Specifically, the role of the feature extraction module is to teach the convolutional neural network (CNN) the parameter configuration that maps images to a specific latent embedding space. Furthermore, the high-level features of a CNN tend to be biased towards specific, previously seen classes during training, which can lead to overfitting. Therefore, the output of the feature extraction module is the mid-level features of the image to better generalize to unseen classes.

[0036] The convolutional structure that makes up the feature information extraction module is divided into 4 feature extraction layers. As the image passes through each feature extraction layer sequentially, each layer outputs the extracted features for the image. When defining features: ; Define the number of channels, height, and width of all feature maps as follows: The second and third layers are both defined middle-layer features. ; The final mid-level features are obtained by concatenating the mid-level features from the second and third layer features that define the mid-level features.

[0037] In one possible implementation, a variational autoencoder is used for domain alignment, and reconstruction loss and alignment loss are calculated, including: obtaining the mid-level features corresponding to the source domain data and the target domain data; and performing global average pooling on the mid-level features corresponding to the source domain data and the target domain data to obtain the prototype feature vector and the target feature vector, respectively. A variational autoencoder is defined by taking the prototype feature vector and the target feature vector as input feature vectors in turn. and decoder Based on the input feature vector, a variational autoencoder is used. and decoder Reconstruct the data; obtain the probability distribution and the restored feature vector. The alignment loss corresponding to the input feature vector is calculated based on the probability distribution. The reconstruction loss corresponding to the input feature vector is calculated based on the restored feature vector and the input feature vector. The alignment loss and reconstruction loss of the mid-level features corresponding to the source domain data, and the alignment loss and reconstruction loss of the mid-level features corresponding to the target domain data are calculated sequentially.

[0038] In one possible implementation, global average pooling is performed on the mid-level features corresponding to the source domain data and the target domain data to obtain the prototype feature vector and the target feature vector, respectively; including: taking the mid-level features of the source domain data as the input of the global average pooling function, and obtaining the corresponding output feature vector, which is denoted as the prototype feature vector; The mid-level features of the target domain data are used as input to the global average pooling function, and the corresponding output feature vector is denoted as the target feature vector. One expression for the global average pooling function is as follows: ; in, The feature vector output by the global average pooling function, and ; Input the mid-layer features of the global average pooling function; c is the number of channels; These represent the height and width of the feature map, respectively.

[0039] In one possible implementation, a variational autoencoder is used based on the input feature vector. and decoder Reconstruction is performed to obtain the probability distribution and the restored feature vector, including: inputting the input feature vector into a variational autoencoder. Output a latent probability distribution in a cross-domain unified embedding space: ; in, The input feature vector is passed through a variational autoencoder The probability distribution of the output, and , It is the mean vector. It is the variance vector; The input feature vector; Variational autoencoder using reparameterization techniques The latent variables are sampled from the output probability distribution: ; in, These are the latent variables obtained through sampling; Let represent a random variable sampled from a normal distribution, and ; via decoder Mapping the latent variables back to their distribution in the original input space yields the reconstructed feature vectors: ; in, To restore the feature vector; and .

[0040] In one possible implementation, the alignment loss corresponding to the input feature vector is calculated based on a probability distribution, including: calculating the alignment loss using KL divergence based on the probability distribution. ; in, This is the alignment loss corresponding to the input feature vector.

[0041] In one possible implementation, the reconstruction loss corresponding to the input feature vector is calculated based on the restored feature vector and the input feature vector, including: calculating the reconstruction loss using mean squared error based on the prototype feature vector and the restored feature vector. ; in, This is the reconstruction loss corresponding to the input feature vector.

[0042] In one possible implementation, the step of obtaining the total reconstruction loss by weighting the reconstruction loss and alignment loss includes: obtaining the alignment loss and reconstruction loss corresponding to the source domain data, and obtaining the total reconstruction loss of the reconstructed source domain image by weighted summation based on the alignment loss and reconstruction loss. Obtain the alignment loss and reconstruction loss corresponding to the target domain data, and perform a weighted summation based on the alignment loss and reconstruction loss to obtain the total reconstruction loss of the reconstructed target domain image.

[0043] Specifically, global average pooling is performed on the mid-level features corresponding to the source domain data to obtain prototype feature vectors, and the corresponding alignment loss and reconstruction loss are calculated based on the prototype feature vectors, including: The mid-level features of the source domain data are used as input to the global average pooling function, and the corresponding output feature vector is denoted as the prototype feature vector. ; ; in, Input the mid-layer features of the global average pooling function; Number of channels; These are the height and width of the feature map, respectively. The variational autoencoder is constructed by inputting the prototype feature vector. Output a latent probability distribution in a cross-domain unified embedding space. : ; in, It is the mean vector. It is the variance vector; The prototype feature vector; Variational autoencoder using reparameterization techniques The latent variables are sampled from the output probability distribution: ; in, These are the latent variables obtained through sampling; Let represent a random variable sampled from a normal distribution, and ; via decoder Mapping the latent variables back to their distribution in the original input space yields the reconstructed feature vectors: ; in, To restore the feature vector; and ; Alignment loss is calculated using KL divergence based on probability distribution. : ; The reconstruction loss is calculated using the mean squared error based on the prototype feature vector and the restored feature vector. : ; The total reconstruction loss of the source domain image is obtained by weighted summation of the alignment loss and reconstruction loss. : ; By reconstructing the mid-level features of the target domain data through the above steps, the total reconstruction loss of the target domain image is obtained. .

[0044] It is understood that in this embodiment, the 'i' and 'j' that appear in the summation process are both counting symbols and have no other meaning.

[0045] Current meta-learning-based few-sample image semantic segmentation methods are difficult to transfer to the field of medical CT image segmentation. This is because current meta-learning methods require a large amount of training data from different test categories within the same domain. However, in the field of medical image semantic segmentation, it is often difficult to collect a large amount of labeled training data due to patient privacy or rare disease issues. Note that other domains, such as the natural image domain, have a large amount of available labeled data. Therefore, by solving the cross-domain segmentation problem, images from other domains can be used for training during the training phase, while the segmentation phase can be transferred to the CT medical image domain.

[0046] To address the cross-domain issue, a small number of CT images are used for training. Domain adaptation techniques are employed to train the feature extraction module, enabling it to map features from different domains to a unified embedding space. In short, the domain alignment module utilizes a variational autoencoder, reconstructing source and target domain images separately and calculating the resulting loss to optimize the model, allowing the feature extraction module to output unified features across domains.

[0047] In one possible implementation, calculating the segmentation loss using the segmentation results includes: calculating the segmentation loss using the cross-entropy loss function on the segmentation results. ; in, To divide the loss, For the segmentation results, This represents the similarity value.

[0048] In one possible implementation, calculating the weighted loss includes: obtaining the total reconstruction loss of the reconstructed source domain image. and the total reconstruction loss of the target domain image ; and segmentation loss Total reconstruction loss based on the reconstructed source domain image and the total reconstruction loss of the target domain image and segmentation loss The total loss is obtained by weighting. : ; in, This is the loss weight.

[0049] This embodiment uses the public dataset PASCAL-5i; the experiment is conducted in a 1-way 1-shot setting, using PyTorch as the artificial intelligence model framework; the feature extraction module is an improvement on the deep convolutional neural network ResNet50, and its composition is shown in Table 1; firstly, this example divides the ResNet50 network into 5 layers, but only uses the features of the third and fourth layers as mid-layer features, and then concatenates the features of the third and fourth layers and reduces the dimensionality to 256 dimensions as the final feature output. , and During training, the pre-trained parameters of the ResNet50 network on the ImageNet dataset are loaded to speed up the training process.

[0050] Table 1. Structure of the feature extraction module in this implementation example

[0051] The encoder Enc structure in the domain alignment module is shown in Table 2; Table 2 Encoder structure in the cross-modal information extraction module

[0052] The decoder Dec structure in the domain alignment module is shown in Table 3; Table 3 Decoder structure in the cross-modal information extraction module

[0053] In this embodiment, during the training and testing phases, the input image resolution is 473×473 pixels, and the size of the feature map output by the feature extraction module is 1 / 8 of the input image size. During training, the training images are randomly cropped and flipped. The SGD optimizer is used to train the entire system, with training lasting 60,000 iterations, where the learning rate is 0.001, the batch size is 8, the momentum is 0.99, and the weight decay rate is 0.0005. In Equations 6-11, α and β are both 1, and in Equations 6-16, δ and ε are 1 and 0.2, respectively.

[0054] After training, the system was tested on a randomly selected small sample segmentation task, and the test results are as follows: Figure 3 As shown, the images are labeled with different colors. Experimental results demonstrate that the system trained in this example can achieve relatively accurate semantic segmentation of cross-domain small-sample CT medical images, and the proposed innovations are truly effective.

[0055] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

[0056] The above embodiments are only used to illustrate the technical methods of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of this application without departing from the spirit and scope of the technical methods of this application.

Claims

1. A domain-adaptive semantic segmentation system for small-sample CT medical images based on variational autoencoders, characterized in that, include: The module includes a data processing module, a feature extraction module, a prototype segmentation module, a domain alignment module, and a loss calculation module. The data processing module is used to process source domain data to obtain a small sample segmentation task set, and to process target domain data to obtain training data. The feature extraction module is used to obtain mid-level features of the source domain image in the source domain data using a convolutional neural network, and to obtain mid-level features of the target domain image in the target domain data using a convolutional neural network; the mid-level features are obtained by concatenating multiple layer features; The prototype segmentation module is used to calculate the segmentation results of a small sample segmentation task set using cosine similarity. The domain alignment module is used to calculate the segmentation loss using the segmentation results, perform domain alignment using a variational autoencoder, and calculate the reconstruction loss and alignment loss; and to obtain the total reconstruction loss by weighting the reconstruction loss and alignment loss. The loss calculation module calculates the weighted loss and optimizes the model; the weighted loss is calculated through segmentation loss, reconstruction loss, and alignment loss.

2. The domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 1, characterized in that, The middle-layer features are obtained by concatenating multiple layer features, including: Obtain the layer features of the second and third feature extraction layers of the feature extraction module, and concatenate the layer features of the second and third feature extraction layers to obtain the middle layer features: ； ； ； in, , and All are mid-level features, and , , ; This represents the convolution operation. This indicates a splicing operation at the channel level; , and This refers to the layer features of the second layer; , and This refers to the layer features of the third layer; The convolutional structure of the feature extraction module includes four feature extraction layers.

3. The domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 1, characterized in that, The process of using a variational autoencoder for domain alignment and calculating reconstruction loss and alignment loss includes: Obtain the mid-level features corresponding to the source domain data and the target domain data; perform global average pooling on the mid-level features corresponding to the source domain data and the target domain data to obtain the prototype feature vector and the target feature vector respectively; A variational autoencoder is defined by taking the prototype feature vector and the target feature vector as input feature vectors in turn. and decoder ; Based on the input feature vector, a variational autoencoder is used and decoder Reconstruct the data; obtain the probability distribution and the restored feature vector. The alignment loss corresponding to the input feature vector is calculated based on the probability distribution. The reconstruction loss corresponding to the input feature vector is calculated based on the restored feature vector and the input feature vector. The alignment loss and reconstruction loss of the mid-level features corresponding to the source domain data, and the alignment loss and reconstruction loss of the mid-level features corresponding to the target domain data are calculated sequentially.

4. The domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 3, characterized in that, The step of performing global average pooling on the mid-level features corresponding to the source domain data and the target domain data to obtain the prototype feature vector and the target feature vector, respectively, includes: The mid-level features of the source domain data are used as input to the global average pooling function, and the corresponding output feature vector is denoted as the prototype feature vector. The mid-level features of the target domain data are used as input to the global average pooling function, and the corresponding output feature vector is denoted as the target feature vector. One expression for the global average pooling function is as follows: ； in, The feature vector output by the global average pooling function, and ; Input the mid-layer features of the global average pooling function; c is the number of channels; These represent the height and width of the feature map, respectively.

5. A domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 3, characterized in that, The variational autoencoder based on the input feature vector and decoder Reconstruction is underway; The probability distribution and the reconstructed feature vector are obtained, including: The input feature vector is input into the variational autoencoder. Output a latent probability distribution in a cross-domain unified embedding space: ； in, The input feature vector is passed through a variational autoencoder The probability distribution of the output, and , It is the mean vector. It is the variance vector; The input feature vector; Variational autoencoder using reparameterization techniques The latent variables are sampled from the output probability distribution: ； in, These are the latent variables obtained through sampling; Let represent a random variable sampled from a normal distribution, and ; via decoder Mapping the latent variables back to their distribution in the original input space yields the reconstructed feature vectors: ； in, To restore the feature vector; and .

6. The domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 3, characterized in that, The alignment loss calculated based on the probability distribution for the input feature vector includes: Alignment loss is calculated using KL divergence based on probability distribution: ； in, This is the alignment loss corresponding to the input feature vector.

7. A domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 3, characterized in that, The process of calculating the reconstruction loss corresponding to the input feature vector based on the restored feature vector and the input feature vector includes: The reconstruction loss is calculated using the mean squared error based on the prototype feature vector and the restored feature vector. ； in, This is the reconstruction loss corresponding to the input feature vector.

8. A domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 3, characterized in that, The total reconstruction loss, obtained by weighting the reconstruction loss and alignment loss, includes: Obtain the alignment loss and reconstruction loss corresponding to the source domain data, and perform a weighted summation based on the alignment loss and reconstruction loss to obtain the total reconstruction loss of the reconstructed source domain image; Obtain the alignment loss and reconstruction loss corresponding to the target domain data, and perform a weighted summation based on the alignment loss and reconstruction loss to obtain the total reconstruction loss of the reconstructed target domain image.

9. A domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 1, characterized in that, The calculation of segmentation loss using the segmentation results includes: The segmentation loss is calculated using the cross-entropy loss function on the segmentation results: ； in, To divide the loss, For the segmentation results, This represents the similarity value.

10. A domain-adaptive small-sample CT medical image semantic segmentation system based on variational autoencoder according to claim 1, characterized in that, The calculation of the weighted loss includes: Obtain the total reconstruction loss of the source domain image. and the total reconstruction loss of the target domain image ; and segmentation loss Total reconstruction loss based on the reconstructed source domain image and the total reconstruction loss of the target domain image and segmentation loss The total loss is obtained by weighting. : ； in, This is the loss weight.

Images