Selective adaptive medical image segmentation method based on few-example domain perception
By adopting a selective adaptive method based on few example domain perception in medical image segmentation, using Transformer encoder and mask decoder combined with high-frequency and multi-scale feature extraction, the problems of data scarcity and domain differences are solved, and the problem of data is achieved. High precision and adaptive medical image segmentation without expert annotation.
Patent Information
- Application Number
- CN202411590254.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2024-12-27
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In medical image segmentation, existing deep learning models face data scarcity and domain differences, resulting in poor performance in medical applications, especially lack of adaptability in unlabeled data and clinical relevance.
Using a selective adaptive medical image segmentation method based on few example domain perception, using a Transformer encoder and mask decoder, combined with high-frequency adapter, multi-scale feature extraction adapter and feature selector, utilizing fast Fourier transform and adaptive pooling The pyramid layer extracts image features and combines feature weights through logistic tis functions and Gumbel noise to enhance the adaptability of the model.
It improves the accuracy and adaptability of medical image segmentation, can effectively identify subtle details and patterns in different spatial ranges under a small amount of data, reduces feature interference, enhances model performance, and achieves good segmentation results without expert annotation.
Smart Images

Figure CN119205821A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image segmentation, and in particular relates to a selective adaptive medical image segmentation method based on few-example domain perception. Background Art
[0002] Medical image segmentation is crucial for diagnosis and treatment planning, and can be used to identify and depict human anatomical structures using MRI, CT, and ultrasound images. The emergence of deep learning (DL) and convolutional neural networks (CNNs), especially the U-Net architecture, enables models to learn from large annotated datasets, thereby improving accuracy and greatly advancing the development and application of automatic image segmentation technology.
[0003] A notable innovation in the field of image segmentation is the Segment Anything (SAM) model, which adopts a hint-based approach similar to large language models such as Generative Pre-Trained Transformers (GPT). SAM uses a Visual Transformer (ViT) backbone for encoding and a mask decoder to generate segmentation masks, and shows remarkable generalization ability on different datasets with minimal task-specific training. However, DL models including SAM face challenges in medical image segmentation due to the scarcity of annotated datasets and inherent domain differences such as changes in color, texture, and anatomical structure. These challenges highlight the necessity of semi-supervised, unsupervised learning, and transfer learning strategies to effectively utilize unlabeled data and knowledge from related fields.
[0004] Recent advances have focused on integrating domain-specific knowledge and using techniques such as attention mechanisms and generative adversarial networks (GANs) to enhance the generalizability of models and generate realistic synthetic training data. Despite the success of SAM in natural image segmentation, its analogous models in the medical domain remain a challenge, motivating ongoing research to overcome data scarcity, enhance model adaptability, and ensure clinical relevance and interpretability in medical image segmentation. Summary of the invention
[0005] In order to enhance the adaptability of the model, the present invention provides a selective adaptive medical image segmentation method based on few-example domain perception.
[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, the present invention provides a selective adaptive medical image segmentation method based on few-example domain perception, comprising the following steps: S1. Obtain a medical image segmentation dataset, the dataset comprising a training set and a test set; S2. construct a selective adaptive medical image segmentation model based on few-example domain perception, wherein the medical image segmentation model comprises a Transformer encoder and a mask decoder, wherein the Transformer encoder comprises a high-frequency adapter, a multi-scale feature extraction adapter, and a feature selector; S3. Using the training set of the medical image segmentation dataset to train the selective adaptive medical image segmentation model based on few-example domain perception to obtain a trained model; S4. Input the image to be tested in the test set into the trained model for inference testing to obtain the medical image segmentation result.
[0007] Furthermore, step S1 specifically includes: The datasets are different public medical imaging datasets, including the brain tumor segmentation BraTS dataset, the liver tumor segmentation LiTS dataset, and the RGB color image dataset Kvasir Seg; The brain tumor segmentation BraTS dataset is an MRI scan of a subject in 3D magnetic resonance imaging (MRI) mode; The liver tumor segmentation LiTS dataset is a CT scan of the liver and liver tumor segmentation tasks; The RGB color image dataset Kvasir Seg is a gastrointestinal polyp image and its corresponding segmentation mask.
[0008] Furthermore, in step S2, the medical images in the training set are subjected to the high-frequency adapter to extract and represent the high-frequency features of the images, and at the same time, the multi-scale feature extraction adapter is used to extract the multi-level features of the images fused at different scales, specifically: The high frequency adapter is based on two-dimensional fast Fourier transform FFT; the medical images in the training set After the high-frequency adapter, the high-frequency characteristics are obtained ; The multi-scale feature extraction adapter includes an adaptive pooling pyramid layer and a spline interpolation function, wherein the adaptive pooling pyramid layer includes global average pooling, Average pooling, Average pooling, Average pooling has four adaptive average pooling layers, and the output sizes of the four adaptive average pooling layers are 1×1, 2×2, 4x4 and 8×8 respectively; the medical images in the training set The medical image feature map is obtained through the ResNet50 network , the medical image feature map Converted into embedding features after Patch Embedding processing , the embedding feature After the multi-scale feature extraction adapter, the multi-scale features are obtained .
[0009] Furthermore, the feature selector in step S2 is specifically: The high frequency characteristics and multi-scale features Converted to high-frequency features through the logits function The corresponding raw score and multi-scale features The corresponding raw score , and Gumbel noise is used as and Add noise and get the score of added noise and , then add the noise fraction and Input into the softmax function to obtain high-frequency features The corresponding decision score and multi-scale features The corresponding decision score , the formula is as follows: , , in, , represents the fraction of added noise, represents the added noise, Represents a learnable control parameter; high-frequency features The corresponding decision score and multi-scale features The corresponding decision score As high frequency features and multi-scale features The weight of high-frequency features and multi-scale features Perform weighted combination to obtain weighted medical image features, fuse the weighted medical image features with the output results of the current Transformer encoder to obtain fused medical image features, and use the fused medical image features as the input of the next Transformer encoder to obtain the final fused medical image features. .
[0010] Furthermore, the high frequency adapter based on two-dimensional fast Fourier transform FFT is specifically: Medical images in the test set After two-dimensional fast Fourier transform, the medical images in the test set are The image information in is converted from the spatial domain to the frequency domain to obtain the spectral components. The formula is as follows: , in, Represents medical images in the test set The middle position is The pixel value of Indicates the position in the frequency domain The spectrum intensity at represents the kernel function of Fourier transform, represents the imaginary unit, and Respectively represent the horizontal and vertical coordinates in the frequency domain, and Respectively represent images size in horizontal and vertical directions; By setting the threshold , get the high frequency components in the spectrum , In , Satisfy the formula: ; The medical images in the test set are transformed by two-dimensional inverse fast Fourier transform The high-frequency image information in the image is converted from the frequency domain to the spatial domain to obtain high-frequency features. , the formula is as follows: , in, Represents the kernel function of the inverse Fourier transform.
[0011] Furthermore, the mask decoder in step S2 is specifically: The mask decoder adopts the SAM segmentation model decoder architecture, including a two-layer Transformer decoder, wherein the Transformer decoder includes a self-attention module, an MLP module, a layer normalization module, and a transposed convolutional layer module; The final fused medical image features are passed through a mask decoder to obtain a medical image segmentation result.
[0012] Furthermore, the mask decoder in step S2 is specifically: The mask decoder adopts the SAM segmentation model decoder architecture, including a two-layer Transformer decoder, wherein the Transformer decoder includes a self-attention module, an MLP module, a layer normalization module, and a transposed convolutional layer module; The final fused medical image features are passed through a mask decoder to obtain a medical image segmentation result.
[0013] Furthermore, the training set of the medical image segmentation dataset in step S3 trains the selective adaptive medical image segmentation model based on few-example domain perception, specifically: During the training process, the model is trained using a dual loss function with Dice loss and cross entropy loss as the objective function. The formula is as follows: , in, represents the Dice loss, represents the cross entropy loss, Represents the weight coefficient of Dice loss, represents the weight coefficient of cross entropy loss, + =1; then the Adam optimization algorithm is used to complete the training.
[0014] In a second aspect, a selective adaptive medical image segmentation device based on few-example domain perception includes: Data acquisition unit: used to acquire medical image segmentation data set; A model building unit: used to build a selective adaptive medical image segmentation model based on few-example domain perception, wherein the medical image segmentation model includes a SAM image encoder and a mask decoder, wherein the SAM image encoder includes a high-frequency adapter, a multi-scale feature extraction adapter, and a feature fusion selector; A model training unit: used for training a selective adaptive medical image segmentation model based on few-example domain perception using a training set of the medical image segmentation data set to obtain a trained model; Medical image segmentation unit: used to input the images to be tested in the test set into the trained model for inference testing to obtain the medical image segmentation results.
[0015] In a third aspect, an electronic device comprises: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a selective adaptive medical image segmentation and recognition method based on few-example domain perception is performed.
[0016] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, executes a selective adaptive medical image segmentation method based on few-example domain perception.
[0017] The advantages of the present invention are: The present invention utilizes the fast Fourier transform (FFT) through a high-frequency adapter to convert image data into the frequency domain, separates and amplifies high-frequency components in the frequency domain, and then inversely transforms the processed information back to the spatial domain, so that the model has the ability to distinguish subtle details, thereby improving accuracy; the spatial hierarchy in the image data is utilized through a multi-scale feature converter, global and local contexts are captured by aggregating features of different scales, and features of different resolutions are extracted using a pyramid pooling scheme, and then these multi-scale features are fused into a cohesive feature map using a convolutional layer, thereby ensuring that the model remains sensitive to features of various sizes and enhancing its ability to recognize patterns in different spatial ranges; the integration strategy of features processed by the multi-scale feature extraction adapter and the high-frequency adapter is dynamically determined through a feature selector. Unlike traditional fusion techniques that indiscriminately combine multiple feature sets, the designed selector uses a decision process based on learning weights to output. This binary selection process ([0,1] or [1,0]) ensures that at any given layer of the transformer block in the image encoder, only the most relevant features are forwarded, thereby alleviating potential feature interference and enhancing the model performance of feature fusion. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.
[0019] Figure 1 is a flow chart of the method of the present invention; Figure 2 It is a comparison chart of the experimental visualization results of the present invention. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0021] Example 1 In this embodiment, Figure 1 As shown, the present invention provides a selective adaptive medical image segmentation method based on few-example domain perception, and the specific steps include: S1. Obtain a medical image segmentation dataset, the dataset comprising a training set and a test set; Specifically, the datasets are different public medical imaging datasets, including the brain tumor segmentation BraTS dataset, the liver tumor segmentation LiTS dataset and the RGB color image dataset Kvasir Seg; the brain tumor segmentation BraTS dataset includes MRI scans of 1,251 subjects in 3D magnetic resonance imaging (MRI) mode; the liver tumor segmentation LiTS dataset includes CT scans of the liver and liver tumor segmentation tasks; the RGB color image dataset Kvasir Seg includes 1,000 gastrointestinal polyp images and their corresponding segmentation masks.
[0022] S2. construct a selective adaptive medical image segmentation model based on few-example domain perception, wherein the medical image segmentation model comprises a Transformer encoder and a mask decoder, wherein the Transformer encoder comprises a high-frequency adapter, a multi-scale feature extraction adapter, and a feature selector; Specifically, the high frequency adapter is based on two-dimensional fast Fourier transform FFT; the medical images in the training set After the high-frequency adapter, the high-frequency characteristics are obtained The multi-scale feature extraction adapter includes an adaptive pooling pyramid layer and a spline interpolation function, wherein the adaptive pooling pyramid layer includes global average pooling, Average pooling, Average pooling, Average pooling has four adaptive average pooling layers, and the output sizes of the four adaptive average pooling layers are 1×1, 2×2, 4x4 and 8×8 respectively; the medical images in the training set The medical image feature map is obtained through the ResNet50 network , the medical image feature map Converted into embedding features after Patch Embedding processing , the embedding feature After the multi-scale feature extraction adapter, the multi-scale features are obtained .
[0023] Specifically, the high frequency feature and multi-scale features Converted to high-frequency features through the logits function The corresponding raw score and multi-scale features The corresponding raw score , and Gumbel noise is used as and Add noise and get the score of added noise and , then add the noise fraction and Input into the softmax function to obtain high-frequency features The corresponding decision score and multi-scale features The corresponding decision score , the formula is as follows: , , in, , represents the fraction of added noise, represents the added noise, Represents a learnable control parameter; high-frequency features The corresponding decision score and multi-scale features The corresponding decision score As high frequency features and multi-scale features The weight of high-frequency features and multi-scale features Perform weighted combination to obtain weighted medical image features, fuse the weighted medical image features with the output results of the current Transformer encoder to obtain fused medical image features, and use the fused medical image features as the input of the next Transformer encoder to obtain the final fused medical image features. .
[0024] Specifically, the medical images in the test set After two-dimensional fast Fourier transform, the medical images in the test set are The image information in is converted from the spatial domain to the frequency domain to obtain the spectral components. The formula is as follows: , in, Represents medical images in the test set The middle position is The pixel value of Indicates the position in the frequency domain The spectrum intensity at represents the kernel function of Fourier transform, represents the imaginary unit, and Respectively represent the horizontal and vertical coordinates in the frequency domain, and Respectively represent images size in horizontal and vertical directions; By setting the threshold , get the high frequency components in the spectrum , In , Satisfy the formula: ; The medical images in the test set are transformed by two-dimensional inverse fast Fourier transform The high-frequency image information in the image is converted from the frequency domain to the spatial domain to obtain high-frequency features. , the formula is as follows: , in, Represents the kernel function of the inverse Fourier transform.
[0025] Specifically, the mask decoder adopts the SAM segmentation model decoder architecture, including a two-layer Transformer decoder, and the Transformer decoder includes a self-attention module, an MLP module, a layer normalization module and a transposed convolutional layer module; the final fused medical image features are passed through the mask decoder to obtain the medical image segmentation result.
[0026] Specifically, the mask decoder adopts the SAM segmentation model decoder architecture, including a two-layer Transformer decoder, and the Transformer decoder includes a self-attention module, an MLP module, a layer normalization module and a transposed convolutional layer module; the final fused medical image features are passed through the mask decoder to obtain the medical image segmentation result.
[0027] S3. Using the training set of the medical image segmentation dataset to train the selective adaptive medical image segmentation model based on few-example domain perception to obtain a trained model; Specifically, during the training process, the model is trained using a dual loss function with Dice loss and cross entropy loss as the objective function. The formula is as follows: , in, represents the Dice loss, represents the cross entropy loss, Represents the weight coefficient of Dice loss, represents the weight coefficient of cross entropy loss, + =1; then the Adam optimization algorithm is used to complete the training.
[0028] S4. Input the image to be tested in the test set into the trained model for inference testing to obtain the medical image segmentation result.
[0029] Example 2 In this embodiment, three different public medical imaging datasets are used to evaluate the performance of the proposed segmentation model in the medical field.
[0030] Experimental Datasets: Each dataset has its own unique challenges and characteristics and is suitable for the segmentation of various anatomical structures and pathologies. Here, the datasets used as well as the validation and testing methods are described. The Brain Tumor Segmentation (BraTS) 2021 Challenge dataset is crucial in evaluating state-of-the-art medical segmentation methods. It includes MRI scans of 1251 subjects under four 3D MRI modalities. Liver Tumor Segmentation (LiTS) 2017 Challenge Dataset. This dataset includes CT scans of the liver and liver tumor segmentation tasks, providing a set of different imaging conditions and tumor appearances. In addition, an RGB color image dataset Kvasir Seg is used, which is dedicated to pixel-level segmentation of colorectal polyps. This dataset includes 1000 gastrointestinal polyp images and their corresponding segmentation masks, all of which have been carefully annotated and verified by gastroenterologists. Experiments on this dataset verify the effectiveness of the proposed method in processing color medical images.
[0031] Implementation details: Experimental environment and hyperparameter settings. The constructed model, also referred to as the proposed model, was implemented using PyTorch and MONAI and trained on a cluster of 4 NVIDIA A100 GPUs. The learning rate was set to 0.0001 and the decay was 0.01. All input images were normalized to have zero mean and unit standard deviation with non-zero pixels. During training, 3D images and labels were sliced using a sliding window method with a two-dimensional size of 256×256. In addition, intensity scaling was performed on the images, adjusting the pixel intensity values from a range of -1000 to 2000 to a standardized range of 0 to 255. Similarly, the label intensity was adjusted in the same output range for consistency and standardization purposes. The batch size for each GPU was set to 1. All models were trained for a total of 100 epochs using the Adam optimizer.
[0032] An early stopping mechanism is used in the experiment to prevent overfitting during model training. The decay rate is applied every 30 epochs.
[0033] Quantitative analysis: In the experiment, this method was used to compare with the mainstream prototype network small sample segmentation models PFENet, PANet, and QNet on three medical datasets: BraTS21 (MRI), LiTS17 (CT), and Kvasir-SEG (RGB). The evaluation indicator was the mean intersection over union (mIoU). In the few example setting, 5 and 10 samples were used for training, respectively, and the test set contained 5 samples. The experimental results show that this method always achieves the best performance. On the BraTS21 dataset, this method achieved an improvement of 8.20 mIoU for 10 samples; on the LiTS17 dataset, it improved by 5.93 mIoU; and on the Kvasir SEG dataset, it improved by 9.48 mIoU. The experimental results are shown in Table 1: Table 1 Comparison of experimental data between the method of the present invention and the existing method Qualitative analysis: Figure 2 As shown, Figure 2 Comparison of existing methods MedSAM, SAM-MED2D and FewMIS methods (5 examples, 10 examples). The first column is the input image, the second column is the image with color mask, the third and fourth columns are the images with color predicted masks by MedSAM and SAM-MED2D. The two columns on the right are the images with color masks predicted by FewMIS method. Figure 2 The results in clearly demonstrate the effectiveness of the proposed method. In comparison, the proposed method shows the best performance even without precise label hints. However, other methods show scattered and ambiguous mask predictions without precise label hints. This shows that the proposed method can achieve medical segmentation well without the help of expert annotations. That is, medical segmentation can still be achieved well without the help of expert annotations. This method can not only adapt well to image data of different modalities, but also rethink and improve the conditions for medical segmentation, that is, without the need for expert annotations. Good prediction results were achieved without clicking and framing under the premise of marking the box.
[0034] Example 3 This embodiment provides a selective adaptive medical image segmentation device based on few-example domain perception, including: Data acquisition unit: used to acquire medical image segmentation data set; A model building unit: used to build a selective adaptive medical image segmentation model based on few-example domain perception, wherein the medical image segmentation model includes a SAM image encoder and a mask decoder, wherein the SAM image encoder includes a high-frequency adapter, a multi-scale feature extraction adapter, and a feature fusion selector; A model training unit: used for training a selective adaptive medical image segmentation model based on few-example domain perception using a training set of the medical image segmentation data set to obtain a trained model; Medical image segmentation unit: used to input the images to be tested in the test set into the trained model for inference testing to obtain the medical image segmentation results.
[0035] Example 4 This embodiment provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, a selective adaptive medical image segmentation and recognition method based on few-example domain perception is performed.
[0036] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0037] Example 5 This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a selective adaptive medical image segmentation method based on few-example domain perception is executed.
[0038] Storage media include: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store programs.
[0039] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A selective adaptive medical image segmentation method based on few-example domain perception, characterized in that: The following steps are involved: S1. Obtain a medical image segmentation dataset, the dataset comprising a training set and a test set; S2. construct a selective adaptive medical image segmentation model based on few-example domain perception, wherein the medical image segmentation model comprises a Transformer encoder and a mask decoder, wherein the Transformer encoder comprises a high-frequency adapter, a multi-scale feature extraction adapter, and a feature selector; S3. Using the training set of the medical image segmentation dataset to train the selective adaptive medical image segmentation model based on few-example domain perception to obtain a trained model; S4. Input the image to be tested in the test set into the trained model for inference testing to obtain the medical image segmentation result.
2. The selective adaptive medical image segmentation method based on few-example domain perception according to claim 1, characterized in that: The step S1 specifically includes: The datasets are different public medical imaging datasets, including the brain tumor segmentation BraTS dataset, the liver tumor segmentation LiTS dataset, and the RGB color image dataset Kvasir Seg; The brain tumor segmentation BraTS dataset is an MRI scan of a subject in 3D magnetic resonance imaging (MRI) mode; The liver tumor segmentation LiTS dataset is a CT scan of the liver and liver tumor segmentation tasks; The RGB color image dataset Kvasir Seg is a gastrointestinal polyp image and its corresponding segmentation mask.
3. The selective adaptive medical image segmentation method based on few-example domain perception according to claim 1, characterized in that: In step S2, the medical images in the training set are subjected to high-frequency adapters to extract and represent high-frequency features of the images, and at the same time, multi-level features of images fused at different scales are extracted through multi-scale feature extraction adapters, specifically: The high frequency adapter is based on two-dimensional fast Fourier transform FFT; the medical images in the training set After the high-frequency adapter, the high-frequency characteristics are obtained ; The multi-scale feature extraction adapter includes an adaptive pooling pyramid layer and a spline interpolation function, wherein the adaptive pooling pyramid layer includes global average pooling, Average pooling, Average pooling, Average pooling has four adaptive average pooling layers, and the output sizes of the four adaptive average pooling layers are 1×1, 2×2, 4x4 and 8×8 respectively; the medical images in the training set The medical image feature map is obtained through the ResNet50 network , the medical image feature map Converted into embedding features after Patch Embedding processing , the embedding feature After the multi-scale feature extraction adapter, the multi-scale features are obtained .
4. The selective adaptive medical image segmentation method based on few-example domain perception according to claim 1, characterized in that: The feature selector in step S2 is specifically: The high frequency characteristics and multi-scale features Converted to high-frequency features through the logits function The corresponding raw score and multi-scale features The corresponding raw score , using Gumbel noise as and Add noise and get and The corresponding noise fraction and , then and Input into the softmax function to obtain high-frequency features The corresponding decision score and multi-scale features The corresponding decision score , the formula is as follows: , , in, , represents the fraction of added noise, represents the added noise, Represents a learnable control parameter; high-frequency features The corresponding decision score and multi-scale features The corresponding decision score As high frequency features and multi-scale features The weight of high-frequency features and multi-scale features Perform weighted combination to obtain weighted medical image features, fuse the weighted medical image features with the output results of the current Transformer encoder to obtain fused medical image features, and use the fused medical image features as the input of the next Transformer encoder to obtain the final fused medical image features. .
5. The selective adaptive medical image segmentation method based on few-example domain perception according to claim 3, characterized in that: The high frequency adapter based on two-dimensional fast Fourier transform FFT is specifically: Medical images in the test set After two-dimensional fast Fourier transform, the medical images in the test set are The image information in is converted from the spatial domain to the frequency domain to obtain the spectral components. The formula is as follows: , in, Represents medical images in the test set The middle position is The pixel value of Indicates the position in the frequency domain The spectrum intensity at represents the kernel function of Fourier transform, represents the imaginary unit, and Respectively represent the horizontal and vertical coordinates in the frequency domain, and Respectively represent images size in horizontal and vertical directions; By setting the threshold , get the high frequency components in the spectrum , In , Satisfy the formula: ; The medical images in the test set are transformed by two-dimensional inverse fast Fourier transform The high-frequency image information in the image is converted from the frequency domain to the spatial domain to obtain high-frequency features. , the formula is as follows: , in, Represents the kernel function of the inverse Fourier transform.
6. The selective adaptive medical image segmentation method based on few-example domain perception according to claim 1, characterized in that: The mask decoder in step S2 is specifically: The mask decoder adopts the SAM segmentation model decoder architecture, including a two-layer Transformer decoder, wherein the Transformer decoder includes a self-attention module, an MLP module, a layer normalization module, and a transposed convolutional layer module; The final fused medical image features are passed through a mask decoder to obtain a medical image segmentation result.
7. The selective adaptive medical image segmentation method based on few-example domain perception according to claim 1, characterized in that: The training set of the medical image segmentation dataset in step S3 trains the selective adaptive medical image segmentation model based on few-example domain perception, specifically: During the training process, the model is trained using a dual loss function with Dice loss and cross entropy loss as the objective function. The formula is as follows: , in, represents the Dice loss, represents the cross entropy loss, Represents the weight coefficient of Dice loss, represents the weight coefficient of the cross entropy loss, + =1; then the Adam optimization algorithm is used to complete the training.
8. A selective adaptive medical image segmentation device based on few-example domain perception, characterized in that: include: Data acquisition unit: used to acquire medical image segmentation data set; A model building unit: used to build a selective adaptive medical image segmentation model based on few-example domain perception, wherein the medical image segmentation model includes a SAM image encoder and a mask decoder, wherein the SAM image encoder includes a high-frequency adapter, a multi-scale feature extraction adapter, and a feature fusion selector; A model training unit: used for training a selective adaptive medical image segmentation model based on few-example domain perception using a training set of the medical image segmentation data set to obtain a trained model; Medical image segmentation unit: used to input the images to be tested in the test set into the trained model for inference testing to obtain the medical image segmentation results.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the selective adaptive medical image segmentation and recognition method based on few-example domain perception as described in any one of claims 1 to 7 is performed.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the selective adaptive medical image segmentation method based on few-example domain perception as described in any one of claims 1 to 7.
Citation Information
Patent Citations
An image blind watermark adding method based on a frequency domain
CN109829842A
Medical image segmentation method based on global attention and multi-scale features
CN118279319A
Zero-sample infrared image coloring method and system based on multilevel representation fusion
CN118823175A
Cited By
Lightweight-based multi-modal medical image segmentation method and system
CN119887807A
