A medical image segmentation method and system based on CNN and ViT hybrid architecture

Abstract

The present invention specifically discloses a medical image segmentation method and system based on a hybrid architecture of CNN and ViT, which relates to the field of medical image processing technology. The method includes: S1, obtaining multiple types of medical image data sets and preprocessing them, and dividing them into training sets, validation sets and test sets in proportion; S2, constructing a segmentation model based on a hybrid architecture of CNN and ViT, the segmentation model uses the U-Net network as the basic framework, and includes an encoder, a decoder and an edge optimization module; S3, training the segmentation model based on the divided training data set; S4, calculating the loss function of the medical image segmentation model, using the Adam optimizer to update the parameters, and saving the optimal model weights during the training process; S5, using the trained segmentation model to test the test set data. This method effectively alleviates the problems of limited receptive field of CNN and expensive ViT calculation cost, and improves the segmentation accuracy of the lesion area.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image segmentation method and system based on a CNN and ViT hybrid architecture. Background Art

[0002] Medical image segmentation, a key technology in computer vision, aims to accurately delineate lesions or regions of interest (ROIs) from medical images, providing support for both quantitative and qualitative analysis. This technology plays a crucial role in medical-assisted diagnosis, treatment planning, and prognosis. While deep learning-based methods have achieved remarkable results, the inherent characteristics of medical images, such as morphological complexity (such as the irregular fractal structure of invasive tumors), low signal-to-noise ratio interference (e.g., high grayscale overlap between lesions and background in ultrasound images), and multi-scale feature mismatches in tiny lesions, remain major obstacles to technological development.

[0003] With the continuous advancement of deep learning technology, convolutional neural networks (CNNs) have become the mainstream architecture in medical image segmentation due to their superior local feature extraction capabilities. They can automatically extract deep features from images without the need for manual feature design or additional intervention. However, CNNs also have significant limitations. Their fixed convolution kernels and limited receptive fields result in poor global context modeling capabilities, making it easy to miss small lesions during segmentation. Specifically, these limitations manifest themselves in two aspects: First, the convolution kernel parameters are fixed during training, lacking the ability to adaptively adjust to different input images; second, the convolution kernels only model pixel relationships within a local area and lack global information. This severely limits the accurate recognition and segmentation of lesion shapes and boundary pixels (such as the boundaries of skin lesions and liver tumors). Therefore, in medical image segmentation tasks, exploiting long-range dependencies between image pixels is crucial to improving segmentation accuracy.

[0004] To overcome these shortcomings of CNNs, numerous medical image segmentation methods based on Visual Transformers (ViTs) have emerged. ViTs can achieve a large receptive field and effectively capture long-range dependencies between different image patches. However, the self-attention mechanism in ViTs has significant drawbacks. It typically requires a large amount of data for training to achieve ideal performance, and its computational complexity increases quadratically with the length of the input sequence, which is inconsistent with the real-time requirements of clinical applications. Furthermore, compared to CNNs, ViTs are less sensitive to local position, which severely impacts segmentation quality when processing small target regions and medical images with complex and varied morphology.

[0005] Although recently proposed methods based on a hybrid CNN and ViT architecture have achieved leading results in benchmark tests, they still face difficulties in dealing with blurred edges and low-contrast areas, and are often accompanied by high computational costs, making it difficult to find an ideal balance between high-precision segmentation and model complexity. Therefore, an innovative network architecture and implementation method is urgently needed to overcome these challenges and further improve medical image segmentation performance. Summary of the Invention

[0006] The purpose of this invention is to propose a medical image segmentation method and system based on a CNN and ViT hybrid architecture, combining the advantages of CNN and ViT to effectively alleviate the problems of limited CNN receptive field and expensive ViT computational cost. While capturing local spatial information and global contextual information, it closely combines channel information modeling capabilities to more efficiently extract medical image features, enhance the correlation between different semantic levels, improve the segmentation accuracy of lesion areas, and provide a more accurate and reliable solution in the field of medical image segmentation.

[0007] To achieve the above objectives, the present invention proposes a medical image segmentation method based on a hybrid architecture of CNN and ViT, the specific steps of which are as follows:

[0008] Step S1: Acquire multiple types of medical image datasets, preprocess the original data, and generate preprocessed images; divide the processed data into training set, validation set, and test set in proportion;

[0009] Step S2: construct a segmentation model based on a hybrid architecture of CNN and ViT, wherein the segmentation model is based on a U-Net network and includes an encoder, a decoder, and an edge optimization module;

[0010] Step S3: performing segmentation model training based on the divided training data set;

[0011] Step S4: Calculate the loss function of the medical image segmentation model, use the Adam optimizer to update the parameters, and save the optimal model weights during the training process;

[0012] Step S5: Use the trained segmentation model to test the test set data.

[0013] Preferably, in step S2, the encoder includes four encoding layers, each layer including a multi-scale feature extraction module and a semantic enhancement module; the decoder includes four decoding layers, each layer also including a multi-scale feature extraction module and a semantic enhancement module; a skip connection is used between the encoding layer and the decoding layer in the same layer, and frequency domain information is introduced at the bottleneck layer; the multi-scale feature extraction module extracts multi-scale feature semantic information from the input image, and the semantic enhancement module enhances the correlation between different semantic information; the output of the last decoding layer is upsampled to obtain a coarse segmentation result, which is then refined by the edge uncertainty guidance module to obtain a final output prediction result;

[0014] In the encoder stage, the input feature map After downsampling through the multi-scale feature extraction module and the semantic enhancement module, it is input to the next encoding layer. After layer-by-layer downsampling of the feature map, the spatial resolution decreases layer by layer and the number of channels increases layer by layer. The downsampling formula at the encoder stage is as follows:

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] in, is a 3×3 convolution operation, is the semantic enhancement module, is a multi-scale feature extraction module, is the output of the first encoding layer, is the output of the second encoding layer, is the output of the third encoding layer, is the output of the fourth encoding layer.

[0020] Preferably, in step S2, additional frequency domain features are introduced at the bottleneck layer, and the input is decomposed into low-frequency and high-frequency components using wavelet transform WT. After deep convolution is performed to extract features, the inverse wavelet transform IWT is used to reconstruct the features. The formula is as follows:

[0021] ;

[0022] in, is the output result after inverse wavelet transform, is the weight vector after the convolution kernel of size k×k.

[0023] Preferably, in step S2, in the decoder stage, the input of each decoding layer is the output feature map of the previous decoding layer, and the upsampling result is combined with the output feature map of the same encoding layer by element point summation for feature fusion. After the feature map is upsampled layer by layer, the spatial resolution is restored layer by layer, and the number of channels is reduced layer by layer, and finally the segmentation model generates a coarse segmentation result.

[0024] Preferably, in step S2, the edge uncertainty guidance module optimizes and reshapes the uncertainty pixels in the boundary fuzzy area in the coarse segmentation result, specifically by generating a refined mask by combining shallow features, uncertainty mapping and local feature matching, optimizing the coarse segmentation result, and obtaining the final predicted segmentation result.

[0025] Preferably, the shallow features are outputs of the first encoding layer;

[0026] The uncertainty map is calculated based on the probability map of the coarse segmentation result, and the formula is as follows:

[0027] ;

[0028] in, For uncertain mapping, the value range is ; 、 Represent the maximum probability value and minimum probability value of the pixel respectively, Calculate the exponential function;

[0029] The local feature is generated by weighted local averaging of the uncertain mapping, and the formula is as follows:

[0030] ;

[0031] ;

[0032] in, is a local feature, represents the normalization factor, is a small value that prevents the denominator from being zero, is a pixel, is the pixel after refinement, The pixel belongs to the target category The probability of It is a shallow feature. Pixels Neighborhood area;

[0033] Generate refined masks by combining shallow features, uncertainty mapping, and local feature matching , the calculation formula is as follows:

[0034] ;

[0035] ;

[0036] in, For pixels The confidence score at is the confidence module;

[0037] The rough segmentation results are optimized to obtain the final predicted segmentation results. The formula is as follows:

[0038] ;

[0039] in, To predict the segmentation results.

[0040] Preferably, the confidence module includes 3×3 convolution, Relu activation function and residual connection; wherein, 3×3 convolution is applied on the main branch for feature extraction, and 3×3 convolution is also applied on the residual branch for feature extraction, and finally the main branch output and the residual branch output are added element by element; the Relu activation function is applied before the convolution operation.

[0041] Preferably, in step S4, the loss function of the segmentation model is calculated, and the formula is as follows:

[0042] ;

[0043] ;

[0044] ;

[0045] in, is the sum of all pixels in the input image, 、 denote the predicted value and target value respectively, is the total loss, is the binary cross entropy loss, is the dice loss, 、 is the weight assigned to each loss.

[0046] The present invention also provides a medical image segmentation system based on a CNN and ViT hybrid architecture, comprising:

[0047] The data acquisition and preprocessing module is used to acquire multiple types of medical image datasets, preprocess the raw data, generate preprocessed images, and divide the processed data into training sets, validation sets, and test sets in proportion;

[0048] A model building module is used to build a segmentation model based on a hybrid architecture of CNN and ViT. The model uses the U-Net network as the basic framework and includes an encoder, a decoder, and an edge optimization module.

[0049] The model training module is used to train the model based on the divided training dataset, calculate the loss function of the medical image segmentation model, update the parameters using the Adam optimizer, and save the optimal model weights during the training process;

[0050] The model testing and evaluation module is used to test the test set data using the trained segmentation model and evaluate the model performance;

[0051] Storage module, used to store medical image data, model parameters and intermediate calculation results.

[0052] Preferably, the encoder constructed by the model construction module includes four encoding layers, each layer includes a multi-scale feature extraction module and a semantic enhancement module; the decoder includes four decoding layers, each layer also includes a multi-scale feature extraction module and a semantic enhancement module; jump connections are used between the encoding layer and the decoding layer at the same layer, and frequency domain information is introduced at the bottleneck layer.

[0053] Therefore, the present invention proposes a medical image segmentation method and system based on a hybrid architecture of CNN and ViT, which has the following beneficial effects:

[0054] (1) Significant improvement in segmentation accuracy: The hybrid architecture combining CNN and ViT can simultaneously capture local spatial information and global contextual information, and significantly improves the segmentation accuracy of small lesions, low-contrast areas, and areas with blurred boundaries.

[0055] (2) Enhanced long-range dependency modeling capabilities: The introduction of ViT’s self-attention mechanism effectively compensates for the shortcomings of traditional CNN in global information modeling and improves the ability to recognize complex shapes and boundaries.

[0056] (3) Good edge refinement effect: The designed edge enhancement perception module can optimize the fuzzy boundary areas in the segmentation results, and significantly improve the segmentation quality when dealing with unclear boundaries or small targets.

[0057] (4) Computational efficiency optimization: Combining the local feature extraction of CNN and the global modeling capability of ViT, compared with the pure ViT method, the computational efficiency is higher, high computational costs are avoided, and it is more suitable for scenarios with high clinical real-time requirements.

[0058] (5) Wide applicability: The hybrid architecture is highly versatile and can be applied to a variety of medical image segmentation tasks, such as liver tumor segmentation, cell segmentation, and skin lesion segmentation, and can achieve high-precision segmentation in different types of medical images.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of a medical image segmentation method based on a hybrid architecture of CNN and ViT in the present invention;

[0061] Figure 2 This is a flow chart of feature extraction by the multi-scale feature extraction module of the present invention;

[0062] Figure 3 This is a flow chart of the refinement operation of the edge uncertainty guidance module in the present invention. DETAILED DESCRIPTION

[0063] To make the technical solutions, advantages, and objectives of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below. The described embodiments are part of the embodiments of the present invention, not all of them. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0065] Example 1

[0066] like Figure 1 FIG. 1 is a flow chart of a medical image segmentation method based on a hybrid architecture of CNN and ViT according to the present invention, and the specific steps are as follows:

[0067] S1. Obtain multiple types of medical image datasets, preprocess the raw data, and generate preprocessed images; divide the processed data into training sets, validation sets, and test sets in proportion;

[0068] S2. Build a segmentation model based on a hybrid architecture of CNN and ViT. The segmentation model uses the U-Net network as the basic framework, including encoder, decoder and edge optimization module;

[0069] like Figure 2As shown in the figure, the encoder includes four encoding layers, each of which includes a multi-scale feature extraction module and a semantic enhancement module; the decoder includes four decoding layers, each of which also includes a multi-scale feature extraction module and a semantic enhancement module; skip connections are used between the encoding and decoding layers in the same layer, and frequency domain information is introduced at the bottleneck layer; the multi-scale feature extraction module extracts multi-scale feature semantic information from the input image, and the semantic enhancement module enhances the correlation between different semantic information; the output of the last decoding layer is upsampled to obtain a coarse segmentation result, which is then refined by the edge uncertainty guidance module to obtain the final output prediction result;

[0070] In the encoder stage, the input feature map After downsampling through the multi-scale feature extraction module and the semantic enhancement module, it is input to the next encoding layer. After layer-by-layer downsampling of the feature map, the spatial resolution decreases layer by layer and the number of channels increases layer by layer. The downsampling formula at the encoder stage is as follows:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] in, is a 3×3 convolution operation, is the semantic enhancement module, is a multi-scale feature extraction module, is the output of the first encoding layer, is the output of the second encoding layer, is the output of the third encoding layer, is the output of the fourth encoding layer.

[0076] Additional frequency domain features are introduced at the bottleneck layer, and the wavelet transform WT is used to decompose the input into low-frequency and high-frequency components. After deep convolution extracts features, the inverse wavelet transform IWT is used to reconstruct the features. The formula is as follows:

[0077] ;

[0078] in, is the output result after inverse wavelet transform, is the weight vector after the convolution kernel of size k×k.

[0079] , , , , Through 、 、 and DWConv is used to obtain multi-scale local information. After convolution, each subset , i =1,2,3,4,5, the output is accumulated in the subsequent stage, which realizes the advance of fine-grained multi-scale features, expands the receptive field of the model, and reduces the loss of feature information flow. The specific formula is as follows:

[0080] ;

[0081] Finally, The multi-scale features obtained by splicing them together according to the channel dimension express: ;

[0082] in, 、 Represent the final output and the output of each feature subset respectively, represents the squeeze excitation module, Indicates the splicing operation according to the channel dimension, is the convolution process, For the The output feature map of each subset after corresponding convolution processing.

[0083] In the decoder stage, the input of each decoding layer is the upsampling result of the output feature map of the previous decoding layer and the output feature map of the encoding layer of the same layer. The feature fusion is performed by element-wise summation. After the feature map is upsampled layer by layer, the spatial resolution is restored layer by layer, and the number of channels is reduced layer by layer. Finally, the segmentation model generates a coarse segmentation result.

[0084] like Figure 3 As shown in the figure, the edge uncertainty guidance module optimizes and reshapes the uncertainty pixels in the fuzzy boundary area of the coarse segmentation result. Specifically, it generates a refined mask by combining shallow features, uncertainty mapping and local feature matching, optimizes the coarse segmentation result, and obtains the final predicted segmentation result.

[0085] Shallow features are the output of the first encoding layer;

[0086] The uncertainty map is calculated based on the probability map of the coarse segmentation result, and the formula is as follows:

[0087] ;

[0088] in, For uncertain mapping, the value range is ; 、 Represent the maximum probability value and minimum probability value of the pixel respectively, Calculate the exponential function;

[0089] The local feature is generated by weighted local averaging of uncertain mapping, and the formula is as follows:

[0090] ;

[0091] ;

[0092] in, is a local feature, represents the normalization factor, is a small value that prevents the denominator from being zero, is a pixel, is the pixel after refinement, The pixel belongs to the target category The probability of It is a shallow feature. Pixels Neighborhood area;

[0093] Generate refined masks by combining shallow features, uncertainty mapping, and local feature matching , the calculation formula is as follows:

[0094] ;

[0095] ;

[0096] in, For pixels The confidence score at is the confidence module;

[0097] The rough segmentation results are optimized to obtain the final predicted segmentation results. The formula is as follows:

[0098] ;

[0099] in, To predict the segmentation results.

[0100] The confidence module includes 3×3 convolution, Relu activation function and residual connection; among them, 3×3 convolution is applied on the main branch for feature extraction, and 3×3 convolution is also applied on the residual branch for feature extraction. Finally, the main branch output and the residual branch output are added element by element; the Relu activation function is applied before the convolution operation.

[0101] S3. Perform segmentation model training based on the divided training data set;

[0102] S4. Calculate the loss function of the medical image segmentation model, use the Adam optimizer to update the parameters, and save the optimal model weights during the training process;

[0103] Calculate the loss function of the segmentation model, the formula is as follows:

[0104] ;

[0105] ;

[0106] ;

[0107] in, is the sum of all pixels in the input image, 、 denote the predicted value and target value respectively, is the total loss, is the binary cross entropy loss, is the dice loss, 、 is the weight assigned to each loss.

[0108] S5. Use the trained segmentation model to test the test set data.

[0109] Example 2

[0110] The present invention also provides a medical image segmentation system based on a CNN and ViT hybrid architecture, comprising:

[0111] The data acquisition and preprocessing module is used to acquire multiple types of medical image datasets, preprocess the raw data, generate preprocessed images, and divide the processed data into training sets, validation sets, and test sets in proportion;

[0112] The model building module is used to build a segmentation model based on the hybrid architecture of CNN and ViT. The segmentation model uses the U-Net network as the basic framework, including encoder, decoder and edge optimization module;

[0113] The encoder constructed by the model building module includes four encoding layers, each of which includes a multi-scale feature extraction module and a semantic enhancement module; the decoder includes four decoding layers, each of which also includes a multi-scale feature extraction module and a semantic enhancement module; jump connections are used between the encoding layer and the decoding layer in the same layer, and frequency domain information is introduced at the bottleneck layer.

[0114] The model training module is used to train the model based on the divided training dataset, calculate the loss function of the medical image segmentation model, update the parameters using the Adam optimizer, and save the optimal model weights during the training process;

[0115] The model testing and evaluation module is used to test the test set data using the trained segmentation model and evaluate the model performance;

[0116] Storage module, used to store medical image data, model parameters and intermediate calculation results.

[0117] It is worth noting that the contents not elaborated in detail in the present invention are all prior art and are well known to those skilled in the art.

[0118] Therefore, the present invention provides a medical image segmentation method and system based on a CNN and ViT hybrid architecture. By combining the advantages of CNN and ViT, it effectively alleviates the problems of limited CNN receptive field and expensive ViT computational cost, improves the segmentation accuracy of the lesion area, and provides a more accurate and reliable solution for the field of medical image segmentation.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A medical image segmentation method based on a hybrid architecture of CNN and ViT, characterized in that: Here are the steps: Step S1: Acquire multiple types of medical image datasets, preprocess the original data, and generate preprocessed images; divide the processed data into training set, validation set, and test set in proportion; Step S2: construct a segmentation model based on a hybrid architecture of CNN and ViT, wherein the segmentation model is based on a U-Net network and includes an encoder, a decoder, and an edge optimization module; Step S3: performing segmentation model training based on the divided training data set; Step S4: Calculate the loss function of the medical image segmentation model, use the Adam optimizer to update the parameters, and save the optimal model weights during the training process; Step S5: Use the trained segmentation model to test the test set data; In step S2, the encoder includes four encoding layers, each including a multi-scale feature extraction module and a semantic enhancement module; the decoder includes four decoding layers, each also including a multi-scale feature extraction module and a semantic enhancement module; skip connections are used between the encoding layer and the decoding layer in the same layer, and frequency domain information is introduced at the bottleneck layer; The multi-scale feature extraction module extracts multi-scale feature semantic information from the input image, and the semantic enhancement module enhances the correlation between different semantic information; the output of the last decoding layer is upsampled to obtain a coarse segmentation result, which is then refined by the edge uncertainty guidance module to obtain the final output prediction result; In the encoder stage, the input feature map After downsampling through the multi-scale feature extraction module and the semantic enhancement module, it is input to the next encoding layer. After layer-by-layer downsampling of the feature map, the spatial resolution decreases layer by layer and the number of channels increases layer by layer. The downsampling formula at the encoder stage is as follows: ； ； ； ； in, is a 3×3 convolution operation, is the semantic enhancement module, is a multi-scale feature extraction module, is the output of the first encoding layer, is the output of the second encoding layer, is the output of the third encoding layer, is the output of the fourth encoding layer; Additional frequency domain features are introduced at the bottleneck layer, and the wavelet transform WT is used to decompose the input into low-frequency and high-frequency components. After deep convolution extracts features, the inverse wavelet transform IWT is used to reconstruct the features. The formula is as follows: ； in, is the output result after inverse wavelet transform, is the weight vector after the convolution kernel of size k×k; In the decoder stage, the input of each decoding layer is the up-sampled output feature map of the previous decoding layer and the output feature map of the encoding layer of the same layer. The feature map is up-sampled layer by layer, the spatial resolution is restored layer by layer, and the number of channels is reduced layer by layer. Finally, the segmentation model generates a coarse segmentation result. The edge uncertainty guidance module optimizes and reshapes the uncertainty pixels in the fuzzy boundary area of the coarse segmentation result. Specifically, it generates a refined mask by combining shallow features, uncertainty mapping and local feature matching, optimizes the coarse segmentation result, and obtains the final predicted segmentation result. The shallow features are the output of the first encoding layer; The uncertainty map is calculated based on the probability map of the coarse segmentation result, and the formula is as follows: ； in, For uncertain mapping, the value range is ; 、 Represent the maximum probability value and minimum probability value of the pixel respectively, Calculate the exponential function; The local feature is generated by weighted local averaging of the uncertain mapping, and the formula is as follows: ； ； in, is a local feature, represents the normalization factor, is a small value that prevents the denominator from being zero, is a pixel, is the pixel after refinement, The pixel belongs to the target category The probability of It is a shallow feature. Pixels Neighborhood area; Generate refined masks by combining shallow features, uncertainty mapping, and local feature matching , the calculation formula is as follows: ； ； in, For pixels The confidence score at is the confidence module; The rough segmentation results are optimized to obtain the final predicted segmentation results. The formula is as follows: ； in, To predict the segmentation results; The confidence module includes 3×3 convolution, Relu activation function and residual connection; wherein, 3×3 convolution is applied on the main branch for feature extraction, and 3×3 convolution is also applied on the residual branch for feature extraction, and finally the main branch output and the residual branch output are added element by element; the Relu activation function is applied before the convolution operation.

2. The medical image segmentation method based on a hybrid architecture of CNN and ViT according to claim 1, characterized in that: In step S4, the loss function of the segmentation model is calculated, and the formula is as follows: ； ； ； in, is the sum of all pixels in the input image, 、 denote the predicted value and target value respectively, is the total loss, is the binary cross entropy loss, is the dice loss, 、 is the weight assigned to each loss.

3. A medical image segmentation system based on a hybrid architecture of CNN and ViT, using the medical image segmentation method based on a hybrid architecture of CNN and ViT as claimed in claim 1, characterized in that: include: The data acquisition and preprocessing module is used to acquire multiple types of medical image datasets, preprocess the raw data, generate preprocessed images, and divide the processed data into training sets, validation sets, and test sets in proportion; A model building module is used to build a segmentation model based on a hybrid architecture of CNN and ViT. The segmentation model uses the U-Net network as the basic framework and includes an encoder, a decoder, and an edge optimization module; The model training module is used to train the model based on the divided training dataset, calculate the loss function of the medical image segmentation model, update the parameters using the Adam optimizer, and save the optimal model weights during the training process; The model testing and evaluation module is used to test the test set data using the trained segmentation model and evaluate the model performance; Storage module, used to store medical image data, model parameters and intermediate calculation results.

4. The medical image segmentation system based on a hybrid architecture of CNN and ViT according to claim 3, characterized in that: The encoder constructed by the model construction module includes four encoding layers, each of which includes a multi-scale feature extraction module and a semantic enhancement module; the decoder includes four decoding layers, each of which also includes a multi-scale feature extraction module and a semantic enhancement module; jump connections are used between the encoding layer and the decoding layer on the same layer, and frequency domain information is introduced at the bottleneck layer.

Images