MTS-Net fundus optic cup and optic disc segmentation method based on deep convolution and attention mechanism
By introducing deep convolution and attention mechanisms into deep learning image segmentation networks, the problems of slow training speed and low segmentation accuracy in the prior art are solved, and more efficient fundus visual cup disc segmentation is achieved.
Patent Information
- Application Number
- CN202411557223.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2025-02-18
AI Technical Summary
When dealing with fundus visual trays, existing deep learning image segmentation networks have problems such as slow training speed and low segmentation accuracy, especially when facing class imbalance problems, it is easy to ignore the segmentation of visual cubes.
After preprocessing the fundus image through the depth convolution and attention mechanism, feature extraction and segmentation is performed through the depth separation convolution module, multi-head self-attention module and decoding module.
It improves training speed and segmentation accuracy, can handle the segmentation of the visual cup and the visual disc more effectively, solves the problem of class imbalance, and achieves more efficient fundus image segmentation.
Smart Images
Figure QLYQS_1 
Figure QLYQS_2 
Figure QLYQS_3
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of deep learning and medical image processing, and in particular to a method for segmenting fundus optic cup and optic disc based on a multi-scale MTS-Net neural network combining deep convolution and attention mechanism. Background Art
[0002] The eye is one of the main sensory organs of the human body, responsible for converting light signals into visual information, helping us understand and perceive the world around us. It plays a key role in identifying objects, distinguishing colors, and perceiving space. Since vision occupies a central position in life, it is very important to maintain eye health. Therefore, early detection and diagnosis of ophthalmic diseases are crucial to prevent vision damage. Fundus image-assisted diagnosis has become the mainstream choice for fundus disease screening due to its non-invasive characteristics, non-traumatic and low-risk advantages.
[0003] The harm of glaucoma lies in its hidden nature and irreversible damage to the optic nerve. With the increasing frequency of using electronic devices such as mobile phones and computers in daily life, the number of people suffering from glaucoma is gradually increasing. It is estimated that the number of glaucoma patients will exceed 100 million in 2040. The hidden nature of glaucoma lies in the fact that the early symptoms of glaucoma patients are not obvious. Patients often realize the problem only after the optic nerve has been severely damaged. If not discovered and treated in time, it will eventually lead to permanent blindness. Therefore, early diagnosis and disease detection of glaucoma are crucial.
[0004] The cup-disc ratio (CDR) is an important clinical indicator for early screening and diagnosis of glaucoma in the field of ophthalmology. The cup-disc ratio of normal people is usually small, between 0.3 and 0.4. In the early stage of glaucoma, as the optic nerve fibers gradually disappear, the optic cup will gradually increase, which is manifested as a gradual increase in the cup-disc ratio. The CDR can be calculated using deep learning image segmentation methods. The image segmentation network based on the U-shaped structure is the mainstream framework of the current deep learning image segmentation method. Since the optic cup usually occupies a small area, and the optic disc and background occupy a large area, the traditional U-shaped network has poor performance in segmenting small areas for the class imbalance problem, and it is easy to ignore the segmentation of the optic cup and prefer the accurate segmentation of the optic disc. In addition, the real-time problem of segmentation training needs to be improved. Some complex networks take a long time to train segmentation. Balancing the accuracy of model segmentation and training speed is also a key challenge. Summary of the invention
[0005] Based on this, the present invention provides an MTS-Net neural network based on deep convolution combined with attention mechanism to jointly segment the fundus optic cup and optic disc, so as to solve the technical problems of slow training speed due to complex network model and balance the low segmentation accuracy.
[0006] To achieve the above object, the present invention adopts the following technical solutions:
[0007] Obtain the original fundus image to be processed and the mask image and input them into the image processing network;
[0008] Preprocess the input original fundus image data in the image processing network and send the preprocessed image into the segmentation network in full resolution form;
[0009] Propose an MTS-Net neural network based on the combination of Vision Transformer and MedNext structures to segment the optic cup and optic disc of fundus images, including a depthwise separable convolution module, a multi-head self-attention module, and a decoding module;
[0010] Further, S2 includes the following steps:
[0011] Create a white grayscale image and a black grayscale image with a size of 512×512 respectively: Define the white grayscale image matrix: W(i,j) = 255, Define the black grayscale image matrix: B(i,j) = 255 where: i,j ∈ {0, ……, 511} represent the row and column coordinates of the matrix W(i,j) represents the value size at the position (i,j), always 255, and B(i,j) represents the value size at the position (i,j), always 0, which is used to paste the cropped original RGB image and the label image;
[0012] The sizes of the original image I and the label image M are H×W, and calculate the row and column coordinates (center_l, center_w) of the median value in the image. Center crop the original image and the label image according to the center position respectively. The size of the cropped image is fixed at 512×512. For images smaller than 512×512, edge cropping and overlaying need to be performed with the created grayscale images respectively to ensure consistent sizes; Median fill the overlaid image, set the threshold as Y, regard the pixels with gray values lower than the threshold in the image as image noise N = {I(i,j) < Y}, calculate the mean value m for the pixels higher than the threshold Y, P = {(i,j)∣I(i,j) ≥ Y}, and fill the noise points with m to obtain the filled image
[0013] Where C represents the number of channels, and W and H are the height and width of the image respectively; The mean formula of pixels is:
[0014] The mean formula of pixels is: Where |P| is the absolute value of the number of non-noise pixels with pixel values higher than the threshold;
[0015] After filling the image For standard normalization, first calculate the mean μ of each channel c and standard deviation σ c , image X c,h,w Normalize The data distribution of each channel is made to have a mean μ of 0, and each channel X′ after normalization c Satisfies Ε[X′ c ]=0,Var[X′ c ]=1, that is, the standard normal distribution with mean 0 and standard deviation 1;
[0016] The MTS-Net network model is further proposed. The preprocessed optic cup and optic disc images are sent to the network. In the encoder network, the optic cup and optic disc features of the fundus image are gradually extracted in three stages.
[0017] The convolution kernel size W used by the deep convolution module c (3×3), with a stride of 2, for each channel X c (i, j) uses a single kernel for convolution operation, does not mix information between channels, and then performs group normalization on the convolution output results, the purpose of which is to extract the core local features and multi-stage local features of the image in the spatial and channel dimensions, including low-level features and high-level features of the optic cup and optic disc;
[0018] The calculation formula of depth convolution is The value range of m and n is 0, 1, 2, corresponding to the 9 pixels X with a convolution kernel size of (3×3) c (i+m,j+n) represents the pixel value of each point in the Cth channel, Y c (i, j) represents the output result after depth convolution;
[0019] The feature expansion layer uses a convolution kernel size of 1×1, achieves channel expansion by setting the number of convolution kernels, and uses a Gaussian error linear unit as a nonlinear activation function. The feature expansion layer is used to improve the expression ability of the model and increase the nonlinear ability, helping the model capture more delicate features.
[0020] The channel reduction module uses a convolution kernel size of 1×1 to compress the number of feature map channels and uses a linear rectifier unit as an activation function. The channel reduction module is used to reduce network model parameters and filter redundant features of the image.
[0021] The point-by-point convolution module is used to introduce a residual connection structure, including a path L1 and a path L2. The path L1 is obtained by processing features through the above-mentioned deep convolution layer. Path L2 is obtained by channel expansion through a point-by-point convolution layer
[0022] The residual connection is used to superimpose the output features of path L1 and path L2 to obtain Enable the network to learn the identity mapping, thereby alleviating the problem of network gradient disappearance;
[0023] The multi-head self-attention module calculates the attention weight using the following formula
[0024] Q is the query vector, which is the feature vector of the current token and is used to query the correlation between the features of the current token and other tokens.
[0025] K is the key vector, which is the feature vector when other tokens query this token. It is used to do a dot product with the query vector Q to calculate the correlation between the two tokens.
[0026] V is a value vector, which is used to perform weight calculation on the calculated correlation degree, thereby enhancing or weakening the relevance and importance of a certain feature;
[0027] d k is the dimension of the key vector, which is used to scale the dot product result, reduce the frequency of extreme values when using softmax later, and stabilize the calculation result so that the model can converge easily;
[0028] The decoding module uses a transposed convolution, deep convolution layer, group normalization operation, channel expansion layer, GELU activation function, and channel compression layer that are symmetrical to the encoding module. Its main function is to restore the spatial resolution and perform feature fusion in three steps, so that image segmentation can be performed later.
[0029] The transposed convolution module first uses a transposed convolution with a kernel size of and a step size of 2 to perform upsampling, and its input feature map is The output is Where in = out, while ensuring that the channel dimension remains unchanged, the spatial dimension is expanded;
[0030] The channel expansion layer uses a convolution operation with a convolution kernel size of 1×1 to transform the input feature map Expand the channel dimension, and its output feature map The number of channels is expanded to twice the original to enhance the learning and expression capabilities of the model;
[0031] The GELU activation function is expressed as: GELU(x) = x·Φ(x), where Φ(x) is the cumulative distribution function of the standard normal distribution;
[0032] The channel compression layer uses another 1×1 convolution to compress the expanded feature map to reduce the computational complexity. The number of channels in the final feature map will be reduced to half of the original number.
[0033] The residual structure symmetrical to the encoder structure is used to connect path L1′ and path L2′, and the superposition is obtained To solve the problem of network gradient disappearance;
[0034] The number of feature map channels after latent space decoding is mapped back to 3 classification channels. Point-by-point convolution is used to reduce the computational complexity. Only channel mapping is involved, and pixels do not affect each other. The segmentation result output from the network is Among them, 512 are the length and width of the segmentation result, which are consistent with the size of the mask image, 3 is the number of channels, and each pixel belongs to three types of prediction results, namely background, optic cup, and optic disc;
[0035] Finally, the segmentation result F′ is combined with the mask image The loss function L is sent to calculate the loss formula and then the model parameters are updated so that the model converges to the minimum value. The optimizer used in the present invention is the Adam optimizer;
[0036] The prediction stage calculates the intersection of the predicted mask A and the true mask B. The Dice coefficient is used to judge the segmentation effect, where |A∩B| represents the number of pixels overlapped by the predicted pixels and the real pixels, and |A∪B| is the total number of pixels overlapped by the predicted pixels and the real pixels, that is, |A∪B|=|A|+|B|-|A∩B|, and |A| and |B| are the predicted segmentation area and the real segmentation area of the label respectively. In theory, the larger the IOU index and the higher the Dice value, the better the segmentation effect, which further evaluates the accuracy of the prediction degree of the real data. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to clearly illustrate the technical solution of the present invention, the drawings required for use in the existing technical description will be introduced below. Ordinary technicians in this field can also obtain other drawings based on these drawings without paying any creative work.
[0038] Figure 1 It is a schematic diagram of the overall framework of the optic cup and optic disc joint segmentation model based on the MTS-Net neural network proposed in the present invention.
[0039] Figure 2 Schematic diagram of some submodules of feature extraction according to the present invention.
[0040] Figure 3A schematic diagram of an attention module in an embodiment of the present invention is shown.
[0041] Figure 4 This is a result diagram of image segmentation in an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The following is a clear and complete description of the technical solution of the present invention in combination with the drawings in the embodiments of the present invention, and the described embodiments are part of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present invention.
[0043] A joint segmentation of optic cup and optic disc in fundus images based on MTS-Net neural network combining VisionTransformer and MedNext structure is performed. Figure 1 As shown, the following steps are included:
[0044] The fundus images are collected using image processing equipment, usually using a clinical fundus camera;
[0045] The collected images are preprocessed by algorithms on the original fundus images, including: The original fundus image and mask image are operated in the spatial dimension, including image center cropping, edge cropping, median filling, data partitioning and normalization operations;
[0046] Create a white grayscale image and a black grayscale image of size 512×512 respectively: Define the white grayscale image matrix: W(i,j)=255, Define the black grayscale image matrix: B(i,j)=255 where: i,j∈{0,……,511} represents the row and column coordinates of the matrix W(i,j) represents the value at position (i,j), which is always 255, and B(i,j) represents the value at position (i,j), which is always 0, and is used to paste the cropped original RGB image and the label image;
[0047] The size of the original image I and the label image M is H×W, and the position of the median value in the image is calculated The row and column coordinates (center_l, center_w) of the original image and the label image are cropped at the center position respectively. The cropped image size is fixed to 512×512. For images smaller than 512×512, the grayscale images created in the previous steps need to be used for edge cropping and superposition to ensure the same size.
[0048] Median fill the superimposed image, set the threshold to Y, and consider the pixels in the image with gray values lower than the threshold as image noise N = {I(i, j) | I(i, j) < Y}. Calculate the mean m for the pixels P = {(i, j) | I(i, j) ≥ Y} higher than the threshold Y, and use m to fill the noise points. The formula for calculating the mean of non-noise pixels is as follows: where |P| is the absolute value of the number of non-noise pixel points with pixel values higher than the threshold;
[0049] Perform standard normalization on the filled image data so that the data distribution of each channel is a standard normal distribution with a mean μ of 0 and a standard deviation σ of 1;
[0050] Before feeding the preprocessed image into the network, first use convolution to implement the embedding layer mapping. The main reason is that the real picture information is high-dimensional sparse data. In order to adapt to the deep learning network, a convolution operation is used in advance to expand the channel dimension, so as to convert the high-dimensional sparse data into low-dimensional dense data, so that the network can capture more features it needs in the latent space;
[0051] Feed the fundus image in the latent space into the MTS-Net neural network for training. The MTS-Net network consists of three parts in total. The depthwise separable convolution module serves as the encoding part, the attention module is used to obtain the global information that is difficult to extract by the convolution structure, and the decoding part that is symmetric to the encoding structure;
[0052] The depthwise separable convolution module contains four sub-modules, namely the Depthwise depth convolution module, the feature expansion layer, the channel reduction module, and the pointwise convolution. As the encoder module of the MTS-Net network, it extracts features in three stages;
[0053] The convolution kernel size W used in the depth convolution module c (3×3), with a stride of 2. For each channel X c (i, j), use a separate kernel for convolution operation, which will not mix information between channels, thereby improving the efficiency of the model and reducing the computational amount and time consumed in the early stage of feature extraction. Its purpose is to extract the core local features and multi-stage local features of the image in the spatial and channel dimensions, including the low-level features and high-level features of the optic cup and optic disc. The formula for depth convolution is as follows: where the value ranges of m and n are 0, 1, 2, corresponding to the 9 pixel points X with a convolution kernel size of (3×3) c (i + m, j + n) represents the pixel value of each point in the Cth channel, Y c(i, j) represents the output result after deep convolution, and the dimension size of the feature map is;
[0054] Then the depth-separable convolution output result Y c (i,j) performs normalization operation grouped by channel,
[0055] The group-normalized feature map is then sent to the feature expansion layer, which uses a convolution kernel size of 1×1. The channel expansion is achieved by setting the number of convolution kernels, and the Gaussian error linear unit is used as the nonlinear activation function. The Gaussian error linear activation unit is defined as: GELU(x)=x·Φ(x), where Φ(x) is the upper distribution function of the standard normal distribution, which can be expressed as: Here erf is the abbreviation of error function, which is defined as: The feature expansion layer is used to improve the model's expressiveness and increase nonlinearity, helping the model capture more delicate features.
[0056] The channel reduction module uses a convolution kernel size of 1×1 to compress the number of feature map channels and uses a linear rectifier unit as an activation function. The channel reduction module is used to reduce network model parameters and filter redundant features of the image.
[0057] The point-by-point convolution module is used to introduce a residual connection structure, including path L1 and path L2. The path L1 processes the features through the above-mentioned deep convolution layer to obtain Y L1 (i,j)=D w (X), path L2 is expanded through a point-by-point convolution layer to obtain Y L2 (i,j)=P c (X);
[0058] The residual connection is used to superimpose the output features of path L1 and path L2 to obtain Y res (i,j)=Y L1 (i,j)+Y L2 (i, j), so that the network can learn the identity mapping, thus alleviating the problem of network gradient disappearance;
[0059] The feature maps extracted by the three encoders and the input sizes are: x1[2,64,256,256],x2[2,128,128,128],x3[2,256,64,64],x4[2,512,32,32]
[0060] The multi-head self-attention module calculates the attention weight using the following formula: Where Q is the query vector, which is the feature vector of the current token and is used to query the correlation between the features of its own token and other tokens; K is the key vector, which is the feature vector when other tokens query this token. It is used to do a dot product with the query vector Q to calculate the correlation between the two tokens. Where V is a value vector, which is used to perform weight calculation on the calculated correlation degree, thereby enhancing or weakening the relevance and importance of a certain feature; where d k is the dimension of the key vector, which is used to scale the dot product result, reduce the frequency of extreme values when using softmax later, and stabilize the calculation result so that the model can converge easily;
[0061] The detailed description of the multi-head attention mechanism is to compare the input image to the word vector token in natural language. Here, each input image data Divide the image into blocks, each block is P×P in size, and then expand the image into a one-dimensional vector token. is divided equally into multiple parts, each with its own query and key vector value vector, and the three vectors are of the same size as x i , and perform synchronous calculations. The final result is then pieced together into a token size Z = Concat (y1, y2, ...), and then restored to the image form of the homologous image O = WZ through linear transformation, and finally superimposed O out =O+I, the image now has global attention weight;
[0062] The decoding module uses a transposed convolution, depthwise separable convolution layer, group normalization operation, channel expansion layer, GELU activation function, and channel compression layer, which are symmetrical to the encoding module. Its main function is to restore the spatial resolution and perform feature fusion in three steps, so that image segmentation can be performed later.
[0063] The transposed convolution module first uses a transposed convolution with a kernel size of 1×1 and a stride of 2 to perform upsampling, and its input feature map is The output is Where in = out, while ensuring that the channel dimension remains unchanged, the spatial dimension is expanded;
[0064] The channel expansion layer uses a convolution operation with a convolution kernel size of 1×1 to transform the input feature map Expand the channel dimension, and its output feature map The number of channels is expanded to twice the original to enhance the learning and expression capabilities of the model;
[0065] The GELU activation function is expressed as: GELU(x) = x·Φ(x), where Φ(x) is the cumulative distribution function of the standard normal distribution;
[0066] The channel compression layer uses another 1×1 convolution to compress the expanded feature map to reduce the computational complexity. The number of channels in the final feature map will be reduced to half of the original number.
[0067] The residual structure symmetrical to the encoder structure is used to connect path L1′ and path L2′, and the superposition is obtained To solve the problem of network gradient disappearance;
[0068] The number of feature map channels after latent space decoding is mapped back to 3 classification channels. Point-by-point convolution is used to reduce the computational complexity. Only channel mapping is involved, and pixels do not affect each other. The segmentation result output from the network is Among them, 512 are the length and width of the segmentation result, which are consistent with the size of the mask image, 3 is the number of channels, and each pixel belongs to three types of prediction results, namely background, optic cup, and optic disc;
[0069] Finally, the segmentation result F′ is combined with the mask image The loss function L is sent to calculate the loss function, and the Adam optimizer is used to update the model parameters. The purpose is to use the loss function to reduce the direction to update the network parameters so that the model converges to the minimum value. At this time, the output result of the model has the highest similarity with the label mask image, the output effect is the best, and the optimal training weight is saved;
[0070] The prediction stage uses the prediction data set. The images of the prediction data set are consistent with those in the training stage. The optimal weights of the training stage are loaded into the network, and the images of the prediction data set are sent to the neural network to obtain the segmentation results. The intersection of the predicted mask A and the real mask B is calculated. and The coefficient is used to judge the segmentation effect, where |A∩B| represents the number of pixels overlapped by the predicted pixels and the real pixels, and |A∪B| is the total number of pixels overlapped by the predicted pixels and the real pixels, that is, |A∪B|=|A|+|B|-|A∩B|, |A| and |B| are the predicted segmentation area and the real segmentation area of the label respectively, so as to evaluate the accuracy of the prediction degree of the real data;
[0071] During the training process, we can see that when using a 3080 graphics card to train the network, the learning rate is set to 1×10 -4 , weight decay is set to 1×10 -8When epoch=1, it takes only one and a half minutes to traverse one round, the average IOU of the optic cup and optic disc can reach 0.822078, the optic cup Dice reaches 0.924851, and the optic disc Dice reaches 0.877988. The optic cup segmentation has basically reached the current average sota level. When epoch=100 rounds, the average IOU can reach 0.894337, the optic cup Dice reaches 0.962012, and the optic disc Dice reaches 0.914380. It takes about 2.5 hours at this time. At this time, the joint segmentation of the optic cup and optic disc can reach the current standard segmentation level, and the time spent is shorter and the efficiency is higher, which meets the requirements of clinical diagnosis.
[0072] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described with reference to the foregoing, a person skilled in the art should understand that the technical solutions described in the foregoing embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. An improved neural network model for Transunet segmentation of optic cup and optic disc in fundus images, characterized in that: The model includes the following steps: S1: Obtain the original fundus image to be processed and the mask image and input them into the image processing network; S2: Preprocess the input original fundus image data in the image processing network and send the preprocessed image to the segmentation network in full resolution form; S3: Propose an MTS-Net neural network based on the combination of Vision Transformer and MedNext structures for joint segmentation of the optic cup and optic disc in fundus images, including a depth convolution module, a multi-head self-attention module, and a decoding module.
2. The network model according to claim 1, characterized in that: In step S2, the specific method for preprocessing the input original fundus image and the corresponding mask image is as follows: S21: Operate on the original fundus image and the mask image in the spatial dimension, including image center cropping, edge cropping, median filling, data partitioning, and normalization operations; S22: Create a white grayscale image and a black grayscale image with a size of 512×512 respectively: Define the white grayscale image matrix: W(i,j) = 255, Define the black grayscale image matrix: B(i,j) = 255 where: i,j ∈ {0, ……, 511} represents the row and column coordinates of the matrix W(i,j) represents the value size at the position (i,j), always 255, and B(i,j) represents the value size at the position (i,j), always 0, which is used to paste the cropped original RGB image and the label image; S23: The size of the original image I and the label image M is H×W, and the position of the median of the image is calculated The row and column coordinates (center_l, center_w) of the original image and the label image are respectively center cropped according to the center position. The size of the cropped image is fixed to 512×512. For images with a size smaller than 512×512, the grayscale image created in step S22 needs to be used for edge cropping and superposition to ensure the same size. S24: Perform median filling on the superimposed image, set the threshold to Y, regard the pixels with gray values lower than the threshold in the image as image noise N = {I(i,j)∣I(i,j) < Y}, calculate the mean m for the pixels P = {I(i,j)∣I(i,j) ≥ Y} higher than the threshold Y, and use m to fill the noise points. The formula for calculating the mean of non-noise pixels is as follows: where |P| is the absolute value of the number of non-noise pixel points with pixel values higher than the threshold; S25: Perform standard normalization on the filled image data so that the data distribution of each channel is a standard normal distribution with a mean μ of 0 and a standard deviation σ of 1.
3. According to the MTS-Net network model of claim 1, it is characterized in that: The specific method of step S3 is as follows: S31: The depth convolution module includes four sub-modules, namely the Depthwise depth convolution module, the feature expansion layer, the channel reduction module, and the pointwise convolution. As the encoder module of the MTS-Net network, it extracts features in three stages; S32: The convolution kernel size W used by the depth convolution module c (3×3), with a stride of 2, for each channel X c (i, j) uses a separate kernel for convolution operation, which does not mix information between channels, and then performs group normalization on the convolution output results. The purpose is to extract the core local features and multi-stage local features of the image in the spatial and channel dimensions, including the low-level features and high-level features of the optic cup and optic disc. The calculation formula for deep convolution is as follows: The value range of m and n is 0, 1, 2, corresponding to the 9 pixels X with a convolution kernel size of (3×3) c (i+m,j+n) represents the pixel value of each point in the Cth channel, Y c (i, j) represents the output result after depth convolution; S33: The feature expansion layer uses a convolution kernel size of 1×1, realizes channel expansion by setting the number of convolution kernels, and uses the Gaussian error linear unit as the non-linear activation function. The feature expansion layer is used to enhance the expression ability of the model and increase the non-linear ability to help the model capture more delicate features; S34: The channel reduction module uses a convolution kernel size of 1×1 to compress the number of channels of the feature map and uses the rectified linear unit as the activation function. The channel reduction module is used to reduce the network model parameters and filter the redundant features of the image; S35: The point-by-point convolution module is used to introduce a residual connection structure, including a path L1 and a path L2. The path L1 processes the features through the above-mentioned deep convolution layer to obtain Y L1 (i,j)=D w (X), path L2 is expanded through a point-by-point convolution layer to obtain Y L2 (i,j)=P c (X), the residual connection is used to superimpose the output features of path L1 and path L2 to obtain Y res (i,j)=Y L1 (i,j)+Y L2 (i, j), so that the network can learn the identity mapping, thus alleviating the problem of network gradient disappearance; S36: The multi-head self-attention module calculates the attention weights through the following formula: Where Q is the query vector, which is the feature vector of the current token and is used to query the correlation between the features of its own token and other tokens; K is the key vector, which is the feature vector when other tokens query this token. It is used to do a dot product with the query vector Q to calculate the correlation between the two tokens. Where V is a value vector, which is used to perform weight calculation on the calculated correlation degree, thereby enhancing or weakening the relevance and importance of a certain feature; where d k is the dimension of the key vector, which is used to scale the dot product result, reduce the frequency of extreme values when using softmax later, and stabilize the calculation result so that the model can converge easily; S37: The decoding module uses a symmetrical structure with the encoding module, including a transposed convolution with a step size of 2 and a kernel size of 1×1, a deep convolution layer with a convolution kernel size of 1×1, a group normalization operation, a channel expansion layer, a GELU activation function, and a channel reduction module. Its main function is to restore the spatial resolution and perform feature fusion in three steps. S38: Finally, the segmented tensor result is restored from the multi-channel of the latent space to the real image, and the resolution is restored to the same size and number of channels as the label image by performing an upsampling operation, thereby achieving complete image segmentation.