Prostate three-dimensional image segmentation model and method based on deep joint transformer

Abstract

The invention discloses a prostate three-dimensional image segmentation model and method based on a deep joint transformer, and belongs to the technical field of computer vision, image segmentation and artificial intelligence, and the prostate three-dimensional image segmentation model mainly comprises the following modules: a deep joint transformer coding module and a pooling fusion attention decoding module. Compared with the prior art, the prostate three-dimensional image segmentation model and method based on the deep joint transformer are designed for prostate three-dimensional image data. According to the model, global and local features of an image are effectively extracted through a deep joint transformer module, and remote semantic information in a context is learned at the same time. Then, the information is transmitted to a pooling fusion attention decoding module for up-sampling, so that features of different scales are fused, potential features of samples are further analyzed, and lost information in the up-and-down sampling process is made up. According to the method, the key gland region in the three-dimensional prostate image can be effectively segmented, and the performance of an image segmentation algorithm is improved.

Description

Technical Field

[0001] The technical field to which the present invention belongs is the field of medical image analysis, and more specifically, it relates to a prostate three-dimensional image segmentation model and method based on a deep joint transformer. Background Art

[0002] Prostate cancer is the second leading cause of cancer death in men. Magnetic resonance imaging (MRI) is often used in clinical examination and diagnosis of prostate diseases due to its high resolution. MRI imaging is formed by the interaction between radio frequency pulses, strong magnetic fields and body tissues, but its images may contain artifacts and noise. Therefore, it is very challenging to accurately identify the prostate in MRI images.

[0003] Early prostate segmentation methods mainly relied on deformable methods, atlas registration, and machine learning. However, these segmentation methods are greatly affected by the performance of equipment and the type of prostate in practical applications, resulting in poor accuracy of segmentation results and high computational complexity. In recent years, deep learning has been widely used in fields such as medical image segmentation and lesion type recognition due to its automatic representation learning ability. Convolutional neural networks are the core networks of deep learning and have strong nonlinear expression capabilities. The key advantage is that the convolutional layer can effectively extract features or reduce dimensions. However, these networks are mainly combined through cascading, and most of the work focuses on feature extraction. The importance of feature analysis to the accuracy of model output is often overlooked.

[0004] In order to solve the above problems, researchers proposed a network model with a symmetrical structure of encoder and decoder, which effectively solved the problem of serial convolutional neural network structure in the field of medical image segmentation. Nevertheless, convolutional neural networks have limited modeling capabilities for long-range dependencies, which limits their ability to fully utilize contextual semantic information in images. With the widespread application of transformers in the field of natural language processing, many scholars have tried to apply them to the field of computer vision, because transformers can effectively obtain long-range semantic information, and their segmentation results are more accurate than traditional convolutional neural networks. Most of the existing transformer-based image segmentation models are designed for two-dimensional images, while medical images are mainly three-dimensional. Slicing studies of three-dimensional medical images may lose important 3D context information.

[0005] In view of the above problems, the present invention proposes a prostate 3D image segmentation model and method based on a deep joint transformer, which adopts a coding and decoding structure with a symmetrical structure, and also uses the idea of ​​a deep joint transformer to extract global context information from the image and enhance the semantic expression ability of the model. Summary of the invention

[0006] Technical problem: The purpose of the present invention is to provide a prostate three-dimensional image segmentation model and method based on deep joint transformer, which is used to fill the research gap in the three-dimensional prostate direction based on deep joint transformer in the current medical image algorithm, and is conducive to solving the problem of further application of medical image segmentation methods.

[0007] Technical solution: First, the prostate image is passed through the data preprocessing module to achieve data enhancement to expand the training set. Then, the pre-trained image is passed through the downsampling encoding path to effectively extract the global and local features of the image, and at the same time learn the long-distance semantic information in the context. Subsequently, the features are further enhanced through jump connections, and then the training image segmentation results are obtained through the upsampling decoding path. The sample results and the actual segmentation target are then calculated with the loss function, and the model parameters are continuously updated. The trained segmentation network model has the function of accurately segmenting the gland. The model not only uses a deep joint transformer in the design of the encoding path, but also embeds a pooling fusion attention method in the decoding path to further analyze the potential features of the sample, and adds a loss function for back propagation, finally realizing a deep joint transformer image segmentation model with a symmetric structure.

[0008] A prostate 3D image segmentation model based on deep joint transformer proposed in the present invention includes a deep joint transformer encoding module and a pooling fusion attention decoding module;

[0009] The deep joint transformer encoding module can effectively extract global and local features of the image and simultaneously learn long-range semantic information in the context;

[0010] The pooling fusion focuses on the decoding module to fuse features of different scales to deeply analyze the potential attributes of the sample to make up for the information lost in the up- and down-sampling process.

[0011] Furthermore, the deep joint transformer encoding module is composed of a joint transformer UTrans, a deep convolutional network module DCN and a grouped convolutional perception module GCP; the patch feature sequence first extracts pixel features locally through a deep convolutional network module, and then uses a joint transformer to explore local important information and global dependencies; finally, the interactive relationship between contextual semantics is perceived through the grouped convolutional perception module, thereby further enhancing the ability to extract target area features.

[0012] Furthermore, the pooling fusion attention decoding module first improves the model's sensitivity to features of different scales through two parallel spatial squeezing and channel excitation modules, and then introduces a residual connection to reduce the information lost when parsing the context; then the features obtained by the parallel modules are element-wise added, and the feature extraction range is expanded through a fully connected layer; in order to keep the dimension unchanged, a convolution dimensionality reduction operation is used later, and a residual connection is introduced to reduce information loss in the process of parsing features.

[0013] Furthermore, the joint transformer consists of a local transformer and a global transformer, the deep convolutional network module consists of a deep convolutional layer, a normalization layer and an activation function, and the grouped convolutional perception module consists of two grouped convolutional layers, a normalization layer and an activation function.

[0014] Furthermore, the spatial squeezing and channel excitation module consists of a pooling layer, a fully connected linear layer, a nonlinear ReLu transformation, a fully connected layer for enhancing the channel dimension, and a Sigmoid activation function.

[0015] After being processed by the above modules in sequence, the three-dimensional prostate image segmentation can be completed.

[0016] The present invention also proposes a prostate three-dimensional image segmentation method based on a deep joint transformer, comprising the following steps:

[0017] S1. Design a deep joint transformer encoding module to effectively extract global and local features of images and learn long-range semantic information in context;

[0018] S2. Design a pooling fusion attention decoding module to fuse features of different scales to further analyze the potential features of the sample and make up for the information lost during upsampling and downsampling.

[0019] Furthermore, the deep joint transformer encoding module in S1 is composed of a joint transformer UTrwns, a deep convolutional network module DCN and a grouped convolutional perception module GCP; the feature sequence first extracts pixel features locally through a deep convolutional network module, and then uses a joint transformer to explore local important information and global dependencies; finally, the interactive relationship between contextual semantics is perceived through the grouped convolutional perception module, further enhancing the ability to extract target area features;

[0020] Among them, the deep convolutional network module consists of a deep convolutional layer, a normalization layer and a ReLu activation function; the grouped convolutional perception module consists of 2 grouped convolutional layers, a normalization layer and a ReLu activation function;

[0021] Among them, the joint transformer includes a local transformer LW and a global transformer GSW;

[0022] The local transformer consists of two normalization layers LN, a local window multi-head self-attention module LW-MSA and a multi-layer perceptron MLP. A residual connection is added after LW-MSA and MLP to alleviate the impact of gradient disappearance. The local window multi-head self-attention module first evenly divides the feature map transformed by the normalization layer into non-overlapping volume windows, and then performs multi-head self-attention learning inside the window.

[0023] Local transformer characteristic result X LW The expression is:

[0024] Q,K,V=LN(Fn(X))

[0025]

[0026] LW-MSA(X)=Attention(Q,K,V)

[0027]

[0028] Among them, X represents the input feature, which is the feature map after preprocessing, Fn represents the window segmentation function, LN represents the normalization layer for linear projection mapping, Q, K, and V represent the mapping result query matrix, key value matrix, and value matrix respectively. d represents the matrix dimension, T represents the matrix transpose, and B represents the deviation parameter of the corresponding function. After a series of operations, the three matrices after mapping are used to obtain the new feature map Attention (Q, K, V) through the nonlinear activation function SoftMax. In the model, LW-MSA (X) is used to represent the operation process of converting the input feature map into a local window multi-head self-attention feature. In order to reduce the loss of information during the sampling process, two layers of residual connections are added to the model. The local window multi-head self-attention feature LW-MSA (X) is added to the input feature X element to obtain the local window residual connection To further extract deep features, the model After normalization, a new feature map is obtained through a multi-layer perceptron Then with Add the elements to get the local transformer feature result X LW , the model uses X LW represents the features after being processed by the local transformer LW;

[0029] Different from the local transformer, the global transformer uses the global moving window multi-head self-attention GSW-MSA instead of LW-MSA; the feature map is divided into multiple volume windows of the same size and non-overlapping, and then the volume window is moved in the three dimensions of height H, width W, and depth D. The top window moves to the bottom, the leftmost window moves to the rightmost, and the topmost window moves to the bottom, thereby forming a new feature map; feature blocks are selected at a specific distance from each dimension of the feature map and kept consistent with the size of the volume window, and then combined into a new global feature map; to ensure the consistency of the local window and global moving window transformer structures, the feature map is again divided into volume windows of the same size and non-overlapping, and its size is the same as the window size in the local transformer; then self-attention learning is performed in each window;

[0030] Global transformer feature results X GSW The expression is:

[0031] X G =Interval(Roll(X Lw ))

[0032] Q, K, V = LN(Fn(X g ))

[0033]

[0034] GSW-MSA(X g ) = Attention(Q, K, V)

[0035]

[0036] Where X LW represents the input feature, which is the local transformer feature result. Roll, Interval, and Fn represent the window moving function, interval selection feature block function, and volume window segmentation function, respectively. g It represents the feature result after the window moving function and the interval selection feature block function are operated. LN represents the normalization layer, which is used for linear projection mapping. Q, K, and V represent the mapping result query matrix, key value matrix, and value matrix respectively. d represents the matrix dimension, T represents the matrix transpose, and B represents the deviation parameter of the corresponding function. After a series of operations, the three matrices after mapping are used through the nonlinear activation function SoftMax to obtain the new feature map Attention (Q, K, V). GSW-MSA (X g ) represents feature X g The global moving window multi-head self-attention feature GSW-MSA(X) is converted into the global moving window multi-head self-attention feature. LWAdd the elements to get the global window residual connection The model will After normalization, a new feature map is obtained through a multi-layer perceptron Then with Add the elements to get the global transformer feature result X GSW , the model uses X GSW Represents the features after processing by the global transformer GSW.

[0037] Furthermore, the pooling fusion attention decoding module in S2 first improves the model's sensitivity to features of different scales through two parallel spatial squeezing and channel excitation modules, and then introduces a residual connection to reduce the information lost when parsing the context; then the features obtained by the parallel modules are element-wise added, and the feature extraction range is expanded through a fully connected layer; in order to avoid the problem of channel dimension expansion in the decoding path, a convolutional dimensionality reduction operation is used to reduce the dimension by half, and a residual connection is introduced to reduce information loss in the process of parsing features.

[0038] Among them, the spatial squeezing and channel excitation modules include a pooling layer, two consecutive linear layers, the ReLu activation function is used between the linear layers, the Sigmoid activation function is used after the second linear layer, and then a residual connection is introduced; the pooling layers in the two parallel modules use average pooling and maximum pooling respectively; two consecutive linear layers are used for channel excitation.

[0039] Beneficial effects of the present invention:

[0040] 1. This paper proposes a prostate 3D image segmentation model based on deep joint transformer, which is the first study to apply deep joint transformer to prostate segmentation. The study shows that the model's ability to extract global semantic features and complex local details has been significantly improved.

[0041] 2. The present invention designs a deep joint transformer encoding module, which effectively combines the deep convolutional network module, the grouped convolutional perception module and the joint transformer to enhance the ability to extract local detail information from the image. This method improves the model's ability to identify and locate the contours of segmented targets.

[0042] 3. The present invention designs a pooling fusion attention decoding module for feature analysis, realizes information fusion in spatial and channel dimensions, and enables the model to collect deeper features and capture key information through multi-angle spatial compression and channel excitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the overall structure of the method of the present invention.

[0044] Figure 2A structural schematic diagram of the deep joint transformer encoding module of the method of the present invention.

[0045] Figure 3 A structural diagram of the pooling fusion attention decoding module of the method of the present invention.

[0046] Wherein, the reference numerals in the accompanying drawings are as follows:

[0047] 000-image sample, 001-output result, 010-data preprocessing module, 020-patch conversion module, 100-deep joint transformer encoding module, 200-pooling fusion attention decoding module, 030-skip connection;

[0048] 110-deep convolutional network module, 120-local transformer, 121-global transformer, 130-joint transformer, 140-grouped convolutional perception module;

[0049] 201-input image sample, 202-output image, 210-average pooling space squeezing and channel excitation module, 220-maximum pooling space squeezing and channel excitation module, 211-average pooling layer, 212-channel squeezing layer, 213-channel excitation layer, 221-maximum pooling layer, 230-fully connected layer, 231-convolutional dimension reduction layer. DETAILED DESCRIPTION

[0050] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific drawings and specific embodiments. Among them, the same or similar reference numerals in the drawings of the specific embodiments of the present invention represent the same or similar elements, or elements with the same or similar functions. It should be understood that the specific embodiments described below are intended to explain the present invention and are not intended to limit the present invention.

[0051] A prostate 3D image segmentation method based on deep joint transformer, mainly including deep joint transformer encoding module and pooling fusion attention decoding module;

[0052] The deep joint transformer encoding module is used to effectively extract the global and local features of the image and learn the long-distance semantic information in the context; the pooling fusion attention decoding module fuses features of different scales to deeply analyze the potential attributes of the sample to make up for the information lost in the up- and down-sampling process;

[0053] The above-mentioned deep joint transformer encoding module includes a joint transformer UTrans130, a deep convolutional network module DCN110 and a grouped convolutional perception module GCP140; the patch feature sequence first extracts pixel features locally through a deep convolutional network module 110, and then uses a joint transformer 130 to explore local important information and global dependencies; finally, the interactive relationship between contextual semantics is perceived through the grouped convolutional perception module 140, thereby further enhancing the ability to extract target area features.

[0054] The above-mentioned pooling fusion attention decoding module first improves the model's sensitivity to features of different scales through two parallel space squeezing and channel excitation modules 210 and 220, and then introduces a residual connection to reduce the information lost when parsing the context; then the features obtained by the parallel modules are element-wise added, and the feature extraction range is expanded through a fully connected layer 230; in order to avoid the problem of channel dimension expansion in the decoding path, a convolutional dimensionality reduction operation 231 is used to reduce the dimension by half, and a residual connection is introduced to reduce information loss in the process of parsing features.

[0055] The following is combined with Figures 1 to 3 A prostate three-dimensional image segmentation method based on deep joint transformer provided by the present invention is described in detail.

[0056] A prostate three-dimensional image segmentation method based on a deep joint transformer is mainly composed of a deep joint transformer encoding module 100 and a pooling fusion attention decoding module 200. The real image 000 is first preprocessed by a data preprocessing module 010, and then the image is converted into a patch sequence by a patch conversion module 020. Then, the features are extracted by a 4-layer deep joint transformer encoding module 100, and then the extracted features are parsed by a pooling fusion attention decoding module 200, and then added to the feature map after the jump connection 030 in the channel dimension for upsampling operation, and finally the segmentation result is obtained after 4 layers of feature parsing and upsampling. The specific structure is as follows Figure 1 shown.

[0057] In the example of the present invention, the data preprocessing module 010 specifically includes random rotation, proportional enlargement and reduction, and adjustment of contrast and brightness. Subsequently, we randomly intercept four volume blocks of the same size on the image to ensure that the volume blocks contain at least 90% of the key area of ​​the prostate. The patch conversion module 020 ensures that the preprocessed data will first be divided into a fixed-size patch sequence through the patch embedding layer before entering the transformer. The jump connection 030 ensures that each layer in the decoding path is spliced ​​by the upsampling of the feature map of the previous layer and the feature map of the corresponding encoding layer, and then passes through the 200 module to make up for the information lost during the upsampling and downsampling process.

[0058] A deep joint transformer encoding module 100 is composed of a joint transformer 130, a deep convolutional network module 110, and a grouped convolutional perception module 140. First, a deep convolutional network module 110 is used to extract pixel features locally; then the joint transformer 130 is used to explore local important information and global dependencies; finally, the grouped convolutional perception module 140 is used to perceive the interactive relationship between contextual semantics, further enhancing the ability to extract target area features. Figure 2 shown.

[0059] In this embodiment, the deep convolutional network module 110 is mainly composed of a deep convolutional layer, a normalization layer and a ReLu activation function. The grouped convolutional perception module 140 is mainly composed of two grouped convolutional layers, a normalization layer and a ReLu activation function. The joint transformer 130 is composed of a local transformer 120 and a global transformer 121.

[0060] The local transformer 120 consists of two normalization layers LN 132, a local window multi-head self-attention LW-MSA 131 module and a multi-layer perceptron MLP 133. Residual connections are added after LW-MSA and MLP to alleviate the impact of gradient disappearance. The local transformer is connected after the deep convolution network module. The features after deep convolution processing are first normalized and then passed to the local window multi-head self-attention module. The feature map is evenly divided into non-overlapping volume windows and multi-head self-attention learning is performed inside the window. After the normalization layer transformation, it is passed to the multi-layer perceptron for processing.

[0061] Local transformer characteristic result X LW The expression is:

[0062] Q, K, V = LN(Fn(X))

[0063]

[0064] LW-MSA(X)=Attention(Q,K,V)

[0065]

[0066] Where X represents the input feature, which is the preprocessed feature map. Fn represents the window segmentation function. LN represents the normalization layer for linear projection mapping. Q, K, and V represent the mapping result query matrix, key value matrix, and value matrix, respectively. d represents the matrix dimension, T represents the matrix transpose, and B represents the bias parameter of the corresponding function. After a series of operations, the three mapped matrices are used to obtain the new feature map Attention (Q, K, V) through the nonlinear activation function SofiMax. In the model, LW-MSA (X) is used to represent the operation process of converting the input feature map into a local window multi-head self-attention feature. In order to reduce the loss of information during the sampling process, two layers of residual connections are added to the model. The local window multi-head self-attention feature LW-MSA (X) is added to the input feature X element to obtain the local window residual connection. To further extract deep features, the model After normalization, a new feature map is obtained through a multi-layer perceptron Then with Add the elements to get the local transformer feature result X LW , the model uses X LW represents the features after being processed by the local transformer LW;

[0067] The difference between the global transformer and the local transformer is that the global moving window self-attention GSW-MSA 141 replaces the LW-MSA 131. The global transformer is connected after the local transformer, first passes through a normalization layer, and then passes into the global moving window self-attention to further extract features. First, the feature map is divided into multiple volume windows of the same size and non-overlapping, and then the volume window is moved in the three dimensions of height H, width W, and depth D. The top window moves to the bottom, the leftmost window moves to the rightmost, and the topmost window moves to the bottom, thereby forming a new feature map. Further, feature blocks are selected at a specific distance from each dimension of the feature map, and are kept consistent with the size of the volume window, and then combined into a new global feature map. In order to ensure the consistency of the local window and global moving window transformer structures, the feature map is again divided into volume windows of the same size and non-overlapping, and its size is the same as the window size in the local window transformer. Then self-attention learning is performed in each window. After that, the self-attention feature map is passed to the normalization layer and the multi-layer perceptron.

[0068] Global transformer feature results X GSW The expression is:

[0069] X G =Interval(Roll(X LW ))

[0070] Q, K, V = LN(Fn(Xg ))

[0071]

[0072] GSW-MSA(X g ) = Attention(Q, K, V)

[0073]

[0074] Where X LW represents the input feature, which is the local transformer feature result. Roll, Interval, and Fn represent the window moving function, interval selection feature block function, and volume window segmentation function, respectively. g It represents the feature result after the window moving function and the interval selection feature block function are operated. LN represents the normalization layer, which is used for linear projection mapping. Q, K, and V represent the mapping result query matrix, key value matrix, and value matrix respectively. d represents the matrix dimension, T represents the matrix transpose, and B represents the deviation parameter of the corresponding function. After a series of operations, the three matrices after mapping are used through the nonlinear activation function SoftMax to obtain the new feature map Attention (Q, K, V). GSW-MSA (X g ) represents feature X g The global moving window multi-head self-attention feature GSW-MSA(X) is converted into the global moving window multi-head self-attention feature. LW Add the elements to get the global window residual connection The model will After normalization, a new feature map is obtained through a multi-layer perceptron Then with Add the elements to get the global transformer feature result X GSW , the model uses X GSW Represents the features after processing by the global transformer GSW.

[0075] A pooling fusion attention decoding module consists of two parallel spatial squeezing and channel excitation modules 210 and 220, a fully connected layer 230 and a convolutional dimension reduction layer 231. In order to improve the model's sensitivity to features of different scales, a residual connection is introduced after the fully connected layer to reduce the information lost when parsing the context. The features obtained by the parallel modules are element-wise added, and the feature extraction range is expanded through a fully connected layer. In order to avoid the problem of channel dimension expansion in the decoding path, a convolutional dimension reduction operation is used later to reduce the dimension by half, and a residual connection is introduced to reduce information loss in the process of parsing features. The specific structure is as follows: Figure 3 shown.

[0076] In this embodiment, the spatial compression and channel excitation modules 210 and 220 include a pooling layer, two consecutive linear layers, the first linear layer squeezes the channel to half, and the second linear layer excites the channel to the original number. The ReLu activation function is used between the linear layers, the Sigmoid activation function is used after the second linear layer, and then a residual connection is introduced. The pooling layers in the two parallel modules use average pooling and maximum pooling respectively. Two consecutive linear layers are used for channel excitation. The feature maps obtained by the parallel modules are element-wise added, passed through a fully connected layer with an unchanged number of channels and associated with the original feature map, and then the number of channels is reduced to half through a convolution dimensionality reduction operation. The output feature map is the result of an upsampling operation.

[0077] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. All equivalent methods or changes that do not deviate from the technical creation of the present invention should be included in the scope of protection of the present invention.

Claims

1. A prostate 3D image segmentation model based on deep joint transformer, characterized in that: Includes deep joint transformer encoding module and pooling fusion attention decoding module; The deep joint transformer encoding module can effectively extract global and local features of the image and learn long-range semantic information in the context; The pooling fusion focuses on the decoding module to fuse features of different scales to deeply analyze the potential attributes of the sample to make up for the information lost in the up- and down-sampling process.

2. A model according to claim 1, characterized in that: The deep joint transformer encoding module includes a joint transformer UTrans, a deep convolutional network module DCN and a grouped convolutional perception module GCP; the patch feature sequence first extracts pixel features locally through a deep convolutional network module, and then uses a joint transformer to explore local important information and global dependencies; finally, the interactive relationship between contextual semantics is perceived by the grouped convolutional perception module, further enhancing the ability to extract target area features; The pooling fusion attention decoding module first improves the model's sensitivity to features of different scales through two parallel spatial squeezing and channel excitation modules, and then introduces a residual connection to reduce the information lost when parsing the context; then the features obtained by the parallel modules are element-wise added, and the feature extraction range is expanded through a fully connected layer; in order to keep the dimension unchanged, a convolutional dimensionality reduction operation is used later, and a residual connection is introduced to reduce information loss in the process of parsing features.

3. A model according to claim 2, characterized in that: The joint transformer consists of a local transformer and a global transformer, the deep convolutional network module consists of a deep convolutional layer, a normalization layer and an activation function, and the grouped convolutional perception module consists of two grouped convolutional layers, a normalization layer and an activation function.

4. A model according to claim 2, characterized in that: The spatial squeezing and channel excitation module consists of a pooling layer, a fully connected linear layer, a nonlinear ReLu transformation, a fully connected layer for enhancing the channel dimension, and a Sigmoid activation function.

5. A prostate 3D image segmentation method based on deep joint transformer, characterized in that: The steps include: S1. Design a deep joint transformer encoding module to effectively extract global and local features of images and learn long-range semantic information in context; S2. Design a pooling fusion attention decoding module to fuse features of different scales to further analyze the potential features of the sample and make up for the information lost during upsampling and downsampling.

6. A method according to claim 5, characterized in that The deep joint transformer encoding module in S1 is composed of a joint transformer UTrans, a deep convolutional network module DCN and a grouped convolutional perception module GCP; the feature sequence first extracts pixel features locally through a deep convolutional network module, and then uses a joint transformer to explore local important information and global dependencies; finally, the interactive relationship between contextual semantics is perceived through the grouped convolutional perception module, further enhancing the ability to extract target area features; Among them, the deep convolutional network module consists of a deep convolutional layer, a normalization layer and a ReLu activation function; the grouped convolutional perception module consists of 2 grouped convolutional layers, a normalization layer and a ReLu activation function; Among them, the joint transformer includes a local transformer LW and a global transformer GSW; The local transformer consists of two normalization layers LN, a local window multi-head self-attention module LW-MSA and a multi-layer perceptron MLP. A residual connection is added after LW-MSA and MLP to alleviate the impact of gradient disappearance. The local window multi-head self-attention module first evenly divides the feature map transformed by the normalization layer into non-overlapping volume windows, and then performs multi-head self-attention learning inside the window. Local transformer characteristic results X LW The expression is: Q,K,V=LN(Fn(X)) LW--MSA(X)=Attention(Q,K,V) Where X represents the input feature, which is the feature map after preprocessing, Fn represents the window segmentation function, LN represents the normalization layer for linear projection mapping, Q, K, and V represent the mapping result query matrix, key value matrix, and value matrix, respectively, d represents the matrix dimension, T represents the matrix transpose, and B represents the bias parameter of the corresponding function. After a series of operations, the three matrices after mapping are used to obtain a new feature map Attention (Q, K, V) through the nonlinear activation function SoftMax. In the model, LW-MSA (X) is used to represent the operation process of converting the input feature map into the local window multi-head self-attention feature. In order to reduce the loss of information during the sampling process, two layers of residual connections are added to the model. The local window multi-head self-attention feature LW-MSA (X) is added to the input feature X element to obtain the local window residual connection To further extract deep features, the model After normalization, a new feature map is obtained through a multi-layer perceptron Then with Add the elements to get the local transformer feature result X LW , the model uses X LW represents the features after being processed by the local transformer LW; Different from the local transformer, the global transformer uses the global moving window multi-head self-attention GSW-MSA instead of LW-MSA; the feature map is divided into multiple volume windows of the same size and non-overlapping, and then the volume window is moved in the three dimensions of height H, width W, and depth D. The top window moves to the bottom, the leftmost window moves to the rightmost, and the topmost window moves to the bottom, thereby forming a new feature map; feature blocks are selected at a specific distance from each dimension of the feature map, and are kept consistent with the size of the volume window, and then combined into a new global feature map; to ensure the consistency of the local window and global moving window transformer structures, the feature map is again divided into volume windows of the same size and non-overlapping, and its size is the same as the window size in the local transformer; then self-attention learning is performed in each window; Global transformer feature result X GSW The expression is: X g =Interval(Roll(X LW )) Q,K,V=LN(Fn(X g )) GSW-MSA(X g )=Attention(Q,K,V) Where X LW represents the input feature, which is the local transformer feature result. Roll, Interval, and Fn represent the window moving function, interval selection feature block function, and volume window segmentation function, respectively. g It represents the feature result after the window moving function and the interval selection feature block function are operated. LN represents the normalization layer, which is used for linear projection mapping. Q, K, and V represent the mapping result query matrix, key value matrix, and value matrix respectively. d represents the matrix dimension, T represents the matrix transpose, and B represents the deviation parameter of the corresponding function. After a series of operations, the three matrices after mapping are used to obtain the new feature map Attention (Q, K, V) through the nonlinear activation function SoftMax. GSW-MSA (X g ) represents feature X g The global moving window multi-head self-attention feature GSW-MSA(X) is converted into the global moving window multi-head self-attention feature. LW Add the elements to get the global window residual connection The model will After normalization, a new feature map is obtained through a multi-layer perceptron Then with Add the elements to get the global transformer feature result X GSW , the model uses X GSW Represents the features after being processed by the global transformer GSW.

7. A method according to claim 5, characterized in that The pooling fusion attention decoding module in S2 first improves the model's sensitivity to features of different scales through two parallel spatial squeezing and channel excitation modules, and then introduces residual connections to reduce the information lost when parsing the context; then the features obtained by the parallel modules are element-wise added, and the feature extraction range is expanded through a fully connected layer; to avoid the problem of channel dimension expansion in the decoding path, a convolutional dimension reduction operation is used to reduce the dimension by half, and a residual connection is introduced to reduce the information loss in the feature parsing process;, Among them, the spatial squeezing and channel excitation modules include a pooling layer, two consecutive linear layers, the ReLu activation function is used between the linear layers, the Sigmoid activation function is used after the second linear layer, and then a residual connection is introduced; the pooling layers in the two parallel modules use average pooling and maximum pooling respectively; two consecutive linear layers are used for channel excitation.

Images