Medical image segmentation method and system integrating external attention and convolution

Abstract

The invention discloses a medical image segmentation method and system integrating external attention and convolution, and the method comprises the steps: constructing a medical image segmentation network which comprises a U-shaped network composed of an encoder and a decoder, two first convolution layers and a second convolution layer, the encoder and the decoder are integrated with paired external attention modules and paired convolution attention modules; inputting the medical image into the medical image segmentation network, extracting global features and local features of the medical image based on the U-shaped network and performing feature fusion, performing convolution operation on the medical image based on the first convolution layer, and performing fusion again on the medical image after convolution operation and feature extraction fusion to obtain a fusion image; and performing convolution operation on the fused medical images in sequence based on the first convolution layer and the second convolution layer. Therefore, by integrating the external attention and the pairwise convolution attention, the potential correlation between the features can be captured, and the segmentation stability of the image segmentation network is improved.

Description

Technical Field

[0001] This invention relates to the field of medical three-dimensional image segmentation technology, and in particular to a medical image segmentation method and system that integrates pairwise external attention and convolution. Background Technology

[0002] Medical image segmentation plays a crucial role in medical image analysis, and accurately identifying regions of interest (ROIs) within medical images is a key step in performing any image segmentation. Convolutional Neural Networks (CNNs) and their derivatives have achieved success in this field. However, due to the inherent limitations of convolution operators, CNN-based medical image segmentation networks can only extract local features. As the network deepens, the convolutional kernel can only focus on a sub-region within the entire image, causing its ability to aggregate long-range information to continuously decline and making it unable to establish long-range dependencies. Because the Vision Transformer (VIT) has the ability to establish long-range dependencies, it has attracted numerous researchers in the medical imaging field to study its applications and explore methods combining CNNs and Transformers, resulting in many medical image segmentation networks that combine CNNs and Transformers. These networks combine the Transformer's ability to establish long-range relationships within a single sample with the CNN's advantage in capturing local image details. To obtain more refined image features, some networks extend their attention from the perspectives of spatial and channel features, improving the model's segmentation performance by acquiring spatial and channel feature maps.

[0003] Because the Transformer derives its attention image by calculating the correlations between different tokens within a single sample, it may overlook potential correlations between tokens across different samples. In medical images, organs of the same type may exhibit differences in grayscale values, shape, and boundaries due to external factors such as patient condition and equipment. Clearly, capturing the correlations between these differing organs of the same type would effectively represent the characteristics of that organ class. Summary of the Invention

[0004] The present invention provides a medical image segmentation method and system that integrates paired external attention and convolution. By integrating paired external attention modules and paired convolutional attention modules, the potential correlation between feature information can be captured, thereby improving the stability of medical image segmentation network segmentation. Firstly, a medical image segmentation method integrating pairwise external attention and convolution is provided, comprising: Acquire the medical image to be segmented; A medical image segmentation network is constructed, comprising a U-shaped network consisting of an encoder and a decoder, two first convolutional layers, and a second convolutional layer. The encoder and the decoder both integrate a pair of external attention modules and a pair of convolutional attention modules. The pair of external attention modules are used to extract global features of the medical image, and the pair of convolutional attention modules are used to extract local features of the medical image. The medical image is input into the medical image segmentation network. The global and local features of the medical image are extracted based on the U-shaped network and fused. The medical image is then convolved based on the first convolutional layer. The medical images after convolution and feature extraction are fused again. Finally, the fused medical images are convolved sequentially based on the second first convolutional layer and the second convolutional layer to output the image segmentation result.

[0005] Secondly, a medical image segmentation system integrating paired external attention and convolution is provided, including: The medical image acquisition module is used to acquire medical images to be segmented. A segmentation network construction module is used to construct a medical image segmentation network. The medical image segmentation network includes a U-shaped network composed of an encoder and a decoder, two first convolutional layers and a second convolutional layer. The encoder and the decoder both integrate a pair of external attention modules and a pair of convolutional attention modules. The pair of external attention modules are used to extract global features of the medical image, and the pair of convolutional attention modules are used to extract local features of the medical image. The image segmentation module is communicatively connected to the medical image acquisition module and the segmentation network construction module. It is used to input the medical image into the medical image segmentation network, extract global and local features of the medical image based on the U-shaped network and perform feature fusion, perform convolution operation on the medical image based on the first convolutional layer, fuse the medical image after convolution operation and feature extraction fusion, and perform convolution operation on the fused medical image sequentially based on the second first convolutional layer and the second convolutional layer to output the image segmentation result.

[0006] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the medical image segmentation method integrating external attention and convolution as described above.

[0007] Fourthly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the medical image segmentation method integrating pairwise external attention and convolution as described above.

[0008] Compared with the prior art, the advantages of the present invention are as follows: by integrating paired external attention modules and paired convolutional attention modules, the potential correlation between feature information can be captured, thereby improving the stability of medical image segmentation network segmentation. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the structure of the medical image segmentation network PECA-Net of the present invention; Figure 2 This is a schematic diagram of the structure of the Paired External Attention Module (PEA) of the present invention; Figure 3 This is a schematic diagram of the paired convolutional attention module (PCA) of the present invention; Figure 4 This is a qualitative comparison diagram of the segmentation performance of various network models on the Synapse dataset; Figure 5 This is a qualitative comparison diagram of the segmentation performance of various network models on the ACDC dataset; Figure 6 This is a schematic diagram illustrating a qualitative comparison of the segmentation performance of various network models on the Tumor dataset. Detailed Implementation

[0010] Referring now to specific embodiments of the invention, examples of which are illustrated in the accompanying drawings. Although the invention will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of the invention as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0011] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0012] Note: The examples described below are merely specific examples and are not intended to limit the embodiments of the present invention to the specific steps, values, conditions, data, order, etc. Those skilled in the art can utilize the concept of the present invention to construct more embodiments not mentioned herein by reading this specification.

[0013] This invention provides a medical image segmentation method integrating pairwise external attention and convolution, comprising: Step S100: Obtain the medical image to be segmented; Step S200: Construct a medical image segmentation network. The medical image segmentation network includes a U-shaped network composed of an encoder and a decoder, two first convolutional layers and a second convolutional layer. The encoder and the decoder both integrate a pair of external attention modules and a pair of convolutional attention modules. The pair of external attention modules are used to extract global features of the medical image, and the pair of convolutional attention modules are used to extract local features of the medical image. Step S300: Input the medical image into the medical image segmentation network, extract global and local features of the medical image based on the U-shaped network and perform feature fusion, perform convolution operation on the medical image based on the first first convolutional layer, fuse the medical image after convolution operation and feature extraction fusion, and perform convolution operation on the fused medical image sequentially based on the second first convolutional layer and the second convolutional layer, and output the image segmentation result.

[0014] See Figure 1 As shown, the encoder includes a first coding layer, a second coding layer, a third coding layer, and a fourth coding layer that are connected in sequence according to the information being transmitted. The decoder includes a first decoding layer, a second decoding layer, a third decoding layer, and a fourth decoding layer that are connected in sequence according to the information transmission, wherein the first decoder is communicatively connected to the fourth encoder; The first encoding layer is used to extract feature information from the medical image and then connect the extracted feature information to the input of the fourth decoding layer. The second encoding layer is used to obtain the feature information output by the first encoding layer, and to connect the extracted feature information to the input of the third decoding layer. The third encoding layer is used to obtain the feature information output by the second encoding layer, and to connect the extracted feature information to the input of the second decoding layer.

[0015] The first encoding layer includes a patch embedding layer and a parallel interaction module connected in sequence according to information transmission; the second, third, and fourth encoding layers each include a downsampling layer and a parallel interaction module connected in sequence according to information transmission. The first decoding layer includes an upsampling layer that is communicatively connected to the parallel interaction module of the fourth encoder. The second decoding layer, the third decoding layer, and the fourth decoding layer all include a parallel interaction module and an upsampling layer that are connected in sequence according to the information transmission. The parallel interaction modules all integrate the paired external attention modules and the paired convolutional attention modules.

[0016] Specifically, in this embodiment, the medical image segmentation network PECA-Net (Paired External and Convolutional Attention Net) includes a U-shaped network composed of an encoder and a decoder, two first convolutional layers, and a second convolutional layer. The first convolutional layer is... Convolutional block; the second convolutional layer is convolution.

[0017] The U-shaped network structure consists of an encoder and a decoder. The encoder comprises four coding layers, each using a Parallel Interactive Module (PIM). In the first coding layer, the Patch Embedding layer segments the input 3D medical image into non-overlapping 3D blocks, and then uses PIM to obtain local and global information from different images. Each of the remaining coding layers begins with a downsampling layer to downsample the size of the input feature map. The channel dimension is expanded to twice the previous size before entering PIM.

[0018] Assuming the size is 3D medical images are input into a U-shaped network. First, the medical images are processed through a PatchEmbedding layer. Divided into non-overlapping 3D blocks The number after division is Then use PIM to extract. Local and global information within a PIM. The structure of a PIM is as follows: Figure 1 As shown on the right, it mainly consists of a Paired External Attention (PEA) module and a Paired Convolutional Attention (PCA) module. PEA is used to extract global image features and correlations between images. PCA adaptively learns the spatial and channel features of the image to obtain local features. Finally, the image features extracted by both are fused together, and the resulting data is connected to the corresponding decoding layer of the decoder via skip connections to recover the information loss caused by downsampling.

[0019] The decoder, corresponding to the encoder, also consists of four layers. The first decoding layer of the decoder is an upsampling layer, which upsamples the size of the input feature map by a factor of 2, while reducing the channel dimension to half of its original size. It is then fused with skip connections and fed into the PIM of the next decoder layer. Subsequent processing steps are similar to those of the encoder and will not be described in detail here.

[0020] The structure of the Parallel Interactive Module (PIM) is as follows: Figure 1 As shown on the right, its main function is to extract spatial and channel feature information of the image, capture strong dependencies between space and channel, and fuse feature information. PIM mainly consists of PEA and PCA. PEA uses dot product attention to capture spatial and channel correlations in the image by sharing weights between the query matrix and key memory units in the spatial and channel branches. PCA uses scaling attention to perform spatial and channel convolutional attention calculations on the image separately, learning to selectively emphasize important information features and suppress unimportant features using global information. In other words, the PIM module fuses the global and local features of the medical image extracted by PEA and PCA. The PEA module is responsible for extracting global features of the image, enabling PEA to extract potential correlations between different images by using memory units. The PCA module is responsible for extracting local features of the image, emphasizing important features in the local area of ​​the image and suppressing irrelevant features through convolution and normalization operations.

[0021] Assuming the input image is After receiving the input feature map, PIM first sends it to the PEA block and PCA to obtain relevant feature information of the image. Then, it fuses the input feature map, the feature map extracted by PEA, and the feature map extracted by PCA. The specific processing formula is as follows:

[0022]

[0023]

[0024] In the formula, For the input image, To output the image, and These are paired external attention modules and paired convolutional attention modules, respectively. It is a 3×3×3 convolution block. It is a 1×1×1 convolution.

[0025] Capturing the latent correlations between sample images makes it easier for the attention mechanism to find regions of interest. Therefore, Paired Convolutional Attention (PEA) uses pure external attention as its backbone and employs memory units in the attention computation. By gradually learning the characteristics of the entire dataset during training through memory units, PEA implicitly considers the correlations between all images. Furthermore, to obtain richer image feature information, we extended PEA from spatial and channel perspectives, designing branches for extracting spatial and channel features.

[0026] The structure of the PEA block is as follows: Figure 2 As shown, PEA first shares the query matrix and key memory units between the spatial and channel branches, enabling it to capture the interdependencies between spatial and channel features in a single image. Second, PEA uses memory units instead of the key and value matrices in self-attention during attention computation, effectively reducing the number of model parameters and computational complexity. Simultaneously, the memory units gradually learn the features of the images in the entire dataset as training progresses. Combining these two methods, PEA can not only acquire spatial and channel feature information and their interdependencies for each image in the dataset, but also obtain potential correlations between spatial and channel feature information across different images.

[0027] Memory cells are crucial for PEA to capture image feature information across the entire dataset. Essentially, they are learnable two-dimensional matrices optimized during training via end-to-end backpropagation. They are independent of individual samples and shared throughout the dataset. The size of the memory cells varies slightly in different branches. In the spatial branch, the size of the memory cell is determined by the projection size and the feature dimension (d). In the channel branch, the size of the memory cell is determined by the image input size and the feature dimension (d).

[0028] In the PEA attention calculation process, a shared [database] is first generated. Matrix. Then AND key memory unit and Perform dot product calculations to obtain the attention maps. and ( Depend on (Generation). The image is then fed into the spatial and channel branches, by... With channel value memory unit Calculate the channel features of the image. Spatial value memory unit The spatial features of the image are calculated. Finally, the spatial features and channel features are fused to obtain the final output image. The specific formula is as follows:

[0029]

[0030]

[0031]

[0032]

[0033] In the formula, and These are the input image and the output image, respectively. For key memory units, and These are spatial value memory units and channel value memory units, respectively. For the first Features and The similarity between the j-th rows, and All of these are different attention maps inferred from prior knowledge of the learned dataset, where T is the transpose.

[0034] Paired Convolutional Attention (PCA) is responsible for acquiring spatial and channel local features of the image, focusing on targets and locations within the image features to enhance feature extraction capabilities. The overall structure of PCA is as follows: Figure 3 As shown in (a), PCA mainly consists of spatial attention branches and channel attention branches. By splicing and fusing spatial feature information and channel feature information, a more refined feature map is obtained.

[0035] Channel attention branch structure as follows Figure 3 As shown in (b), the spatial information of the image is compressed to 1 using an average pooling layer. A compression and activation block consisting of two convolutional layers extracts the spatial feature information of the image. Finally, the extracted feature information is normalized, and corresponding weights are assigned to each channel feature to represent the importance of different channel features. The feature map of each channel is multiplied by the corresponding weight to obtain different levels of attention.

[0036] Spatial attention branch structure such as Figure 3 As shown in (c), a compression firing module composed of convolutional layers compresses the image channels to 1. The extracted spatial feature information is normalized, and corresponding weights are assigned to each spatial feature to represent the importance of different spatial features.

[0037] Assume the feature map of the input PCA is ,in The feature map is then processed by spatial attention and channel attention branches, respectively. In the channel branch, it first undergoes an average pooling layer to compress the spatial size of the feature map to 1. Then two The convolutional layers of the convolutional kernels perform compression and activation operations. The first convolutional layer compresses the channels to... The second convolutional layer restores the channels to their original state. Next, normalization is performed, and weights are assigned to the channel features to obtain the image's channel attention map. Finally, the original image is multiplied by the weights to obtain the corresponding image channel features. In the spatial branch, the image enters... Convolutional layers with convolutional kernels compress image channels to 1, i.e. Then, through normalization, weights are assigned to the spatial features to obtain the spatial attention map of the image. Finally, the original image is multiplied by the weights to obtain the corresponding image spatial features. The specific formula is as follows:

[0038]

[0039]

[0040] In the formula, and These are spatial features and channel features, respectively. and These are the input image and the output image, respectively. , and The output channels are 1 and 2 respectively. and Convolutional layers, This is an average pooling layer.

[0041] I. The following is a comparative experimental analysis of the present invention and other existing technologies.

[0042] 1. Experimental results on the Synapse dataset Table (1) presents the quantitative experimental results on the Synapse dataset. The average Dice value for PECA-Net is 86.69%, and for HD95 it is 9.82%. The results are superior to other methods. UNETR++ achieved second place on this dataset, while the proposed method achieved a Dice value 0.88% higher than UNETR++ and an HD95 value 1.38 lower than UNETR++. .

[0043] Table (1)

[0044] Table (1) shows a quantitative comparison of segmentation performance on the Synapse dataset. The evaluation metrics are DSC (%) and HD95 (mm). The best results are indicated in bold, and the second best results are indicated by underline.

[0045] Figure 4 This demonstrates the qualitative experimental results of the present invention on the Synapse dataset. Figure 4 As can be seen in (a), nnU-Net and MISSFormer have some slight biases in the segmentation of image boundary details, while UNETR++ has segmentation errors. Figure 4 (b) Figure 4 In (d), nnU-Net, MISSFormer, and UNETR++ all exhibited instances of missing segmentation in the stomach portion. Figure 4 (c) shows that nnU-Net and MISSFormer perform incorrect Aorta segmentation. Figure 4 In (e), nnU-Net, MISSFormer, and UNETR++ all perform inaccurate boundary segmentation of Pancreas, resulting in missing or added parts.

[0046] 2. Experimental results on the ACDC dataset Table (2) shows the experimental results for the ACDC dataset. The average Dice value of the PECA-Net network is 91.77%, which is higher than the average Dice value of other models. It is 0.64% higher than nn-UNet and 0.53% higher than the second-best UNETR++.

[0047] Table (2)

[0048] Table (2) shows a quantitative comparison of segmentation performance on the ACDC dataset. The evaluation metric is DSC (%). RV represents the right ventricle, Myo represents the myocardium, and LV represents the left ventricle. The best result is indicated in bold, and the second best result is indicated in underline.

[0049] In the ACDC dataset, the current state-of-the-art (SOTA) network can achieve an average Dice score of 90%, which means that in most cases, the results of SOTA models in segmenting images are very similar. Figure 5 In the more challenging cases (a) and (b), all networks segmented the right ventricle of the image incorrectly. However, the network in this invention can segment more completely compared to other networks. Figure 5 In (c), the right ventricular segmentation is missing in UNETR++. Figure 5 In (d), right ventricular segmentation was also missing in nnU-Net.

[0050] 3. Experimental results on the Tumor dataset Table (3) presents the experimental results for the Tumor dataset. The average Dice value of the PECA-Net network is 85.85%, and the HD95 value is 4.10. In both of these values, the Dice value is 1.19% higher than UNETR++, and HD95 is 0.54% higher than UNETR++. It is worth noting that the PECA-Net network has the highest Dice value across all projects.

[0051] Table (3)

[0052] Table (3) shows a quantitative comparison of segmentation performance on the Tumor dataset. The evaluation metrics are DSC (%) and HD95 (mm). The best results are indicated in bold, and the second best results are indicated by underline.

[0053] In the Tumor dataset, the blurred boundaries of regions of interest pose a significant challenge to segmentation. The irregularities, blurriness, and clutter of the boundaries to be segmented are clearly visible in the labeled images. Thanks to the PECA-Net network's ability to capture the spatial channel dependencies of the entire dataset, the performance in handling such challenges is greatly improved. Figure 6 As can be seen, nnU-Net segmented the Tumor image with numerous errors such as additions and omissions. UNETR and UNETR++ failed to segment complex boundaries accurately.

[0054] 4. Parameter Analysis Table (4) shows the model parameters and DSC of different medical image segmentation methods. Since Swin-UNet and MISSFormer are used for segmenting two-dimensional images, they have fewer model parameters. Excluding the above 2D segmentation networks, the PECA-Net network model has the fewest training parameters. Compared to UNETR++, it reduces the number of parameters by 8.91M, while the DSC increases from 91.24% to 91.77%, an improvement of 0.53%. These results indicate that the PECA-Net network has a significant advantage in terms of parameter computation.

[0055] Table (4)

[0056] Table (4) compares the number of parameters and DSC for different models. UNETR++ is the baseline network. Underlined networks represent 2D segmentation networks. The best result is indicated in bold, and the second best result is indicated in underline.

[0057] II. Ablation Experiments - Impact of PEA, PCA, and PIM on Network Segmentation Performance The experimental results are shown in Table (5). Among them, Cascaded PCA means that sSE (spatial attention) and cSE (channel attention) are serially connected to the spatial branch and channel branch of EPA (Efficient Paired-Attention Block), respectively; Parallel PCA means that EPA and PCA are connected in parallel; XX in Encoder means that XX is used only in the encoder; XX in Decoder means that XX is used in both the encoder and decoder.

[0058] Table (5)

[0059] Furthermore, this invention also provides a medical image segmentation system integrating pairwise external attention and convolution, comprising: The medical image acquisition module is used to acquire medical images to be segmented. A segmentation network construction module is used to construct a medical image segmentation network. The medical image segmentation network includes a U-shaped network composed of an encoder and a decoder, two first convolutional layers, and a second convolutional layer. Both the encoder and the decoder integrate paired external attention modules and paired convolutional attention modules. The paired external attention modules are used to extract global features of the medical image, and the paired convolutional attention modules are used to extract local features of the medical image. The image segmentation module is communicatively connected to the medical image acquisition module and the segmentation network construction module. It is used to input the medical image into the medical image segmentation network, extract global and local features of the medical image based on the U-shaped network and perform feature fusion, perform convolution operation on the medical image based on the first convolutional layer, fuse the medical image after convolution operation and feature extraction fusion, and perform convolution operation on the fused medical image sequentially based on the second first convolutional layer and the second convolutional layer to output the image segmentation result.

[0060] In summary, existing medical image segmentation networks based on a hybrid of Transformer and CNN typically suffer from the following problems: First, Transformer ignores the potential correlation between different samples. When external factors cause interference in the image, it will lead to a decrease in the stability of the network model in segmenting the image.

[0061] Second, Transformer-based methods have high computational complexity and a large number of parameters.

[0062] Third, networks that extract medical image features from spatial and channel perspectives often overlook the potential correlation between spatial and channel features of an image.

[0063] To address the above three issues, this technology proposes PECA-Net, a hybrid network architecture that integrates external attention and convolution: First, PECA-Net uses Paired External Attention (PEA) with external attention as its backbone, which captures the potential correlation between different samples through memory units, thereby improving the stability of the network model's segmentation.

[0064] Second, external attention has linear complexity, while Transformer complexity has quadratic complexity. Therefore, PEA has lower computational complexity and fewer computational parameters.

[0065] Third, PEA adopts a parallel structure design, which enables PEA to capture the potential correlation between spatial and channel features of an image by sharing the key matrix and value matrix between spatial and channel branches.

[0066] Specifically, this embodiment corresponds one-to-one with the above method embodiments. The functions of each module have been described in detail in the corresponding method embodiments, so they will not be repeated here.

[0067] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the method steps of the above method.

[0068] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0069] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory and a processor. The memory stores a computer program that runs on the processor. When the processor executes the computer program, it implements all or part of the method steps described above.

[0070] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting all parts of the computer device through various interfaces and lines.

[0071] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0072] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0073] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method of medical image segmentation integrating pair-wise external attention with convolution, the method comprising: The method comprises the following steps: acquiring a medical image to be segmented; constructing a medical image segmentation network, wherein the medical image segmentation network comprises a U-shaped network composed of an encoder and a decoder, two first convolutional layers and a second convolutional layer, the encoder and the decoder each integrate a pair of external attention modules and a pair of convolutional attention modules, the pair of external attention modules are used to extract global features of the medical image, and the pair of convolutional attention modules are used to extract local features of the medical image; inputting the medical image into the medical image segmentation network, extracting global features and local features of the medical image based on the U-shaped network and performing feature fusion, performing convolutional operation on the medical image based on the first first convolutional layer, fusing the medical image which has undergone convolutional operation and feature extraction fusion again, and sequentially performing convolutional operation on the fused medical image based on the second first convolutional layer and the second convolutional layer, and outputting an image segmentation result. 2.The method of claim 1, wherein, The encoder comprises a first encoding layer, a second encoding layer, a third encoding layer and a fourth encoding layer connected in sequence according to information transmission; The decoder comprises a first decoding layer, a second decoding layer, a third decoding layer and a fourth decoding layer connected in sequence according to information transmission, wherein the first decoder and the fourth encoder are in communication connection; The first encoding layer is used to extract feature information of the medical image, and the extracted feature information is jump-connected to an input end of the fourth decoding layer; The second encoding layer is used to acquire the feature information output by the first encoding layer, and the extracted feature information is jump-connected to an input end of the third decoding layer; The third encoding layer is used to acquire the feature information output by the second encoding layer, and the extracted feature information is jump-connected to an input end of the second decoding layer. 3.The method of claim 2, wherein, The first encoding layer comprises a patch embedding layer and a parallel interaction module connected in sequence according to information transmission, and the second encoding layer, the third encoding layer and the fourth encoding layer each comprise a down-sampling layer and a parallel interaction module connected in sequence according to information transmission; The first decoding layer comprises an up-sampling layer in communication connection with the parallel interaction module of the fourth encoder, and the second decoding layer, the third decoding layer and the fourth decoding layer each comprise a parallel interaction module and an up-sampling layer connected in sequence according to information transmission; The parallel interaction module integrates the pair of external attention modules and the pair of convolutional attention modules. 4.The method of claim 3, wherein, The processing method of the parallel interaction module is as follows: wherein is an input image, is an output image, and are a pair-wise external attention module and a pair-wise convolution attention module, respectively, is a 3x3x3 convolution block, is a 1x1x1 convolution. 5.The method of claim 1, wherein, The processing method of the pair of external attention modules is as follows: wherein, and are input and output images, respectively, is a key memory unit, and are spatial and channel value memory units, respectively, is the similarity between the i-th feature and the j-th row of and are different attention maps inferred from the learned dataset prior knowledge, and T is the transpose.​​ 6.The method of claim 1, wherein, The processing method of the pair of convolutional attention modules is as follows: In the formula, and These are spatial features and channel features, respectively. and These are the input image and the output image, respectively. , and The output channels are 1 and 2 respectively. and Convolutional layers, This is an average pooling layer. 7.The method of claim 1, wherein, The first convolutional layer is configured to Convolutional block; The second convolutional layer is configured to convolution.

8. A medical image segmentation system integrating pair-wise external attention with convolution, the system comprising: The method comprises the following steps: a medical image acquisition module is configured to acquire a medical image to be segmented; The segmentation network construction module is configured to construct a medical image segmentation network, the medical image segmentation network comprising a U-shaped network composed of an encoder and a decoder, two first convolutional layers, and a second convolutional layer, wherein the encoder and the decoder each integrate a pair of external attention modules and a pair of convolutional attention modules, the pair of external attention modules being configured to extract global features of the medical image, and the pair of convolutional attention modules being configured to extract local features of the medical image; The image segmentation module is in communication connection with the medical image acquisition module and the segmentation network construction module, configured to input the medical image into the medical image segmentation network, extract global features and local features of the medical image based on the U-shaped network, and perform feature fusion, perform convolutional operation on the medical image based on a first of the first convolutional layers, fuse the medical image that has undergone convolutional operation and feature extraction fusion again, and sequentially perform convolutional operation on the fused medical image based on a second of the first convolutional layers and the second convolutional layer, and output an image segmentation result.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the medical image segmentation method integrating a pair of external attention and convolution as claimed in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the program, implements the medical image segmentation method integrating a pair of external attention and convolution as claimed in any one of claims 1 to 7.

Images