A medical image automatic segmentation method based on a double-aggregation hybrid neural network

Abstract

The application discloses a medical image automatic segmentation method based on a double-aggregation hybrid neural network, which comprises the following steps: inputting a medical image to be segmented into a pre-trained medical image segmentation network to output a medical image segmentation result; the medical image segmentation network adopts a double-aggregation hybrid U-shaped network, which comprises two parallel encoders and a single decoder, and the two parallel encoders are a convolutional neural network and a Transformer respectively; local features and global features of the medical image are extracted by the convolutional neural network and the Transformer respectively; multi-scale feature fusion is performed on the extracted local features and global features by a collaborative feature aggregation module CFA and a non-collaborative feature aggregation module NCFA; and the features subjected to the multi-scale feature fusion are input into the decoder to output the medical image segmentation result. The application can realize multi-scale feature fusion, enhances the segmentation precision of the medical image segmentation result, and improves the performance of the medical image segmentation network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image segmentation, and particularly relates to a medical image automatic segmentation method based on a double-aggregation hybrid neural network. BACKGROUND

[0002] The purpose of medical image segmentation is to extract the region of interest in the medical image for computer-aided diagnosis. In addition, it can also help healthcare professionals to visualize injuries, monitor diseases and treatment plans.

[0003] Convolutional Neural Networks (CNNs) have powerful representation extraction capabilities and have achieved great success in medical image analysis. Among them, the U-Net is the most widely used, which is composed of a symmetric encoder-decoder network, and different scale high-resolution features are aggregated by jump connections to enhance detail preservation. Its derivative network models (such as UNet++, Attention UNet, Res-UNet) have successively achieved the state-of-the-art performance in various segmentation tasks, but the CNN-based method has poor global modeling ability due to the inherent inductive bias. Especially for segmentation performance in terms of texture, shape and size, which involves global vision.

[0004] Transformer, a brand-new architecture, only needs attention mechanism to model global dependencies without any convolution operation, and has achieved great success in Natural Language Processing (NLP). Researchers have tried to introduce Transformer into the field of Computer Vision (CV), and the results show that under the condition of large-scale dataset pre-training, Transformer provides better performance than CNN in computer vision CV, and has become an alternative architecture. Among them, Vision Transformer (ViT) divides the image into a series of tokens to meet the input requirements, and models their global dependencies by stacking Transformer blocks. ViT has achieved impressive performance in image classification tasks, but needs large datasets for pre-training. The results of ViT in image classification are outstanding, but due to its low-resolution feature mapping and the quadratic growth of complexity with the size of the image, its structure is not suitable as a general backbone network other than image classification, such as dense visual tasks or high-resolution input images. SUMMARY

[0005] To solve the above technical problems, the application provides a medical image automatic segmentation method based on a double-aggregation hybrid neural network, which comprises: inputting a medical image to be segmented into a pre-trained medical image segmentation network to output a medical image segmentation result.

[0006] The medical image segmentation network adopts a double-aggregation hybrid U-shaped network, which respectively includes two parallel encoders and a single decoder, and the two parallel encoders are a convolutional neural network (CNN) and a Transformer respectively;

[0007] Local features and global features of the medical image are extracted by the CNN and the Transformer respectively at the same time;

[0008] The extracted local features and global features are subjected to multi-scale feature fusion by a collaborative feature aggregation module (CFA) and a non-collaborative feature aggregation module (NCFA);

[0009] The features subjected to the multi-scale feature fusion are input into the decoder, and a medical image segmentation result is output.

[0010] The training process of the medical image segmentation network includes:

[0011] S1: A medical image set is collected, and after being labeled and preprocessed, it is saved to a medical image database;

[0012] S2: Images are selected from the medical image database, and after data augmentation, a training set and a test set are obtained;

[0013] S3: The medical image and its segmentation image are selected from the training set, and are input into the medical image segmentation network respectively, and the global features and the local features of the medical image are extracted by the CNN and the Transformer respectively after 4 times of down-sampling;

[0014] S4: The extracted global features and local features are subjected to feature fusion by the CFA and the NCFA;

[0015] S5: The features subjected to the feature fusion are input into the decoder, and the predicted segmentation result of the medical image and its segmentation image is obtained respectively;

[0016] S6: The loss between the predicted segmentation result of the medical image and its segmentation image is calculated, and the medical image segmentation network is optimized by the loss through back propagation.

[0017] The beneficial effects of the present application are:

[0018] Firstly, the present application adopts a U-shaped network architecture with double encoders and a single decoder for medical image segmentation, wherein the double encoders include a CNN and a Transformer; this architecture can combine the edge local semantic information of the CNN with the global context interaction of the Transformer, thereby enhancing the integration of complementary features and obtaining better segmentation performance. In medical image segmentation, this method performs better when dealing with irregular and challenging boundaries.

[0019] Secondly, the application provides a heterogeneous hierarchical down-sampling feature cooperative fusion and non-cooperative fusion method, that is, the hierarchical features collected by CNN and Transformer are fused and processed in a cooperative and non-cooperative manner based on a multi-attention mechanism, which effectively fuses coarse-grained and fine-grained feature representations and significantly improves the accuracy of medical image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 FIG. 1 is a structural schematic diagram of a medical image segmentation network in an embodiment of the application;

[0021] Figure 2 FIG. 2 is a network structure schematic diagram of a CNN in an embodiment of the application;

[0022] Figure 3 FIG. 3 is a network structure schematic diagram of a cooperative feature aggregation module CFA in an embodiment of the application;

[0023] Figure 4 FIG. 4 is a network structure schematic diagram of a non-cooperative feature aggregation module NCFA in an embodiment of the application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0025] An embodiment of the application provides a medical image automatic segmentation method based on a double-aggregation hybrid neural network, as shown in FIG. 1, which comprises the following steps: Figure 1 inputting a medical image to be segmented into a pre-trained medical image segmentation network, and outputting a medical image segmentation result;

[0026] The medical image segmentation network adopts a double-aggregation hybrid U-shaped network, which comprises two parallel encoders and a single decoder, and the two parallel encoders are a convolutional neural network CNN and a Transformer, respectively.

[0027] The convolutional neural network CNN and the Transformer are used to extract local features and global features of the medical image, respectively;

[0028] A cooperative feature aggregation module CFA and a non-cooperative feature aggregation module NCFA are used to perform multi-scale feature fusion on the extracted local features and global features;

[0029] The features subjected to the multi-scale feature fusion are input into the decoder, and the medical image segmentation result is output.

[0030] Figure 1 Structure diagram of a medical image segmentation network in an embodiment of the present application. Figure 1 In the embodiment, the medical image segmentation network adopts a dual-aggregation hybrid U-shaped network, which includes two parallel encoders and a single decoder. The picture input end is connected to the two parallel encoders, which are a convolutional neural network (CNN) and a Transformer, respectively. After embedding processing, the input picture is input into the Transformer, which includes SwinTransformer and Patch merging in the embodiment of the present application, to extract the local features and global features of the input picture data. The input picture is input into the CNN to extract the local features and global features of the input picture data. After the first downsampling, the sampled picture data is input into the first layer Swin Transformer and the first layer CNN, respectively, and the first layer Swin Transformer and the first layer CNN output the global features and the local features, respectively, which are input into the first collaborative feature aggregation module (CFA) for feature fusion. The outputs of the second layer Swin Transformer, the second layer CNN, and the first CFA are input into the second CFA, and the process is repeated in sequence, and after at least 4 times of downsampling processing, the fused features are obtained. The outputs of each layer SwinTransformer and each layer CNN are taken as non-collaborative feature aggregation modules (NCFA), and after at least 4 times of downsampling processing, the fused features are obtained. The fused features output by the CFA are subjected to feature reconstruction, and the reconstructed features and the fused features output by the at least 4 NCFA are input into the decoder to obtain the image segmentation result.

[0031] Reference Figure 1As shown, the backbone network Swin Transformer (ST) of the Transformer in the embodiment of the present application includes at least 4 layers of Swin Transformer Layer (STL), the first three layers of STL are each composed of two Swin Transformer Block (STB) and Patch Merging, and the last layer of STL is only composed of two STB. The Swin Transformer (ST) block is composed of a window-based multi-head self attention (W-MSA) module or a shifted window-based multi-head self attention (SW-MSA) module, and a 2-layer MLP. A LN (Layer Norm) layer is applied before each multi-head self attention (MSA) module and each MLP module, and a residual connection is applied after each module. In two consecutive SBT blocks, the W-MSA module or the SW-MSA module is used alternately to realize the cross-window connection while maintaining the efficient calculation of the non-overlapping window, and the calculation formula is as follows:

[0032]

[0033] wherein, represents the output of the lth layer (S)W-MSA module, (S)W-MSA(.) represents the (moving) window multi-head self-attention mechanism, LN(.) represents layer normalization, MLP(.) represents the multi-layer perception, x l represents the output of the lth layer MLP module, and x l+1 represents the above operation of the l+1th layer.

[0034] The calculation of the self-attention mechanism is as follows:

[0035]

[0036] wherein, Attention(Q,K,V) represents the calculation of the multi-head self-attention mechanism on Q,K,V, Q,K,V represent the query matrix, the key matrix and the value matrix respectively, p represents the number of patches in the window, d represents the dimension of the query or the key, and the value in B is taken from the bias matrix

[0037] Patch Merging down-samples the input feature map with dimension [N, C, H, W] in H and W dimensions respectively, starting from 0 and 1, with a step of 2, to obtain 4 feature maps with dimension [N, C, H / 2, W / 2]. The 4 feature maps are concatenated in the channel dimension to obtain a feature map with dimension [N, C*4, H / 2, W / 2]. In order to align the output feature channel of the CNN module, a linear layer is applied after channel concatenation to unify the output feature dimension to twice the input dimension, obtaining a feature map with dimension [N, C*2, H / 2, W / 2].

[0038] Figure 2 The network structure of the CNN in the embodiment of the application is shown in the figure.

[0039] As some preferred embodiments, referring to Figure 2 The convolutional neural network CNN includes a root layer and three body layers. The root layer includes a 7*7 convolution kernel conv, a group normalization layer (GN), and a Relu activation function layer. The body layer includes N bottleneck blocks. Each bottleneck block includes three layers of cascaded depthwise separable convolution kernels (DWConv), a group normalization layer GN, and a Relu activation function.

[0040] Figure 2 The CNN includes a root layer and three body layers. The root layer uses a standard convolution with a convolution kernel of 7, a step of 2, and padding of 3 to learn feature representation and down-sample. The three body layers first use DWConv with a step of 2 to learn feature representation and down-sample, and then use multiple DWConv to learn more in-depth feature representation. Group normalization (GN) is used after all convolution operations, and an activation function Relu is performed once at the end of each layer. Finally, the output features are linearly embedded and position encoding is added for subsequent feature fusion.

[0041] Figure 3 The network structure of the collaborative feature aggregation module CFA in the embodiment of the application is shown in the figure.

[0042] As some preferred embodiments, referring to Figure 3As shown, the collaborative feature aggregation module CFA includes N layers of layer normalization layer LN and multi-head channel attention (MCA), and a connection layer normalization layer (LN) and a multilayer perceptron (MLP).

[0043] Figure 3 In the specific implementation, the features output by the ST (Swin Transformer), the features output by the CNN, and the features output by the previous CFA are embedded into the same dimension after LN, and the three embedded features are additionally concatenated in the channel and input into the MCA together, and the output is obtained after N times of calculation. Finally, the global and local features extracted by collaborative learning are processed by LN and MLP, and E1-E3 represent three different embedded features.

[0044] Figure 4 FIG. 2 is a schematic diagram of a network structure of a non-collaborative feature aggregation module NCFA in an embodiment of the present application.

[0045] The non-collaborative feature aggregation module NCFA includes a concatenation layer contact, N layers of layer normalization layer LN, multi-head self-attention (MSA), layer normalization layer LN, and multilayer perceptron MLP, and a connection reconstructing and restoring layer Reconstruct, and S1-S4 represent four non-collaborative output features.

[0046] Figure 4 In the specific implementation, the hierarchical down-sampling features collected by the Transformer branch and the CNN branch are processed by embedding layers respectively, then concatenated in the channel, and then input into the feature aggregation module composed of N layers of LN, MSA, residual connection, and MLP for processing, and then restored to the original shape and connected with the input in the residual connection to keep the features stable, and finally output as the non-collaborative features.

[0047] In some preferred embodiments, the training process of the medical image segmentation network includes:

[0048] S1: Collect a medical image set, label and pre-process it, and save it to a medical image database.

[0049] Cooperate with hospitals or medical research institutions to obtain a medical image dataset of patients.

[0050] In some preferred embodiments, the medical image set specifically includes X-ray images, CT images, MRI images, and ultrasound images containing lesions.

[0051] Specifically, the medical image data includes X-ray (such as chest X-ray), CT (computed tomography) (such as brain or lung CT), MRI (magnetic resonance imaging) (such as brain or spine MRI image), ultrasound (such as fetal ultrasound, cardiac ultrasound, etc.). Due to the diversity and complexity of medical images, the collected images need to be preprocessed to ensure the consistency of data quality.

[0052] In some preferred embodiments, the collected medical images are manually annotated, which specifically includes:

[0053] Setting the pixel class ID of the medical image background to 0 value;

[0054] Segmenting the medical image into different class segmentation images, and setting the pixel class ID of different classes to 1, 2, …, N in turn, where N is the number of segmentation classes.

[0055] Specifically, various algorithms are used to remove noise in the image, such as artifacts, interference introduced by imaging devices, etc., to improve the clarity of the image. Due to different imaging devices and imaging parameters, the resolution and contrast of medical images may differ, so the image needs to be standardized to have consistent resolution and grayscale range in the dataset. The image is cropped or resized as needed to adapt to the subsequent model input format. For example, all images are adjusted to 224x224 pixel resolution, the pixel class ID of the background is set to 0 value, and the pixel class ID of different classes in the segmentation is set to 1, 2, …, N in turn. Where N is the number of segmentation classes (excluding background).

[0056] S2: Select images from the medical image database, and obtain the training set and test set after data augmentation.

[0057] In some preferred embodiments, the collected medical images are preprocessed, which specifically includes:

[0058] Removing noise from the collected medical images;

[0059] Standardizing the medical image after removing noise, adjusting the resolution and grayscale range of the medical image.

[0060] Specifically, some classes with obvious features are selected from the medical image dataset, and similar, repetitive and single-background classes are removed, and a series of data augmentation operations such as horizontal flip, vertical flip, 45-degree rotation, contrast enhancement, and noise addition are performed on the selected images to form a medical image dataset required for training. The medical image dataset is divided into a training set and a test set according to a ratio of 9:1.

[0061] S3: Selecting a medical image and its segmentation image from the training set, inputting the medical image segmentation network respectively, and extracting the local features and global features of the medical image after 4 times of down-sampling through CNN and Transformer respectively.

[0062] The medical image is specifically a lesion medical image containing a lesion of a patient. The segmentation image of the medical image is a lesion region segmentation mask annotated by a doctor.

[0063] Specifically, the Root layer uses a standard convolution with a convolution kernel of 7, a stride of 2, and padding of 3 to learn feature representation and down-sample. Each convolution calculates the weighted sum of the local pixel region by sliding on the picture (i.e. moving stride), to generate a new feature map (Feature Map). The convolution kernel extracts low-level features by detecting local patterns of the picture (such as edges, textures, color changes, etc.). As the network deepens, the convolution kernel will gradually extract higher-level features (such as shape, object parts, etc.) from low-level features (such as edges). For a multi-channel input image (such as an RGB image) and a corresponding convolution kernel, the formula needs to sum the convolution results of each channel. Assuming that the input image has C channels, and the size of each channel's convolution kernel is k h ×k w , then the pixel value F(i,j) of position (i,j) in the feature map is calculated by the formula

[0064]

[0065] where I c is the cth channel of the input image. K c is the convolution kernel corresponding to the cth channel. C is the number of channels of the input image.

[0066] Each convolution kernel generates a corresponding feature map, and the feature maps of multiple convolution kernels are stacked into a three-dimensional tensor.

[0067] The three Body layers first use DWConv with a step size of 2 for feature representation learning and down-sampling, and then use multiple DWConv for more in-depth feature representation learning. GN is used after all convolution operations, and Relu is performed once at the end of each layer. Finally, the output features are linearly embedded and added with position encoding for subsequent feature fusion. Compared with standard convolution, DWConv significantly reduces the amount of calculation and the number of parameters. Depthwise convolution can extract spatial information of each channel separately. In addition, pointwise convolution realizes cross-channel information fusion through linear combination between channels, and this separation operation can more efficiently extract features. Compared with Batch Normalization (BN), GN needs to calculate the mean and variance through the entire mini-batch, which is not good in the case of small batch. The normalization operation of GN is based on the channel grouping of each sample and does not depend on the batch size, so it performs more stably in small batch or single sample inference.

[0068] Specifically, unlike the convolution kernel of traditional CNN, Swin Transformer first divides the input image into fixed-size non-overlapping blocks (Patch), and flattens each block into a vector.

[0069] Input image size: H x W x C (height, width, number of channels).

[0070] The image is cut into non-overlapping blocks of size P x P, and each block has a size of P 2 x C.

[0071] These blocks are flattened and mapped to a fixed dimension through a linear transformation to obtain the initial Patch Embedding.

[0072] The final output is an embedding vector in sequence form, with a size of:

[0073]

[0074] where D is the dimension of the embedding.

[0075] Swin Transformer uses a fixed-size window of 7x7 to perform local self-attention calculation on the picture in each layer. The pixels within each window obtain the dependency relationship through the self-attention mechanism, realizing local feature extraction, without the need for global calculation as in traditional Transformers. The feature extraction of Swin Transformer is performed layer by layer, each layer is composed of multiple Transformer blocks, and a downsampling operation (similar to the pooling operation of CNN) is introduced in each stage, gradually reducing the resolution and increasing the channel number. After downsampling from high resolution and low channel number, the resolution is halved and the channel number is increased, further downsampling, the resolution continues to halve, and finally the resolution becomes one-sixteenth of the global feature. Downsampling is achieved through the Patch Merging operation. Adjacent blocks are merged and the channel number is adjusted through linear transformation. The advantage of Swin Transformer over traditional CNN and standard Vision Transformer (ViT) is the use of local window self-attention, which avoids the high computational cost of global self-attention. Secondly, the sliding window mechanism gradually captures long-distance dependencies.

[0076] S4: Feature fusion is performed on the extracted global features and local features through the collaborative aggregation module CFA and the non-collaborative aggregation module NCFA.

[0077] Specifically, the Swin Transformer module, the CNN module, and the output of the previous CFA module output 3 features as the current CFA input. The shapes of the 3 collaborative features input 4 times are [B,2*C,H / 2,W / 2], [B,4*C,H / 4,W / 4], [B,8*C,H / 8,W / 8], and [B,16*C,H / 16,W / 16]. The 3 collaborative features input 4 times are processed by LN and linearly embedded into [B,L,2*C], [B,L,4*C], [B,L,8*C], and [B,L,16*C] with the same height and width. Each time, the 3 embedded features are concatenated into 1 feature [B,L,30*C], and then the 4 embedded features are input into LN and MLP after feature aggregation through N-layer multi-head channel attention layers. The final collaborative feature is obtained. The calculation of the channel attention mechanism is as follows:

[0078]

[0079] wherein, represents the query matrix, the key matrix, and the value matrix. d is the sequence length and C i (i = 1, 2, 3, 4) four jump connection channel dimensions, specifically C1 = 64, C2 = 128, C3 = 256, and C4 = 512.

[0080] The cooperative feature shapes of 4 outputs are [B, 2*C, H / 2, W / 2], [B, 4*C, H / 4, W / 4], [B, 8*C, H / 8, W / 8], and [B, 16*C, H / 16, W / 16], respectively. Compared with the traditional jump connection feature fusion mode, the CFA can simultaneously utilize the global and local feature information extracted by the Transformer and CNN, avoid falling into low decomposition accuracy caused by using single features, and thus improve the segmentation accuracy

[0081] Specifically, the Swin Transformer module and the CNN module respectively output 4 down-sampling features at different stages as the NCFA input, and the 4 non-cooperative feature input shapes are [B, 2*C, H / 2, W / 2], [B, 4*C, H / 4, W / 4], [B, 8*C, H / 8, W / 8], and [B, 16*C, H / 16, W / 16]. The 4 input features are linearly embedded into [B, L, 2*C], [B, L, 4*C], [B, L, 8*C], and [B, L, 16*C] with unchanged height and width. Then, the four embedded features are spliced into a feature [B, L, 30*C], and after passing through N Transformer layers for feature aggregation, the aggregated features are reconstructed into the input 4 down-sampling feature shapes at different scales and are connected with the input features as four output features. The calculation of the self-attention mechanism in the Transformer is as follows:

[0082]

[0083] wherein Attention(Q, K, V) represents the calculation of the multi-head self-attention mechanism on Q, K, and V, Q, K, and V represent the query matrix, the key matrix, and the value matrix, respectively, p represents the number of patches in the window, d represents the dimension of the query or key, and the value in B is taken from the bias matrix

[0084] The 4 non-cooperative feature output shapes are [B, 2*C, H / 2, W / 2], [B, 4*C, H / 4, W / 4], [B, 8*C, H / 8, W / 8], and [B, 16*C, H / 16, W / 16], respectively. Compared with the traditional up-sampling interpolation mode, the NCFA independently extracts the hierarchical down-sampling features using the Transformer and the CNN, and in the subsequent up-sampling recovery process, the global and local features can be utilized to assist in recovering the image, without relying on interpolation up-sampling, which can further improve the segmentation accuracy.

[0085] S5: inputting the feature fused feature into the decoder to obtain the predicted segmentation result of the medical image and the segmentation image thereof, respectively.

[0086] The collaborative features output by the collaborative aggregation module CFA are reconstructed, and then combined with the four groups of fusion features output by the non-collaborative aggregation module NCFA, and then decoded by four times of up-sampling decoding in the decoder, to output the prediction segmentation results of the medical image and the segmentation image thereof.

[0087] S6: Calculate the loss between the prediction segmentation results of the medical image and the segmentation image thereof, and optimize the medical image segmentation network through the loss. The loss is a hybrid cross-entropy and Dice loss.

[0088] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, which can include ROM, RAM, magnetic disk or optical disk, etc.

[0089] Although the embodiments of the present application have been shown and described, it should be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A medical image automatic segmentation method based on a double-aggregation hybrid neural network, characterized in that, The application relates to a medical image segmentation network and a training method thereof. The application comprises the following steps: inputting a medical image to be segmented into a pre-trained medical image segmentation network to output a medical image segmentation result; the medical image segmentation network adopts a double-aggregation hybrid U-shaped network, which comprises two parallel encoders and a single decoder, and the two parallel encoders are respectively a convolutional neural network (CNN) and a Transformer; the convolutional neural network (CNN) and the Transformer are used to simultaneously extract local features and global features of the medical image respectively; multi-scale feature fusion is performed on the extracted local features and global features through a collaborative feature aggregation module (CFA) and a non-collaborative feature aggregation module (NCFA); the collaborative feature aggregation module (CFA) comprises N layers of layer normalization layers (LN) and multi-head channel attention (MCA), and the layer normalization layers (LN) and a multi-layer perceptron (MLP) are connected respectively; the non-collaborative feature aggregation module (NCFA) comprises a contact layer, N layers of layer normalization layers (LN), multi-head self-attention (MSA), a layer normalization layer (LN) and a multi-layer perceptron (MLP), and finally a reconstruct recovery layer (Reconstruct) is used to recover the original scale; 2. The automatic medical image segmentation method based on a double-aggregation hybrid neural network according to claim 1, characterized in that, the features subjected to the multi-scale feature fusion are input into the decoder to output the medical image segmentation result. The training process of the medical image segmentation network comprises the following steps: collecting a medical image set, labeling and pre-processing the medical image set and saving the medical image set to a medical image database; screening images from the medical image database, and obtaining a training set and a test set after data enhancement; selecting medical images and segmentation images from the training set, inputting the medical images and the segmentation images into the medical image segmentation network respectively, and extracting global features and local features of the medical images through the CNN and the Transformer respectively after 4 times of down-sampling; performing feature fusion on the extracted global features and local features through the collaborative aggregation module (CFA) and the non-collaborative aggregation module (NCFA); inputting the features subjected to the feature fusion into the decoder to obtain the predicted segmentation results of the medical images and the segmentation images respectively; 3. The automatic medical image segmentation method based on a dual-aggregation hybrid neural network according to claim 1 or 2, characterized in that, The convolutional neural network CNN comprises a root layer and three body layers, the root layer comprises a convolution kernel conv, a group normalization layer GN and a Relu activation function layer; the body layer comprises N bottleneck blocks; each bottleneck block comprises three layers of cascaded depth separable convolution kernels DWConv, a group normalization layer GN and a Relu activation function.

4. The automatic medical image segmentation method based on a double-aggregation hybrid neural network according to claim 2, characterized in that, calculating the loss between the predicted segmentation results of the medical images and the segmentation images, and optimizing the medical image segmentation network through the loss.

5. The automatic medical image segmentation method based on a double-aggregation hybrid neural network according to claim 2, characterized in that, The medical image set is specifically X-ray images, CT images, MRI images and ultrasonic images containing lesions. The collected medical images are manually labeled, and the specific steps are as follows: setting the pixel class ID of the background of the medical image to 0; 6. The automatic medical image segmentation method based on a double-aggregation hybrid neural network according to claim 2, characterized in that, segmenting the medical image into different class segmentation images, and setting the pixel class ID of each class to 1, 2,..., N in sequence, wherein N is the number of segmentation classes. The collected medical images are pre-processed, and the specific steps are as follows: removing noise in the collected medical images; 7. The automatic medical image segmentation method based on a double-aggregation hybrid neural network according to claim 2, characterized in that, performing standardization processing on the medical images after the noise is removed, and adjusting the resolution and the gray scale range of the medical images. screening images from the medical image database, and performing data enhancement on the images, specifically: removing similar, repeated and single-background classes in the medical images, and performing horizontal flipping, vertical flipping, 45-degree rotation, contrast enhancement or noise addition on the medical images.

Images