A tumor medical image segmentation modeling method for liver multimodal MRI

Abstract

The present invention relates to a tumor medical image segmentation modeling method for liver multimodal MRI, belonging to the field of electronic digital data processing technology. First, the data is preprocessed and the spatial coordinates are aligned; then the data is sent to the modality-specific encoder for feature extraction, and the encoders are fused and exchanged; secondly, in the high-level semantic feature stage, the transformer block is used to model long-distance dependencies and the long-distance fusion of high-level semantic information based on the mutual attention feature fusion block; then the decoder is decoded to obtain a mask map of the tumor area; finally, the model is trained based on the loss function. Through a coding and decoding network based on cross-modal feature fusion, combined with the input multimodal medical image data, the segmentation accuracy of the tumor area and the fineness of the edge of the segmented area are improved, and the problem of insufficient segmentation accuracy and smooth edge of the segmented area of ​​liver multimodal MRI is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of electrical digital data processing, and in particular relates to a tumor medical image segmentation modeling method for liver multimodal MRI. Background Art

[0002] MRI is a technology based on the nuclear magnetic resonance phenomenon of certain atomic nuclei in human tissues, which processes the obtained radio frequency signals through electronic computers for imaging. It has the characteristics of no ionizing radiation, safety for the human body, and obvious differences in soft tissues. It is the preferred imaging examination method for the definitive diagnosis of patients with abnormal liver ultrasound or serum AFP screening. MRI has a variety of imaging methods, such as diffusion-weighted imaging (DWI), which is related to the Brownian motion of free water molecules in tissues, and malignant tumors often show high signals; T2-weighted images (T2-weighted images), which mainly show the water content in tissues. Contrast-enhanced magnetic resonance imaging (CE-MRI) is a series of images obtained after MRI scanning after intravenous injection of a certain contrast agent. For example, enhanced MRI images of the liver often include T1-weighted imaging (PRE), T1 arterial phase (AP), T1 portal venous phase (PVP), T1 transitional phase (TP), and T1 hepatobiliary phase (HBP). In T1-weighted images, substances with fast relaxation times appear bright, while substances with slow relaxation times appear dark. In the arterial phase, the contrast agent enters the hepatic artery, which is enhanced. In the portal venous phase, the contrast agent reaches the portal vein system of the liver, which is enhanced. In the transitional phase, the contrast agent is gradually taken up by hepatocytes, and the degree of enhancement of the liver parenchyma increases. In the hepatobiliary phase, the signal intensity of the liver and bile duct increases significantly, reflecting whether the lesion contains tissue with normal hepatocyte function. Doctors need to integrate information from each modality of images when reading films for diagnosis, which not only greatly increases the doctor's reading time, but also requires the doctor to have very professional imaging knowledge.

[0003] In today's medical imaging tumor segmentation field, deep learning has become the mainstream research method in this field due to its powerful feature extraction capabilities. A model for tumor segmentation is obtained by building a neural network and training it with a certain number of labeled multimodal enhanced MRI data samples. After the MRI image is fed into the model, the segmentation result can be quickly obtained.

[0004] In "SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks", LingxiaoYang proposed a parameter-free attention module based on neuroscience theory and by optimizing the energy function to find the importance of each neuron.

[0005] Nowadays, multimodal medical image tumor segmentation solves the problem of feature exchange and fusion between modalities. For the contour map after segmentation, it is necessary to combine multi-scale information to improve the accuracy of the tumor segmentation contour.

[0006] Therefore, at this stage, it is necessary to design a tumor medical image segmentation modeling method for liver multimodal MRI to solve the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a tumor medical image segmentation modeling method for liver multimodal MRI, which is used to solve the limitations of the application scenarios of single-modality segmentation networks and the problem of feature exchange between multi-modal MRI image modalities; that is, the present invention improves the segmentation accuracy of tumor medical image regions and the fineness of segmentation region edge through a coding and decoding network based on cross-modal feature fusion, combined with input multimodal medical image data, and solves the problems of insufficient segmentation accuracy and smooth segmentation region edge of liver multimodal MRI.

[0008] To achieve the above object, the technical solution of the present invention is:

[0009] A tumor medical image segmentation modeling method for liver multimodal MRI includes the following steps:

[0010] S1: Preprocessing of abdominal enhanced MRI data;

[0011] S2: Divide the preprocessed data set in step S1 into a training set and a validation set at a ratio of 8:2;

[0012] S3: Establish a liver MRI multimodal UNet network, which inputs 5 modal MRI images to jointly extract features and decode the semantic segmentation mask map corresponding to each modality;

[0013] S4: Calculate the loss of each modal output and perform back-propagation to update the parameters; the loss function uses a combination of binary cross entropy loss and overlap loss;

[0014] S5: Model training based on loss function.

[0015] Furthermore, step S1 includes the following sub-steps:

[0016] S1.1: Read the medical digital imaging and communication format file of each modality of CE-MRI, resample it, and adjust the spatial voxel size to 1mm*1mm*1mm;

[0017] S1.2: Map the resampled data from the image coordinate system to the world coordinate system so that the positions of the pixels in each modality image in the world coordinate system are the same;

[0018] S1.3: Compare the data of each modality of the same patient based on the world coordinate system, fill the missing parts of some modalities in the world coordinate system with 0 to ensure that the length, width and number of slices of each modality image of the same patient are consistent, and adjust the size of each modality to 128*128*128;

[0019] S1.4: Map the pixel value range from 0-65535 to 0-255 and save it in npy format;

[0020] S1.5: The annotation files corresponding to each modality are processed in the same way as steps S1.1, S1.2 and S1.3, and the points with non-zero image values, i.e., the annotated tumor areas, are assigned a value of 255 and saved in npy format.

[0021] Furthermore, step S3 includes the following sub-steps:

[0022] S3.1: The encoder adopts a UNet-like encoder structure. The encoder has a five-layer structure. The feature blocks of each layer of the encoder are extracted by the feature extraction block, and the extracted features are fused with the features corresponding to other modalities and added back to the corresponding position to realize the exchange of feature information between modalities, and then downsampled to the next layer;

[0023] S3.2: Perform feature fusion and long-range correlation calculation on the high-level semantic information extracted by each modality encoder; use the feature fusion block based on the attention mechanism to exchange high-level semantic features, send the high-level semantic features of each modality into the transformer, and perform long-range correlation calculation to overcome the defect of local feature extraction by convolution;

[0024] S3.3: Decode the high-level features of each modality; the decoding operation is performed in a form symmetrical to the encoder, and the decoder uses a decoder with shared parameters to decode each modality.

[0025] Furthermore, step S3.1 includes the following sub-steps:

[0026] S3.1.1: The encoder consists of two parts: the visual geometry group and the parameter-free attention module; the VGG block consists of two convolutions, each followed by batch normalization and a linear rectification activation; after extracting features through the VGG block, the feature block is sent to simAM for attention calculation;

[0027] S3.1.2: Fusion of features from different modalities; concatenation of features from the five modalities on the channels, and then calculation of channel attention to encourage the model to learn important channels after concatenation; then restoration of the number of channels to the number of channels in the current layer after one convolution, and calculation of attention using the simAM parameter-free attention block; calculation of spatial attention for features from each modality and calculation of attention using the simAM parameter-free attention block, and calculation of the maximum value in the modality for each position in the space, and finally obtaining feature fusion information and adding it to each modality to achieve preliminary exchange of information between modalities;

[0028] S3.1.3: Use maximum pooling to downsample the features.

[0029] Furthermore, step S3.2 includes the following sub-steps:

[0030] S3.2.1: For the high-level semantic information of each modality, the current modality is linearly mapped to query Q, the inter-modal fusion block is linearly mapped to key K and value V, and attention is calculated on Q, K and V;

[0031] S3.2.2: The high-level semantic information of each modality is linearly mapped into three vectors Q, K, and V, and self-attention, i.e., transformer calculation, is performed on the three vectors Q, K, and V. Finally, the output of the self-attention is activated by one convolution and ReLU, and the number of channels is restored to the number of channels output by the modality encoder.

[0032] Furthermore, step S3.3 includes the following sub-steps:

[0033] S3.3.1: Each layer of the decoder uses VGG blocks to decode features and feeds them into SimAM blocks. When each modality is used in the decoder, it is concatenated with the jump connection features of the modality on the channel, and a single convolution and ReLU activation are used to restore the number of feature channels;

[0034] S3.3.2: For the skip connection of each layer of features of each modality, multi-scale skip connections are performed to enhance the detailed contour information contained in the skip part; the information of other layers is upsampled or downsampled to the size of the current layer and feature fused, the fused features are passed through a gating mechanism to suppress useless information, and are fused with the information of the current layer to achieve the fusion of multi-scale information in the skip connection;

[0035] S3.3.3: Upsample the features of each layer.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] First, the data is preprocessed and spatial coordinates are aligned; then the data is sent to the modality-specific encoder for feature extraction, and the parameter-free attention block is used in the modality encoding to improve the ability to capture useful features. Feature fusion and exchange are performed between encoders to achieve the capture of complementary information between modalities during the encoding process; secondly, in the high-level semantic feature stage, the transformer block is used to model long-distance dependencies and the long-distance fusion of high-level semantic information based on the mutual attention feature fusion block; then the decoder is used to decode to obtain the mask map of the tumor area; finally, the model is trained based on the loss function. Through an encoder-decoder network based on cross-modal feature fusion, combined with the input multimodal medical image data, the segmentation accuracy of the tumor area and the fineness of the edge of the segmented area are improved, solving the problem of insufficient segmentation accuracy and smooth edge of the segmented area in liver multimodal MRI. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart of the SVGG block for feature extraction and decoding used in the present invention.

[0039] Figure 2 This is an internal structure diagram of the feature information fusion block used in the encoder stage of the present invention.

[0040] Figure 3 This is a flow chart of a high-level feature information fusion block used in the high-level feature stage of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the embodiments described are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0042] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention. It should be noted that relational terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0043] Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0044] A tumor medical image segmentation modeling method for liver multimodal MRI includes the following steps:

[0045] Step 1: Preprocess the abdominal enhanced MRI data;

[0046] Step 1.1: Read the DICOM format file of each modality in the abdominal CE-MRI, resample it, and adjust its spatial voxel size to 1mm*1mm*1mm;

[0047] Step 1.2: Read the values ​​of the three tags (0018,5100), (0020,0032) and (0020,0037) in the DICOM file of each modality, and map the resampled data from the image coordinate system to the world coordinate system according to the values ​​in the three tags, so that the positions of the pixels at the corresponding positions in each modality are the same in the world coordinate system;

[0048] Step 1.3: Compare the data of each modality of the same patient based on the world coordinate system, fill the missing values ​​of some modalities in the world coordinate system with 0 to ensure that the length, width and number of slices of each modality image are consistent, and change the size of each modality to 128*128*128;

[0049] Step 1.4: Read the two tag values ​​(0028, 1050) and (0028, 1050) in each modality DICOM file, and map the pixel value range from 0-65535 to 0-255 according to the tag value, and save it in npy format;

[0050] Step 1.5: Process the annotation files corresponding to each modality in the same way as steps 1.1, 1.2, and 1.3, assign a value of 255 to the points with non-zero image values, i.e., the annotated tumor areas, and save them in npy format.

[0051] Step 2: Divide the data set into a training set and a validation set at a ratio of 8:2;

[0052] Step 3: Establish a liver MRI multimodal UNet network. The network inputs five modal MRI images to jointly extract features and decode the semantic segmentation mask map corresponding to each modality.

[0053] Step 3.1: Build a dedicated encoder for each input modality. The encoder adopts a UNet-like encoder structure, that is, Figure 1 As shown in Figure 1, the encoder has a five-layer structure. In each layer of the encoder, semantic information is extracted through the SVGG block, and the extracted semantic information is sent to the feature information fusion block to obtain semantic information from other modalities, and finally sent to the next layer of the encoder after maximum pooling, that is, downsampling.

[0054] Step 3.1.1: Figure 1 As shown in Figure 1, the encoder consists of two parts: a VGG block and a simple parameter-free attention module (SIMAM). The VGG block consists of two convolutions, each followed by batch normalization and a linear rectification activation. After extracting features through the VGG block, the feature block is fed into the SIMAM for attention calculation.

[0055] Step 3.1.2: The information blocks of each modality are sent to the feature information fusion block at each layer of the encoder, so that each modality can receive possible useful semantic information from other modalities to achieve the complementarity of useful features. Figure 2 As shown in Figure 1, the feature blocks of each modality are fed into the modality feature fusion block, which fully fuses the feature blocks of each modality and outputs a fused feature information block. The fused feature information block is added back to each modality through residual connection to achieve the exchange of semantic information between modalities.

[0056] The internal result of the modal feature fusion block concatenates the feature blocks of the five input modalities on the channel, and then calculates the channel attention of the concatenated feature blocks to encourage the model to learn the important channels after concatenation. The feature blocks with channel attention information are then sent to the convolution to restore the number of channels to the number of channels of each modality in the current layer before being sent to the modal fusion block. Finally, they are multiplied with the spatial maximum feature block after the spatial attention calculation after the simAM parameter-free attention block, and finally output after being activated by a sigmod function.

[0057]

[0058] where ƒ attn is the spatial attention calculation, ƒ simAM is a parameter-free attention block, F PRE is the modal feature block during the T1 sweep period, F AP is the AP, i.e. T1 arterial phase modality feature block, F PVP PVP is the T1 gate static mode feature block, F TP PVP is the modal feature block of T1 transition period, F HBP is the HBP or T1 hepatobiliary phase feature block, and max is the maximum value of each position in the tensor among the five modes. The spatial maximum feature block after spatial attention calculation is obtained by the above formula from the five input modes. The feature blocks of the five modes will be obtained by the spatial attention block and the parameterless attention module to obtain the feature blocks containing spatial attention. The final max function will return a tensor with the same shape as each input mode. The value of each position of the tensor is the maximum value of the five modal tensors for the position.

[0059] Step 3.1.3: For the feature blocks of each mode output by the feature information fusion block, use the maximum pooling (MaxPooling) to downsample the features.

[0060] Step 3.2: The high-level semantic information extracted by each modality encoder is self-attention calculated and high-level semantic information feature fused. The high-level semantic information of each modality will be sent to the transformer for long-distance correlation calculation to overcome the defect of local feature extraction by convolution. The high-level feature information fusion block adopts a feature fusion method based on the attention mechanism.

[0061] Step 3.2.1: When calculating the self-attention of high-level semantic information, first perform convolution on the feature blocks of each modality, and adjust the number of channels to the number of patches required by the transformer. The dimension of each patch is the feature dimension in each channel. Linearly map the patch vector into three vectors Q, K, and V, and perform self-attention calculations. Repeat the multi-head self-attention calculation 12 times. Restore the number of channels of the output content to the number of channels before sending it to the transformer through a convolution, and finally activate the feature block through ReLU output.

[0062] Step 3.2.2: Perform feature fusion on high-level feature information, such as Figure 3 As shown, the input of each modality is sent to the attention fusion block for fusion of high-level feature blocks, and the output of the five fused modalities output by this module is added to the five modalities input by the high-level feature information fusion block in a residual manner and output.

[0063] Step 3.2.3: The attention fusion block is a feature fusion method based on attention. The five input modalities will be spliced ​​on the channel, similar to the modal fusion method of other layers. After the channel attention calculation, a convolution is performed again to generate a preliminary feature fusion block, which is linearly mapped to the K and V required for attention. The five modal vectors input to the attention fusion block are linearly mapped to Q, and the attention calculation is performed with the K and V obtained by the preliminary feature fusion block. In this way, the current modality is used as a query to obtain the semantic information in the preliminary feature fusion block that is useful for the semantic segmentation of the current modality, and the final feature fusion block of each modality is obtained and output.

[0064] Step 3.3: Decode the high-level feature blocks of each modality. The decoding operation is symmetrical to the encoder. Unlike the independent encoder for each modality, the decoder consists of two parts. One part is the SVGG decoder with shared parameters, which is used to decode the 2-5 layer feature blocks and upsample them to the next layer; the other part is the shallow feature decoder exclusive to each modality, which uses the SVGG decoder with independent parameters and finally outputs.

[0065] Step 3.3.1: After obtaining the skip connection information from the corresponding modality, each layer of the decoder uses the same SVGG block decoding features as each layer of the encoder, that is, first through VGG and then sent to the SimAM block.

[0066] Step 3.3.2: In the decoding process of semantic segmentation, in order to obtain the resolution information of the downsampling loss in the encoding stage, the feature block of the corresponding layer of the decoder, namely the jump connection, will be obtained. Among them, the feature block of each layer in the encoder stage will be sent to the multi-scale information fusion block, which outputs the multi-scale information block of the corresponding resolution in the four-layer jump connection process. Each layer of jump connection will add the multi-scale information fusion block in a residual manner and send it to the corresponding decoder layer.

[0067] In the multi-scale information fusion block, for each scale output, the multi-scale information fusion block converts the scale of feature blocks of other scales to the current scale by upsampling or downsampling, and fully fuses the multi-scale information through the SVGG block. The current scale input and the multi-scale fused information are sent to the gate mechanism (Gate) block and finally output. The gate mechanism block is in the form of convolution, which suppresses the useless information in the multi-scale fused information as much as possible according to the current scale input.

[0068] Step 3.3.3: Upsample the feature blocks of each layer.

[0069] Step 4: Calculate the loss of each modality output and perform backpropagation to update the parameters:

[0070] Step 4.1: The loss is a combination of BCEWithLogitsLoss and IoU loss.

[0071] =

[0072] L (BCEWithLogitsLoss) is the commonly used loss function in pixel-level semantic segmentation, namely the binary unactivated cross entropy loss function, where σ is the sigmoid activation function, N is the number of samples, is the label value of the i-th sample, is the model prediction value of the i-th sample.

[0073]

[0074]

[0075] Intersection over Union (IoU) is the ratio of the intersection and union of the predicted area and the true area, where A and B represent the predicted expected area and the true area respectively, and L (IoU) is the loss value based on IoU.

[0076]

[0077]

[0078] Where L a is the total loss of this mode, L (BCEWithLogitsLoss)a is the binary unactivated cross entropy loss value of the modality, L (IoU)a is the IoU loss value of the modality, L LLMUNet is the total loss value of the model training, that is, the weighted sum of all modality losses. For the calculation of each modality loss, the binary cross entropy loss and IoU loss between the output of the current modality and the mask map are calculated, and the coefficients of the two losses are set to 0.5. For the overall model loss, the loss of each modality is accumulated and averaged.

[0079] For the calculation of each modality loss, the binary cross entropy loss and IoU loss between the output of the current modality and the mask map are calculated, and the coefficients of the two losses are set to 0.5. For the overall model loss, the loss of each modality is accumulated and averaged.

[0080] Step 5: Train the model based on the loss function.

[0081] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A tumor medical image segmentation modeling method for liver multimodal MRI, characterized in that: The following steps are involved: S1: Preprocessing of abdominal enhanced MRI data; S2: Divide the preprocessed data set in step S1 into a training set and a validation set at a ratio of 8:2; S3: Establish a liver MRI multimodal UNet network, which inputs 5 modal MRI images to jointly extract features and decode the semantic segmentation mask map corresponding to each modality; S4: Calculate the loss of each modal output and perform back-propagation to update the parameters; the loss function uses a combination of binary cross entropy loss and overlap loss; S5: Model training based on loss function; Step S1 includes the following sub-steps: S1.1: Read the medical digital imaging and communication format file of each modality of CE-MRI, resample it, and adjust the spatial voxel size to 1mm*1mm*1mm; S1.2: Map the resampled data from the image coordinate system to the world coordinate system so that the positions of the pixels in each modality image in the world coordinate system are the same; S1.3: Compare the data of each modality of the same patient based on the world coordinate system, fill the missing parts of some modalities in the world coordinate system with 0 to ensure that the length, width and number of slices of each modality image of the same patient are consistent, and adjust the size of each modality to 128*128*128; S1.4: Map the pixel value range from 0-65535 to 0-255 and save it in npy format; S1.5: The annotation files corresponding to each modality are processed in the same way as steps S1.1, S1.2 and S1.3, and the points with non-zero image values, i.e., the annotated tumor areas, are assigned a value of 255 and saved in npy format; Step S3 includes the following sub-steps: S3.1: The encoder adopts a UNet-like encoder structure. The encoder has a five-layer structure. The feature blocks of each layer of the encoder are extracted by the feature extraction block, and the extracted features are fused with the features corresponding to other modalities and added back to the corresponding position to realize the exchange of feature information between modalities, and then downsampled to the next layer; S3.2: Perform feature fusion and long-range correlation calculation on the high-level semantic information extracted by each modality encoder; use the feature fusion block based on the attention mechanism to exchange high-level semantic features, send the high-level semantic features of each modality into the transformer, and perform long-range correlation calculation to overcome the defect of local feature extraction by convolution; S3.3: Decode the high-level features of each modality; the decoding operation is performed in a form symmetrical to the encoder, and the decoder uses a decoder with shared parameters to decode each modality; Step S3.1 includes the following sub-steps: S3.1.1: The encoder consists of two parts: the visual geometry group and the parameter-free attention module; the VGG block consists of two convolutions, each followed by batch normalization and a linear rectification activation; after extracting features through the VGG block, the feature block is sent to simAM for attention calculation; S3.1.2: Fusion of features from different modalities; concatenation of features from the five modalities on the channels, and then calculation of channel attention to encourage the model to learn important channels after concatenation; then restoration of the number of channels to the number of channels in the current layer after one convolution, and calculation of attention using the simAM parameter-free attention block; calculation of spatial attention for features from each modality and calculation of attention using the simAM parameter-free attention block, and calculation of the maximum value in the modality for each position in the space, and finally obtaining feature fusion information and adding it to each modality to achieve preliminary exchange of information between modalities; S3.1.3: Use maximum pooling to downsample the features.

2. The tumor medical image segmentation modeling method of liver multimodal MRI according to claim 1, characterized in that: Step S3.2 includes the following sub-steps: S3.2.1: For the high-level semantic information of each modality, the current modality is linearly mapped to query Q, the inter-modal fusion block is linearly mapped to key K and value V, and attention is calculated on Q, K and V; S3.2.2: The high-level semantic information of each modality is linearly mapped into three vectors Q, K, and V, and self-attention, i.e., transformer calculation, is performed on the three vectors Q, K, and V. Finally, the output of the self-attention is activated by one convolution and ReLU, and the number of channels is restored to the number of channels output by the modality encoder.

3. The tumor medical image segmentation modeling method of liver multimodal MRI according to claim 1, characterized in that: Step S3.3 includes the following sub-steps: S3.3.1: Each layer of the decoder uses VGG blocks to decode features and feeds them into SimAM blocks. When each modality is used in the decoder, it is concatenated with the jump connection features of the modality on the channel, and a single convolution and ReLU activation are used to restore the number of feature channels; S3.3.2: For the skip connection of each layer of features of each modality, multi-scale skip connections are performed to enhance the detailed contour information contained in the skip part; the information of other layers is upsampled or downsampled to the size of the current layer and feature fused, the fused features are passed through a gating mechanism to suppress useless information, and are fused with the information of the current layer to achieve the fusion of multi-scale information in the skip connection; S3.3.3: Upsample the features of each layer.

Images