Ldct image denoising method based on multi-scale self-attention generative adversarial network

Abstract

The present application relates to a LDCT image denoising method based on a multi-scale self-attention generative adversarial network. The present application constructs a CT image dataset required for training; then constructs a denoising model based on a GAN framework, constructs a generator based on a Transformer encoder-decoder structure and a multi-scale convolutional network discriminator; uses a training set and a validation set to train and verify the constructed denoising model, and obtains optimized model parameters; finally, the generator of the denoising network model with the highest accuracy in the test process is used to process the input low-dose CT image, and a denoised low-dose CT image is output; the present application can remove the noise and artifacts in the low-dose CT image, while also retaining the overall structure and local texture details and edge blur problems of the original image, preventing image over-smoothing and loss of local information, and facilitating clinical diagnosis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image denoising, and relates to a generative adversarial network (GAN) LDCT image denoising method based on multi-scale self-attention. BACKGROUND

[0002] Computed tomography is a reliable and non-invasive medical image imaging mode, which is helpful for discovering pathological abnormalities, tumors, cardiovascular diseases, lung nodules, internal injuries and fractures of the human body. In addition to the diagnostic aspect, CT is also useful in guiding various clinical treatments, such as radiotherapy and surgery.

[0003] However, the X-ray radiation in the repeated CT scanning process may be harmful to the human body, may cause immune function to decrease, metabolism to be abnormal, reproductive organs to be damaged, and the risk of leukemia, cancer and genetic diseases to increase. Therefore, it is necessary to reduce the X-ray radiation dose as much as possible while ensuring that the CT image quality meets the diagnostic requirements. However, when CT scanning is performed, if the photons penetrating through the patient to the detector are insufficient, the generated CT image will produce serious stripe artifacts and speckle noise, specifically, bright or dark straight lines, and this situation is more obvious when low-dose CT scanning is performed. The decline in the quality of the CT image seriously affects the accuracy of diagnosis, especially for the diagnosis of small-area, shape-microscopic early lesions. Therefore, it is necessary to analyze and research the denoising technology in the preprocessing of medical images, accurately distinguish the noise area and the microstructure texture under the lowest possible radiation dose, and efficiently denoise the noise area, so as to obtain a CT image with a quality similar to that of a conventional dose CT image (NDCT).

[0004] In the past decades, many low-dose CT (LDCT) restoration methods have been proposed, which can be divided into three categories: sinogram domain filtering, iterative reconstruction and image domain restoration. These traditional LDCT denoising algorithms play an important role in improving the quality of denoised images. With the rise of deep learning, convolutional neural networks (CNN) are widely used in image denoising. Due to the powerful feature learning and feature mapping capabilities of CNN, LDCT image denoising networks based on CNN have achieved good results, but there are still problems such as over-smoothing of denoised images leading to loss of key details or introduction of new noise. The choice of loss function plays a decisive role in the denoising effect of the image. Through the traditional pixel-level loss function such as mean square error (MSE), the mean of the sum of the squares of the errors of the corresponding pixels of the generated denoised image and the NDCT image is calculated, aiming to take the average of the possible output to adapt to the uncertainty in prediction. Because the noise in the LDCT image presents different scales, different directions and different density distributions, etc., the denoised image generated by this way usually has the problems of blurring, over-smoothing and lack of texture details. SUMMARY

[0005] The present application aims to provide a LDCT image denoising method based on multi-scale self-attention generative adversarial network; using the adversarial loss of generative adversarial network GAN, dynamically measure the similarity between the denoised image and the normal dose image during training, combine mean square error and mean absolute error (MAE), ensure the denoising effect of the denoised image while retaining more structure and texture information, and improve the denoising effect of low-dose CT image.

[0006] The generator using the encoder-decoder structure, a local enhanced Transformer (Leformer) based on Transformer is constructed to replace the original convolutional layer; while retaining the multi-head attention mechanism of Transformer, a local enhancement module is proposed, so that the network can not only learn global information but also capture local information.

[0007] A discriminator with a multi-scale feature extraction module is constructed to expand the receptive field of the module by extracting multi-scale features at the same image scale. The Canny edge detection algorithm is used to calculate the image gradient, and the mean absolute error between the generated denoised image and the conventional dose image is calculated in the image gradient domain. The adversarial loss of the generative adversarial network and the pixel-level loss in the image domain are combined to enhance the denoising network's ability to remove artifacts and obtain image edge information.

[0008] The specific steps of this invention are as follows:

[0009] Step 1: Construct a low-dose CT image paired dataset:

[0010] Multiple sets of conventional-dose CT images of different body parts were acquired, and Poisson noise was added to each image to simulate low-dose CT images corresponding to the conventional-dose images; a CT image dataset (I) was constructed. LD ,I ND ), where I LD These are low-dose CT images, I ND These are conventional dose CT images that match low-dose images;

[0011] Step 2: Construct a low-dose CT image denoising model:

[0012] A denoising model is constructed based on the GAN framework, comprising a generator structure and a discriminator structure; the generator G converts low-dose CT images I... LD Mapped to the corresponding conventional dose CT image I ND This achieves the purpose of noise removal, i.e., I ND ≈I gen =G(I LD ), I gen This represents the image after denoising.

[0013] Step 1: Build a denoising model generator:

[0014] The generator is a Transformer-based encoder-decoder structure. The encoder consists of a tokenization module, two consecutive Leformer modules combined with downsampling layers, and the decoder consists of two consecutive upsampling layers combined with Leformer modules and a deTokenization module. The encoder and decoder are connected through a Leformer module. There are skip connections between corresponding Leformer modules in the encoder and decoder to avoid the gradient vanishing problem and preserve more image structure and texture details in the decoder stage.

[0015] The tokenization module stretches the input CT image into a token sequence composed of n one-dimensional tokens: where n represents the number of tokens, d a is the length of each token;

[0016] The Leformer module based on the Transformer structure can not only learn global information but also capture local information; the Leformer module has two processing stages:

[0017] In the first stage, the token sequence passes through a layer normalization layer (Layer Norm) and then enters a multi-head self-attention module (MSA), and the output is Specifically, in the multi-head self-attention module, the token sequence T a is multiplied by three different weight matrices W Q , W K , and W V to obtain three types of tensors, respectively referred to as Q, K, and V, and then the output expression of the multi-head self-attention module is: where MSA represents the multi-head self-attention mechanism, is the length of the tensor K;

[0018] In the second stage, the token sequence passes through a layer normalization layer and then enters the local enhancement module (LocalEnhance). First, a linear projection layer (MLP) is used to increase the feature dimension, then the token sequence is converted into a two-dimensional feature map using the detokenization module, then the local information in the two-dimensional feature is obtained using a combination of two residual 3x3 convolutions and Leaky ReLU, then the two-dimensional feature map is restored to the token sequence using the tokenization module, and finally the feature dimension is restored using a linear projection layer, and the output is The expression of the token sequence passing through the Leformer module is:

[0019] T′ b = MSA (LN (T a ))+ T a ; T b = LE (MLP (T′ b ))+ T′ b ;

[0020] In the above formula, LN is the layer normalization layer, and LE represents the local enhancement module.

[0021] In the down-sampling layer, a convolution layer with a convolution kernel size of 3x3 and a step size of 2 is used to realize the down-sampling operation; in the up-sampling layer, an inverse convolution layer with a convolution kernel size of 3x3 and a step size of 2 is used to realize the up-sampling operation;

[0022] Step 2, constructing the denoising model discriminator:

[0023] The discriminator includes three consecutive modules:

[0024] The first module includes a convolution layer with a kernel size of 3x3 and a step of 2, a batch normalization layer, and a Leaky ReLU activation function, which reduces the computational load of the network while expanding the receptive field;

[0025] The second module is composed of a multi-scale feature extraction module, which extracts multi-scale features of the same scale of the image to expand the receptive field in the module and improve the discriminant ability of the discriminator. Specifically, first, a convolution layer with an output channel of 128 and a kernel size of 1x1 is used for convolution operation to obtain a feature X a with a channel number of 128; X a is evenly divided into four groups of features with a channel number of 32: X b , X c , X d , X e ; the first group of features X b remains unchanged and is directly used as output to obtain X' b , the second group of features X c is calculated after a convolution layer with a kernel size of 3x3, a step of 1, and padding of 1 to obtain output X' c ; the third group of features X d is added to the output X' c of the second group and calculated after the same convolution layer with a kernel size of 3x3, a step of 1, and padding of 1 to obtain output X' d ; similarly, the fourth group of features X e is added to the output X' d of the third group of features and calculated after the same convolution layer with a kernel size of 3x3, a step of 1, and padding of 1 to obtain output X' e ; the outputs of the four groups of features are stacked in the channel dimension; finally, a convolution layer with an output channel of 1 and a kernel size of 1x1 is used to reduce the dimension of the stacked features to obtain output X' a , which finally completes the fusion of multi-scale information; in this way, the feature X a is approximately calculated by multiple different size convolution operations, obtaining different size receptive fields, i.e., being able to obtain multiple different scale features under the same scale;

[0026] The third module comprises a convolution layer with a kernel size of 3*3 and a step of 1 and a sigmoid activation function; the output is the judgment of the discriminator on the input image, the more similar the input image of the discriminator is to the conventional dose CT image, the closer the output is to 1.

[0027] Step three, data preprocessing:

[0028] In the data preprocessing stage, the data set is divided into a training set, a validation set and a test set; in order to better obtain the local information of the image and expand the sample size, each pair of images in the training set and the validation set is randomly cropped to a set size of image block;

[0029] Step four, training the denoising model and updating the parameters:

[0030] The denoising network is trained according to the training mode of the GAN framework, the low dose CT image is input into the generator to obtain the generated denoising image, then the conventional dose CT image and the denoising image are input into the discriminator for processing, and finally the true or false judgment of the denoising image is output, the least square loss function is used to calculate the adversarial loss, and the loss function expression of the generator of the denoising network is:

[0031] L G =αL gen +βL canny +γL pixel ;

[0032]

[0033]

[0034]

[0035] In the above formula, L G represents the overall loss function of the generator G, and α, β and γ are hyperparameters, representing the weights of L gen , L canny and L pixel respectively, and the importance of the three loss functions is controlled by adjusting their values; L gen represents the loss function of the generator, represents the i-th denoising image generated by G in a data batch; L canny represents the L1 loss function of the gradient image processed by the Canny edge detection algorithm, and canny() is the Canny edge detection algorithm, represents the i-th conventional dose CT image in a data batch; L pixel represents and MSE loss function;

[0036] Loss function of discriminator of denoising network

[0037] In the above formula, L D denotes the loss function of the discriminator, D is the image domain discriminator, denotes the i-th regular dose CT image in a data batch, is the i-th denoised image generated by G in a data batch;

[0038] The Adam optimizer is used to update the weight parameters of the network, and the generator and the discriminator are alternately trained during the training process; the objective standard and the subjective judgment are combined to verify the denoising effect of the denoising model after training, and the learning rate and the corresponding hyperparameters are continuously adjusted to optimize the model;

[0039] Step five, using the denoising model with the highest test precision in the test set, inputting any low-dose CT image needing denoising processing into the denoising model, and outputting the denoised low-dose CT image.

[0040] Compared with the prior art, the above technical scheme has the following innovations and advantages:

[0041] The encoder-decoder structure generator including the Leformer module is constructed, compared with the prior art, the means of perceiving the local area and extracting the high-level features seriously depends on the network depth, the Leformer module based on the Transformer can calculate the relationship between any two positions, fully utilize the similarity of the global area, and the local enhancement module is used to enhance the ability of the Transformer structure to learn the local information, thereby enhancing the ability of the generator to learn the multi-scale information, so that the generated denoised image can better preserve the original overall structure and local texture details.

[0042] The discriminator based on the multi-scale convolutional network is constructed, the multi-scale feature extraction module in the discriminator can extract multi-scale features under the same scale to expand the receptive field of the network, so that the discriminator can more stably extract the overall organ structure information of the image under the condition of not increasing the number of convolutional layers too much, thereby enhancing the ability of the discriminator to distinguish the true and false of the input image, and then assisting the generator to generate realistic denoised images through the adversarial loss.

[0043] The loss function combined with the adversarial loss, the image domain loss and the image gradient domain loss is adopted, the image domain loss is used as the main loss function to train the denoising network to generate denoised images with excellent denoising effect, the adversarial loss is supplemented to enhance the structure and texture details of the denoised images, in addition, the Canny edge detection algorithm is used to calculate the image gradient, and the mean absolute error of the CT image in the image gradient domain is calculated to enhance the edge details of the denoised images.

[0044] This invention can remove noise and artifacts in low-dose CT images while preserving the overall structure, local texture details, and edge blurring of the original image, preventing excessive smoothing and loss of local information, thus facilitating clinical diagnosis. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall network structure of the present invention;

[0046] Figure 2 This is a schematic diagram of the generator module structure of the present invention;

[0047] Figure 3 yes Figure 2 A schematic diagram of the self-attention module structure with local enhancement;

[0048] Figure 4 This is a schematic diagram of the discriminator module structure of the present invention;

[0049] Figure 5 This is a schematic diagram of low-dose CT images in the embodiment;

[0050] Figure 6 yes Figure 5 A schematic diagram of a low-to-medium dose CT image after denoising. Detailed Implementation

[0051] The present invention will be explained in detail below with reference to the accompanying drawings.

[0052] like Figure 1 As shown, the LDCT image denoising method based on multi-scale self-attention generative adversarial networks has the following specific steps:

[0053] Step 1: Preprocess the CT image dataset:

[0054] The CT image dataset was divided into a training set, a validation set, and a test set. Each pair of paired images in the training and validation sets was randomly cropped into 10 64x64 image patches to obtain local information of the images and expand the sample size.

[0055] Step 2: Optimize the discriminator and generator:

[0056] The processed low-dose CT image is input into the generator to generate a denoised image, such as... Figure 2 As shown, specifically:

[0057] In the encoder stage: input low-dose CT image X a First, the tokenization module produces a token sequence T consisting of n one-dimensional tokens. a Then, Ta Input into the locally enhanced self-attention module (Leformer);

[0058] like Figure 3 As shown, in the Leformer module, T a It goes through two processing stages:

[0059] In the first stage, T a After passing through the Layer Normalization layer, the image enters the Multi-Head Self-Attention (MSA) module. This module utilizes the self-attention mechanism to calculate the relationship between any two locations within the features, fully leveraging the similarity of global regions to learn global information, thereby enhancing the generator's ability to learn the overall image structure. The output of this stage is T′. b ;

[0060] In the second stage, T′ b After the normalization layer, the code enters the Local Enhancement module. First, a Linear Projection Layer (MLP) is used to increase the feature dimension. Then, an inverse lexicalization module converts the word sequence into a two-dimensional feature map. Next, two consecutive 3x3 convolutions with residual connections combined with Leaky ReLU are used to extract local information from the two-dimensional features, thereby enhancing the generator's ability to learn local texture details of the image. Subsequently, a lexicalization module restores the two-dimensional feature map to a word sequence, and finally, a linear projection layer restores its feature dimension. The output at this stage is T. b ;

[0061] Next, we move to the downsampling layer; first, we use the anti-word morphology module to convert T... b Convert to two-dimensional feature map X b ∈R C×H×W Where C is the feature map X b The number of channels, H and W represent the height and width, respectively; a downsampling operation is performed using a convolutional layer with a kernel size of 3x3 and a stride of 2, at which point the size of the two-dimensional feature map becomes The lexicalization module is then used to restore the two-dimensional feature map into a lexical sequence T. c ;

[0062] T c Then, after passing through another identical Leformer module, the output T is obtained. d Then, after passing through a downsampling layer, the encoder output T is obtained. e ;

[0063] During the bottleneck phase, a Leformer module is used to capture a wider range of image structural information, T e After this stage, the output is T. f ;

[0064] Then the decoder stage is entered, T f First, an up-sampling layer is entered, using the de-tokenization module to convert T f After conversion into a two-dimensional feature map, a de-convolutional layer with a kernel size of 3x3 and a step size of 2 is used to perform up-sampling operation, and then the tokenization module is used to restore the two-dimensional feature map to the token sequence T g ;

[0065] There is a skip connection between the corresponding positions of the encoder and the decoder, which can avoid the problem of gradient disappearance and preserve more image structure and texture details in the decoder stage; therefore, T g is added to the output of the second Leformer module in the encoder T d , and then the Leformer module is entered for image restoration processing to obtain the output T h ;

[0066] Similarly, after an up-sampling layer, the output is added to the output of the first Leformer module in the encoder T b , and then the result is input to the last Leformer module, and then the output is restored to a two-dimensional feature map using the de-tokenization module, and finally the output is added to the low-dose CT image to generate the final denoised image;

[0067] The denoised image obtained and the corresponding regular-dose CT image are input into the discriminator to optimize the discriminator with the goal of minimizing the loss function of the discriminator, as shown in Figure 5 The discriminator includes three consecutive modules, specifically:

[0068] The first module is a convolutional layer with a kernel size of 3x3 and a step size of 2, a batch normalization layer (BN), and a Leaky ReLU activation function; it plays the role of a down-sampling layer, reducing the computational load of the network while also expanding the receptive field;

[0069] The second module is a multi-scale feature extraction module that extracts multi-scale features at the same scale to expand the receptive field in the module, thereby improving the discrimination ability of the discriminator;

[0070] The third module is a convolutional layer with a kernel size of 3x3 and a step size of 1, and a sigmoid activation function;

[0071] After the denoised image generated by the generator and the corresponding regular-dose CT image enter the discriminator, the final output is the discriminator's judgment of the input image's authenticity; the more similar the input image is to the regular-dose CT image, the closer the output is to 1; optimizing the discriminator means minimizing the loss function of the discriminator

[0072] In the above formula, L D represents the loss function of the discriminator, D is the image domain discriminator, represents the i-th regular dose CT image in a data batch, represents the i-th denoised image generated by G in a data batch;

[0073] The generator is optimized to minimize the generator loss, the image domain mean square error and the image gradient domain mean absolute error, and the loss function expression of the generator of the denoising network is:

[0074] L G = αL gen + βL canny + γL pixel ;

[0075]

[0076]

[0077]

[0078] In the above formula, L G represents the overall loss function of the generator G, α, β, γ are hyperparameters, respectively representing the weights of L gen , L canny and L pixel , and the importance of the three loss functions is controlled by adjusting their values; L gen represents the loss function of the generator, represents the i-th denoised image generated by G in a data batch; L canny represents the L1 loss function of the gradient image processed by the Canny edge detection algorithm, and canny() is the Canny edge detection algorithm, represents the i-th regular dose CT image in a data batch; L pixel represents the mean square error of and .

[0079] Step three, alternately train the discriminator and the generator:

[0080] Repeat step two, which involves generating a denoised image using the optimized generator, inputting the denoised image into the discriminator, optimizing the discriminator, and then using the optimized discriminator to optimize the generator. Repeat the above operations to train the discriminator and generator alternately until the maximum number of iterations set in the experiment is reached. Verify the denoising effect of the trained denoising model using a combination of objective standards and subjective judgment, and continuously adjust the learning rate and corresponding hyperparameters to optimize the model.

[0081] Step 4: Denoise the low-dose CT images:

[0082] Using a trained denoising network to perform... Figure 5 The low-dose CT images shown are denoised to generate images like... Figure 6 The CT image shown has good noise reduction.

[0083] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1.A method for LDCT image denoising based on a multi-scale self-attention generative adversarial network, characterized in that: Specifically comprising the following steps: Step one, constructing a low-dose CT image paired dataset: Acquire multiple sets of regular dose CT images of different parts, add Poisson noise to each image to simulate low dose CT images corresponding to the regular dose images; construct a CT image dataset (I LD , ND ), where I LD is a low dose CT image, I ND is a regular dose CT image matched with the low dose image; Step two, constructing a low-dose CT image denoising model: The denoising model is constructed based on a GAN framework, that is, the denoising model comprises a generator and a discriminator; a low-dose CT image I LD is mapped to a corresponding regular-dose CT image I ND by the generator G, to obtain a denoised image I gen ; Step 1, constructing a denoising model generator: The generator is a Transformer-based encoder-decoder structure: the encoder includes a tokenization module, two consecutive local enhanced self-attention modules combined with a down-sampling layer, the decoder includes two consecutive up-sampling layers combined with a local enhanced self-attention module and a de-tokenization module, and the encoder and the decoder are connected through a local enhanced self-attention module; there is a skip connection between the corresponding local enhanced self-attention modules in the encoder and the decoder to avoid the problem of gradient disappearance, and at the same time, more image structure and texture details can be preserved in the decoder stage; Step 2, constructing a denoising model discriminator: The discriminator includes three modules: the first module is a convolutional layer with a kernel size of 3x3 and a step of 2, a batch normalization layer and a Leaky ReLU activation function, which reduces the computational load of the network while also expanding the receptive field; the second module is a multi-scale feature extraction module, which extracts multi-scale features of the same scale of the image to expand the receptive field in the module, thereby improving the discrimination ability of the discriminator; the third module is a convolutional layer with a kernel size of 3x3 and a step of 1 and a sigmoid activation function; the output is the discriminator's judgment of the authenticity of the input image, and the more similar the input image is to the conventional dose CT image, the closer the output is to 1; Step three, data preprocessing: divide the dataset constructed in step one into training set, validation set and test set; randomly crop each pair of images in the training set and the validation set to a set size image block to obtain local information of the image and expand the sample size; Step four, training the denoising model and optimizing the model: According to the training mode of the GAN framework, the denoising model is trained, the low-dose CT image is input into the generator to obtain the generated denoised image, then the conventional dose CT image and the denoised image are input into the discriminator for processing, and finally the authenticity of the denoised image is output. The least squares loss function is used to calculate the adversarial loss, and the loss function expression of the generator of the denoising network is: L G = aL gen + bL canny + gL pixel ; In the above formula, L G represents the overall loss function of the generator G, α, β, γ are hyperparameters, respectively representing the weights of L gen , L canny and L pixel , and the importance of the three loss functions is controlled by adjusting the values thereof; L gen represents the loss function of the generator, represents the i-th denoised image generated by G in a data batch; L canny represents the L1 loss function of the gradient image processed by the Canny edge detection algorithm, and canny() is the Canny edge detection algorithm, represents the i-th regular dose CT image in a data batch; L pixel represents and MSE loss function; Loss function of the discriminator of the denoising network In the above formula, L D represents the loss function of the discriminator, D is the image domain discriminator, represents the i-th regular dose CT image in a data batch, is the i-th denoised image generated by G in a data batch; The Adam optimizer is used to update the weight parameters of the network, and the generator and the discriminator are trained alternately during the training process; the objective standard and subjective judgment are combined to verify the denoising effect of the denoising model after training, and the learning rate and the corresponding hyperparameters are adjusted to optimize the model; Step five, using the denoising model with the highest test accuracy in the test set, inputting any low-dose CT image that needs to be denoised into the denoising model, and outputting the denoised low-dose CT image. 2.The LDCT image denoising method based on multi-scale self-attention generative adversarial network of claim 1, wherein: The tokenization module in step 1 stretches the input CT image into a token sequence composed of n one-dimensional tokens, and the local enhanced self-attention module has two processing stages: In the first stage, the word sequence passes through the layer normalization layer and enters the multi-head self-attention module to output T' b In the multi-head self-attention module, the word sequence T a is multiplied by three different weight value matrices W4, W K , W V to obtain three types of tensors, respectively referred to as Q, K, and V, and then the output expression of the multi-head self-attention module is: Wherein, MSA represents the multi-head self-attention mechanism, is the length of the tensor K; In the second stage, the token sequence enters the local enhancement module after a layer normalization layer, first uses a linear projection layer to increase its feature dimension, then uses the detokenization module to convert the token sequence into a two-dimensional feature map, then uses two consecutive residual 3x3 convolution and Leaky ReLU combination to obtain local information in the two-dimensional feature, then uses the tokenization module to restore the two-dimensional feature map to the token sequence, and finally uses a linear projection layer to restore its feature dimension, output The expression of the token sequence through the Leformer module is: T' b = MSA(LN(T a ))+T a ; T b = LE(MLP(T' b ))+T' b ; in the above formula, LN is a layer normalization layer, and LE represents a local enhancement module; In the down-sampling layer, the down-sampling operation is implemented by using a convolution layer with a convolution kernel size of 3x3 and a step of 2; in the up-sampling layer, the up-sampling operation is implemented by using a de-convolution layer with a convolution kernel size of 3x3 and a step of 2. 3.The LDCT image denoising method based on multi-scale self-attention generative adversarial network of claim 1, wherein: The multi-scale feature extraction module described in step 2 specifically involves: firstly, performing convolution operations using a convolutional layer with 128 output channels and a kernel size of 1x1 to obtain feature X with 128 channels. a ; X a The feature is divided into four groups, each with 32 channels: X b X c X d X e The first set of features X b Keeping it unchanged, we can directly use it as the output to obtain X′. b The second set of features X c The output X′ is obtained after computation through a convolutional layer with a kernel size of 3x3, a stride of 1, and padding of 1. c The third set of features X d With the output X′ of the second group c After addition, the result is processed through a convolutional layer with the same kernel size of 3x3, stride of 1, and padding of 1 to obtain the output X′. d Similarly, the fourth set of features X e The output X′ of the third set of features d After addition, the result is processed through a convolutional layer with the same kernel size of 3x3, stride of 1, and padding of 1 to obtain the output X′. e The outputs of these four sets of features are stacked along the channel dimension; finally, a convolutional layer with 1 output channel and a kernel size of 1x1 is used to reduce the dimensionality of the stacked features to obtain the output X′. a This ultimately completes the fusion of multi-scale information; in this way, feature X... a It is approximately equivalent to simultaneously performing multiple convolution calculations of different sizes, obtaining receptive fields of different sizes, that is, being able to acquire features of multiple different scales at the same scale.

Images