Multi-mode optical fiber imaging method based on dual-encoder network and attention mechanism

Abstract

The invention discloses a multimode optical fiber imaging method based on a double-encoder network and an attention mechanism. A parallel double-encoder structure is formed by combining a Transform encoder and a convolution encoder, and meanwhile, a CBAM attention mechanism is introduced, so that a DEANet network model is constructed. According to the method, a convolution encoder is adopted to capture local details, and meanwhile, a Transform encoder is utilized to establish global feature association and high-frequency feature distribution to make up for the problem that the sensing range is limited. In addition, collaborative extraction and fusion of multi-scale information of channels and spaces are performed in a feature space through an attention module (CBAM). Compared with a traditional convolutional network, the method has the advantages that local details and a global structure are considered, high-fidelity reconstruction of a complex target structure and edge details in multimode optical fiber imaging is realized, and good robustness is kept after system disturbance.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computational imaging, and particularly relates to a multi-mode optical fiber imaging method based on a double-encoder network and an attention mechanism. BACKGROUND

[0002] In the field of minimally invasive medical devices, multi-mode optical fibers have become an important solution to realize ultra-thin endoscopes due to their excellent light guiding performance and miniaturization advantages. However, due to mode dispersion, multi-mode optical fibers have two inherent defects. One is that the transmitted light field exhibits strong scattering medium characteristics, which ultimately degrades the target optical information into a random speckle pattern. The second is that small disturbances such as temperature fluctuations and mechanical vibrations in the experimental environment will cause significant changes in the propagation parameters. Traditional optical processing methods such as transmission matrix, wavefront shaping, and phase compensation, although can achieve a certain degree of image restoration, all face the technical bottlenecks of complicated calibration process and high measurement repeatability requirements, which seriously restrict the practical performance of the system.

[0003] While the speckle decoding technology based on deep learning has made breakthrough progress and can achieve end-to-end image reconstruction, there are still two key defects: one is the lack of edge sharpness and texture fidelity of the reconstructed image; the second is the lack of robustness of the system to external disturbances in dynamic detection scenarios. Among them, the traditional single-encoder convolutional neural network can effectively extract local features of speckle images, but has limitations in modeling global structure and high-frequency feature distribution. These problems seriously hinder the practical application of multi-mode optical fiber imaging in precision medical diagnosis. SUMMARY

[0004] The application proposes a multi-mode optical fiber imaging method based on a double-encoder network and an attention mechanism, which solves the problem that the prior art cannot balance image local details and global structure when imaging multi-mode optical fibers, realizes high-fidelity reconstruction of complex target structures and edge details in multi-mode optical fiber imaging, and maintains good robustness after system disturbance.

[0005] The technical solution of the application is as follows: a multi-mode optical fiber imaging method based on a double-encoder network and an attention mechanism, comprising the following steps:

[0006] Step 1: Use images in the MNIST dataset as original images, adjust the size of all original images to h*w, obtain a target image dataset, and go to step 2.

[0007] Step 2, use K target images in the target image data set as labels; K target images projected by the spatial light modulator SLM in turn form K speckles at the exit end after being transmitted by the single multimode optical fiber imaging system without disturbance, K original speckle images are collected, 500 < K < 20000; turn to step 3.

[0008] Step 3, process the collected original speckle images, first adjust the size to h * w, then normalize to obtain normalized speckle images; use K pairs of data formed by labels and normalized speckle images to construct a speckle data set, 500 < K < 20000; turn to step 4.

[0009] Step 4, on the basis of UNet, combine the Transformer encoder and the CBAM attention mechanism to construct a double-encoder network DEANet with an attention module.

[0010] DEANet includes a convolutional encoder, a Transformer encoder, a CBAM attention module and a convolutional decoder;

[0011] Turn to step 5.

[0012] Step 5, divide the speckle data set into a training data set and a verification data set in a ratio of 8:2, train DEANet using the training data set, determine the disturbance-free model weight, and then obtain the trained DEANet model; turn to step 6.

[0013] Step 6, verify the accuracy and generalization ability of the DEANet model using the verification data set, and turn to step 7.

[0014] Step 7, under the disturbance condition, use the single multimode optical fiber imaging system to collect speckle images for testing, which are preprocessed and sent into the DEANet model to test the robustness of the DEANet model.

[0015] Compared with the prior art, the present application has the following advantages:

[0016] (1) The double-encoder structure is used to extract local and global features in parallel, the attention mechanism is combined for feature information fusion, and high-fidelity reconstruction of complex structure edges and texture details is realized.

[0017] (2) The system shows good robustness under complex conditions such as different laser powers, different speckle image resolutions, and different multimode optical fiber deformation states. BRIEF DESCRIPTION OF DRAWINGS

[0018] Fig. 1 A flowchart of a multimode optical fiber imaging method based on a double-encoder network and an attention mechanism.

[0019] Fig. 2 Figure 1 is a diagram of an optical system of the present application.

[0020] Fig. 3 Figure 2 is a diagram of the structure of a DEANet network of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0022] It should be noted that all directionality indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directionality indications also change accordingly.

[0023] In addition, the description such as "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features; in the description of the present application, "a plurality of" means at least two, for example, two, three, etc., unless otherwise explicitly and specifically limited.

[0024] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixing" and the like should be understood in a broad sense, for example, "fixing" can be fixed connection, or detachable connection, or integral; "connection" can be mechanical connection, or electrical connection; for those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0025] In addition, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, and is not within the scope of protection claimed by the present application.

[0026] The specific embodiments and the technical difficulties and points of the present application will be further introduced below in combination with the design examples;

[0027] In combination with Figs. 1-3 , the multi-mode optical fiber imaging method based on a double-encoder network and an attention mechanism provided by the present application has the following steps:

[0028] Step 1, using the images in the MNIST dataset as original images, resizing all original images to h*w to obtain a target image dataset, and go to Step 2.

[0029] Step 2, using K target images in the target image dataset as labels; the K target images projected by the spatial light modulator SLM in turn are transmitted through the undisturbed single multimode optical fiber imaging system to form K speckles at the exit end, and K original speckle images are collected, 500<K<20000.

[0030] The K target images projected by the spatial light modulator SLM in turn are transmitted through the undisturbed single multimode optical fiber imaging system to form K speckles at the exit end, and K original speckle images are collected, combined with Fig. 2 Specifically as follows:

[0031] The single multimode optical fiber imaging system comprises,

[0032] A laser for emitting continuous wave laser with a wavelength of 532 nm;

[0033] A beam expander, a polarizer, and a spatial light modulator are sequentially arranged along the light emitting direction of the laser.

[0034] A first lens, a second lens, a first objective lens, a single multimode optical fiber, a manual displacement stage placed below the multimode optical fiber, a second objective lens, and a CMOS camera are sequentially arranged along the light emitting direction of the spatial light modulator.

[0035] The continuous wave laser with a wavelength of 532 nm is used as an illumination light source; the laser beam is expanded and collimated by the expander to obtain a collimated beam, ensuring the uniformity of the beam before reaching the next stage; the collimated beam then passes through the polarizer and is incident on the spatial light modulator, which displays the target image and modulates the incident light accordingly; the modulated beam is transmitted through the 4F imaging system composed of the first lens and the second lens; the above 4F system guides the beam to the first objective lens, which couples the light into the single multimode optical fiber; the light is scattered in the multimode optical fiber, and the scattered light is collected by the second objective lens to form a speckle; finally, the speckle is captured by the CMOS camera to obtain K original speckle images.

[0036] Go to Step 3.

[0037] Step 3, process the collected original speckle images, first resize them to h*w, then normalize them to obtain normalized speckle images; use K pairs of data formed by labels and normalized speckle images to construct a speckle dataset, 500<K<20000; go to Step 4.

[0038] Step 4, combination Fig. 3 On the basis of UNet, a dual-encoding network DEANet with attention module is constructed by combining a Transformer encoder and a CBAM attention mechanism.

[0039] The DEANet includes a convolutional encoder, a Transformer encoder, a CBAM attention module, and a convolutional decoder.

[0040] The UNet includes four down-sampling and four up-sampling stages, each of which is composed of a convolutional module, a skip connection and a max-pooling layer, and a deconvolution operation; each convolutional module includes one linear activation unit and two convolutional kernels; the down-sampling factor of the max-pooling layer is 2; the up-sampling factor of the deconvolution operation is 2; the inter-layer skip connection is achieved by concatenating the corresponding high-dimensional feature images in the channel direction in the up-sampling and down-sampling processes; the down-sampling part of the UNet serves as the convolutional encoder of the DEANet; the output feature image thereof, together with the output feature image of the Transformer encoder, is input into the CBAM attention module to obtain an output result after feature enhancement; the up-sampling part of the UNet serves as the convolutional decoder to decode the output result and obtain the output image of the DEANet.

[0041] The Transformer encoder is specifically as follows:

[0042] Firstly, the normalized speckle image is divided into a plurality of non-overlapping blocks of fixed size, each of which is flattened into a one-dimensional vector and then mapped to a fixed-dimensional feature space through linear projection to form an initial block embedding; subsequently, to preserve the spatial information of the image, a learnable position encoding is added to each initial block embedding to constitute a final input block embedding sequence; finally, all block embedding sequences are sequentially input into a 12-layer standard Transformer encoder stack, each layer of which includes a layer normalization module, a multi-head self-attention mechanism MSA, and a multi-layer perceptron MLP; the output of the last layer stack is used as the global feature representation of the entire encoder branch; the CBAM attention mechanism is composed of two sub-modules, a channel attention module CAM and a spatial attention module SAM; the channel attention module CAM is used to extract channel attention weights, and the spatial attention module SAM is used to extract spatial attention weights; the input is sequentially weighted and summed to obtain an output result after feature enhancement.

[0043] Go to step 5.

[0044] Step 5, divide the speckle dataset into a training dataset and a validation dataset in a ratio of 8:2, use the training dataset to train the DEANet, determine the undisturbed model weight, and then obtain the trained DEANet model.

[0045] The DEANet is trained by using a training data set to determine the undisturbed model weight, and then a trained DEANet model is obtained, specifically as follows:

[0046] The normalized speckle image is simultaneously sent into the convolutional encoder and the Transformer encoder; in the convolutional encoder, 4 times of down-sampling are experienced, and after each operation, the size of the picture is reduced and the number of feature channels is increased, and finally at the bottom of the network, a down-sampled feature map A l :

[0047] A k =Pool(σ1(Conv(A k-1 )), k = 1, 2, 3, 4,

[0048] Pool represents the maximum pooling operation, σ1 represents the Relu activation function, and Conv represents the convolutional block; k represents the number of down-sampling times of the UNet.

[0049] In the Transformer encoder, the normalized speckle image is subjected to image slice and block embedding, position embedding and 12-layer standard Transformer encoding stack to obtain a feature map B l :

[0050] B' l =MSA(LN(B l-1 ))+B l-1 , l = 1, 2, 3, … 12,

[0051] B l =MLP(LN(B' l ))+B' l , l = 1, 2, 3, … 12,

[0052] wherein B' l represents the output feature of the multi-head self-attention MSA, and l represents the number of layers of the Transformer encoding stack.

[0053] LN is a normalization algorithm, and the feature map B l represents the encoded image of each patch.

[0054] The feature map B l output by the Transformer encoder will be spliced with the feature map A l output by the convolutional encoder along the channel direction to obtain a spliced feature map F:

[0055] F = Cat(A l , B l ),

[0056] Cat(·) represents a concatenation operation, and the concatenated feature map F will be input into the CBAM attention mechanism; in the CBAM attention mechanism, F first passes through the CAM output channel attention weight M c (F), as follows:

[0057] M c (F) = σ2(MLP(AvgPool(F)) + MLP(MaxPool(F))),

[0058] wherein σ2 represents a Sigmoid activation function, AvgPool(·) represents a global average pooling, MaxPool(·) represents a global maximum pooling; MLP(·) represents a multi-layer perceptron; Mc is used for channel-wise weighting on the concatenated feature map F to obtain a channel-enhanced feature map F'; the SAM performs maximum pooling and average pooling on F' in the channel dimension to obtain two spatial mapping maps respectively; next, the two spatial mappings are concatenated and fed into a 7x7 convolution; the convolution result is subjected to a Sigmoid activation function to obtain a spatial attention weight Ms:

[0059] M s (F') = σ2(f 7×7 ([AvgPool(F'); MaxPool(F')]),

[0060] wherein f 7×7 represents a 7x7 convolution operation; [:;:] represents concatenation along the channel dimension; Ms is used for weighted summation on F' to generate a final attention-enhanced feature map E; subsequently, E is input into a convolutional decoder and undergoes 4 times of upsampling, each time of upsampling including twice of convolution plus an activation function and once of deconvolution operation, so as to realize the enlargement of the size of the feature map and the reduction of the number of feature channels; in addition, after one time of upsampling, the obtained result will be merged with the feature map corresponding to the downsampling; the expression is as follows:

[0061] E l = TConv(σ1(Conv(Cat(A 4-g ,E g-1 )), g = 1, 2, 3, 4,

[0062] wherein TConv represents a deconvolution operation, σ1 represents a Relu activation function, Conv represents a convolution block, and g represents the number of times of upsampling of the UNet; Cat(·) represents concatenation along the channel; the output result E l undergoes a 1x1 convolution kernel to reduce the number of channels to 1 to obtain a final output result E out ; during training, a mean square error loss function Loss MSE is used:

[0063]

[0064] wherein, is the average value of the output result, E GT is the target image, i.e. the label, (x, y) is the pixel coordinate, m is the total number of pixel points in the length direction of the image, and n is the total number of pixel points in the width direction of the image.

[0065] The training process is a plurality of iterations after back propagation and loss function calculation; during the network training process, the learning rate of the network is set to 0.0001, and the dropout is set to 0.1; a total of 100 epochs are trained.

[0066] Step 6 is entered.

[0067] Step 6, the accuracy and generalization ability of the DEANet model are verified by using the verification data set, and step 7 is entered.

[0068] Step 7, under the disturbance condition, the speckle image used for testing is collected by using the single multimode optical fiber imaging system, and after preprocessing, it is sent into the DEANet model to test the robustness of the DEANet model.

[0069] Under the disturbance condition, the speckle image used for testing is collected by using the single multimode optical fiber imaging system, and after preprocessing, it is sent into the DEANet model to test the robustness of the DEANet model. The specific content is as follows: after adjusting the laser power of the laser, changing the resolution of the speckle image collected by the CMOS camera, using a manual displacement table to change the deformation state of the multimode optical fiber, the spatial light modulator projects the target image data set in real time, and the original speckle image used for testing is collected after passing through the multimode optical fiber. After preprocessing, the input speckle image used for testing is obtained. The input speckle image used for testing is sent into the trained DEANet model for calculation and imaging to test the robustness of the DEANet model.

[0070] Embodiment 1

[0071] Step 1, using the images in the MNIST data set as original images, adjusting the size of all original images, and uniformly adjusting the size to 256*256, obtaining a target image data set, and entering step 2.

[0072] Step 2, using 10,000 target images in the target image data set as labels; 10,000 target images projected by the spatial light modulator SLM in turn are transmitted by the single multimode optical fiber imaging system without disturbance to form 10,000 speckles at the exit end, and 10,000 original speckle images are collected; step 3 is entered.

[0073] Step 3, the collected original speckle image is processed, first the size is adjusted to 256*256, then normalized to obtain a normalized speckle image; 10000 pairs of data formed by labels and normalized speckle images are used to construct a speckle dataset; go to step 4.

[0074] Step 4, a double-encoding network DEANet with attention module is constructed based on UNet combined with Transformer encoder and CBAM attention mechanism.

[0075] DEANet includes a convolutional encoder, a Transformer encoder, a CBAM attention module, and a convolutional decoder.

[0076] Go to step 5.

[0077] Step 5, the speckle dataset is divided into training dataset and validation dataset in the ratio of 8:2, DEANet is trained using the training dataset, the undisturbed model weight is determined, and then the trained DEANet model is obtained; go to step 6.

[0078] Step 6, the accuracy and generalization ability of the DEANet model are verified using the validation dataset, and go to step 7.

[0079] Step 7, under the disturbance condition, the speckle image for testing is collected using a single multimode optical fiber imaging system, and after preprocessing, it is sent into the DEANet model to test the robustness of the DEANet model.

[0080] Under the disturbance condition, the speckle image for testing is collected using a single multimode optical fiber imaging system, and after preprocessing, it is sent into the DEANet model to test the robustness of the DEANet model. The specific content is as follows: adjust the laser power of the laser, change the resolution of the speckle image collected by the CMOS camera, use a manual displacement table to change the deformation state of the multimode optical fiber, and then the spatial light modulator projects the target image dataset in real time. After the multimode optical fiber, the original speckle image for testing is collected. After preprocessing, the input speckle image for testing is obtained. The input speckle image for testing is sent into the trained DEANet model for calculation and imaging to test the robustness of the DEANet model.

Claims

1. A multi-modality fiber imaging method based on dual-encoder network and attention mechanism, characterized in that, The method comprises the following steps: Step 1, using the images in the MNIST dataset as original images, resizing all the original images to h*w to obtain a target image dataset, and going to step 2; Step 2, using K target images in the target image dataset as labels; K target images projected by a spatial light modulator (SLM) in turn are transmitted through a single multimode optical fiber imaging system without disturbance to form K speckles at the exit end, and K original speckle images are collected, 500 < K < 20000; go to step 3; Step 3, processing the collected original speckle images, first resizing to h*w, then normalizing to obtain normalized speckle images; using K pairs of data formed by labels and normalized speckle images, constructing a speckle dataset, 500 < K < 20000; go to step 4; Step 4, constructing a double-encoding network (DEANet) with an attention module based on UNet combined with a Transformer encoder and a CBAM attention mechanism; The DEANet comprises a convolutional encoder, a Transformer encoder, a CBAM attention module and a convolutional decoder; Go to step 5; Step 5, dividing the speckle dataset into a training dataset and a validation dataset in a ratio of 8:2, training the DEANet using the training dataset to determine the disturbance-free model weight, and then obtaining the trained DEANet model; go to step 6; Step 6, verifying the accuracy and generalization ability of the DEANet model using the validation dataset, and going to step 7; Step 7, under the condition of disturbance, using a single multimode optical fiber imaging system to collect speckle images for testing, and sending the preprocessed speckle images into the DEANet model to test the robustness of the DEANet model.

2. The multi-modal fiber imaging method based on dual-encoder network and attention mechanism of claim 1, wherein, In step 2, K target images projected by a spatial light modulator (SLM) in turn are transmitted through a single multimode optical fiber imaging system without disturbance to form K speckles at the exit end, and K original speckle images are collected, specifically as follows: The single multimode optical fiber imaging system comprises, a laser for emitting continuous laser with a wavelength of 532 nm; a beam expander, a polarizer and a spatial light modulator arranged in turn along the light emitting direction of the laser; a first lens, a second lens, a first objective lens, a single multimode optical fiber, a manual displacement stage placed below the multimode optical fiber, a second objective lens and a CMOS camera arranged in turn along the light emitting direction of the spatial light modulator; A 532 nm continuous wave laser is used as the illumination source; the laser beam is expanded and collimated by an expander to obtain a collimated beam, ensuring the uniformity of the beam before reaching the next stage; the collimated beam then passes through a polarizer and is incident on a spatial light modulator, which displays the target image and modulates the incident light accordingly; the modulated beam is transmitted through a 4F imaging system composed of a first lens and a second lens; the 4F system directs the beam to a first objective lens, which couples the light into a single multimode optical fiber; the light scatters in the multimode optical fiber during transmission, and the output scattered light is collected by a second objective lens to form a speckle; finally, the speckle is captured by a CMOS camera to obtain K original speckle images.

3. The method of claim 2, wherein the method is based on a dual-encoder network with attention mechanism. In step 4, the UNet includes four down-sampling and four up-sampling stages, each consisting of a convolution module, a max-pooling layer, and a deconvolution operation; each convolution module includes a linear activation unit and two convolution kernels; the down-sampling factor of the max-pooling layer is 2; the up-sampling factor of the deconvolution operation is 2; the inter-layer skip connection is achieved by concatenating the corresponding high-dimensional feature images in the channel direction during down-sampling and up-sampling; the down-sampling part of the UNet serves as the convolutional encoder of the DEANet; The output feature map is fused with the output feature map of the Transformer encoder and input into the CBAM attention module to obtain the output result after feature enhancement; the up-sampling part of the UNet serves as the convolutional decoder to decode the output result and obtain the output image of the DEANet.

4. The multi-modal fiber imaging method based on dual-encoder network and attention mechanism of claim 3, wherein, In step 4, the Transformer encoder is as follows: First, the normalized speckle image is divided into several fixed-size non-overlapping blocks, each block is flattened into a one-dimensional vector and then mapped to a fixed-dimensional feature space through linear projection to form an initial block embedding; Then, to preserve the spatial information of the image, a learnable position encoding is added to each initial block embedding to form a final block embedding sequence; finally, all block embedding sequences are input into a 12-layer standard Transformer encoder stack, each layer including a layer normalization module, a multi-head self-attention mechanism MSA, and a multi-layer perceptron MLP; the output of the last layer stack is used as the global feature representation of the entire encoder branch; The CBAM attention mechanism consists of two sub-modules: a channel attention module CAM and a spatial attention module SAM; the channel attention module CAM is used to extract channel attention weights, and the spatial attention module SAM is used to extract spatial attention weights; the input is weighted and summed to obtain the output result after feature enhancement.

5. The method of claim 4, wherein the method is based on a dual-encoder network with attention mechanism. In step 5, the DEANet is trained using the training data set to determine the undisturbed model weight, and then the trained DEANet model is obtained, as follows: The normalized speckle image is simultaneously sent into a convolutional encoder and a Transformer encoder; in the convolutional encoder, 4 times of down-sampling are experienced, the size of the picture is reduced after each operation, the number of feature channels is increased, and finally at the bottom of the network, a down-sampled feature map A is obtained l : A k = Pool(σ1(Conv(A k-1 )), k = 1,2,3,4, Pool represents the max-pooling operation, sigma1 represents the Relu activation function, and Conv represents the convolution block; k represents the number of down-sampling of UNet; In the Transformer encoder, the normalized speckle image manager image slices with block embeddings, position embeddings, and a 12-layer standard Transformer encoding stack to get feature maps B l : B' l = MSA(LN(B l-1 ))+ B l-1 , l = 1, 2, 3,... 12, B l = MLP(LN(B l ))+ B l , l = 1, 2, 3,... 12, wherein B l denotes the output feature of multi-head self-attention MSA, and l denotes the number of layers of the Transformer encoding stack. LN is a normalization algorithm, feature map B l represents the encoded image of each patch; Feature map B output by the Transformer encoder l Feature map A output by the convolutional encoder l Spliced along the channel direction to obtain a spliced feature map F: F = Cat(A l , B l ), Cat(·) denotes a concatenation operation, and the concatenated feature map F will be input to the CBAM attention mechanism; in the CBAM attention mechanism, F first passes through the CAM output channel attention weight M c (F), as follows: M c (F) = σ2(MLP(AvgPool(F)) + MLP(MaxPool(F))), Wherein, σ2 represents the Sigmoid activation function, AvgPool(·) represents global average pooling, MaxPool() represents global maximum pooling; MLP(·) represents multi-layer perceptron; Mc is used for channel-by-channel weighting on the spliced feature map F to obtain a channel-enhanced feature map F'; SAM performs maximum pooling and average pooling on F' in the channel dimension to obtain two spatial mapping maps respectively; Next, the two spatial mappings are spliced and fed into a 7*7 convolution; the convolution result is subjected to a Sigmoid activation function to obtain a spatial attention weight Ms: M s (F') = σ2(f 7×7 ([AvgPool(F'); MaxPool(F'))], where f 7×7 denotes a 7x7 convolution operation; [:; :] denotes concatenation along the channel dimension; F' is weighted summed using Ms to generate the final attention enhanced feature map E; subsequently E is input to a convolutional decoder, which goes through 4 upsamplings, each of which includes twice convolution plus activation function and once deconvolution operation, to realize the enlargement of the size of the feature map and the reduction of the number of feature channels; in addition, after each upsampling, the obtained result is merged with the feature map corresponding to the downsampling; which is expressed as follows: E l = TConv(σ1(Conv(Cat(A 4-g ,E g-1 )), g = 1,2,3,4, Wherein, TConv represents the deconvolution operation, σ1 represents the Relu activation function, Conv represents the convolution block, g represents the number of up-sampling of UNet; Cat(·) represents the channel splicing; the output result E l After a 1x1 convolution kernel is experienced to reduce the channel number to 1, the final output result E is obtained out ; during training, the mean square error loss function Loss MSE : wherein, is the average of the output results, E GT is the target image, i.e. the label, (x, y) are the pixel coordinates, m is the total number of pixel points in the length direction of the image, and n is the total number of pixel points in the width direction of the image.

6. The method of claim 5, wherein the method is based on a dual-encoder network with attention mechanism. The training process is a plurality of iterations after reverse propagation and loss function calculation; during the network training process, the learning rate of the network is set to 0.0001, and the dropout is set to 0.1; a total of 100 epochs are trained.

Images