Saliency detection and edge guidance optimization method based on multi-scale feature fusion

Abstract

The invention discloses a saliency detection and edge guidance optimization method based on multi-scale feature fusion, and solves the problems that a target saliency map is difficult to detect, the edge of the saliency map is fuzzy and the edge is unsmooth under a complex background in the existing saliency detection method. The saliency map detection work is completed by introducing a multi-scale feature fusion module; in consideration of weak edges and background noise, a CFRL edge loss function is designed, the loss function not only can ensure that the model has detection capability on edges with different sizes and proportions, but also adjusts an edge weighting mode to avoid meaningless weight increase on a background area, and edge detection is completed through a U-Net network based on the loss function; and in consideration of the problems of fuzzy boundary and unsmooth transition of the predicted saliency map obtained by the multi-scale feature fusion module, saliency map optimization is carried out through an ES-fusion module, and finally an optimized saliency map is obtained.

Description

Technical Field

[0001] The present invention relates to a method for detecting and processing an image, and in particular to a method for saliency detection and edge guidance optimization based on multi-scale feature fusion. Background Art

[0002] Identifying salient stimuli in the visual field is a key attention mechanism for humans. When viewing freely, our eyes tend to focus on areas in the scene that have unique changes in visual stimuli, such as bright colors, unique textures, or more complex semantic aspects. If a familiar face or any sudden movement appears, this mechanism will guide our eyes to focus on the salient information areas in the scene. In recent years, saliency detection based on this mechanism has played an increasingly important role in applications such as object recognition, image segmentation, and scene understanding. By accurately locating the salient areas in the image that the human eye or algorithm pays attention to, it can not only optimize the processing effect of subsequent tasks, but also significantly improve the overall computing efficiency.

[0003] The features extracted by traditional saliency detection methods lack sufficient semantic information and often show certain limitations in complex scenes. Because these methods mainly rely on contrast calculation and statistical analysis of low-level visual features (such as color, texture, brightness, etc.), although they can effectively handle scenes with simple backgrounds and single targets, the detection accuracy and robustness will be significantly reduced when dealing with complex backgrounds, occlusions and multi-target scenes.

[0004] In recent years, saliency detection methods based on deep learning have gradually become mainstream. By building an end-to-end deep neural network model, high-level features with strong representation capabilities can be learned in large-scale data, thereby greatly improving the accuracy and robustness of saliency detection. In 2017, researchers proposed a salient target detection model based on the encoder-decoder structure, UCF. However, the encoder will lose some effective salient features while filtering out irrelevant features, and the decoder will inevitably introduce some noise information during upsampling. In 2018, researchers combined the cyclic calculation structure of the recursive neural network with the encoder-decoder network, added a time series on the basis of UCF, and made the basic network repeatedly trained at different times. The saliency map output by the network at the previous moment was regarded as the prior knowledge of the network training at the next moment, and it was sent to the encoder together with the original image for processing to obtain the saliency map, which achieved good results. In order to further improve the accuracy of boundary detection of salient objects, in 2019, researchers maintained a bidirectional feedback connection between each submodule of the encoding and decoding parts of the U-Net model. At the same time, the model learned to extract boundaries from the saliency predictions of its own output to enhance its ability to process boundaries. With the development of deep neural networks, researchers have gradually applied deep feature information to salient object detection. However, in complex natural scenes, existing saliency detection methods based on deep learning still have problems such as difficulty in detecting saliency maps of targets in complex backgrounds, blurred edges of saliency maps, and uneven edges. Summary of the invention

[0005] The purpose of the present invention is to solve the technical problems of the existing saliency detection methods, such as the difficulty in detecting target saliency maps under complex backgrounds, blurred edges of saliency maps and uneven edges, and to provide a saliency detection and edge guidance optimization method based on multi-scale feature fusion.

[0006] In order to achieve the above-mentioned invention object, the present invention provides the following technical solutions: A saliency detection and edge guidance optimization method based on multi-scale feature fusion, which is special in that it includes the following steps: S1. Select the VGG-16 network without a fully connected layer as the backbone network, input the original image into it for preprocessing, and pass the preprocessed original image into the multi-scale feature fusion module; in the multi-scale feature fusion module, perform saliency map detection on the preprocessed original image to obtain a predicted saliency map; S2, based on the U-Net network, the CFRL edge loss function is used to perform edge detection on the original image to obtain an edge map; S3. The predicted saliency map and edge map are optimized through the ES-fusion module to obtain the optimized saliency map, completing the saliency detection and edge-guided optimization based on multi-scale feature fusion.

[0007] Furthermore, step S1 is specifically as follows: S1.1. Select the VGG-16 network without fully connected layers as the backbone network, input the original image into it for preprocessing, and send the preprocessed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer and block5_conv layer in the VGG-16 network to the multi-scale feature fusion module respectively; S1.2. In the multi-scale feature fusion module, 1×1 and 3×3 convolution kernels are used to perform convolution operations on the pre-processed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer and block5_conv layer, and batch normalization and relu function activation are performed to obtain the tensors y0 and y1 of each layer respectively; S1.3, use separable convolution to perform 5×5, 7×7, 9×9, 11×11, 13×13, and 15×15 convolution operations on the preprocessed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer, and block5_conv layer, respectively, and use the relu function to activate, and obtain the tensors y2, y3, y4, y5, y6, and y7 of each layer respectively; S1.4, first add the tensors y2 and y5, y3 and y6, y4 and y7 of each layer to synthesize the feature information of different levels, and then concatenate the added results with the tensors y0 and y1 of each layer to form feature maps of each layer containing multiple feature scales; S1.5, input the feature maps of each layer containing multiple feature scales into the channel attention module of the CBAM module in the multi-scale feature fusion module, perform global maximum pooling and global average pooling on the feature maps of each layer containing multiple feature scales in the spatial dimension, compress the spatial dimension to 1, and retain the channel information, so as to obtain the feature maps of each layer after global maximum pooling and global average pooling; S1.6, input the feature maps of each layer after global maximum pooling and global average pooling into the shared multi-layer perceptron to extract features; S1.7, add the features extracted from each layer, and activate them through the sigmoid function to obtain the final channel attention weights of each layer; S1.8, multiply the channel attention weight of each layer by the feature map of the corresponding layer after global maximum pooling and global average pooling, and obtain the weighted feature map of each layer after being processed by the channel attention module to focus on the important information of the channel; S1.9, input the weighted feature maps of each layer into the spatial attention module of the CBAM module, perform maximum pooling and average pooling on them in the channel dimension, compress the channel dimension to 1, and retain the spatial information, so as to obtain the weighted feature maps of each layer after maximum pooling and average pooling; S1.10, the weighted feature maps of each layer after maximum pooling and average pooling are connected along the channel dimension, and the channel dimension increases accordingly. Then, a convolution layer is used to extract features, and then the channel dimension is compressed to 1. Finally, the final spatial attention weights of each layer are obtained by sigmoid function activation; S1.11. Multiply the spatial attention weights of each layer by the weighted feature maps of the corresponding layers after maximum pooling and average pooling to obtain the weighted feature maps of each layer processed by the spatial attention module to focus on the spatial location of important information; S1.12. The weighted feature maps of each layer processed by the spatial attention module are gradually upsampled and fused at different scales to finally generate a high-resolution predicted saliency map.

[0008] Furthermore, in step S2, the CFRL edge loss function Including cross entropy loss function , multi-scale dice loss function And the DAFT loss function ; The cross entropy loss function The formula is expressed as:

[0009] in: is the total number of original images to be processed, For the A manually labeled mask image, For the Zhang predicted edge map; The multi-scale dice loss function The formula is expressed as:

[0010] in: is the weight, is the corresponding scale list, is an element in the corresponding scale list, for Dice loss at scale; The DAFT loss function The formula is expressed as:

[0011] in: is the weight after introducing the edge enhancement term, is the number of positive samples correctly identified, The number of negative samples identified as positive samples, that is, the number of false positive negative samples; The number of positive samples identified as negative samples, that is, the number of positive samples that are missed; To adjust Dynamic weight of the ratio, To adjust Dynamic weight of the ratio, is a constant, is a constant; The CFRL marginal loss function The formula is expressed as: .

[0012] Furthermore, step S3 is specifically as follows: S3.1, the predicted saliency map and edge map are first input into the channel attention module in the ES-fusion module and activated by the sigmoid function to obtain the channel weight; S3.2, multiply the channel weight by the predicted saliency map to obtain a weighted saliency map, input the weighted saliency map into the spatial attention module of the ES-fusion module and activate it with a sigmoid function to obtain a saliency map spatial mask; S3.3, multiplying the channel weights by the edge map to obtain a weighted edge map; S3.4. Add the result of multiplying the weighted edge map and the spatial mask of the saliency map to the weighted saliency map to obtain the optimized saliency map, thus completing the saliency detection and edge-guided optimization based on multi-scale feature fusion.

[0013] Furthermore, the corresponding scale list =[0.5,1,2], which means the size is halved, the original size, and the size is doubled respectively; Said The value is ; Said The value is 2.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a saliency detection and edge guidance optimization method based on multi-scale feature fusion. The VGG-16 network without a fully connected layer is used as the backbone network. The saliency map detection work is completed by introducing a multi-scale feature fusion module. Considering the weak edges and background noise, a CFRL edge loss function is designed. The loss function can not only ensure that the model has the ability to detect edges of different sizes and proportions, but also adjust the edge weighting method to avoid the meaningless weight increase of the background area. Based on the loss function, edge detection is completed through a U-Net network. Considering that the predicted saliency map obtained by the multi-scale feature fusion module has blurred boundaries and uneven transitions, an ES-fusion algorithm is used to optimize the edge guidance optimization method. The module optimizes the saliency map, puts the predicted saliency map and edge map into the channel attention module of the ES-fusion module and activates them through the sigmoid function to obtain the channel weight, multiplies the channel weight with the predicted saliency map to obtain the weighted saliency map, inputs the weighted saliency map into the spatial attention module of the ES-fusion module and activates them through the sigmoid function to obtain the saliency map spatial mask, then multiplies the channel weight with the edge map to obtain the weighted edge map, and finally adds the result of multiplying the weighted edge map with the saliency map spatial mask to the weighted saliency map to obtain the optimized saliency map. The saliency map generated in this way is more uniform, the target edge contour is clearer, and the segmentation effect of the target and the surrounding environment is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention; Figure 2 It is a schematic diagram of a CBAM module in an embodiment of a saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention; Figure 3 It is a schematic diagram of a channel attention module in a CBAM module in an embodiment of a saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention; Figure 4 It is a schematic diagram of a spatial attention module in a CBAM module in an embodiment of a saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention; Figure 5 A schematic diagram of a multi-scale feature fusion module in an embodiment of a saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention; Figure 6 FIG. 1 is a comparison diagram of edge detection effects using different loss functions in an embodiment of a saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention, wherein Figure 6 (a1) and Figure 6 (b1) is the original image, Figure 6 (a2) and Figure 6(b2) is the edge detection effect diagram after processing using the cross entropy loss function combined with the multi-scale dice loss function. Figure 6 (a3) and Figure 6 (b3) is the edge detection effect image processed by CFRL edge loss function; Figure 7 It is a schematic diagram of the ES-fusion module in the embodiment of the saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention; Figure 8 : is a comparison diagram of the effects of the original image, the manually marked mask image, the predicted saliency map and the optimized saliency map in the embodiment of the saliency detection and edge guidance optimization method based on multi-scale feature fusion of the present invention, wherein Figure 8 (a1), Figure 8 (a2) and Figure 8 (a3) is the original image, Figure 8 (b1), Figure 8 (b2) and Figure 8 (b3) is the manually marked mask image, Figure 8 (c1), Figure 8 (c2) and Figure 8 (c3) is the predicted saliency map, Figure 8 (d1), Figure 8 (d2) and Figure 8 (d3) is the optimized saliency map. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings and exemplary embodiments.

[0017] Reference Figure 1 As shown in the flowchart, the saliency detection and edge guidance optimization method based on multi-scale feature fusion provided by the present invention comprises the following steps: S1. Select the VGG-16 network without a fully connected layer as the backbone network, input the original image into it for preprocessing, and pass the preprocessed original image into the multi-scale feature fusion module; in the multi-scale feature fusion module, perform saliency map detection on the preprocessed original image to obtain a predicted saliency map; More specifically, step S1 is: S1.1. Select the VGG-16 network without fully connected layers as the backbone network, input the original image into it for preprocessing, and send the preprocessed original image output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer and block5_conv layer in the VGG-16 network to the multi-scale feature fusion module respectively; the schematic diagram of the multi-scale feature fusion module (MSFF Module) is shown as follows Figure 5 As shown; S1.2. In the multi-scale feature fusion module, 1×1 and 3×3 convolution kernels are used to perform convolution operations on the pre-processed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer and block5_conv layer, and batch normalization and relu function activation are performed to obtain the tensors y0 and y1 of each layer respectively; S1.3, use separable convolution to perform 5×5, 7×7, 9×9, 11×11, 13×13, and 15×15 convolution operations on the preprocessed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer, and block5_conv layer, respectively, and use the relu function to activate, and obtain the tensors y2, y3, y4, y5, y6, and y7 of each layer respectively; S1.4, first add the tensors y2 and y5, y3 and y6, y4 and y7 of each layer to synthesize the feature information of different levels, and then concatenate the added results with the tensors y0 and y1 of each layer to form feature maps of each layer containing multiple feature scales; S1.5, the feature maps of each layer containing multiple feature scales are respectively input into the channel attention module of the CBAM module in the multi-scale feature fusion module, and the feature maps of each layer containing multiple feature scales are respectively subjected to global maximum pooling and global average pooling in the spatial dimension, and the spatial dimension is compressed to 1, and the channel information is retained, so as to obtain the feature maps of each layer after global maximum pooling and global average pooling; wherein the CBAM module is as follows Figure 2 As shown, it includes a channel attention module and a spatial attention module connected in series, where the schematic diagram of the channel attention module is as follows Figure 3 As shown; S1.6, input the feature maps of each layer after global maximum pooling and global average pooling into the shared multi-layer perceptron to extract features; S1.7, add the features extracted from each layer, and activate them through the sigmoid function to obtain the final channel attention weights of each layer; S1.8, multiply the channel attention weight of each layer by the feature map of the corresponding layer after global maximum pooling and global average pooling, and obtain the weighted feature map of each layer after being processed by the channel attention module to focus on the important information of the channel; S1.9. Input the weighted feature maps of each layer into the spatial attention module of the CBAM module, perform maximum pooling and average pooling on them in the channel dimension, compress the channel dimension to 1, and retain the spatial information to obtain the weighted feature maps of each layer after maximum pooling and average pooling; the schematic diagram of the spatial attention module is as follows Figure 4 As shown; S1.10, the weighted feature maps of each layer after maximum pooling and average pooling are connected along the channel dimension, and the channel dimension increases accordingly. Then, a convolution layer is used to extract features, and then the channel dimension is compressed to 1. Finally, the final spatial attention weights of each layer are obtained by sigmoid function activation; S1.11. Multiply the spatial attention weights of each layer by the weighted feature maps of the corresponding layers after maximum pooling and average pooling to obtain the weighted feature maps of each layer processed by the spatial attention module to focus on the spatial location of important information; S1.12. The weighted feature maps of each layer processed by the spatial attention module are gradually upsampled and fused at different scales to finally generate a high-resolution predicted saliency map.

[0018] S2, based on the U-Net network, the CFRL edge loss function is used to perform edge detection on the original image to obtain an edge map; In step S2, the present invention designs a CFRL (Comprehensive Feature Refinement Loss) edge loss function , CFRL edge loss function It consists of three parts, including the cross entropy loss function , multi-scale dice loss function and DAFT loss function .

[0019] The cross entropy loss function It is used to measure the accuracy of pixel-level prediction, and its formula is expressed as:

[0020] in: is the total number of original images to be processed, For the A manually labeled mask image, For the Zhang predicted edge map; Multi-scale dice loss function The purpose is to introduce multi-scale image adjustment to enhance the sensitivity to edges of different scales. (manually labeled mask map) and (Predicted edge map) is resampled at different scales, corresponding to each scale , calculate the intersection sum:

[0021] in: For manually labeled mask images at scale The resampled image on To predict the edge map at scale Resampled image on ; Then for each scale calculate (Right now Dice loss at different scales):

[0022] in: is a smoothing term, and its value is 1 here to avoid the denominator being 0; Combined weight For different scales Weighted summation gives the formula for the multi-scale dice loss function:

[0023] in: is the weight, default , is a list of corresponding scales, and =[0.5,1,2], which respectively means the size is reduced by half, the original size, and the size is doubled. is an element in the corresponding scale list, for Dice loss at scale; DAFT Loss Function The design concept combines the idea of ​​dynamic weight adjustment of special difficult and easy samples in edge detection and the adaptive balance mechanism of positive and negative samples. It is suitable for tasks with a serious imbalance in the ratio of positive and negative samples and is very sensitive to small edge structures.

[0024] First, we introduce the basic Tversky Index function, which is used express:

[0025] in: is the number of positive samples correctly identified, is the number of negative samples identified as positive samples, that is, the number of false positive negative samples. is the number of positive samples identified as negative samples, that is, the number of positive samples that are missed. for The weight of for The weight of Because the weight of the basic Tversky Index function and is a fixed value, here we introduce dynamic weights: According to the false positive ( ) and false negatives ( ) is calculated to adjust Dynamic weight of proportion and used to adjust Dynamic weight of proportion :

[0026]

[0027] in: and is a constant with a value of 0.5. is a constant, and its value is ; Introducing dynamic weights based on the Tversky Index function:

[0028] The edge enhancement term is then introduced to give greater weight to the edge area:

[0029] in: is the weight after introducing the edge enhancement term, is a constant, with a value of 2. is the edge graph; Then, the Focal Loss function is combined to adjust the weight of the difficult samples, and the formula of the DAFT loss function is expressed as:

[0030] Right now:

[0031] in: To adjust Dynamic weight of the ratio, To adjust Dynamic weight of the ratio, is a constant, with a value of 2; Combining the above three loss functions, we can obtain the CFRL edge loss function designed by the present invention: : .

[0032] like Figure 6 As shown in the figure, it is a comparison chart of edge detection effects using different loss functions. Figure 6 (a1) and Figure 6 (b1) is the original image, Figure 6 (a2) and Figure 6(b2) is the edge detection effect diagram after processing using the cross entropy loss function combined with the multi-scale dice loss function. Figure 6 (a3) and Figure 6 (b3) is the edge detection effect diagram processed by CFRL edge loss function. It can be seen that CFRL edge loss function Compared with the cross entropy loss function With multi-scale dice loss function The combination effect is better, the ability to suppress background noise is relatively good, and the generated edge map is more coherent.

[0033] S3. The predicted saliency map and edge map are optimized through the ES-fusion module to obtain the optimized saliency map, completing the saliency detection and edge-guided optimization based on multi-scale feature fusion.

[0034] Step S3 is specifically as follows: S3.1. The predicted saliency map and edge map are first input into the channel attention module in the ES-fusion module and activated by the sigmoid function to obtain the channel weight. The schematic diagram of the ES-fusion module is shown in Figure 7 As shown; S3.2, multiply the channel weight by the predicted saliency map to obtain a weighted saliency map, input the weighted saliency map into the spatial attention module of the ES-fusion module and activate it with a sigmoid function to obtain a saliency map spatial mask; S3.3, multiplying the channel weights by the edge map to obtain a weighted edge map; S3.4. Add the result of multiplying the weighted edge map and the spatial mask of the saliency map to the weighted saliency map to obtain the optimized saliency map, thus completing the saliency detection and edge-guided optimization based on multi-scale feature fusion.

[0035] like Figure 8 As shown in Figure 1, it is a comparison of the original image, the manually marked mask image, the predicted saliency map, and the optimized saliency map. Figure 8 (a1), Figure 8 (a2) and Figure 8 (a3) is the original image, Figure 8 (b1), Figure 8 (b2) and Figure 8 (b3) is the manually marked mask image, Figure 8 (c1), Figure 8 (c2) and Figure 8 (c3) is the predicted saliency map, Figure 8 (d1), Figure 8 (d2) and Figure 8(d3) is the optimized saliency map. It can be seen that the optimized saliency map obtained after being processed by the saliency detection based on multi-scale feature fusion and edge-guided optimization method of the present invention is more uniform, the target edge contour is clearer, and the segmentation effect with the surrounding environment is better. This method proposes a new solution for the accuracy of saliency map detection and the retention of edge details, which is suitable for target segmentation and edge optimization needs in complex scenes.

[0036] The embodiments described above are merely descriptions of specific implementation methods of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A saliency detection and edge guidance optimization method based on multi-scale feature fusion, characterized in that: The following steps are involved: S1. Select the VGG-16 network without a fully connected layer as the backbone network, input the original image into it for preprocessing, and pass the preprocessed original image into the multi-scale feature fusion module; in the multi-scale feature fusion module, perform saliency map detection on the preprocessed original image to obtain a predicted saliency map; S2, based on the U-Net network, the CFRL edge loss function is used to perform edge detection on the original image to obtain an edge map; S3. The predicted saliency map and edge map are optimized through the ES-fusion module to obtain the optimized saliency map, completing the saliency detection and edge-guided optimization based on multi-scale feature fusion.

2. The method for saliency detection and edge guidance optimization based on multi-scale feature fusion according to claim 1, characterized in that: Step S1 is specifically as follows: S1.

1. Select the VGG-16 network without fully connected layers as the backbone network, input the original image into it for preprocessing, and send the preprocessed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer and block5_conv layer in the VGG-16 network to the multi-scale feature fusion module respectively; S1.

2. In the multi-scale feature fusion module, 1×1 and 3×3 convolution kernels are used to perform convolution operations on the pre-processed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer and block5_conv layer, and batch normalization and relu function activation are performed to obtain the tensors y0 and y1 of each layer respectively; S1.3, use separable convolution to perform 5×5, 7×7, 9×9, 11×11, 13×13, and 15×15 convolution operations on the preprocessed original images output by the block2_conv2 layer, block3_conv3 layer, block4_conv3 layer, and block5_conv layer, respectively, and use the relu function to activate, and obtain the tensors y2, y3, y4, y5, y6, and y7 of each layer respectively; S1.4, first add the tensors y2 and y5, y3 and y6, y4 and y7 of each layer to synthesize the feature information of different levels, and then concatenate the added results with the tensors y0 and y1 of each layer to form feature maps of each layer containing multiple feature scales; S1.5, input the feature maps of each layer containing multiple feature scales into the channel attention module of the CBAM module in the multi-scale feature fusion module, perform global maximum pooling and global average pooling on the feature maps of each layer containing multiple feature scales in the spatial dimension, compress the spatial dimension to 1, and retain the channel information, so as to obtain the feature maps of each layer after global maximum pooling and global average pooling; S1.6, input the feature maps of each layer after global maximum pooling and global average pooling into the shared multi-layer perceptron to extract features; S1.7, add the features extracted from each layer, and activate them through the sigmoid function to obtain the final channel attention weights of each layer; S1.8, multiply the channel attention weight of each layer by the feature map of the corresponding layer after global maximum pooling and global average pooling, and obtain the weighted feature map of each layer after being processed by the channel attention module to focus on the important information of the channel; S1.9, input the weighted feature maps of each layer into the spatial attention module of the CBAM module, perform maximum pooling and average pooling on them in the channel dimension, compress the channel dimension to 1, and retain the spatial information, so as to obtain the weighted feature maps of each layer after maximum pooling and average pooling; S1.10, the weighted feature maps of each layer after maximum pooling and average pooling are connected along the channel dimension, and the channel dimension increases accordingly. Then, a convolution layer is used to extract features, and then the channel dimension is compressed to 1. Finally, the final spatial attention weights of each layer are obtained by sigmoid function activation; S1.

11. Multiply the spatial attention weights of each layer by the weighted feature maps of the corresponding layers after maximum pooling and average pooling to obtain the weighted feature maps of each layer processed by the spatial attention module to focus on the spatial location of important information; S1.

12. The weighted feature maps of each layer processed by the spatial attention module are gradually upsampled and fused at different scales to finally generate a high-resolution predicted saliency map.

3. The method for saliency detection and edge guidance optimization based on multi-scale feature fusion according to claim 2, characterized in that: In step S2, the CFRL edge loss function Including cross entropy loss function , multi-scale dice loss function And the DAFT loss function ; The cross entropy loss function The formula is: ； in: is the total number of original images to be processed, For the A manually labeled mask image, For the Zhang predicted edge map; The multi-scale dice loss function The formula is: ； in: is the weight, is the corresponding scale list, is an element in the corresponding scale list, for Dice loss at scale; The DAFT loss function The formula is: ； in: is the weight after introducing the edge enhancement term, is the number of positive samples correctly identified, The number of negative samples identified as positive samples, that is, the number of false positive negative samples; The number of positive samples identified as negative samples, that is, the number of positive samples that are missed; To adjust Dynamic weight of the ratio, To adjust Dynamic weight of the ratio, is a constant, is a constant; The CFRL marginal loss function The formula is: 。 4. The method for saliency detection and edge guidance optimization based on multi-scale feature fusion according to claim 3, characterized in that: Step S3 is specifically as follows: S3.1, the predicted saliency map and edge map are first input into the channel attention module in the ES-fusion module and activated by the sigmoid function to obtain the channel weight; S3.2, multiply the channel weight by the predicted saliency map to obtain a weighted saliency map, input the weighted saliency map into the spatial attention module of the ES-fusion module and activate it with a sigmoid function to obtain a saliency map spatial mask; S3.3, multiplying the channel weights by the edge map to obtain a weighted edge map; S3.

4. Add the result of multiplying the weighted edge map and the spatial mask of the saliency map to the weighted saliency map to obtain the optimized saliency map, thus completing the saliency detection and edge-guided optimization based on multi-scale feature fusion.

5. The method for saliency detection and edge guidance optimization based on multi-scale feature fusion according to claim 3, characterized in that: The corresponding scale list =[0.5,1,2], which means the size is halved, the original size, and the size is doubled respectively; Said The value is ; Said The value is 2.

Images