An edge enhancement based wide decoder salient object detection method and system

Abstract

The application discloses a kind of based on edge enhancement's wide decoder saliency target detection method and system, comprising: obtaining high-resolution image dataset and carrying out downsampling processing to target high-resolution image in dataset, obtain target downsampling image;Target high-resolution image and target downsampling image are respectively input into saliency target detection model, saliency target detection model carries out feature extraction to image by double-channel network, obtains different levels of features, splices the deepest layer feature of double-channel network and the splicing result is input into multi-scale attention module to obtain first feature;Based on first feature, the first three layer features of double-channel network are sequentially fused using multi-level wide decoder module to obtain second feature, and the final saliency target detection result is obtained by feature decoding second feature.The application can carry out fine processing to complex geometric shape in high-resolution image under limited computing resources, and accurately extract edge information to realize high-quality segmentation.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a salient target detection method and system based on edge enhancement wide decoder. Background Technology

[0002] Saliency object detection, a hot research topic in computer vision, has multifaceted research value. For example, in image editing, saliency object detection can be used to automatically crop or enhance main targets in images; in autonomous driving, it can help vehicles identify and track important targets such as pedestrians and vehicles on the road; and in medical image analysis, it can assist doctors in identifying and locating disease sites. Therefore, saliency object detection technology has broad application prospects.

[0003] Traditional salient object detection methods are often designed and optimized for low-resolution images, primarily aiming to solve the problem of object localization. However, with the advancement of camera technology and the widespread availability of high-definition cameras, images captured are clearer and richer in detail. These high-resolution images not only provide more information but also present greater challenges to object detection algorithms. When processing high-resolution images, traditional salient object detection algorithms have revealed some shortcomings. For example, some algorithms downsample high-resolution images to a fixed small size for identification. During the scaling process, the effective pixels for small-scale objects become fewer, resulting in poor detection performance for small targets. Furthermore, for large-scale objects, traditional algorithms cannot effectively model their global information, making accurate detection and localization difficult when the target is large. Additionally, the richer detail information in high-resolution images leads to poor performance of traditional algorithms in object edge segmentation and detail extraction, often resulting in blurred, overlapping, and incomplete edge segmentation. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a salient target detection method and system based on edge enhancement wide decoder to solve the problems mentioned in the background art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a salient target detection method based on edge enhancement wide decoder, comprising: acquiring a high-resolution image dataset and downsampling the target high-resolution image in the dataset to obtain a target downsampled image;

[0009] The high-resolution image and the downsampled image of the target are respectively input into the salient target detection model. The salient target detection model extracts features from the image through a dual-channel network to obtain features at different levels. The dual-channel network includes a local feature extraction network and a semantic feature extraction network.

[0010] The deepest features of the local feature extraction network and the semantic feature extraction network are concatenated, and the concatenation result is input into the multi-scale attention module in the salient object detection model to obtain the first feature;

[0011] Based on the first feature, the first three layers of features from the local feature extraction network and the semantic feature extraction network are sequentially fused using the multi-level wide decoder module in the salient target detection model to obtain the second feature. The final salient target detection result is obtained by decoding the second feature.

[0012] As a preferred embodiment of the salient target detection method based on edge enhancement wide decoder described in this invention, it further includes: extracting features from the high-resolution image of the target through a local feature extraction network, and inputting the first two layers of features into the differential convolution module in the salient target detection model; the differential convolution region of the differential convolution module includes multiple convolution branches;

[0013] The differential convolution module extracts convolutional features from the first two layers of input features and concatenates the extracted convolutional features in the dimensional direction. The concatenated convolutional features are then used for dimensionality reduction to obtain differential features. These differential features are then added to the first two layers of input features via skip connections and output to obtain the first branch features and the second branch features of the target high-resolution image.

[0014] The first branch feature and the second branch feature are also inputs to one side of the multi-level wide decoder module, respectively.

[0015] As a preferred embodiment of the edge-enhanced wide decoder salient object detection method described in this invention, the method for obtaining the first feature by inputting the concatenation result into the multi-scale attention module of the salient object detection model includes: after inputting the concatenation result into the multi-scale attention module, obtaining four layers of features {F} at different scales through continuous convolutional pooling operations. i i = 1, 2, 3, 4};

[0016] Starting with the deepest features, features from adjacent scales are input into the attention calculation module layer by layer. The low-resolution deep features F... i Transformed into a two-dimensional vector key sequence Key, and high-resolution shallow features F i-1 The query sequence (Query) and value sequence (Value) are transformed into two-dimensional vectors. Through attention calculation, semantic information from deep features guides shallow features, preserving detailed information about salient targets.

[0017]

[0018] Among them, F i Q and F i V These are the two-dimensional vector query sequence and value sequence obtained by flattening the shallow features, respectively, F. j K The key sequence obtained by flattening deep features;

[0019] Calculate the similarity between the query sequence and the key sequence, normalize the similarity using the Softmax activation function to obtain the attention weights; multiply the attention weights by the value sequence to obtain the weighted value sequence; add the obtained attention features to the concatenated input through skip connections to output the first feature.

[0020] As a preferred embodiment of the edge-enhanced wide decoder salient target detection method of the present invention, the method comprises: using the multi-level wide decoder module in the salient target detection model to sequentially fuse the first three layers of features of the local feature extraction network and the semantic feature extraction network to obtain the second feature, including: the multi-level wide decoder module receiving feature streams from three branches, wherein one side of the multi-level wide decoder module is the output feature of the local feature extraction network, one side is the output feature of the semantic feature extraction network, and the other side is the output feature of the previous level decoder;

[0021] The three different features are adjusted to the same size and then spliced ​​together in the channel dimension to generate a composite feature map that integrates multiple feature information.

[0022] The concatenated composite feature map is input into the wide convolution module. The features processed by the wide convolution module are added to the ordinary convolution branch features of the fused features, and the added features are input into the wide convolution module for secondary wide convolution module processing.

[0023] The features processed and output by the second-order wide convolution module are added to the features before input through skip connections, and the output is obtained by upsampling.

[0024] As a preferred embodiment of the edge-enhanced wide decoder salient target detection method described in this invention, it further includes: the fusion optimization process of the multi-level wide decoder modules is expressed as follows:

[0025]

[0026] F i D =WC(F i D )+Conv(F i D )

[0027] F i D =Up(WC(F) i D )+F i D )

[0028] Among them, F i R F is the feature output of the i-th layer of the local feature extraction network. i S Let Up(·) be the feature output of the i-th layer of the semantic feature extraction network, where Up(·) is the bilinear interpolation upsampling operation, and F i D Let represent the features output by the i-th layer decoder. Concat(·) indicates feature concatenation in the channel direction, WC(·) is a wide convolutional module, and Conv(·) is a regular convolution.

[0029] As a preferred embodiment of the edge-enhanced wide decoder salient object detection method described in this invention, the wide convolution module includes: concatenating and fusing three different input features, and then reshaping the features through four different convolution branches; wherein the first branch uses a 1×1 convolution, the second branch uses a 3×3 convolution kernel, the third branch uses a 7×7 convolution kernel, and the fourth branch uses a 3×3 dilated convolution; the features of the four branches are added together and output for secondary fusion optimization.

[0030] As a preferred embodiment of the edge-enhanced wide decoder salient object detection method described in this invention, it further includes: the feature processing of the wide convolution module is expressed as follows:

[0031] F b1 =Conv 1×1 (F i )

[0032] F b2 =BN(Conv 3×3 (F i ))

[0033] F b3 =BN(Conv 7×7 (F i ))

[0034] F b4 =BN(Conv 3×3,d＝3 (F i ))

[0035] F = F b1 +F b2 +F b3 +F b4

[0036] Among them, F i For input features, F bi Let be the output feature of the i-th branch, and BN(·) be the batch normalization operation.

[0037] Secondly, the present invention provides a wide decoder salient target detection system based on edge enhancement, comprising: an image acquisition and processing module, used to acquire a high-resolution image dataset and perform downsampling processing on the target high-resolution image in the dataset to obtain a target downsampled image;

[0038] The feature extraction module is used to input the target high-resolution image and the target downsampled image into the salient target detection model, respectively. The salient target detection model extracts features from the image through a dual-channel network to obtain features at different levels. The dual-channel network includes a local feature extraction network and a semantic feature extraction network.

[0039] The feature processing module is used to concatenate the deepest features of the local feature extraction network and the semantic feature extraction network, and input the concatenation result into the multi-scale attention module in the salient object detection model to obtain the first feature;

[0040] The detection module is used to obtain a second feature based on the first feature by sequentially fusing the first three layers of features from the local feature extraction network and the semantic feature extraction network using the multi-level wide decoder module in the salient target detection model, and to obtain the final salient target detection result by performing feature decoding on the second feature.

[0041] Thirdly, the present invention provides a computing device, comprising:

[0042] Memory and processor;

[0043] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the edge-enhanced wide decoder salient target detection method.

[0044] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the edge-enhanced wide decoder saliency target detection method.

[0045] Compared with existing technologies, the advantages of this invention are as follows: When processing high-resolution images, this invention employs a lightweight residual network for detail feature extraction, while utilizing the Swing Transformer to process downsampled images and extract semantic information. This ensures that the computational load remains within a reasonably acceptable range while fully extracting image features. To ensure that detail features and semantic features can be properly fused and complemented, a multi-scale attention module is introduced to re-correct the fused features at different scales and suppress unnecessary background noise. In the decoder stage, a multi-level wide decoder with large convolutional kernels is used to expand the original receptive field and better model the context of the features, thereby improving the model's image understanding ability. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0047] Figure 1 This is a model overall architecture diagram of a wide decoder-based salient target detection method and system based on edge enhancement, according to an embodiment of the present invention.

[0048] Figure 2 This is a structural diagram of the differential convolution module of a wide decoder salient target detection method and system based on edge enhancement according to an embodiment of the present invention;

[0049] Figure 3 This is a differential template diagram of three types of a wide decoder-based salient target detection method and system based on edge enhancement, as described in one embodiment of the present invention.

[0050] Figure 4 This is a structural diagram of a multi-scale attention module of a wide decoder-based salient target detection method and system based on edge enhancement, according to an embodiment of the present invention.

[0051] Figure 5 This is a structural diagram of a multi-level wide decoder module of a salient target detection method and system based on edge enhancement according to an embodiment of the present invention.

[0052] Figure 6 This is a structural diagram of a wide convolutional module of a wide decoder-based salient target detection method and system based on edge enhancement, according to an embodiment of the present invention.

[0053] Figure 7 This is a comparison of segmentation results of different models of a salient target detection method and system based on edge enhancement and wide decoder according to an embodiment of the present invention. Detailed Implementation

[0054] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0055] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0056] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0057] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0058] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0059] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] Example 1

[0061] Reference Figures 1-6 As one embodiment of the present invention, this embodiment provides a salient target detection method based on edge enhancement wide decoder, comprising:

[0062] It should be noted that for high-resolution salient object detection tasks, using a small-scale model, such as a residual network, alone will result in insufficient model encoding and decoding capabilities, thus failing to achieve high-quality detection and segmentation results. Using a larger-scale model, such as the Vision Transformer, will lead to enormous computational costs. Therefore, this application uses a dual-stream backbone network structure to resolve this contradiction. Specifically, a ResNet residual network is used to extract detailed features from high-resolution images, followed by image downsampling, and then a Swin Transformer network is used to extract global semantic features. This approach ensures high-quality detection performance for the salient object detection model while avoiding excessive computational overhead.

[0063] S1: Obtain a high-resolution image dataset and perform downsampling processing on the target high-resolution image in the dataset to obtain the target downsampled image;

[0064] It should be noted that the image resolution of the high-resolution image dataset obtained in this application embodiment is not less than 1024x1024 pixels.

[0065] S2: Input the high-resolution image and the downsampled image of the target into the salient target detection model. The salient target detection model extracts features from the image through a dual-channel network to obtain features at different levels. The dual-channel network includes a local feature extraction network and a semantic feature extraction network.

[0066] In an optional embodiment, to improve the detection capability of targets at different scales, the local feature extraction network of this embodiment utilizes the 4-layer features of the residual network for output, denoted as {R}. ii = 2, 3, 4, 5}. Based on the convolutional computing mechanism, the shallow features R2 and R3 in the first two layers contain rich image details, such as texture, color, and edge shape. By introducing differential convolution operations, richer edge information can be added to the shallow features, enabling the network to more accurately capture the boundaries and contours of objects, thereby improving the final segmentation quality of the salient object detection model. This not only effectively improves edge detection performance in image segmentation tasks but also enhances overall segmentation performance, providing strong support for more accurate object recognition and localization.

[0067] Similarly, in an optional embodiment, the semantic feature extraction network of this application selects the features from the last 4 layers of the SwingTransformer network {S i The output is after size adjustment for i = 2, 3, 4, 5. To fuse features of the same dimension from corresponding layers of the residual network and the Swin Transformer network, the difference in feature types between the two needs to be addressed. The residual network tends to extract detailed information, while the Swin Transformer is better at extracting semantic information. Therefore, a multi-scale attention module is introduced to re-align and correct the two types of features by learning the feature relationships at different scales. This further achieves complementarity between detailed and semantic information, suppresses redundant background noise, and improves the performance of the final model.

[0068] After refinement by the multi-scale attention module, the features are input into the multi-level wide decoder module for feature decoding. Unlike the classic decoding paradigm in traditional salient object detection methods, the multi-level wide decoder module incorporates large-size convolutional kernels during the progressive upsampling process. These large convolutional kernels can obtain a wider original receptive field, thereby capturing more comprehensive contextual information. Therefore, the network can more effectively extract abstract high-level features, enabling it to better understand the semantic content in the image.

[0069] Furthermore, a layer-by-layer supervision method was employed during the model training phase. When the feature flow is at different scales, the ground truth labels are scaled down to the same size. Then, the features are convolved and a sigmoid activation function is applied to calculate the foreground or background classification probability for each pixel. Finally, the gradient is generated by calculating the loss between the predicted map and the labels. Layer-by-layer supervision, by adding supervisory signals to the intermediate layers of the network, ensures effective gradient propagation, helps mitigate the vanishing or exploding gradient problem, and accelerates the network's convergence speed. Moreover, layer-by-layer supervision effectively avoids feature drift by promptly correcting the feature distribution, thus improving the model's generalization ability.

[0070] In one optional embodiment, the algorithm flow is shown in Table 1:

[0071]

[0072] It should be noted that in this embodiment, the Sigmoid function is used to generate the predicted image, mapping the network output to a value between 0 and 1, that is, the probability that the corresponding pixel is a significant foreground object.

[0073] Furthermore, it also includes: extracting features from the high-resolution image of the target through a local feature extraction network, and then inputting the first two layers of features into the differential convolution module in the salient target detection model; the differential convolution region of the differential convolution module includes multiple convolution branches;

[0074] The differential convolution module extracts convolutional features from the first two layers of input features and concatenates the extracted convolutional features along the dimensional direction. The concatenated convolutional features are then used for dimensionality reduction to obtain differential features. These differential features are then added to the first two layers of input features via skip connections and output to obtain the first and second branch features of the target high-resolution image.

[0075] The first branch features and the second branch features are also inputs to one side of the multi-level wide decoder module, respectively.

[0076] In an optional implementation, the shallow features {R} flowing out of the residual network are... i The input to the differential convolution module (i=2,3) involves features passing through multi-branch differential convolution regions. Central difference, angular difference, and radial difference are used to convolve the features, extracting different types of detail information through various convolution methods, with a focus on enhancing edge features. The four features are concatenated along their dimensionality, then dimensionality is reduced using convolution. Finally, the convolutions are added to the original features via skip connections before outputting the final result. By weightedly adding the differential features to the original features, local details and edge features of the image can be further enhanced, improving the visual quality and contrast of the predicted image.

[0077] In this embodiment, the differential convolution module uses three different types of differential convolution: central differential convolution, circular differential convolution, and radial differential convolution. The calculation principle of differential convolution is based on calculating pixel differences using pre-designed differential modules. The three different differential templates are as follows: Figure 3 As shown. For each pixel, the difference module calculates the difference between it and its neighboring pixels, thus highlighting changes in edge features. The resulting pixel difference image is multiplied one by one with the weights of the convolution kernel, and the results of all multiplications are summed. The summation result forms the numerical value in the feature map, which reflects the feature intensity or degree of change at different locations in the image. The calculation process is represented as follows:

[0078]

[0079] Where, x i and x i ′ represents the number of pixels in the input image, w i This represents the weights of a k×k convolution kernel.

[0080] S3: Concatenate the deepest features of the local feature extraction network and the semantic feature extraction network, and input the concatenation result into the multi-scale attention module in the salient object detection model to obtain the first feature;

[0081] Furthermore, inputting the concatenated result into the multi-scale attention module of the salient object detection model to obtain the first feature includes: after the concatenated result is input into the multi-scale attention module, through continuous convolutional pooling operations, four layers of features at different scales {F} are obtained. i i = 1, 2, 3, 4};

[0082] Starting with the deepest features, features from adjacent scales are input into the attention calculation module layer by layer. The low-resolution deep features F... i Transformed into a two-dimensional vector key sequence Key, and high-resolution shallow features F i-1 The query sequence (Query) and value sequence (Value) are transformed into two-dimensional vectors. Through attention calculation, semantic information from deep features guides shallow features, preserving detailed information about salient targets.

[0083]

[0084] Among them, F i Q and F i V These are the two-dimensional vector query sequence and value sequence obtained by flattening the shallow features, respectively, F. j K The key sequence obtained by flattening deep features.

[0085] Calculate the similarity between the query sequence and the key sequence, normalize the similarity using the Softmax activation function to obtain the attention weights; multiply the attention weights by the value sequence to obtain the weighted value sequence; add the obtained attention features to the concatenated input through skip connections to output the first feature.

[0086] In an optional embodiment, this application employs a multi-scale attention module to compute attention at different resolutions. The multi-scale attention module continuously decouples the input features, creating a hierarchical structure of features to better capture multi-layered information in the image. Attention computation is performed during this process, using its inherent global semantic information to match and enhance the detailed information in each layer of features. After layer-by-layer filtering and enhancement, feature information related to salient targets can be obtained, while noise in shallow features is suppressed. The feature fusion correction and enhancement process in the multi-scale attention module effectively improves the model's detection performance for salient targets, enabling the model to more accurately locate and identify target regions.

[0087] S4: Based on the first feature, the first three layers of features from the local feature extraction network and the semantic feature extraction network are sequentially fused using the multi-level wide decoder module in the salient target detection model to obtain the second feature. The final salient target detection result is obtained by decoding the second feature.

[0088] Furthermore, by utilizing the multi-level wide decoder module in the salient object detection model, the first three layers of features from the local feature extraction network and the semantic feature extraction network are sequentially fused to obtain the second feature, which includes:

[0089] The multi-level wide decoder module receives feature streams from three branches. One side of the multi-level wide decoder module receives the output features of the local feature extraction network, another side receives the output features of the semantic feature extraction network, and the third side receives the output features of the previous level decoder.

[0090] Three different features are adjusted to the same size and then stitched together along the channel dimension to generate a composite feature map that integrates multiple feature information.

[0091] The concatenated composite feature map is input into the wide convolution module. The features processed by the wide convolution module are added to the ordinary convolution branch features of the fused features, and the added features are input into the wide convolution module for secondary wide convolution module processing.

[0092] The features processed by the second-order wide convolution module are added to the features before input through skip connections, and the output is obtained by upsampling.

[0093] It should be noted that the multi-level wide decoder module of this application receives feature streams from three branches as input. These three branches originate from the residual network in the encoder, the Swin Transformer network, and the output features of the previous stage decoder, respectively. After receiving these three features, their dimensions need to be unified first. To achieve this, sampling operations are used to scale the three features to the same size, and feature concatenation is performed along the channel dimension for subsequent processing.

[0094] The concatenated features are further integrated and optimized using a wide convolutional module to ensure the salient object detection model can better understand image content. The adjusted features are added to the features from the regular convolutional branch of the fused features to obtain richer and more diverse feature representations. The resulting features are then fed back into the wide convolutional module to further enhance their expressive power. The output features are added to the previous input features via skip connections to preserve original information, and finally output through upsampling. This multi-level wide decoder module design fully utilizes feature information extracted from different branches, and by progressively integrating and optimizing features, further improves the performance of the salient object detection model in the task.

[0095] Furthermore, the fusion optimization process of the multi-level wide decoder module is represented as follows:

[0096]

[0097] F i D =WC(F i D )+Conv(F i D )

[0098] F i D =Up(WC(F) i D )+F i D )

[0099] Among them, F i R F is the feature output of the i-th layer of the local feature extraction network. i S Let Up(·) be the feature output of the i-th layer of the semantic feature extraction network, where Up(·) is the bilinear interpolation upsampling operation, and F i D Let represent the features output by the i-th layer decoder. Concat(·) indicates feature concatenation in the channel direction, WC(·) is a wide convolutional module, and Conv(·) is a regular convolution.

[0100] Furthermore, the wide convolution module includes: concatenating and fusing three different input features, and then reshaping the features through four different convolutional branches; the first branch uses a 1×1 convolution, the second branch uses a 3×3 convolution kernel, the third branch uses a 7×7 convolution kernel, and the fourth branch uses a 3×3 dilated convolution; the features from the four branches are added together and output for secondary fusion optimization.

[0101] Furthermore, the feature processing of the wide convolutional module is represented as follows:

[0102] F b1 =Conv 1×1 (F i )

[0103] F b2 =BN(Conv 3×3 (F i ))

[0104] F b3 =BN(Conv 7×7 (F i ))

[0105] F b4 =BN(Conv 3×3,d＝3 (F i ))

[0106] F = F b1 +F b2 +F b3 +F b4

[0107] Among them, F i For input features, F bi Let be the output feature of the i-th branch, and BN(·) be the batch normalization operation.

[0108] It's important to note that one common model architecture for salient object detection methods is the U-shaped structure, such as the U-Net model. This type of model typically consists of an encoder and a decoder cascaded together. In the encoder, convolution and downsampling operations are used to extract local and global information at different scales, gradually reducing the feature size. In the decoder, to ensure the final model outputs a salient prediction map of the same size as the original image, upsampling is usually performed to gradually restore the features to their initial size. If only common upsampling methods, such as bilinear upsampling, are used, their lack of learnability and ineffective information transfer may lead to the loss of some information in the original features. This results in spatial distortion of the features, causing the feature map to be over-smoothed, thus losing some subtle details and negatively impacting subsequent prediction output.

[0109] To address these issues, some methods introduce convolutional and normalization layers during the upsampling process to adjust the feature distribution, thereby restoring or enhancing detailed information and mitigating the impact of continuous upsampling. However, these convolutional layers typically use small kernels, which, due to limitations in the receptive field, cannot effectively handle long-range dependencies. This may cause the model to lose important contextual information during the recovery of detailed information, thus affecting the final prediction results.

[0110] In an optional embodiment, this application designs a decoder module with wide convolutions, the structure of which is as follows: Figure 6 As shown:

[0111] The input features are restructured through four different convolutional branches. First, the 1×1 convolution in the first branch primarily reduces the feature channel dimension, decreasing computational cost and parameter count, and aiding in the extraction of more abstract features. Second, the second branch uses a 3×3 convolution kernel to capture features of small-scale targets. Since small-scale information is easily lost during wide convolutions, using 3×3 convolution helps preserve these subtle features, improving the model's ability to perceive small-scale targets. The third branch uses a 7×7 convolution kernel; the larger kernel covers a wider spatial range, capturing more local and global information. This is crucial for contextual understanding during feature extraction, enhancing the model's perception and understanding of the target's surrounding environment. Finally, the fourth branch uses a 3×3 dilated convolution. Compared to the other branches, it assists in extracting long-distance dependencies in the feature map without increasing the number of model parameters. Dilated convolution can flexibly adjust the receptive field size by adjusting the dilation rate, thereby better capturing long-distance dependencies and improving the model's ability to perceive global information.

[0112] The above is an illustrative scheme of a salient target detection method based on edge enhancement wide decoder according to this embodiment. It should be noted that the technical solution of this salient target detection system based on edge enhancement wide decoder belongs to the same concept as the technical solution of the aforementioned salient target detection method based on edge enhancement wide decoder. Details not described in detail in the technical solution of the salient target detection system based on edge enhancement wide decoder in this embodiment can be found in the description of the technical solution of the aforementioned salient target detection method based on edge enhancement wide decoder.

[0113] This embodiment presents a wide decoder-based salient target detection system with edge enhancement, comprising:

[0114] The image acquisition and processing module is used to acquire a high-resolution image dataset and perform downsampling processing on the target high-resolution image in the dataset to obtain the target downsampled image.

[0115] The feature extraction module is used to input the high-resolution image and the downsampled image of the target into the salient target detection model. The salient target detection model extracts features from the image through a dual-channel network to obtain features at different levels. The dual-channel network includes a local feature extraction network and a semantic feature extraction network.

[0116] The feature processing module is used to concatenate the deepest features of the local feature extraction network and the semantic feature extraction network, and input the concatenation result into the multi-scale attention module in the salient object detection model to obtain the first feature;

[0117] The detection module is used to obtain a second feature based on the first feature by sequentially fusing the first three layers of features from the local feature extraction network and the semantic feature extraction network using the multi-level wide decoder module in the salient target detection model. The final salient target detection result is obtained by decoding the second feature.

[0118] This embodiment also provides a computing device suitable for edge-enhanced wide decoder salient target detection methods, including:

[0119] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the edge-enhanced wide decoder salient target detection method proposed in the above embodiments.

[0120] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the edge-enhanced wide decoder salient target detection method as proposed in the above embodiments.

[0121] The storage medium proposed in this embodiment and the implementation of the edge-enhanced wide decoder salient target detection method proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0122] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0123] Example 2

[0124] Referring to Tables 2 to 7, this is one embodiment of the present invention. Unlike the first embodiment, this embodiment trains and tests each strategy and module on five mainstream public datasets to verify the necessity and effectiveness of each module of the network. These include two low-resolution datasets, DUTS and DUT-OMRON, and three high-resolution datasets, HRSOD, DAVIS-S, and UHRSD. The five datasets used for testing in this embodiment encompass common scenarios in salient object detection tasks, covering multiple categories of salient objects, and also include various scenes such as cities, forests, beaches, and indoor environments.

[0125] In this embodiment, four different evaluation metrics are used to measure the performance of the salient object detection model from different perspectives: the F-measure, S-measure, E-measure, and mean absolute error (MAE). The F-measure comprehensively evaluates the model's performance by harmonicizing precision and recall; the S-measure measures the structural similarity between the detected object and the ground truth; and the E-measure comprehensively measures model performance based on both structural similarity and detail differences. The mean absolute error measures the average absolute deviation between the model's predicted values ​​and the ground truth values.

[0126] The specific ablation experiment is as follows:

[0127] 1. Effectiveness of the differential convolution module

[0128] In this embodiment, an ablation experiment was conducted on the differential convolution module, and three different training methods were defined: Method 1: No differential convolution module was used; Method 2: Differential convolution module was used for deep features of the residual network; Method 3: Differential convolution module was used for shallow features of the residual network.

[0129] Keeping all other training parameters consistent, the three methods were trained for 50 epochs on the DUTS dataset, and performance tests were conducted on both low-resolution and high-resolution datasets to observe the impact of introducing differential convolution modules at different locations on model performance; as shown in Tables 2 and 3:

[0130] Table 2. Experimental comparison results of different methods on low-resolution datasets.

[0131]

[0132] Table 3. Experimental comparison results of different methods on high-resolution datasets.

[0133]

[0134] As shown in Tables 2 and 3, Methods 2 and 3, which utilize differential convolution modules, outperform Method 1. The performance improvement is particularly significant on high-resolution datasets, indicating that as image clarity increases, differential convolution modules better highlight edge details, thus improving segmentation quality. Compared to Method 2, which adds differential convolution modules to deeper layers, Method 3 introduces this module at shallower layers with a more significant performance improvement. Since shallow features contain most of the detailed information, while deep features mainly contain contextual semantic information, adding differential convolution modules at shallower layers is a better choice from both efficiency and benefit perspectives.

[0135] 2. Effectiveness of multi-scale attention modules

[0136] In this embodiment, two sets of control experiments were set up: Method 1: without using the multi-scale attention module; Method 2: with the multi-scale attention module added. All other structures and training parameters remained unchanged in both sets of experiments to ensure accurate evaluation of the independent impact of the multi-dimensional attention fusion enhancement module on model performance; as shown in Tables 4 and 5:

[0137] Table 4. Experimental comparison results of different methods on low-resolution datasets.

[0138]

[0139] Table 5. Experimental comparison results of different methods on high-resolution datasets.

[0140]

[0141] As shown in Tables 4 and 5, Method 2, which adds a multi-scale attention module, achieves a significant performance improvement in salient object detection compared to Method 1. This indicates that the introduction of the multi-scale attention module into the salient object detection model plays a crucial role in improving performance. The multi-scale attention module can effectively guide semantic information towards detailed information and suppress redundant noise by calculating attention at different scales. This helps to strengthen the detailed information in features, correct potential conflicts in fused features, thereby improving the model's representation of features and ultimately enhancing performance. Therefore, the multi-scale attention module can effectively achieve complementarity between detailed and semantic information, making a significant contribution to improving model performance.

[0142] 3. Effectiveness of the multi-level wide decoder module

[0143] To verify the effectiveness of the designed multi-level wide decoder module and layer-by-layer supervision strategy, four control networks were set up. In Method 1, the multi-level wide decoder module was not used, and no layer-by-layer supervision was performed; in Method 2, although the multi-level wide decoder module was not used, layer-by-layer supervision was performed; in Method 3, the multi-level wide decoder module was used, but no layer-by-layer supervision was performed; and in Method 4, both the multi-level wide decoder module and layer-by-layer supervision were used. These four sets of experiments aimed to systematically study the impact of the multi-level wide decoder module and layer-by-layer supervision on model performance.

[0144] Table 6. Experimental comparison results of different methods on low-resolution datasets.

[0145]

[0146]

[0147] Table 7. Experimental comparison results of different methods on high-resolution datasets.

[0148]

[0149] As shown in Tables 6 and 7, using the multi-level wide decoder module significantly improves model performance compared to methods without it; simultaneously, layer-by-layer auxiliary supervision also positively impacts model performance. Further analysis reveals that method four, combining the multi-level wide decoder module with layer-by-layer auxiliary supervision, achieves the best performance, demonstrating the complementarity and effectiveness of these two techniques.

[0150] Example 3

[0151] Reference Figure 7 This is one embodiment of the present invention. This embodiment verifies the beneficial effects of the present invention through comparative experiments.

[0152] The comparison networks in this embodiment include Local Cascaded Decoder Network (CPD), Stacked Cross-Refining Network (SCRN), Depth-Aware Satisfactory Object Detection Network (DAS), Fusion Feedback and Focusing Network (F3Net), Global Context-Aware Progressive Aggregation Network (GCPA), Interactive Dual-Stream Decoder Network (ITSD), Label Decoupling Framework (LDF), Complementary Triple Decoder Network (CTD), Pyramid Feature Shrinking Network (PFS), and Pyramid Grafting Network (PGNet). All networks were trained on the DUTS dataset, and their performance metrics on five public datasets were compared. The specific results are shown in Tables 8 and 9. In the tables, "D" indicates that the model was trained on the DUTS dataset, "DH" indicates mixed training on the DUTS and HRSOD datasets, and bold numbers represent the best performance value on each dataset.

[0153] Table 8 compares the performance of each method on low-resolution datasets.

[0154]

[0155] Table 9. Experimental comparison results of different methods on high-resolution datasets.

[0156]

[0157]

[0158] As shown in Tables 8 and 9, the method of this invention performs significantly well on five public datasets. Notably, the method shows a more significant improvement in metrics on the three high-resolution datasets compared to the two low-resolution datasets. This highlights the effectiveness of each module in the method, especially when dealing with high-resolution images, where the salient object detection model can learn richer detail information. Through the combined action of the differential convolution module, the multi-scale attention module, and the multi-level wide decoder module, the model can achieve more accurate salient object segmentation, which helps improve model performance and segmentation quality. Therefore, the method of this invention has significant advantages in processing high-resolution images and can effectively address the challenges posed by images of different resolutions.

[0159] To demonstrate the superiority of the method of this invention on high-resolution datasets, this embodiment visually compares the predicted images of the method of this invention with those of other methods. For example... Figure 7 As shown in the figure, the first column is the original image in the dataset, the second column is the ground truth annotation corresponding to the image, the third column is the prediction result after 50 rounds of training of the method of this invention, and the fourth to the last columns are other methods for comparison.

[0160] In the complex scene shown in row 1 of the figure, the comparative methods failed to detect small-scale targets, while the method of this invention successfully detected and segmented the person, but it incorrectly segmented the person's shadow as well. In the scenes in rows 2, 3, and 4, the comparative methods failed to correctly understand salient targets, such as the white stairs in row 2 and the person in front of the airplane in row 3. However, as can be seen from rows 5, 6, and 7, the method of this invention performs excellently in segmenting objects with complex structures, such as the outline of the leaves in row 5 and the gaps between the fences in row 6. In contrast, the comparative methods suffer from problems such as failure to segment, multiple segmentation, partial segmentation, and blurred edges, which seriously affect the detection quality. This indicates that the differential convolution module proposed in this invention effectively improves the expressive power of edge features, while the multi-scale attention module and multi-level wide decoder module help the model effectively focus on targets of different scales, thereby improving the model's object understanding ability and detection accuracy.

[0161] This invention integrates a residual network with a differential convolution module to focus on object edge features, combined with rich global semantic features extracted by a Swin Transformer network from another branch. A multi-scale attention module is incorporated during feature fusion to comprehensively learn image features at different scales from different feature types, achieving information complementarity at both type and scale, suppressing background noise features, and improving the model's generalization ability. A multi-level wide decoding module is applied in the decoder, utilizing large convolutional kernels to expand the original receptive field, better performing contextual modeling, and further improving network performance.

[0162] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A salient target detection method based on edge enhancement wide decoder, characterized in that, include: A high-resolution image dataset is acquired, and the target high-resolution image in the dataset is downsampled to obtain the target downsampled image. The high-resolution image and the downsampled image of the target are respectively input into the salient target detection model. The salient target detection model extracts features from the image through a dual-channel network to obtain features at different levels. The dual-channel network includes a local feature extraction network and a semantic feature extraction network. The deepest features of the local feature extraction network and the semantic feature extraction network are concatenated, and the concatenation result is input into the multi-scale attention module in the salient object detection model to obtain the first feature; Based on the first feature, the first three layers of features from the local feature extraction network and the semantic feature extraction network are sequentially fused using the multi-level wide decoder module in the salient target detection model to obtain the second feature. The final salient target detection result is obtained by decoding the second feature. Feature extraction is performed on the high-resolution image of the target through a local feature extraction network. The first two layers of features are then input into the differential convolution module in the salient target detection model. The differential convolution region of the differential convolution module includes multiple convolution branches. The differential convolution module extracts convolutional features from the first two layers of input features and concatenates the extracted convolutional features in the dimensional direction. The concatenated convolutional features are then used for dimensionality reduction to obtain differential features. These differential features are then added to the first two layers of input features via skip connections and output to obtain the first branch features and the second branch features of the target high-resolution image. The first branch feature and the second branch feature are also inputs to one side of the multi-level wide decoder module, respectively; The process of inputting the concatenated result into the multi-scale attention module of the salient object detection model to obtain the first feature includes: after the concatenated result is input into the multi-scale attention module, it undergoes continuous convolutional pooling operations to obtain four layers of features at different scales. ; Starting with the deepest features, features from adjacent scales are input into the attention calculation module layer by layer, reducing the low-resolution deep features. Transformed into a two-dimensional vector key sequence Key, high-resolution shallow features The query sequence (Query) and value sequence (Value) are transformed into two-dimensional vectors. Through attention calculation, semantic information from deep features guides shallow features, preserving detailed information about salient targets. ； in, and The shallow features are flattened to obtain two-dimensional vector query sequences and value sequences, respectively. The key sequence obtained by flattening deep features; Calculate the similarity between the query sequence and the key sequence, normalize the similarity using the Softmax activation function to obtain the attention weights; multiply the attention weights by the value sequence to obtain the weighted value sequence; add the obtained attention features to the concatenated input through skip connections and output the first feature. The second feature is obtained by sequentially fusing the first three layers of features from the local feature extraction network and the semantic feature extraction network using the multi-level wide decoder module in the salient object detection model. The features include: The multi-level wide decoder module receives feature streams from three branches, wherein one side of the multi-level wide decoder module is the output feature of the local feature extraction network, one side is the output feature of the semantic feature extraction network, and the other side is the output feature of the previous level wide decoder. The three different features are adjusted to the same size and then spliced ​​together in the channel dimension to generate a composite feature map that integrates multiple feature information. The concatenated composite feature map is input into the wide convolution module. The features processed by the wide convolution module are added to the ordinary convolution branch features of the fused features, and the added features are input into the wide convolution module for secondary wide convolution module processing. The features processed and output by the second-order wide convolution module are added to the features before input through skip connections, and the output is obtained by upsampling.

2. The salient target detection method based on edge enhancement wide decoder as described in claim 1, characterized in that, Also includes: The fusion optimization process of the multi-level wide decoder module is represented as follows: ； ； ； in, For the local feature extraction network Features of layer output, For the semantic feature extraction network Features of layer output, This is a bilinear interpolation upsampling operation. For the first Features of the layer-width decoder output This indicates that feature stitching operations are performed along the channel direction. For wide convolutional modules, This is a regular convolution.

3. The salient target detection method based on edge enhancement wide decoder as described in claim 2, characterized in that, The wide convolutional module includes: concatenating and fusing three different input features, and then reshaping the features through four different convolutional branches; the first branch uses... Convolution, the second branch uses The convolution kernel, the third branch uses The convolution kernel, the fourth branch uses Dilated convolution: The features of the four branches are summed and output for secondary fusion optimization.

4. The salient target detection method based on edge enhancement wide decoder as described in claim 3, characterized in that, Also includes: The feature processing of a wide convolutional module is represented as follows: ； ； ； ； ； in, For input features, For the first The output characteristics of the branch This is for batch normalization operations.

5. A salient target detection system based on edge enhancement wide decoder, based on the salient target detection method based on edge enhancement wide decoder according to any one of claims 1 to 4, characterized in that, include: The image acquisition and processing module is used to acquire a high-resolution image dataset and perform downsampling processing on the target high-resolution image in the dataset to obtain the target downsampled image. The feature extraction module is used to input the target high-resolution image and the target downsampled image into the salient target detection model, respectively. The salient target detection model extracts features from the image through a dual-channel network to obtain features at different levels. The dual-channel network includes a local feature extraction network and a semantic feature extraction network. The feature processing module is used to concatenate the deepest features of the local feature extraction network and the semantic feature extraction network, and input the concatenation result into the multi-scale attention module in the salient object detection model to obtain the first feature; The detection module is used to obtain a second feature based on the first feature by sequentially fusing the first three layers of features from the local feature extraction network and the semantic feature extraction network using the multi-level wide decoder module in the salient target detection model, and to obtain the final salient target detection result by performing feature decoding on the second feature.

6. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 4.

Images