A salient object detection method, system, electronic device and storage medium

Abstract

This invention discloses a salient target detection method, system, electronic device, and storage medium, relating to the field of salient target detection technology. The method involves training a constructed salient target detection model based on multiple sample images. In the constructed salient target detection model, multiple local detail features and multiple global semantic features of the target sample image are extracted through a dual-branch encoder. After interactively fusing some and all local detail features, multiple enhanced local detail features and multiple enhanced global detail features are obtained. Each enhanced local detail feature and its corresponding enhanced global detail feature are concatenated. All concatenated features and the remaining local detail features are then processed by an edge-aware enhancement module and a decoder to obtain the salient target detection result. The trained salient target detection model is then used to accurately detect salient targets in the target image.

Description

Technical Field

[0001] This invention relates to the field of salient target detection technology, and in particular to a salient target detection method, system, electronic device, and storage medium. Background Technology

[0002] Salient Object Detection (SOD) aims to mimic human visual perception by automatically identifying and segmenting the most salient objects or regions from natural images. An object's "salientity" can manifest in various aspects such as shape, size, color, and spatial location. Due to its powerful capabilities, salient object detection has wide applications in fields such as autonomous driving, image retrieval, video segmentation, image cropping, semantic segmentation, and object recognition.

[0003] Early salient object detection methods relied on hand-designed features, utilizing manually extracted colors and textures to uncover contrasting regions. However, these methods failed to extract high-level semantic information and performed poorly in complex scenarios. The development of deep learning has significantly overcome the limitations of hand-designed methods and will continue to make steady progress.

[0004] Deep learning-based salient object detection methods primarily rely on deep neural networks to extract discriminative features and segment salient regions with high contrast within the surrounding environment. While existing methods have achieved significant breakthroughs, challenges remain, such as the irregular topological structures of salient objects and cluttered backgrounds. Specifically, salient objects may exhibit complex geometric structures such as non-rigid deformations, internal voids, bifurcations, or adhesion, and the color or texture differences at the boundary between the salient object and the background may be small. This leads to blurry prediction results and unclear segmentation edges.

[0005] Convolutional neural network (CNN)-based methods can extract discriminative features through multiple convolutional layers. However, due to their limited depth, they cannot capture long-range dependencies, fail to produce good results in complex scenes, and compromise the overall integrity of objects. Transformer-based methods can extract contextual semantic information from a global perspective. However, due to their global self-attention mechanism, they cannot capture sufficient local details, and the prediction results may be ambiguous when dealing with small targets. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the shortcomings of the prior art, and specifically provides a salient target detection method, system, electronic device, and storage medium, as detailed below:

[0007] 1) In a first aspect, the present invention provides a salient target detection method, the specific technical solution of which is as follows:

[0008] Based on multiple sample images, a constructed salient object detection model is trained to obtain a trained salient object detection model. The constructed salient object detection model includes a dual-branch encoder, a cross-model interactive fusion module, an edge-aware enhancement module, and a decoder. The dual-branch encoder extracts multiple local detail features and multiple global semantic features from the target sample image. Some local detail features and all global semantic features are then interactively fused by a cross-model interactive fusion module according to a preset correspondence to obtain multiple enhanced local detail features and multiple enhanced global detail features. Each enhanced local detail feature and its corresponding enhanced global detail feature are then concatenated to obtain multiple concatenated features. All concatenated features and the remaining local detail features are then processed by the edge-aware enhancement module and the decoder to obtain the salient object detection result. The target sample image can be any target sample image.

[0009] The trained salient object detection model is used to detect salient objects in the target image.

[0010] The beneficial effects of the salient target detection method provided by this invention are as follows:

[0011] The dual-branch encoder can extract local detail features and global semantic features separately. Local detail features help capture complex geometric structures of targets, such as non-rigid deformations and internal voids, while global semantic features help grasp the overall scene and resist interference from cluttered backgrounds. Furthermore, the cross-model interaction fusion module further enhances these features, enabling the salient object detection model to identify targets more accurately, avoid fuzzy prediction results, and effectively cope with the challenges of irregular topological structures and cluttered backgrounds of salient objects. Through cross-model interaction fusion and subsequent processing modules, the salient object detection model achieves full interaction and integration of local and global information, breaking through depth limitations and better understanding the scene from a global perspective while taking into account local information to ensure the integrity of the object. Even in complex scenes, it can generate higher-quality results and solve the problem of being unable to capture long-distance dependencies due to limited depth. The edge perception enhancement module can strengthen edge detail information. Combined with the local detail feature extraction of the dual-branch encoder, it can accurately capture the edges of salient objects. Even when the color or texture difference between the target and the background is small, it can make the segmentation edges clearer and improve the overall effect of salient object detection. Especially when dealing with small targets, it can effectively avoid fuzzy prediction results.

[0012] Based on the above scheme, the salient target detection method of the present invention can be further improved as follows.

[0013] Furthermore, the first branch of the dual-branch encoder includes N sequentially arranged first network layers, and the second branch of the dual-branch encoder includes M sequentially arranged second network layers.

[0014] Multiple local detail features and multiple global semantic features are extracted from the target sample image through a dual-branch encoder. These local detail features and all global semantic features are then fused separately through a cross-model interactive fusion module according to a predefined correspondence, resulting in multiple enhanced local detail features and multiple enhanced global detail features. Each enhanced local detail feature and its corresponding enhanced global detail feature are then concatenated to obtain multiple concatenated features, including:

[0015] After inputting the target sample image into the first first network layer, the local detail features output by the first to nth first network layers are obtained;

[0016] The local detail features output by the (n-1)th first network layer are input into the first second network layer to obtain the global semantic features output by the first second network layer. Then, the local detail features output by the nth first network layer and the global semantic features output by the first second network layer are interactively fused through the first cross-model interaction fusion module. This integrates the local detail information from the nth first network layer into the global semantic features output by the first second network layer, resulting in the first enhanced global semantic feature. Finally, the global context information from the global semantic features output by the first second network layer is integrated into the local detail features output by the nth first network layer, resulting in the first enhanced global semantic feature. For local detail features, the first enhanced global semantic feature and the first enhanced local detail feature are concatenated to obtain the first concatenated feature. The first enhanced global semantic feature is used as the input to the second second network layer, and the first enhanced local detail feature is used as the input to the (n+1)th first network layer. The output of the second second network layer and the output of the (n+1)th first network layer are interactively fused through the second cross-model interaction fusion module to obtain the second enhanced global semantic feature and the first enhanced local detail feature, which are then concatenated to obtain the second concatenated feature. This process continues until the Mth concatenated feature is obtained, where n and M are both positive integers, and N = M + n - 1.

[0017] The beneficial effects of adopting the above-mentioned further scheme are as follows: Through the multi-layer network structure of the dual-branch encoder, local detail features and global semantic features of the image can be systematically extracted from different levels, ensuring the richness and completeness of the features; the design of the cross-model interactive fusion module realizes the dynamic interactive fusion of local detail features and global semantic features. This interactive mechanism not only enhances the semantic understanding ability of local features but also improves the detail accuracy of global features, enabling the model to capture key information more accurately when processing complex targets; the feature concatenation operation organically combines the enhanced local and global features, providing a more comprehensive feature representation for subsequent processing, improving the accuracy and robustness of salient target detection, especially when processing targets with complex structures.

[0018] Furthermore, by combining all the stitched features and the remaining local detail features through the edge perception enhancement module and the decoder, the salient target detection results are obtained, including:

[0019] Each concatenated feature is processed through a convolutional layer to obtain the convolution result corresponding to each concatenated feature;

[0020] The convolution result corresponding to the Mth concatenated feature is input into the ASPP module to obtain multi-scale features. The multi-scale features and the convolution result corresponding to the Mth concatenated feature are input into the first decoder. The output of the first decoder and the convolution result corresponding to the Mth concatenated feature are input into the first edge perception enhancement module to enhance the edge information of the salient target. The (M-1)th concatenated feature and the output of the first edge perception enhancement module are input into the second decoder. The output of the second decoder and the convolution result corresponding to the (M-1)th concatenated feature are input into the second edge perception enhancement module. This process continues until the output of the mth edge perception enhancement module is obtained, where m is a positive integer, m = M - n + 1.

[0021] The output of the m-th edge perception enhancement module and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th decoder. The output of the (m+1)-th decoder and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th edge perception enhancement module. The output of the (m+1)-th edge perception enhancement module and the local detail features output by the (n-2)-th first network layer are input into the (n+2)-th decoder, until the output of the N-th decoder is obtained. The output of the N-th decoder is used as the salient target detection result.

[0022] The beneficial effects of adopting the above-mentioned further scheme are as follows: By processing each concatenated feature separately through a convolutional layer and inputting it into the ASPP (Atrous Spatial Pyramid Pooling) module, this technique can capture multi-scale information. The ASPP module uses convolutional kernels with different dilation rates to extract features in parallel, effectively solving the problem of target scale variation and enhancing the detection capability for salient targets of different sizes. The features processed by the decoder are input into the edge perception enhancement module, significantly improving the detection accuracy of target edges. This solves the problem of blurred and unclear edges in existing technologies, making the boundaries of the detection results more accurate and detailed. The cascaded structure design of the decoder allows the features to continuously integrate information from different levels during the progressive decoding process, combining with the remaining local detail features, comprehensively improving the model's ability to grasp details, making the detection results more complete and detailed.

[0023] Furthermore, the cross-model interaction and fusion module is specifically used for:

[0024] After receiving local detail features and global semantic features, the received local detail features are processed sequentially by the first channel attention module and the first spatial attention module to obtain the first feature. The received local detail features and the received global semantic features are processed by the first cross-model spatial attention module to obtain the second feature. The first feature and the second feature are added element by element to obtain the enhanced local detail features.

[0025] The received global detail features are processed sequentially by the second channel attention module and the second spatial attention module to obtain the third feature. The received local detail features and the received global semantic features are processed by the second cross-model spatial attention module to obtain the fourth feature. The third feature and the fourth feature are added element-wise to obtain the enhanced global semantic features.

[0026] The beneficial effects of adopting the above-mentioned further solutions are as follows: By combining channel attention and spatial attention modules, local detail features are processed in multiple dimensions to further extract and enhance key information, thereby improving the discriminative ability of features; by utilizing a cross-model spatial attention mechanism, local detail features and global semantic features are combined to achieve effective fusion of the two, enhancing the semantic understanding of local features and the detail accuracy of global features; by fusing features obtained from different path processing through element-wise addition, feature complementarity is achieved, strengthening the key features of salient targets and improving detection accuracy; the enhanced local detail features and global semantic features more accurately represent salient targets, improving the performance of the detection model in complex scenes, especially when dealing with targets with complex structures and backgrounds.

[0027] Furthermore, the edge awareness enhancement module is specifically used to: obtain the boundary map in the received convolution result or local detail features, calculate the Hadamard product of the boundary map and the output of the received decoder, process the Hadamard product through a convolutional layer to obtain the convolution result corresponding to the Hadamard product, and add the convolution result corresponding to the Hadamard product and the output of the received decoder element by element to obtain the output of the edge awareness enhancement module.

[0028] The beneficial effects of adopting the above-mentioned further scheme are as follows: by extracting the boundary map from the convolution result and calculating the Hadamard product, the focus is on the target edge information, which strengthens the boundary features of salient targets and makes the edges clearer and more distinct; the result of the convolution process of the Hadamard product is added element by element to the decoder output to achieve the fusion of multi-level features, optimize feature expression, and improve the model's ability to grasp the overall and detailed aspects of the target; the enhanced edge information helps the salient target detection results to more accurately match the actual target boundary, improve the detection accuracy and robustness, and is particularly effective when dealing with complex backgrounds and targets with blurred edges.

[0029] Furthermore, it also includes: in training the constructed salient object detection model, a first loss function is used to supervise the training of the boundary map, and a second loss function is used to supervise the training of the salient object detection results.

[0030] The beneficial effects of adopting the above-mentioned further scheme are: by using the first loss function to supervise the boundary map and the second loss function to supervise the saliency detection results, multi-faceted optimization of the saliency target detection model can be achieved. This can improve boundary accuracy, making the boundaries of saliency targets clearer and more accurate; at the same time, it can enhance the accuracy and robustness of target detection, improve the detection capability of the saliency target detection model in complex scenarios, and enhance the generalization performance of the saliency target detection model.

[0031] 2) In a second aspect, the present invention also provides a salient target detection system, the specific technical solution of which is as follows:

[0032] It includes a model training module and a salient object detection module;

[0033] The model training module is used to: train the constructed salient object detection model based on multiple sample images to obtain the trained salient object detection model. The constructed salient object detection model includes: a dual-branch encoder, a cross-model interactive fusion module, an edge-aware enhancement module, and a decoder. The dual-branch encoder extracts multiple local detail features and multiple global semantic features from the target sample image. After some local detail features and all global semantic features are interactively fused by a cross-model interactive fusion module according to a preset correspondence, multiple enhanced local detail features and multiple enhanced global detail features are obtained. Each enhanced local detail feature and its corresponding enhanced global detail feature are concatenated to obtain multiple concatenated features. After combining all concatenated features and the remaining local detail features, the edge-aware enhancement module and the decoder process the results to obtain the salient object detection result. The target sample image can be any target sample image.

[0034] The salient object detection module is used to perform salient object detection on target images using a trained salient object detection model.

[0035] Based on the above scheme, the salient target detection system of the present invention can be further improved as follows.

[0036] Furthermore, the first branch of the dual-branch encoder includes N sequentially arranged first network layers, and the second branch of the dual-branch encoder includes M sequentially arranged second network layers; after inputting the target sample image into the first first network layer, the local detail features output by the first to nth first network layers are obtained;

[0037] The local detail features output by the (n-1)th first network layer are input into the first second network layer to obtain the global semantic features output by the first second network layer. Then, the local detail features output by the nth first network layer and the global semantic features output by the first second network layer are interactively fused through the first cross-model interaction fusion module. This integrates the local detail information from the nth first network layer into the global semantic features output by the first second network layer, resulting in the first enhanced global semantic feature. Finally, the global context information from the global semantic features output by the first second network layer is integrated into the local detail features output by the nth first network layer, resulting in the first enhanced global semantic feature. For local detail features, the first enhanced global semantic feature and the first enhanced local detail feature are concatenated to obtain the first concatenated feature. The first enhanced global semantic feature is used as the input to the second second network layer, and the first enhanced local detail feature is used as the input to the (n+1)th first network layer. The output of the second second network layer and the output of the (n+1)th first network layer are interactively fused through the second cross-model interaction fusion module to obtain the second enhanced global semantic feature and the first enhanced local detail feature, which are then concatenated to obtain the second concatenated feature. This process continues until the Mth concatenated feature is obtained, where n and M are both positive integers, and N = M + n - 1.

[0038] Furthermore, each concatenated feature is processed through a convolutional layer to obtain the convolution result corresponding to each concatenated feature. The convolution result corresponding to the Mth concatenated feature is input into the ASPP module to obtain multi-scale features. The multi-scale features and the convolution result corresponding to the Mth concatenated feature are input into the first decoder. The output of the first decoder and the convolution result corresponding to the Mth concatenated feature are then input into the first edge perception enhancement module to enhance the edge information of salient targets. The (M-1)th concatenated feature and the output of the first edge perception enhancement module are input into the second decoder. The output of the second decoder and the convolution result corresponding to the (M-1)th concatenated feature are then input into the second decoder. Input the second edge perception enhancement module until the output of the m-th edge perception enhancement module is obtained, where m is a positive integer, m=M-n+1; input the output of the m-th edge perception enhancement module and the local detail features output by the (n-1)-th first network layer into the (m+1)-th decoder; input the output of the (m+1)-th decoder and the local detail features output by the (n-1)-th first network layer into the (m+1)-th edge perception enhancement module; input the output of the (m+1)-th edge perception enhancement module and the local detail features output by the (n-2)-th first network layer into the (n+2)-th decoder, until the output of the N-th decoder is obtained, and the output of the N-th decoder is used as the salient target detection result.

[0039] Furthermore, the cross-model interaction and fusion module is specifically used for:

[0040] After receiving local detail features and global semantic features, the received local detail features are processed sequentially by the first channel attention module and the first spatial attention module to obtain the first feature. The received local detail features and the received global semantic features are processed by the first cross-model spatial attention module to obtain the second feature. The first feature and the second feature are added element by element to obtain the enhanced local detail features.

[0041] The received global detail features are processed sequentially by the second channel attention module and the second spatial attention module to obtain the third feature. The received local detail features and the received global semantic features are processed by the second cross-model spatial attention module to obtain the fourth feature. The third feature and the fourth feature are added element-wise to obtain the enhanced global semantic features.

[0042] Furthermore, the edge awareness enhancement module is specifically used to: obtain the boundary map in the received convolution result or local detail features, calculate the Hadamard product of the boundary map and the output of the received decoder, process the Hadamard product through a convolutional layer to obtain the convolution result corresponding to the Hadamard product, and add the convolution result corresponding to the Hadamard product and the output of the received decoder element by element to obtain the output of the edge awareness enhancement module.

[0043] Furthermore, the model training module is also used to: supervise the training of the boundary map using the first loss function and supervise the training of the salient object detection results using the second loss function during the training of the constructed salient object detection model.

[0044] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so as to enable the electronic device to implement any of the above-mentioned salience target detection methods.

[0045] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements any of the above-described methods for detecting salient targets.

[0046] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below:

[0048] Figure 1 This is a flowchart illustrating a salient target detection method according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the network structure of a salient object detection model;

[0050] Figure 3 This is a schematic diagram of the network structure of the cross-model interaction and fusion module;

[0051] Figure 4 This is a schematic diagram of the network structure of the edge perception enhancement module;

[0052] Figure 5 A schematic diagram illustrating the international standard dataset used to evaluate the technical effects of this invention;

[0053] Figure 6 A schematic diagram illustrating the evaluation indicators used to assess the technical effects of this invention;

[0054] Figure 7 This is a schematic diagram illustrating the comparison model;

[0055] Figure 8 This is one of the comparative diagrams of salient target detection results;

[0056] Figure 9 The second illustration shows a comparison of the results of salient target detection.

[0057] Figure 10 This is a schematic diagram of the structure of a salient target detection system according to an embodiment of the present invention;

[0058] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0059] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0060] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0061] like Figure 1As shown, a salient target detection method according to an embodiment of the present invention includes the following steps:

[0062] S1. Based on multiple sample images, the constructed salient target detection model is trained to obtain a trained salient target detection model. The constructed salient target detection model includes: a dual-branch encoder, a cross-model interactive fusion module, an edge-aware enhancement module, and a decoder. The dual-branch encoder extracts multiple local detail features and multiple global semantic features from the target sample image. Some local detail features and all global semantic features are interactively fused by a cross-model interactive fusion module according to a preset correspondence to obtain multiple enhanced local detail features and multiple enhanced global detail features. Each enhanced local detail feature and its corresponding enhanced global detail feature are concatenated to obtain multiple concatenated features. All concatenated features and the remaining local detail features are combined and processed by the edge-aware enhancement module and the decoder to obtain the salient target detection result. The target sample image is any target sample image.

[0063] S2. Use the trained salient object detection model to perform salient object detection on the target image.

[0064] The target images are those from the fields of image and video compression technology, image content understanding technology, autonomous driving technology, visual tracking technology, or image retrieval technology. Images from other fields may also be selected as target images depending on the specific circumstances.

[0065] 1) In the field of image and video compression technology, the target image can be an image containing natural scenery, including blue sky, green trees, grassland, and distant mountains. The detected green trees, blue-green grassland, and clearly defined distant mountains are considered salient targets. These areas are usually the focus of human visual attention and contain rich texture and color information. During image and video compression, the quality of these salient target areas is prioritized. For non-salient areas, such as the sky or other relatively monochromatic areas, the resolution can be appropriately reduced or compressed more significantly. This reduces data volume, improves compression efficiency, and saves storage space and transmission bandwidth without affecting the viewer's perception of the main landscape.

[0066] 2) In the field of image content understanding technology, the target image is a city street scene image containing elements such as pedestrians, vehicles, buildings, and roads. Detected pedestrians and vehicles are used as salient target detection results. Pedestrians may stand out due to their bright clothing or unique movements, while vehicles become the focus due to their large size and movement on the road. This facilitates rapid semantic understanding of the image content. For example, in intelligent security systems, by detecting the salientity of pedestrians and vehicles in the street scene, traffic flow and pedestrian behavior patterns can be analyzed in real time. Furthermore, it can be combined with other technologies, such as pedestrian re-identification, to track specific individuals or monitor traffic violations, such as speeding.

[0067] 3) In the field of autonomous driving technology, the target image is the road condition image ahead of the vehicle, including vehicles in other lanes, traffic signs, road markings, etc. Vehicles traveling in the same direction in the lane ahead, especially those that are close by, are detected as salient targets. Simultaneously, traffic signs on both sides of the road (such as speed limit signs, turn signs, etc.) are also detected. For autonomous vehicles, recognizing vehicles ahead is crucial for maintaining a safe following distance and timely braking. Salient target detection helps the vehicle quickly and accurately locate the position and status of surrounding vehicles. Traffic sign detection provides the vehicle with information on driving rules, such as adjusting speed according to speed limit signs and preparing to turn in advance according to turn signs, thereby improving the safety and reliability of autonomous driving.

[0068] 4) In the field of visual tracking technology, the target image is a sequence of video images from a basketball game during a sports competition. The images include basketball players, the basketball, referees, and spectators. In these image sequences, the basketball itself and the player holding the ball are detected as salient targets. The basketball stands out due to its fast movement and bright color (usually orange), while the player holding the ball becomes the focus due to their central role in the action (such as dribbling or shooting). In video analysis of sports events, visual tracking can be performed more effectively by detecting salient targets such as the basketball and the player holding the ball. For example, automatically tracking the trajectory of the basketball and analyzing the player's dribbling, passing, and shooting actions can provide coaches and spectators with more in-depth game analysis data. Furthermore, in multi-target tracking scenarios, prioritizing the tracking of these salient targets can improve the accuracy and efficiency of the tracking.

[0069] 5) In the field of image retrieval technology, the target image is a user-uploaded image containing flowers, including blooming blossoms, stems, and green leaves. The detected flowers in the target image are used as the salient target detection result. Flowers typically have vibrant colors and unique shapes, easily attracting attention in the target image. In image retrieval systems, extracting features of salient targets (such as flowers) allows for more accurate image matching. When a user wants to find images of similar flowers, the system searches its database based on the flower's salient features (color, shape, etc.), improving the accuracy and relevance of the retrieval and providing more precise results when users search for specific types of flower images or identify flower species online.

[0070] In the field of salient object detection, building customized datasets based on the needs of different application scenarios is crucial. For example, in medical image analysis, a large number of images labeled with lesion areas can be collected to train a model capable of accurately identifying lesions, assisting doctors in diagnosis. In the field of autonomous driving, establishing image datasets containing various road conditions and obstacles allows for the training of models focused on detecting key targets such as pedestrians and vehicles, improving driving safety. This targeted model building approach enables salient object detection technology to more accurately serve specific domains, achieving deep integration of technology and practical applications, and promoting the intelligent development of various industries.

[0071] Optionally, in the above technical solution, the first branch of the dual-branch encoder includes N sequentially arranged first network layers, and the second branch of the dual-branch encoder includes M sequentially arranged second network layers.

[0072] The value of N can be 2, 3, 4 or 5, etc., and can be set according to the actual situation. The value of M can be 1, 2, 3 or 4, etc.

[0073] The network structure used to extract local detail features constitutes the first branch, and the network structure used to extract global detail features constitutes the second branch. The network structure used to extract local detail features can be a convolutional neural network, a DenseNet model, or a U-Net model, and the network structure used to extract global detail features can be a Transformer, a Dense Attention network, or an LSTM model. Alternatively, the network structure used to extract global detail features and the network structure used to extract local detail features can be selected according to the actual situation.

[0074] Specifically, when the network structure used to extract local detail features is a convolutional neural network, the VGG-16 network (which belongs to convolutional neural networks) can be used. The first layer of the VGG-16 network is a convolutional block (VggBlock). When the network structure used to extract global detail features is a Transformer, the Swin-Transformer (which belongs to the Swin-Transformer family) can be used. The second layer of the Swin-Transformer is a SwinBlock.

[0075] Multiple local detail features and multiple global semantic features are extracted from the target sample image through a dual-branch encoder. These local detail features and all global semantic features are then fused separately through a cross-model interactive fusion module according to a predefined correspondence, resulting in multiple enhanced local detail features and multiple enhanced global detail features. Each enhanced local detail feature and its corresponding enhanced global detail feature are then concatenated to obtain multiple concatenated features, including:

[0076] S10. After inputting the target sample image into the first first network layer, the local detail features output by the first to nth first network layers are obtained.

[0077] S11. Input the local detail features output by the (n-1)th first network layer into the first second network layer to obtain the global semantic features output by the first second network layer. Then, through the first cross-model interaction fusion module, interact and fuse the local detail features output by the nth first network layer and the global semantic features output by the first second network layer. This integrates the local detail information in the local detail features output by the nth first network layer into the global semantic features output by the first second network layer, resulting in the first enhanced global semantic feature. Finally, the global context information in the global semantic features output by the first second network layer is integrated into the nth first network layer. From the local detail features output by a network layer, the first enhanced local detail feature is obtained. The first enhanced global semantic feature is used as the input of the second network layer, and the first enhanced local detail feature is used as the input of the (n+1)th first network layer. The output of the second network layer and the output of the (n+1)th first network layer are interactively fused through the second cross-model interaction fusion module to obtain the second enhanced global semantic feature and the first enhanced local detail feature. The features are then concatenated to obtain the second concatenated feature, and so on until the Mth concatenated feature is obtained, where n and M are both positive integers, and N = M + n - 1.

[0078] like Figure 2As shown, taking "N=5, M=3, n=3" as an example, the process of "obtaining multiple splicing features" is explained as follows:

[0079] ① After inputting the target sample image into the first network layer VggBlock-1, the first local detail feature is obtained. The first local detail feature After inputting the second first network layer VggBlock-2, the second local detail feature is obtained. The second local detail feature After inputting the third first network layer VggBlock-3, the third local detail feature is obtained. Among them, the first local detail feature The size is 224×224×64, the second local detail feature. The size is 112×112×128, the third local detail feature. The size is: 56×56×256.

[0080] ② The local detail features output by the second first network layer VggBlock-2 Input the first second-level network layer, SwinBlock-1, to obtain the global semantic features output by the first second-level network layer, SwinBlock-1. The local detail features output by the third first network layer, VggBlock-3, are then integrated through the first cross-model interaction fusion module, CMIM-1. The global semantic features output by the first second network layer, SwinBlock-1 Interactive fusion is performed to enhance the local detail features output by the third first network layer, VggBlock-3. Local details are integrated into the global semantic features output by the first second network layer, SwinBlock-1. In this process, the first enhanced global semantic feature is obtained. And make the global semantic features output by the first second network layer SwinBlock-1 The global context information is incorporated into the local detail features output by the third first network layer, VggBlock-3. In the process, the first enhanced local detail feature is obtained. The first enhanced global semantic feature And the first enhanced local detail features After feature concatenation, the first concatenated feature is obtained. This includes the global semantic features output by the first second network layer, SwinBlock-1. The size is: 56×56×256.

[0081] ③ Enhance the first local detail feature As the fourth first network layer, VggBlock-4, it incorporates the first enhanced global semantic features. As the input to the second second network layer SwinBlock-2, the fourth first network layer VggBlock-4 outputs the fourth local detail feature. The second global semantic feature is output by the second network layer, SwinBlock-2. The fourth local detail feature and the second global semantic feature Inputting the second cross-model interactive fusion module CMIM-2 yields the second enhanced local detail feature. and the second enhanced global semantic feature The second enhanced local detail feature And the second enhanced global semantic feature After feature concatenation, the second concatenated feature is obtained, including the fourth local detail feature. The size is 28×28×256, the second global semantic feature. The size is: 28×28×512.

[0082] ④ The second enhanced local detail feature As the fifth first network layer, VggBlock-5, it incorporates the second enhanced global semantic features. As the input to the third second network layer, SwinBlock-3, the fifth first network layer, VggBlock-5, outputs the fifth local detail feature. The third global semantic feature is output by the third second network layer, SwinBlock-3. The fifth local detail feature and the third global semantic feature Inputting the third cross-model interactive fusion module CMIM-3 yields the third enhanced local detail feature. and the third enhanced global semantic feature The third enhanced local detail feature And the third enhanced global semantic feature After feature concatenation, the third concatenated feature is obtained, including the fifth local detail feature. The size is 14×14×256, the third global semantic feature The size is: 14×14×1024.

[0083] Optionally, in the above technical solution, after combining all splicing features and the remaining local detail features through the edge perception enhancement module and the decoder, the salient target detection result is obtained, including:

[0084] S12. Process each concatenated feature separately through a convolutional layer to obtain the convolution result corresponding to each concatenated feature. Specifically:

[0085] The first concatenated feature is processed through a convolutional layer (conv), and the resulting convolution is denoted as... The second concatenated feature is processed through a convolutional layer (conv), and the resulting convolution is denoted as... The third concatenated feature is processed through a convolutional layer (conv), and the resulting convolution is denoted as... .

[0086] The above feature concatenation process, and the process of using convolutional layers to process each concatenated feature, can be represented by the following formula:

[0087]

[0088] Indicates to and Perform feature splicing. Represents the i-th enhanced local detail feature. Let i represent the i-th enhanced global semantic feature, where i is a positive integer and i ≤ M. Indicates: To Perform convolution operations, specifically... Convolution operation.

[0089] S13. Input the convolution result corresponding to the Mth concatenated feature into the ASPP module to obtain multi-scale features. Input the multi-scale features and the convolution result corresponding to the Mth concatenated feature into the first decoder. Input the output of the first decoder and the convolution result corresponding to the Mth concatenated feature into the first edge perception enhancement module to enhance the edge information of salient targets. Input the (M-1)th concatenated feature and the output of the first edge perception enhancement module into the second decoder. Input the output of the second decoder and the convolution result corresponding to the (M-1)th concatenated feature into the second edge perception enhancement module, until the output of the mth edge perception enhancement module is obtained, where m is a positive integer, m=M-n+1. At this time, m=5-3+1=3, then:

[0090] ① The convolution result corresponding to the third concatenated feature Input the ASPP module to obtain multi-scale features; convolve the multi-scale features with the concatenated third feature. Input to the first decoder (Decoder1), and the convolution result of the first decoder (Decoder1) on the multi-scale features and the third concatenated features. After processing, the output of the first decoder, Decoder1, is obtained. The output of the first decoder, Decoder1. The convolution result corresponding to the third concatenated feature Input the first edge perception enhancement module EEM1 and obtain the output of the first edge perception enhancement module EEM1.

[0091] ② The convolution result corresponding to the second concatenated feature The output of the first edge-aware enhancement module EEM1 is input into the second decoder Decoder2 to obtain the output of the second decoder Decoder2. The convolution result corresponding to the second concatenated feature and the output of the second decoder, Decoder2 Input the second edge perception enhancement module EEM2 to obtain the output of the second edge perception enhancement module EEM2.

[0092] ③ The convolution result corresponding to the first concatenated feature The output of the second edge-aware enhancement module (EEM2) is input into the third decoder (Decoder3) to obtain the output of the third decoder (Decoder3). The convolution result corresponding to the first concatenated feature and the output of the third decoder, Decoder3 Input the third edge perception enhancement module EEM3 to obtain the output of the third edge perception enhancement module EEM3.

[0093] S14. Input the output of the m-th edge perception enhancement module and the local detail features output by the (n-1)-th first network layer into the (m+1)-th decoder. Input the output of the (m+1)-th decoder and the local detail features output by the (n-1)-th first network layer into the (m+1)-th edge perception enhancement module. Input the output of the (m+1)-th edge perception enhancement module and the local detail features output by the (n-2)-th first network layer into the (n+2)-th decoder, until the output of the N-th decoder is obtained. Use the output of the N-th decoder as the salient target detection result. Specifically:

[0094] The output of the third edge-aware enhancement module EEM3 and the local detail features output by the second first network layer are combined. Input the 4th decoder (Decoder4) and get the output of the 4th decoder (Decoder4). The output of the 4th decoder, Decoder4. and the local detail features output by the second first network layer Input the fourth edge perception enhancement module, and input the output of the fourth edge perception enhancement module and the local detail features output by the first network layer. The fifth decoder, Decoder5, uses the output of the fifth decoder as the salient object detection result.

[0095] Optionally, in the above technical solutions, such as Figure 3 As shown, the cross-model interaction and fusion module is specifically used for:

[0096] S31. After receiving local detail features and global semantic features, the received local detail features are processed sequentially by the first channel attention module and the first spatial attention module to obtain the first feature. The received local detail features and the received global semantic features are processed by the first cross-model spatial attention module to obtain the second feature. The first feature and the second feature are added element-wise to obtain the enhanced local detail features. Specifically:

[0097] S310, When local detail features are received and global semantic features Then, the received local detail features The first channel attention module CA-1 is input, and the first channel attention module CA-1 processes the received local detail features. A series of pooling and convolution operations are performed to obtain the channel attention weights. And utilize channel attention weights Enhance local detail features The channel information is used to obtain the first intermediate feature. Specifically, it can be expressed by the following formula:

[0098]

[0099]

[0100] in, express function, Indicates to Perform global average pooling. Indicates to Perform a global max pooling operation. Indicates: To conduct Convolution operation. The symbol for the Hadamard product is... , It is a positive integer. .

[0101] S311, the first intermediate feature After processing by the first spatial attention module SA-1, the first feature is obtained. Specifically, the first spatial attention module SA-1 obtains Spatial attention weights And utilize spatial attention weights Enhance the first intermediate feature The important spatial location information in the text can be represented by the following formula:

[0102]

[0103]

[0104] in, Indicates the first intermediate feature conduct Convolution operation.

[0105] S312, Received local detail features and global semantic features After processing by the first cross-model spatial attention module SA_CM-1, the second feature is obtained. The first cross-model spatial attention module, SA_CM-1, obtains the globally semantically related spatial attention weights. Utilizing spatial attention weights Enhance received local detail features The global semantic information in the data can be represented by the following formula:

[0106]

[0107]

[0108] in, Indicates: To conduct Convolution operation.

[0109] S313, the first feature Second feature Element-wise addition is performed to obtain enhanced local detail features. Specifically, it can be expressed by the following formula:

[0110]

[0111] The branch with enhanced local detail features can be called the SwinToVgg branch. The SwinToVgg branch can introduce contextual information from the global semantic features in the Swin-Transformer into the VGG-16 network.

[0112] S32. The received global detail features are processed sequentially by the second channel attention module and the second spatial attention module to obtain the third feature. The received local detail features and the received global semantic features are processed by the second cross-model space attention module to obtain the fourth feature. The third feature and the fourth feature are added element-wise to obtain the enhanced global semantic feature. Specifically:

[0113] S320, Receive the global semantic features After processing by the second-channel attention module CA-2, the second intermediate feature is obtained. , specifically:

[0114] The second-channel attention module CA-2 processes the received global semantic features. A series of pooling and convolution operations are performed to obtain the channel attention weights. And utilize channel attention weights Enhance the received global semantic features The channel information is used to obtain the second intermediate feature. Specifically, it can be expressed by the following formula:

[0115]

[0116]

[0117] in, Indicates to Perform global average pooling. Indicates to Perform a global max pooling operation. Indicates: To conduct Convolution operation.

[0118] S321, Second Intermediate Feature After processing by the second spatial attention module SA-2, the third feature is obtained. , specifically:

[0119] The second spatial attention module SA-2 obtains the third feature. Spatial attention weights And utilize spatial attention weights Enhance the third feature The important spatial location information in the text can be represented by the following formula:

[0120]

[0121]

[0122] in, Indicates to conduct Convolution operation.

[0123] S322, Received local detail features and global semantic features After processing by the second cross-model spatial attention module SA_CM-2, the fourth feature is obtained. Specifically, the second cross-model spatial attention module SA_CM-2 obtains the spatial attention weights related to local details. Utilizing spatial attention weights Enhance global semantic features Local details in the image can be represented by the following formula:

[0124]

[0125]

[0126] in, Indicates: To conduct Convolution operation.

[0127] S323, the third feature and the fourth feature By performing element-wise addition, the enhanced global semantic features are obtained. Specifically, it can be expressed by the following formula:

[0128]

[0129] The branch with the enhanced global semantic features described above can be called the VggToSwin branch. The VggToSwin branch can introduce local detail information from the local detail features of the VGG-16 network into the Swin-Transformer.

[0130] Optionally, in the above technical solution, the edge perception enhancement module is specifically used to: obtain the boundary map in the received convolution result or local detail features, calculate the Hadamard product of the boundary map and the output of the received decoder, process the Hadamard product through a convolutional layer to obtain the convolution result corresponding to the Hadamard product, and add the convolution result corresponding to the Hadamard product and the output of the received decoder element by element to obtain the output of the edge perception enhancement module.

[0131] The input to the edge awareness enhancement module includes... and ,when hour, Indicates the first The convolution result corresponding to each concatenated feature; when hour, Indicates: the Local detail features output by the first network layer; Indicates: the The output of each decoder.

[0132] like Figure 4 As shown, in different decoding stages, the edge information of salient targets is enhanced by the edge perception enhancement module. The specific data processing process of the edge perception enhancement module is as follows:

[0133] S40, Calculate along the channel dimension The maximum and mean values, and the calculated values By concatenating the maximum and mean values, we obtain... The splicing features are processed by convolutional layers. Perform a 3×3 convolution operation on the concatenated features, and input the result of the convolution operation into... The function calculates the result, and then subtracts that result from 1 to get the result. Corresponding boundary map Specifically, it can be expressed by the following formula:

[0134]

[0135] Indicates calculation along the channel dimension The mean, Calculate along the channel dimension The maximum value; Indicates: Through convolutional layers The spliced ​​features are subjected to a 3×3 convolution operation. Indicates: will enter The calculation result obtained by the function.

[0136] S41, Calculate the boundary diagram and The Hadamard product is calculated, and then processed through a convolutional layer to obtain the corresponding convolution result. The convolution result is then compared with... The edge perception enhancement module outputs the result by adding elements one by one. Specifically, it can be expressed by the following formula:

[0137]

[0138] in, Represents the output of the q-th edge perception enhancement module.

[0139] Optionally, the above technical solution also includes:

[0140] To ensure accurate boundary localization, each boundary map generated in the intermediate stages is supervised using a first loss function. This first loss function can be a binary cross-entropy (BCE) loss function, a mean squared error loss function, or a multi-class cross-entropy loss function, etc. This application uses the binary cross-entropy loss function for illustration, which can be expressed by the following formula:

[0141]

[0142] in, Represents: Binary cross-entropy loss function. express: The corresponding actual boundary map, Specifically, this can be obtained from the true saliency map corresponding to the preset sample images using the Canny algorithm. Indicates the number of boundary maps generated. Represents: the sum of the binary cross-entropy losses corresponding to all boundary graphs.

[0143] Optionally, it also includes: performing a convolution operation on the output of each decoder through convolutional layers, and utilizing... The function yields the salient object detection result corresponding to the output of each decoder, which can be expressed by the following formula:

[0144]

[0145] in, Indicates: Through convolutional layers, the first... The output of the decoder Perform convolution operations; Indicates: will enter function, express: The corresponding salient target detection results.

[0146] It should be noted that in S2, the trained salient object detection model is used to perform salient object detection on the target image. The salient object detection result obtained by passing through the output of the last decoder is selected as the final salient object detection result and then provided to the user.

[0147] Optionally, in the above technical solution, a second loss function is used to train and supervise the detection results of salient targets.

[0148] To effectively train and optimize the salient object detection model, a second loss function can be constructed by combining multiple loss functions. That is, the loss calculated by the second loss function is the sum of the losses calculated by the multiple loss functions. In this embodiment, the second loss function is constructed by using the first loss function, the binary cross-entropy loss function, and the intersection over union (IoU) loss function. In other words, the loss calculated by the second loss function is the sum of the losses calculated by the first loss function, the binary cross-entropy loss function, and the intersection over union (IoU) loss function. It should be noted that the first loss function, in addition to supervising the boundary map, also supervises the training and optimization of the salient object detection model.

[0149] based on Figures 5 to 9 This paper describes the salient object detection performance of the trained salient object detection model, specifically using... Figure 5 The international standard datasets shown (DUTS-TE dataset, ECSSD dataset, and DUT-OMRON dataset) were used to train the salient object detection model and contrastive models (including RCSB and DPNet, etc.) of this application. , , and The evaluation metrics are used to compare the salient object detection performance of the trained salient object detection model with that of the trained control model. The comparison results are as follows: Figure 7 As shown, the table compares the salient object detection results obtained by the trained salient object detection model with the salient object detection results obtained by the trained contrast model. The comparison results are as follows: Figure 8 As shown, the trained salient object detection model in this application has good salient object detection accuracy. The comparison model is described below. Figure 7 As shown.

[0150] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0151] like Figure 10 As shown, a salient target detection system 200 according to an embodiment of the present invention includes a model training module 201 and a salient target detection module 202;

[0152] The model training module 201 is used to: train the constructed salient target detection model based on multiple sample images to obtain the trained salient target detection model. The constructed salient target detection model includes: a dual-branch encoder, a cross-model interactive fusion module, an edge-aware enhancement module, and a decoder. The dual-branch encoder extracts multiple local detail features and multiple global semantic features from the target sample image. After some local detail features and all global semantic features are interactively fused by a cross-model interactive fusion module according to a preset correspondence, multiple enhanced local detail features and multiple enhanced global detail features are obtained. Each enhanced local detail feature and its corresponding enhanced global detail feature are concatenated to obtain multiple concatenated features. After all concatenated features and the remaining local detail features are processed by the edge-aware enhancement module and the decoder, the salient target detection result is obtained. The target sample image is any target sample image.

[0153] The salient object detection module 202 is used to: perform salient object detection on the target image using the trained salient object detection model.

[0154] Optionally, in the above technical solution, the first branch of the dual-branch encoder includes N sequentially arranged first network layers, and the second branch of the dual-branch encoder includes M sequentially arranged second network layers; after inputting the target sample image into the first first network layer, the local detail features output by the first to nth first network layers are obtained;

[0155] The local detail features output by the (n-1)th first network layer are input into the first second network layer to obtain the global semantic features output by the first second network layer. Then, the local detail features output by the nth first network layer and the global semantic features output by the first second network layer are interactively fused through the first cross-model interaction fusion module. This integrates the local detail information from the nth first network layer into the global semantic features output by the first second network layer, resulting in the first enhanced global semantic feature. Finally, the global context information from the global semantic features output by the first second network layer is integrated into the local detail features output by the nth first network layer, resulting in the first enhanced global semantic feature. For local detail features, the first enhanced global semantic feature and the first enhanced local detail feature are concatenated to obtain the first concatenated feature. The first enhanced global semantic feature is used as the input to the second second network layer, and the first enhanced local detail feature is used as the input to the (n+1)th first network layer. The output of the second second network layer and the output of the (n+1)th first network layer are interactively fused through the second cross-model interaction fusion module to obtain the second enhanced global semantic feature and the first enhanced local detail feature, which are then concatenated to obtain the second concatenated feature. This process continues until the Mth concatenated feature is obtained, where n and M are both positive integers, and N = M + n - 1.

[0156] Optionally, in the above technical solution, each spliced ​​feature is processed through a convolutional layer to obtain the convolution result corresponding to each spliced ​​feature; the convolution result corresponding to the Mth spliced ​​feature is input into the ASPP module to obtain multi-scale features; the multi-scale features and the convolution result corresponding to the Mth spliced ​​feature are input into the first decoder, and the output of the first decoder and the convolution result corresponding to the Mth spliced ​​feature are input into the first edge perception enhancement module to enhance the edge information of the salient target through the first edge perception enhancement module; the (M-1)th spliced ​​feature and the output of the first edge perception enhancement module are input into the second decoder, and the output of the second decoder is input into the convolution result corresponding to the (M-1)th spliced ​​feature. The convolution result is input into the second edge perception enhancement module, until the output of the m-th edge perception enhancement module is obtained, where m is a positive integer, m=M-n+1; the output of the m-th edge perception enhancement module and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th decoder, the output of the (m+1)-th decoder and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th edge perception enhancement module, the output of the (m+1)-th edge perception enhancement module and the local detail features output by the (n-2)-th first network layer are input into the (n+2)-th decoder, until the output of the N-th decoder is obtained, and the output of the N-th decoder is used as the salient object detection result.

[0157] Optionally, in the above technical solution, the cross-model interaction and fusion module is specifically used for:

[0158] After receiving local detail features and global semantic features, the received local detail features are processed sequentially by the first channel attention module and the first spatial attention module to obtain the first feature. The received local detail features and the received global semantic features are processed by the first cross-model spatial attention module to obtain the second feature. The first feature and the second feature are added element by element to obtain the enhanced local detail features.

[0159] The received global detail features are processed sequentially by the second channel attention module and the second spatial attention module to obtain the third feature. The received local detail features and the received global semantic features are processed by the second cross-model spatial attention module to obtain the fourth feature. The third feature and the fourth feature are added element-wise to obtain the enhanced global semantic features.

[0160] Optionally, in the above technical solution, the edge perception enhancement module is specifically used to: obtain the boundary map in the received convolution result or local detail features, calculate the Hadamard product of the boundary map and the output of the received decoder, process the Hadamard product through a convolutional layer to obtain the convolution result corresponding to the Hadamard product, and add the convolution result corresponding to the Hadamard product and the output of the received decoder element by element to obtain the output of the edge perception enhancement module.

[0161] Optionally, in the above technical solution, the model training module is further used to: supervise the training of the boundary map using a first loss function and supervise the training of the salient target detection results using a second loss function during the training of the constructed salient target detection model.

[0162] It should be noted that the beneficial effects of the salient target detection system 200 provided in the above embodiments are the same as those of the salient target detection method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0163] The salient target detection system of the present invention can be a computer program (including program code) running on a computer device. For example, the salient target detection system of the present invention is an application software that can be used to execute the corresponding steps in the salient target detection method of the present invention.

[0164] In some embodiments, the salient target detection system of the present invention can be implemented in a combination of hardware and software. As an example, the salient target detection system of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the salient target detection method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0165] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0166] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned salient target detection methods. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the salient target detection method shown in any embodiment of the present invention by calling the computer program.

[0167] In one alternative embodiment, an electronic device is provided, such as Figure 11 As shown, Figure 11 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0168] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0169] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 11 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0170] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0171] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0172] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0173] It should be noted that, Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0174] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described salient target detection methods.

[0175] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0176] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the aforementioned salient target detection methods.

[0177] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0178] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0179] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EEPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0180] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0181] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0182] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0183] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0184] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A salient target detection method, characterized in that, include: Based on multiple sample images, a constructed salient object detection model is trained to obtain a trained salient object detection model. The constructed salient object detection model includes a dual-branch encoder, a cross-model interaction fusion module, an edge perception enhancement module, and a decoder. The first branch of the dual-branch encoder includes N sequentially arranged first network layers, and the second branch includes M sequentially arranged second network layers. After inputting the target sample image into the first first network layer, the local detail features output by the first to nth first network layers are obtained. The local detail features output by the (n-1)th first network layer are input into the first second network layer to obtain the output of the first second network layer. The global semantic features are generated, and the local detail features output by the nth first network layer and the global semantic features output by the first second network layer are interactively fused through the first cross-model interaction fusion module. This integrates the local detail information in the local detail features output by the nth first network layer into the global semantic features output by the first second network layer, resulting in the first enhanced global semantic feature. Similarly, the global context information in the global semantic features output by the first second network layer is integrated into the local detail features output by the nth first network layer, resulting in the first enhanced local detail feature. Finally, the first enhanced global semantic feature and the first enhanced local detail feature are concatenated. The process begins by obtaining the first concatenated feature. This first enhanced global semantic feature is then used as the input to the second layer of the second network, and the first enhanced local detail feature is used as the input to the (n+1)th layer of the first network. The outputs of the second layer and the (n+1)th layer are then fused through a second cross-model interaction fusion module to obtain the second enhanced global semantic feature and the second enhanced local detail feature. These features are then concatenated to obtain the second concatenated feature, and so on, until the Mth concatenated feature is obtained, where n and M are positive integers, and N = M + n - 1. Each concatenated feature is then processed through a convolutional layer to obtain the convolution result corresponding to each concatenated feature. The Mth concatenated feature is then... The convolution result corresponding to the first concatenated feature is input into the ASPP module to obtain multi-scale features. The convolution result of the multi-scale features and the Mth concatenated feature is input into the first decoder. The output of the first decoder and the convolution result corresponding to the Mth concatenated feature are input into the first edge perception enhancement module to enhance the edge information of the salient target. The (M-1)th concatenated feature and the output of the first edge perception enhancement module are input into the second decoder. The output of the second decoder and the convolution result corresponding to the (M-1)th concatenated feature are input into the second edge perception enhancement module. This process continues until the output of the mth edge perception enhancement module is obtained, where m is a positive integer, m = M - n + 1.The output of the m-th edge perception enhancement module and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th decoder. The output of the (m+1)-th decoder and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th edge perception enhancement module. The output of the (m+1)-th edge perception enhancement module and the local detail features output by the (n-2)-th first network layer are input into the (n+2)-th decoder, until the output of the N-th decoder is obtained. The output of the N-th decoder is used as the salient target detection result, where the target sample image is any target sample image. The cross-model interaction fusion module is specifically used for: after receiving local detail features and global semantic features, processing the received local detail features sequentially through the first channel attention module and the first spatial attention module to obtain the first feature; processing the received local detail features and the received global semantic features sequentially through the first cross-model spatial attention module to obtain the second feature; and adding the first feature and the second feature element by element to obtain the enhanced local detail features; processing the received global detail features sequentially through the second channel attention module and the second spatial attention module to obtain the third feature; processing the received local detail features and the received global semantic features sequentially through the second cross-model spatial attention module to obtain the fourth feature; and adding the third feature and the fourth feature element by element to obtain the enhanced global semantic features. The trained salient object detection model is used to detect salient objects in the target image.

2. The salient target detection method according to claim 1, characterized in that, The edge awareness enhancement module is specifically used to: acquire the boundary map in the received convolution result or local detail features, calculate the Hadamard product of the boundary map and the output of the received decoder, process the Hadamard product through a convolutional layer to obtain the convolution result corresponding to the Hadamard product, and add the convolution result corresponding to the Hadamard product and the output of the received decoder element by element to obtain the output of the edge awareness enhancement module.

3. The salient target detection method according to claim 2, characterized in that, Also includes: In training the constructed salient object detection model, a first loss function is used to supervise the training of the boundary map, and a second loss function is used to supervise the training of the salient object detection results.

4. A salient target detection system, characterized in that, It includes a model training module and a salient object detection module; The model training module is used to: train the constructed salient object detection model based on multiple sample images to obtain the trained salient object detection model. The constructed salient object detection model includes: a dual-branch encoder, a cross-model interaction fusion module, an edge perception enhancement module, and a decoder. The first branch of the dual-branch encoder includes N sequentially arranged first network layers, and the second branch of the dual-branch encoder includes M sequentially arranged second network layers. After inputting the target sample image into the first first network layer, the local detail features output by the first to nth first network layers are obtained; the local detail features output by the (n-1)th first network layer are input into the first second network layer to obtain the... The global semantic features output by the first second network layer are combined with the local detail features output by the first nth first network layer and the global semantic features output by the first second network layer through the first cross-model interaction fusion module. This process integrates the local detail information from the local detail features output by the first nth first network layer into the global semantic features output by the first second network layer, resulting in the first enhanced global semantic feature. Similarly, the global context information from the global semantic features output by the first second network layer is integrated into the local detail features output by the first nth first network layer, resulting in the first enhanced local detail feature. Finally, the first enhanced global semantic feature and the first enhanced local detail feature are combined... After concatenating the row features, the first concatenated feature is obtained. The first enhanced global semantic feature is used as the input to the second second network layer, and the first enhanced local detail feature is used as the input to the (n+1)th first network layer. The outputs of the second second network layer and the (n+1)th first network layer are interactively fused through the second cross-model interaction fusion module to obtain the second enhanced global semantic feature and the second enhanced local detail feature. These are then concatenated to obtain the second concatenated feature, and so on, until the Mth concatenated feature is obtained, where n and M are positive integers, and N = M + n - 1. Each concatenated feature is processed through a convolutional layer to obtain the convolution result corresponding to each concatenated feature. The convolution results corresponding to the M concatenated features are input into the ASPP module to obtain multi-scale features. The multi-scale features and the convolution result corresponding to the Mth concatenated feature are input into the first decoder. The output of the first decoder and the convolution result corresponding to the Mth concatenated feature are input into the first edge perception enhancement module to enhance the edge information of the salient target. The (M-1)th concatenated feature and the output of the first edge perception enhancement module are input into the second decoder. The output of the second decoder and the convolution result corresponding to the (M-1)th concatenated feature are input into the second edge perception enhancement module. This process continues until the output of the mth edge perception enhancement module is obtained, where m is a positive integer, m = M - n + 1.The output of the m-th edge perception enhancement module and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th decoder. The output of the (m+1)-th decoder and the local detail features output by the (n-1)-th first network layer are input into the (m+1)-th edge perception enhancement module. The output of the (m+1)-th edge perception enhancement module and the local detail features output by the (n-2)-th first network layer are input into the (n+2)-th decoder, until the output of the N-th decoder is obtained. The output of the N-th decoder is used as the salient target detection result, where the target sample image is any target sample image. The cross-model interaction fusion module is specifically used for: after receiving local detail features and global semantic features, processing the received local detail features sequentially through the first channel attention module and the first spatial attention module to obtain the first feature; processing the received local detail features and the received global semantic features sequentially through the first cross-model spatial attention module to obtain the second feature; and adding the first feature and the second feature element by element to obtain the enhanced local detail features; processing the received global detail features sequentially through the second channel attention module and the second spatial attention module to obtain the third feature; processing the received local detail features and the received global semantic features sequentially through the second cross-model spatial attention module to obtain the fourth feature; and adding the third feature and the fourth feature element by element to obtain the enhanced global semantic features. The salient object detection module is used to: perform salient object detection on the target image using the trained salient object detection model.

5. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a salient target detection method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements a salient target detection method according to any one of claims 1 to 3.

Images