A method for detecting salient objects based on semantic information guidance

Abstract

The present application relates to the technical field of image processing, in particular to a salient object detection method based on semantic information guidance. The present application constructs a salient object detection model based on semantic information guidance; divides the pictures in a salient image data set into a training set, a validation set and a test set of the salient object detection model; trains the salient object detection model; inputs the test set into the trained salient object detection model to obtain four evaluation indexes; when the four evaluation indexes meet the actual application requirements, the corresponding salient object detection model is used for salient object detection of images; otherwise, the learning rate is adjusted, the salient object detection model is retrained until the four evaluation indexes of the salient object detection model meet the actual application requirements. The salient object detection model constructed by the present application realizes the purpose of completely segmenting out salient objects and keeping accurate details, and improves the overall feature extraction capability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a salient target detection method based on semantic information guidance. Background Technology

[0002] Saliency object detection is a hot topic in computer vision, aiming to simulate the behavior of the human visual system by automatically identifying and locating the most salient object regions in images. Saliency object detection has wide applications in many practical fields, such as semantic segmentation, object tracking, and image retrieval. Traditional salient object detection methods are mainly based on manually designed features or heuristic rules, typically requiring manually labeled training samples of salient objects, which is extremely time-consuming and labor-intensive. Furthermore, they can become very difficult to handle large-scale datasets and struggle to adapt to complex and varied scenarios and tasks. With the development of deep learning technology, more and more researchers are adopting deep learning-based methods. These methods can automatically learn features from large amounts of data and are generally more accurate and efficient than traditional methods.

[0003] Chinese invention patent CN111209918B discloses an image saliency detection method based on multi-graph model priors and short-connection network optimization for salient object detection. This method first calculates the corresponding KNN graph model and K-regular graph model for each input RGB image using color and position information, then fuses them at the pixel level, and finally optimizes the initial saliency map using a short-connection network to obtain the final saliency map of the original image. This method improves the performance of salient object detection to some extent, but there is still a gap between its performance and practical application. He Wei and Pan Chen, in their paper "Saliency Detection Using Attention-Guided Networks," *Journal of Image and Graphics*, 2022, Vol. 27, No. 4, pp. 1176-1190, disclose a new saliency detection model, AGNet. By combining channel attention and spatial attention mechanisms, it selectively and gradually aggregates deep and shallow feature information, better handling the transmission and aggregation of features at different levels, while avoiding the influence of redundant background information on saliency mapping. This method improves the performance of salient object detection, but the overall feature extraction capability of the model is limited. Summary of the Invention

[0004] To address the limited overall feature extraction capability of current salient object detection models, this invention aims to provide a salient object detection method guided by semantic information. The specific technical solution adopted is as follows:

[0005] Construct a salient target detection model guided by semantic information;

[0006] Create a dataset of RGB saliency images; crop all images in the dataset to the same size, and use all cropped images as the saliency image dataset; divide the images in the saliency image dataset into training, validation, and test sets for the saliency object detection model;

[0007] Train and construct a salient object detection model;

[0008] The test set is input into the trained salient object detection model to test its performance and obtain four evaluation metrics.

[0009] If the four evaluation metrics meet the practical application requirements, the corresponding salient object detection model is used for salient object detection in images; otherwise, the learning rate during the training of the salient object detection model is adjusted, and the salient object detection model is retrained until the four evaluation metrics of the salient object detection model meet the practical application requirements, and then the corresponding salient object detection model is used for salient object detection in images.

[0010] Furthermore, the construction of the semantically guided salient object detection model includes:

[0011] Step 1.1: Input the RGB image into the PVTv2 backbone network and extract the image features of the four stages to generate feature representations of the salient images of the first, second, third and highest layers;

[0012] Step 1.2: Introduce an attention module at the highest layer, which includes a channel attention module and a spatial attention module, to extract the semantic information rich in the features of the highest layer and generate the highest layer feature map;

[0013] Step 1.3: Construct a high-level feature guidance module. The feature representations of the generated first, second, and third layer salient images, along with the semantic information extracted in Step 1.2, are fed into the high-level feature guidance module to enhance the low-level feature representations and generate salient feature maps at three levels.

[0014] Step 1.4: Construct an adaptive feature fusion module, whose inputs include high-level features from the highest layer output and three levels of salient feature maps generated by the high-level feature guidance module. By adding learnable weight coefficients during the fusion process, coarse salient features are generated.

[0015] Step 1.5: Construct a top-down correlation aggregation module, which guides the coarse salient features generated by the adaptive feature fusion module through the feature map generated in Step 1.1, thereby improving the accuracy of salient feature location and local details.

[0016] Furthermore, the high-level feature guidance module takes the feature representations of the generated first, second, and third layer salient images and the semantic information extracted in step 1.2 as inputs to the high-level feature guidance module, enhances the low-level feature representations, and generates three levels of salient feature maps, including:

[0017] Step 1.3.1: Combine the highest-level feature f(i,j) with the features F of each of the other layers. (n) Input into the module, where n represents the layer number and n∈{1,2,3};

[0018] Step 1.3.2: Apply global adaptive pooling to the highest-level feature f(i,j) to capture the global information of the entire feature map, as shown in formula (1):

[0019]

[0020] in, The channel statistics represent the highest-level feature f(i,j); In the diagram, C represents the number of channels; i and j are the horizontal and vertical coordinates of the highest-level feature f(i,j) in each layer; H represents the number of channels. h and W h H represents the height and width of the highest-level feature map; h and W h In this context, 'h' represents the highest floor number.

[0021] Step 1.3.3: Multiply the obtained semantic information element-wise with the low-level feature representation and sum them along the channel dimension to obtain the correlation matrix; generate probability coefficients in space using the Sigmoid function, and then multiply them with the low-level feature representation to obtain the guided low-level features, as shown in formula (2):

[0022]

[0023] Where A (n) These are the low-level features after guidance. conv(·) is a 1×1 convolutional layer, and δ(·) is the Sigmoid operation. F represents element-wise multiplication; (n) This represents the features of the nth layer.

[0024] Furthermore, the adaptive feature fusion module receives inputs including high-level features from the highest-level output and three hierarchical salient feature maps generated by the high-level feature guidance module. By incorporating learnable weight coefficients during the fusion process, coarse salient features are generated, including:

[0025] Step 1.4.1: Input the high-level features from the highest layer output and the three levels of salient feature maps generated by the high-level feature guidance module into the adaptive feature fusion module;

[0026] Step 1.4.2: During the fusion process, the input features in the adaptive feature fusion module are multiplied by the adaptive weight coefficient α. n and β n , where {α n ,β n}∈[0,1], and the salient features after fusion are obtained, as shown in formula (3):

[0027]

[0028] Among them B (n) f represents the coarse salient features after fusion. (n+1) This represents the high-level features from the previous layer's output during the fusion process, and up(·) represents the upsampling operation. This indicates element-wise addition; A (n) These are features obtained by the lower layers through this process.

[0029] Furthermore, the top-down correlation aggregation module, through the feature map generated in step 1.1, guides the coarse salient features generated by the adaptive feature fusion module, improving the accuracy of salient feature locations and local details, including:

[0030] Step 1.5.1: Extract the coarse salient features from the input. The features are passed to three different convolutional layers to obtain feature maps X, Y, and Z, where...

[0031] Step 1.5.2: Perform a sigmoid operation on the feature map generated in Step 1.1 and multiply it by X, Y, Z to construct matrices K, Q, and V, where...

[0032] Step 1.5.3: Reshape the K, Q, V matrices as follows Where N = H × W is the number of pixels, then the transpose of Q is multiplied by K and passed through a softmax layer to obtain the attention map. As shown in formula (4):

[0033]

[0034] Among them, S ij This represents the correlation between the i-th position and the j-th position; exp is an exponential function with the natural constant e as the base. K is the value of the element at the i-th position of the transpose of matrix Q; j Let J be the value of the element at the j-th position in matrix K;

[0035] Step 1.5.4: Multiply the obtained feature map S by the V matrix, and then reshape it into... Multiply by the scale parameter γ and combine with feature B (n) Perform element-wise summation to obtain the final output.

[0036] Furthermore, the training to construct the salient object detection model includes:

[0037] Step 3.1: Set training parameters; set the training batch size to Batch size = 8, the initial learning rate to Ir = 0.0001, and the number of training iterations to Epoch = 200;

[0038] Step 3.2: Input the established saliency image dataset into the saliency target detection model and train it according to the parameters set in Step 3.1;

[0039] Step 3.3: Use the Adam stochastic optimization algorithm to perform stochastic gradient descent on the salient object detection model trained in Step 3.2 to continuously optimize the loss function. Based on the loss change trend of cross-validation of the training set and validation set, until the loss change gradually tends to a stable state, determine the optimal weight.

[0040] Furthermore, the test set is input into the trained salient object detection model to test its performance, resulting in four evaluation metrics, including:

[0041] The test set is input into the trained salient object detection model to test its performance and obtain four evaluation metrics: MAE, F-measure, E-measure, and S-measure.

[0042] The embodiments of the present invention have at least the following beneficial effects:

[0043] First, this invention uses PVTv2 as the encoder of the network. Compared with CNN convolutional neural networks, PVT establishes dependencies between pixels globally through a self-attention mechanism, which can capture the contextual information of salient objects in the image and generate a global receptive field suitable for maintaining the structural integrity of salient objects.

[0044] Second, the present invention constructs a high-level feature guidance module, which supplements the low-level features by obtaining the importance of each spatial location and feature channel of the highest-level semantic features.

[0045] Third, the present invention constructs an adaptive feature fusion module, which improves the accuracy of significant targets by adaptively adding a set of weight coefficients to features from different layers, and at the same time reduces redundant features in the fusion process.

[0046] Fourth, this invention constructs a top-down correlation aggregation module, which guides the lower-level feature maps to establish a correlation matrix through global feature information from high-level feature maps, and aggregates them to obtain salient feature maps, guiding the network layer by layer to generate the final salient objects. The salient object detection model constructed in this invention achieves the goal of completely segmenting salient objects while maintaining accurate details, thus improving the overall feature extraction capability of the salient object detection model. Attached Figure Description

[0047] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 The flowchart illustrates a semantically guided salient target detection method according to an embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of a network structure provided in one embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of a high-level feature guidance module provided in one embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of a top-down correlation aggregation module provided in one embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram illustrating the visualization results of a semantically guided salient target detection model provided in an embodiment of the present invention in a real-world application scenario. Detailed Implementation

[0053] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a semantically guided salient target detection method proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0055] This invention provides a specific implementation method for a semantically guided salient object detection method, applicable to salient object detection scenarios. To address the limited overall feature extraction capability of current salient object detection models, this invention constructs a high-level feature guidance module, an adaptive feature fusion module, and a top-down correlation aggregation module to build and train a salient object detection module. The salient object detection model constructed by this invention achieves the goal of completely segmenting salient objects while preserving accurate details, thus improving the overall feature extraction capability of the salient object detection model.

[0056] The following description, in conjunction with the accompanying drawings, details a specific scheme for a semantically guided salient target detection method provided by the present invention.

[0057] Please see Figure 1 The diagram illustrates a flowchart of a semantically guided saliency target detection method according to an embodiment of the present invention, which includes the following steps:

[0058] Step S100: Construct a salient target detection model guided by semantic information.

[0059] Please see Figure 2 , Figure 2 This is a schematic diagram of the network structure of the present invention. The specific steps for constructing the salient object detection model are as follows:

[0060] Step 1.1: Input the RGB image into the PVTv2 backbone network and extract the image features of the four stages to generate feature representations of the first, second, third and top layer salient images.

[0061] Step 1.2: Introduce an attention module at the highest layer. This attention module includes a channel attention module and a spatial attention module to extract rich semantic information from the highest-layer features and generate a highest-layer feature map. Specifically, the attention module models the semantic relationships between different channels and positions on the highest-layer feature map F, reweights the feature maps in different channels, and estimates the accurate location information of salient objects, resulting in more robust global semantic features, denoted as . h represents the highest level, H h and W h This represents the height and width of the highest-level feature map.

[0062] Step 1.3: Construct a high-level feature guidance module. The feature representations of the generated first, second, and third layer salient images, along with the semantic information extracted in Step 1.2, are fed into the high-level feature guidance module to enhance the low-level feature representations and generate three levels of salient feature maps. (See also...) Figure 3 , Figure 3This is a schematic diagram of the high-level feature guidance module of the present invention.

[0063] Step 1.3 specifically includes:

[0064] Step 1.3.1: Combine the highest-level feature f(i,j) with the features F of each of the other layers. (n) Input into the module, where n represents the layer number and n∈{1,2,3};

[0065] Step 1.3.2: Apply global adaptive pooling to the highest-level feature f(i,j) to capture the global information of the entire feature map, as shown in formula (1):

[0066]

[0067] in, The channel statistics represent the highest-level feature f(i,j); In the diagram, C represents the number of channels; i and j are the horizontal and vertical coordinates of the highest-level feature f(i,j) in each layer; H represents the number of channels. h and W h H represents the height and width of the highest-level feature map; h and W h In this context, 'h' represents the highest level.

[0068] Step 1.3.3: Multiply the obtained semantic information element-wise with the low-level feature representation and sum them along the channel dimension to obtain the correlation matrix; generate probability coefficients in space using the Sigmoid function, and then multiply them with the low-level feature representation to obtain the guided low-level features, as shown in formula (2):

[0069]

[0070] Where A (n) These are the features obtained from the lower layers through this process, i.e., the guided lower-layer features. Conv(·) is a 1×1 convolutional layer, and δ(·) is the Sigmoid operation. F represents element-wise multiplication; (n) This represents the features of the nth layer.

[0071] Low-level features are supplemented and enhanced by the high-level feature guidance module. The features generated by the high-level feature guidance module are represented as A. (1) A (2) A (3) .

[0072] Since the feature details obtained from each layer are different, the proportions they contribute during feature fusion will also differ. To efficiently utilize the feature information from each layer, this embodiment of the invention employs a selective fusion strategy to suppress redundant information and constructs an adaptive feature fusion module.

[0073] Step 1.4: Construct an adaptive feature fusion module. Its input includes high-level features from the highest layer output and three levels of salient feature maps generated by the high-level feature guidance module. By adding learnable weight coefficients during the fusion process, redundant information is reduced and salient features are further enhanced to generate coarse salient features.

[0074] Specifically:

[0075] Step 1.4.1: Input the high-level features from the highest layer output and the three levels of salient feature maps generated by the high-level feature guidance module into the adaptive feature fusion module.

[0076] Step 1.4.2: During the fusion process, the input features in the adaptive feature fusion module are multiplied by the adaptive weight coefficient α. n and β n , where {α n ,β n}∈[0,1], and the salient features after fusion are obtained, as shown in formula (3):

[0077]

[0078] Among them B (n) f represents the coarse salient features after fusion. (n+1) This represents the high-level features from the previous layer's output during the fusion process, and up(·) represents the upsampling operation. This indicates element-wise addition; A (n) These are features obtained by the lower layers through this process.

[0079] The lower layer obtains a coarse salient result through the adaptive feature fusion module. If the result is output directly, the salient region obtained will rarely match the real salient region. It lacks complete information about the salient target. Therefore, a top-down correlation aggregation module is constructed.

[0080] Step 1.5: Construct a top-down correlation aggregation module. This module guides the coarse salient features generated by the adaptive feature fusion module using the feature map generated in Step 1.1, improving the accuracy of salient feature locations and local details. (See also...) Figure 4 , Figure 4 This is a schematic diagram of the top-down correlation aggregation module of the present invention.

[0081] Step 1.5 specifically includes:

[0082] First, bilinear interpolation upsampling is performed on the high-level output feature map. Then, the Sigmoid function is used to map the values ​​to the range [0, 1], converting them into probability values. This probability is used to adjust the range and intensity of the attention region in the low-level feature map. The closer the probability is to 1, the more likely the pixel is to belong to the foreground target, while the closer it is to 0, the more likely the pixel is to belong to the background pixel.

[0083] Step 1.5.1: Extract the coarse salient features from the input. The features are passed to three different convolutional layers to obtain feature maps X, Y, and Z, where...

[0084] Step 1.5.2: Perform a sigmoid operation on the feature map generated in Step 1.1 and multiply it by X, Y, Z to construct matrices K, Q, and V, where...

[0085] Step 1.5.3: Reshape the K, Q, V matrices as follows Where N = H × W is the number of pixels, then the transpose of Q is multiplied by K and passed through a softmax layer to obtain the attention map. As shown in formula (4):

[0086]

[0087] Where S ij This represents the correlation between the i-th position and the j-th position; exp is an exponential function with the natural constant e as the base. K is the value of the element at the i-th position of the transpose of matrix Q; j Let be the value of the element at the j-th position in matrix K.

[0088] Step 1.5.4: Multiply the obtained feature map S by the V matrix, and then reshape it into... Multiply by the scale parameter γ and combine with feature B (n) Perform element-wise summation to obtain the final output. As shown in formula (5):

[0089]

[0090] Where γ is an adaptive parameter, initialized to 0. (C) at each position (n) ) j It is a weighted sum of cross-space global features and original category features; S ij B represents the correlation between the i-th position and the j-th position; (n) The coarse, salient features after fusion; V iThis represents the value of the element at the i-th position in matrix V. The top-down correlation aggregation module, through learned global context information, can more accurately focus on the features of salient regions, improving the integrity of salient targets.

[0091] Step S200: Establish a dataset of RGB saliency images; crop all images in the dataset to the same size, and use all cropped images as the saliency image dataset; divide the images in the saliency image dataset into training set, validation set and test set for the saliency object detection model.

[0092] First, all images in the dataset are randomly cropped to the same size, and all cropped images are used as the saliency image dataset. Then, the images in the saliency image dataset are used as the training set, validation set, and test set of the model in a ratio of 7:2:1.

[0093] Step S300: Train and construct a salient object detection model.

[0094] Specifically:

[0095] Step 3.1: Set training parameters. Set the batch size to Batchsize = 8, the initial learning rate to Ir = 0.0001, and the number of training iterations to Epoch = 200.

[0096] Step 3.2: Input the established saliency image dataset into the saliency target detection model and train it according to the parameters set in Step 3.1.

[0097] Step 3.3: Use the Adam stochastic optimization algorithm to perform stochastic gradient descent on the salient object detection model trained in Step 3.2 to continuously optimize the loss function. Based on the loss change trend of cross-validation of the training set and validation set, until the loss change gradually tends to a stable state, determine the optimal weight.

[0098] Step S400: Input the test set into the trained salient object detection model, test the performance of the salient object detection model, and obtain four evaluation metrics.

[0099] The test set is input into the trained semantic information-guided salient object detection model to test its performance. Based on the test results, the performance of the semantic information-guided salient object detection model is evaluated on three public datasets using four evaluation metrics: MAE, F-measure, E-measure, and S-measure.

[0100] Step S500: When the four evaluation metrics meet the practical application requirements, the corresponding salient object detection model is used for salient object detection in the image; otherwise, the learning rate during the training of the salient object detection model is adjusted, and the salient object detection model is retrained until the four evaluation metrics of the salient object detection model meet the practical application requirements, and the corresponding salient object detection model is used for salient object detection in the image.

[0101] Determine whether the MAE, F-measure, E-measure, and S-measure of the evaluation results of the semantic information-guided salient target detection model meet the requirements of practical applications.

[0102] If the evaluation results are not ideal, that is, they do not meet the actual application requirements, adjust the learning rate of the semantic information-guided salient object detection model in step S3, and retrain the salient object detection model until the four evaluation indicators of the salient object detection model meet the actual application requirements, and then use the corresponding salient object detection model for salient object detection in images.

[0103] Once the four evaluation metrics meet the requirements of practical applications, the semantically guided salient object detection model will be used for salient object detection in real-world image scenarios. Please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram illustrating the visualization results of the semantically guided salient object detection model in a real-world application scenario. It can be seen that the salient object detection model obtained based on this invention can completely segment salient objects in an image while preserving accurate details.

[0104] In summary, this invention relates to the field of image processing technology. The method first constructs a salient object detection model guided by semantic information; establishes a dataset of RGB salient images; crops all images in the dataset to the same size, using these cropped images as the salient image dataset; divides the images in the salient image dataset into training, validation, and test sets for the salient object detection model; trains and constructs the salient object detection model; inputs the test set into the trained salient object detection model to test its performance, obtaining four evaluation metrics; when the four evaluation metrics meet the practical application requirements, the corresponding salient object detection model is used for salient object detection in images; otherwise, the learning rate during training is adjusted, and the salient object detection model is retrained until the four evaluation metrics of the salient object detection model meet the practical application requirements, at which point the corresponding salient object detection model is used for salient object detection in images. This invention achieves the goal of completely segmenting salient objects while preserving accurate details.

[0105] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A salient target detection method based on semantic information guidance, characterized in that, The method includes the following steps: Construct a salient target detection model guided by semantic information; Create a dataset of RGB saliency images; crop all images in the dataset to the same size, and use all cropped images as the saliency image dataset; divide the images in the saliency image dataset into training, validation, and test sets for the saliency object detection model; Train and construct a salient object detection model; The test set is input into the trained salient object detection model to test its performance and obtain four evaluation metrics. If the four evaluation metrics meet the requirements of practical application, the corresponding salient object detection model is used for salient object detection in images; otherwise, the learning rate during the training of the salient object detection model is adjusted, and the salient object detection model is retrained until the four evaluation metrics of the salient object detection model meet the requirements of practical application, and then the corresponding salient object detection model is used for salient object detection in images. The construction of a salient object detection model guided by semantic information includes: Step 1.1: Input the RGB image into the PVTv2 backbone network and extract the image features of the four stages to generate feature representations of the salient images of the first, second, third and highest layers; Step 1.2: Introduce an attention module at the highest layer, which includes a channel attention module and a spatial attention module, to extract the semantic information rich in the features of the highest layer and generate the highest layer feature map; Step 1.3: Construct a high-level feature guidance module. The feature representations of the generated first, second, and third layer salient images, along with the semantic information extracted in Step 1.2, are fed into the high-level feature guidance module to enhance the low-level feature representations and generate salient feature maps at three levels. Step 1.4: Construct an adaptive feature fusion module, whose inputs include high-level features from the highest layer output and three levels of salient feature maps generated by the high-level feature guidance module. By adding learnable weight coefficients during the fusion process, coarse salient features are generated. Step 1.5: Construct a top-down correlation aggregation module, which uses the feature map generated in Step 1.1 to guide the coarse salient features generated by the adaptive feature fusion module, thereby improving the accuracy of salient feature location and local details; Step 1.3 further includes: Step 1.3.1: Extract the highest-level features Features from other layers respectively Input into the module, where Indicates the number of layers and ; Step 1.3.2: For the highest-level features Global adaptive pooling is used to capture global information of the entire feature map, as shown in Equation (1): （1） in, Represents the highest level features Channel statistics; middle Indicates the number of channels; These are the highest level features The horizontal and vertical coordinates of each layer; and This represents the height and width of the highest-level feature map; and In Indicates the highest floor number; Step 1.3.3: Multiply the obtained semantic information element-wise with the low-level feature representation and sum them along the channel dimension to obtain the correlation matrix; generate probability coefficients in space using the Sigmoid function, and then multiply them with the low-level feature representation to obtain the guided low-level features, as shown in formula (2): （2） in These are low-level features after guidance. yes Convolutional layer It is the Sigmoid operation. This indicates element-wise multiplication; This represents the features of the nth layer.

2. The salient target detection method based on semantic information guidance according to claim 1, characterized in that, The adaptive feature fusion module is constructed, and its inputs include high-level features from the highest-level output and three hierarchical salient feature maps generated by the high-level feature guidance module. By adding learnable weight coefficients during the fusion process, coarse salient features are generated, including: Step 1.4.1: Input the high-level features from the highest layer output and the three levels of salient feature maps generated by the high-level feature guidance module into the adaptive feature fusion module; Step 1.4.2: During the fusion process, the input features in the adaptive feature fusion module are multiplied by adaptive weight coefficients. and ,in The salient features obtained after fusion are shown in formula (3): （3） in These are the coarse, salient features after fusion. This represents high-level features from the output of the previous layer during the fusion process. Indicates an upsampling operation. This indicates element-wise addition. These are features obtained by the lower layers through this process.

3. The salient target detection method based on semantic information guidance according to claim 1, characterized in that, The top-down correlation aggregation module, through the feature map generated in step 1.1, guides the coarse salient features generated by the adaptive feature fusion module, improving the accuracy of salient feature locations and local details. This includes: Step 1.5.1: Extract the coarse salient features from the input. The features are passed to three different convolutional layers to obtain feature maps. , , ,in ; Step 1.5.2: Perform a Sigmoid operation on the feature map generated in Step 1.1 and then... , , Constructing by multiplication , , Matrix, where ; Step 1.5.3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] , , Matrix reshaping ,in For the number of pixels, then... transpose and Multiply and then pass through a softmax layer to obtain the attention map. As shown in formula (4): （4） in, Indicates the first The position and the first The correlation of each position; exp is an exponential function with the natural constant e as the base; Let be the value of the element at the i-th position of the transpose of the Q matrix; Let J be the value of the element at the j-th position in matrix K; Step 1.5.4: Obtain the feature map and Matrix multiplication, then reshaping Multiply by scale parameter and features Perform element-wise summation to obtain the final output. .

4. The salient target detection method based on semantic information guidance according to claim 1, characterized in that, The training process constructs a salient object detection model, including: Step 3.1: Set training parameters; set the training batch size to Batch size = 8, the initial learning rate to Ir = 0.0001, and the number of training iterations to Epoch = 200; Step 3.2: Input the established saliency image dataset into the saliency target detection model and train it according to the parameters set in Step 3.1; Step 3.3: Use the Adam stochastic optimization algorithm to perform stochastic gradient descent on the salient object detection model trained in Step 3.2 to continuously optimize the loss function. Based on the loss change trend of cross-validation of the training set and validation set, until the loss change gradually tends to a stable state, determine the optimal weight.

5. The salient target detection method based on semantic information guidance according to claim 1, characterized in that, The test set is input into the trained salient object detection model to test its performance, resulting in four evaluation metrics, including: The test set is input into the trained salient object detection model to test its performance and obtain four evaluation metrics: MAE, F-measure, E-measure, and S-measure.

Images