A multi-task prompting-based optical remote sensing image salient object detection method

Abstract

The application discloses a kind of based on multi-task prompt optical remote sensing image saliency target detection method, to improve the detection precision of saliency target, solve the problem of edge blur and information loss in prior art.The method first maximizes the difference between saliency task and edge task by designing task prompt, reduces the gradient conflict between the two, optimizes the cooperation between the two.Then propose MTP module to learn and feature interaction of multiple tasks, improve the utilization effect of edge information by fusing fine edge features.In order to better align the saliency region features, SEALoss is introduced, and the alignment of saliency region is optimized by edge perception mask to improve the detection accuracy.Meanwhile, PRA is introduced, which significantly reduces the computational load and memory consumption, and improves the operation efficiency of the model.The method can robustly and accurately detect saliency targets in optical remote sensing images, adapt to complex scenes, and has strong practicality and popularization value.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and relates to the improvement of salient target detection models, specifically a salient target detection method for optical remote sensing images based on multi-task prompts. Background Technology

[0002] With the development of salient target detection, it has recently become a hot topic in the field of remote sensing image processing. The main goal of this technology is to quickly and accurately detect salient targets, such as ships, aircraft, and buildings, from a large number of remote sensing images. However, due to the complexity and diversity of remote sensing images, as well as the diversity of the targets themselves, salient target detection remains a significant challenge.

[0003] Remote sensing images typically contain a wealth of background information, such as clouds, oceans, and forests, which can interfere with target detection. Furthermore, the appearance of targets can vary due to factors such as image resolution, lighting conditions, and viewing angle, further increasing the difficulty of target detection. Therefore, it is necessary to develop specialized remote sensing image detection algorithms capable of handling various targets to better process complex and diverse optical remote sensing images. Summary of the Invention

[0004] 1. Purpose of the invention:

[0005] To address the shortcomings of existing technologies, this invention provides a salient target detection method for optical remote sensing images based on multi-task prompts. This solves the problem of poor detection performance of current optical remote sensing image salient target detection models in some complex scenes.

[0006] 2. Technical Solution:

[0007] This invention proposes a salient target detection method for optical remote sensing images based on multi-task prompts. The specific implementation steps of this method are as follows:

[0008] Step (1): Introduce task cues from the encoder into the backbone network, and extract multi-scale feature maps f from the backbone network. 1 f 2 f 3 f 4 An MTP module is added to the decoder, and task prompts from the decoder are introduced into the MTP module. The multi-scale feature map is input into the MTP module. The MTP module captures the intrinsic correlation between edge tasks and salient tasks in the multi-scale feature map by interacting between specific task tokens. The MTP module includes a cross-layer fusion block, a task learning block, a task interaction block, and a task-specific prediction block.

[0009] Step (2): The multi-scale feature map f obtained in step (1) 1 f 2 f 3 f 4 The input is fed into the cross-layer fusion block, which fuses features at different scales to obtain a more comprehensive and richer multi-scale feature map.

[0010] Step (3): The multi-scale feature map obtained in step (2) The input is fed into the task learning block, where task learning is performed through a main task-specific learning branch and another auxiliary task-specific learning branch. Then, the PRA (Personal Recognition Analyzer) is used to learn the respective task tokens and prompts, thereby adaptively distinguishing the importance of tasks, realizing the intrinsic connection between tasks and task prompts, and obtaining new tasks. and general characteristics of the task Meanwhile, the construction of joint feature representations and MHSA enable the model to share information across different tasks and learn richer feature representations with long-distance dependencies;

[0011] Step (4): The task obtained in step (3) The task interaction block is input into the task interaction block for interaction processing. The task-related token is used to perform corresponding task interaction, and finally mapped to the corresponding task features to obtain the token after task interaction.

[0012] Step (5): Combine the token obtained in step (4) after task interaction with the general task feature obtained in step (3). The input is fed into the task-specific prediction block, and multiple prediction maps are obtained by performing predictions on the salient object detection and the marginal object detection in the main task-specific learning branch and another auxiliary task-specific learning branch, respectively. The prediction maps include a salient object detection prediction map and a marginal object detection prediction map, wherein the salient object prediction map is obtained through SEALoss supervision.

[0013] Preferably, the specific implementation steps of the task prompt are as follows: by introducing a similarity loss function, the difference between salient tasks and marginal tasks is explicitly maximized, and the importance of each task in the detection process is adaptively distinguished, thereby improving the effectiveness of inter-task collaboration.

[0014] Preferably, the SEALoss improves detection accuracy and robustness by aligning salient region features with edge-aware masks.

[0015] Preferably, the PRA significantly reduces the computational load and memory consumption of the model through multi-scale pooling, and then introduces a linear attention mechanism to achieve efficient and accurate salient object detection.

[0016] 3. Beneficial effects:

[0017] (1) This invention solves the gradient conflict problem caused by the edge detection task of salient target by using task prompts, and learns the inherent correlation between tasks through the MTP module.

[0018] (2) The model proposed in this invention has superior performance. On the two optical remote sensing image datasets EORSSD and ORSSD, the algorithm of this invention is better than the existing advanced algorithms. Attached Figure Description

[0019] Figure 1 This is the overall process framework of the present invention;

[0020] Figure 2 This is a flowchart illustrating the implementation of the MTP module of the present invention;

[0021] Figure 3 This is a comparison chart of the PRA complexity and self-attention complexity of the present invention;

[0022] Figure 4 This is a visualization of the challenge situation of the present invention on the EORSSD dataset. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. This embodiment applies a Swing-Transformer encoder, which employs a Swing-Tiny backbone network, and provides a method for salient target detection in optical remote sensing images based on multi-task cues. The specific implementation steps are as follows:

[0024] (S1): Design task prompts

[0025] Features at encoder 1 / 4, 1 / 8, 1 / 16, and 1 / 32 were extracted from the Swin-Tiny backbone network, and task-specific cues were introduced into each encoder block. The task prompt is concatenated with the features in the encoder block to form a feature block, which exists as a spatial task prompt.

[0026] Given image features f of a specific block in a Swin-Transformer encoder i And use window attention and feature f i Classified into window features Where M represents the window size, l i / M 2That's the number of windows. Then, copy the prompt for each window. and obtain Where N i c represents the number of learnable hint tokens. i This represents the number of channels. Next, these are appended to the patch feature token in each window, and self-attention is performed within each window, as shown in Equation (1):

[0027]

[0028] W-MHSA and SW-MHSA are multi-head self-attention modules with regular and shifted window configurations, respectively. Residual connections and layer normalization are omitted here. Subsequently, the layers are split from each window. And calculate their average value to obtain encoder task hints. Finally, the output window features f iw The feature f of reassembling into the next block i+1 .

[0029] An MTP module is added to the decoder, in which task cues are introduced. The MTP module includes a cross-layer fusion block, a task learning block, a task interaction block, and a task-specific prediction block. Regarding task-specific cues in the decoder, learnable cues are simply appended to specific blocks from the decoder. Decoder feature tag f i+1 Then, self-attention is applied according to the following formula (2) to obtain the task cue for the i-th decoder. and the cross-layer features of the (i+1)th decoder

[0030]

[0031] MHSA stands for Multi-head Self-Attention, and MLP stands for Multiple Perceptron.

[0032] (S2) Cross-layer fusion block

[0033] like Figure 2 As shown in the (cross-layer fusion block), the specific implementation process of the cross-layer fusion block is as follows: for the input feature f i-1 and f i Processing is then performed. The primary operation is to convert the lower-sized features f... i Upsampling to f i-1By using the same size, features at different scales are aligned and integrated to facilitate subsequent feature fusion and interaction. Next, the two inputs are concatenated along the channel dimension, organically fusing feature information from both scales to construct a richer and more diverse feature representation. Then, a linear projection maps the channel dimension to the C-dimensional dimension. This mapping operation adjusts and unifies the feature dimensions, providing a more unified feature representation space for subsequent feature processing and task adaptation. Finally, a multi-scale feature map is obtained. This provides the model with more comprehensive and richer feature representations to facilitate subsequent task processing and information transmission.

[0034] (S3) Task Learning Block

[0035] like Figure 2 As shown in the (task learning block), the specific implementation process of the task learning block is as follows: receiving multi-scale feature maps. Next, initialize the tokens for the salient task and the edge task as follows: and And randomly initialize salient task cues and edge task cues. For learnable parameters, and Connecting along the channel dimension yields the joint feature representation X. f As shown in the formula below:

[0036]

[0037] Where Concat represents the connection operation on the channel dimension, which is updated and iterated in each round of training. Then the joint feature representation X f Utilize PRA for the token and features of the required task Make predictions to obtain a new joint feature representation X′ f The process is as follows:

[0038] X′ f =LN(PRA(X9)) (4)

[0039] Here, LN represents layer normalization. Next, to learn a more accurate joint feature representation and to learn the intrinsic relationship between task tokens, task cues, and tasks, the following processing is performed:

[0040] X″ f =MLP(X″) f )+X″ f +X f (5)

[0041] After this process, a new task assistance token is obtained: and general characteristics of the task Subsequently, in order to learn the respective feature representations for the corresponding tasks, MHSA in Transformer is used, with general features of the task as queries and task-specific tokens as keys and values, to obtain salient features f″. s i and edge features f″ e i The process is as follows:

[0042]

[0043] To prevent potential interference between task interactions, a task-specific learning branch is added for supervision to increase model stability. Figure 2 As shown, the task-specific features of another branch were obtained.

[0044] (S4) Cross-task interaction block

[0045] like Figure 2 As shown in the (cross-task interaction block), the specific implementation process of the cross-task interaction block is as follows: First, concatenate the tokens from different tasks. Then, apply a multi-head self-attention mechanism to capture the global relationships and long-distance dependencies between features. Through this process, the model can more comprehensively understand the intrinsic connections between tasks. Finally, use MLP to further refine the tokens after self-attention processing to obtain... This enhances its expressive power, and the process can be described as follows:

[0046]

[0047] Then, the interactively processed features are separated along the channel dimension to obtain the tokens t′ for the two tasks. s i ,t′ e i As shown in the formula below:

[0048]

[0049] The tokens for the obtained salient and marginal tasks are interactively processed. Unlike other methods, this embodiment uses task-related tokens to perform corresponding task interactions and finally maps them to the corresponding task features. Since the feature sequence length L is much larger than the token sequence length L′, this method can obtain deep-level information in task interactions while greatly reducing the amount of computation.

[0050] (S5) Specific Task Prediction Block

[0051] like Figure 2 As shown in the (task-specific prediction block), the specific implementation process of the task-specific prediction block is as follows: The final feature map is obtained through a two-branch task-specific prediction. In the first branch, the tokens after task interaction and the corresponding task-specific features are processed by LN and transformed by MLP, respectively. Subsequently, their outputs are fused through matrix multiplication. The final task-specific features are then transformed into a single-channel representation, which is used for the final prediction to obtain the saliency prediction map s. i With edge prediction graph e i This process can be represented by the following formula:

[0052]

[0053] Here, ⊙ represents matrix multiplication. This process ensures the layer-by-layer refinement and fusion of task-specific information. LN independently normalizes each input to eliminate input bias and scaling issues; MLP enhances the complexity of feature representation through nonlinear transformations; and matrix multiplication deeply fuses the processed tokens with features, further strengthening information exchange between tasks. Finally, dimensionality transformation simplifies the feature map, making it suitable for prediction.

[0054] The other branch (auxiliary branch) directly performs LN and MLP operations on task-specific features, and then applies linear layer mapping to obtain an auxiliary prediction map. and This can be expressed as the following formula:

[0055]

[0056]

[0057] Through this process, the model can gain a deeper understanding and representation of the complex relationships and long-distance dependencies between tasks, thereby improving overall performance and making the final prediction.

[0058] (S6)PRA

[0059] like Figure 3 As shown, the complexity of PRA is compared with that of traditional self-attention. It can be clearly seen that PRA reduces the computational complexity to linear, such as... Figure 3 As shown in (b), the specific implementation process of the PRA design is as follows: given input First, the query matrix Q is obtained through a linear transformation, as follows:

[0060]

[0061] in These are learnable parameters. Then Q is reshaped into... Where C = h × d, h represents the number of attention heads, and d is the number of channels per attention head. Before performing the pooling operation, the input X is transferred from... Remodeling Where H and W represent the height and width of the feature map, multi-scale pooling operations are then performed along the spatial dimensions:

[0062] P1 = AvgPool1(X) - ),P2=AvgPool2(X - ),…,P n =AvgPool n (X - (17)

[0063] Where P1, P2, ..., P n The generated pyramid feature maps correspond to different pooling layers, with AvgPool representing adaptive average pooling. These multi-scale pyramid feature maps are then input into a deep convolutional network for relative position encoding.

[0064]

[0065] Where DWConv represents the depthwise convolution operation, with a kernel size of 3×3. For the encoded pyramid feature map P i .

[0066] Next Reshape back to the same shape as the input:

[0067]

[0068] Where N i This represents the number of tokens remaining after processing by the i-th pooling layer. Then, these encoded pyramid feature maps are concatenated:

[0069]

[0070] Finally obtained Where M = (N1 + N2 + ... + N) n ),therefore

[0071] like Figure 3 As shown in (a), when the pooling window is large enough, the P sequence is significantly shorter than the input sequence. This significantly reduces computational complexity. Furthermore, P captures contextual information about the input X, thus P can serve as an efficient substitute for X when computing MHSA. Given input The PRA attention mechanism can be described as follows:

[0072]

[0073] Where Q = XW q K = PW k V = PW Q , It is a learnable parameter, A i This represents the i-th row of the output matrix. This corresponds to the standard softmax attention mechanism, Q. i ,K j V j This represents the query, key, and value.

[0074] In this embodiment, as Figure 3 As shown in (b), the PRA uses a ReLU-based linear attention mechanism to achieve a global receptive field and linear computational complexity. In ReLU linear attention, the similarity function is defined as:

[0075] Sim(Q,K)=ReLU(Q i )ReLU(K j ) T (twenty two)

[0076] At this point, the PRA attention mechanism in this embodiment is restated as follows:

[0077]

[0078] By utilizing the associative properties of matrix multiplication, the computational complexity and memory usage can be reduced from quadratic to linear without changing its functionality. As shown in the equation above, only the following calculation is required: and Once, then they can be reused for each query, thus requiring only Computational overhead and This allows for a more efficient balance between computational cost and performance when learning long-range dependencies in images.

[0079] (S7)SEALoss

[0080] The specific implementation process of SEALoss is as follows: (e.g.) Figure 1 As shown. First, by calculating the dilation and erosion of the predicted image, an edge mask is generated to highlight the edge regions of the predicted image:

[0081] Edge=Dilation(Pred)-Erosion(Pred) (24)

[0082] This allows us to focus on the registration of image boundaries in subsequent calculations. Next, through local average pooling, we extract the average features (AvgFeatures, AvgPred) for each window from both the ground and predicted images.

[0083] AvgFeatures=AvgPool(Label) (25)

[0084] AvgPred = AvgPool(Pred) (26)

[0085] Here, Label represents the ground truth image, and the size of this window is determined by a parameter r, with a diameter equal to 2 × r + 1. Then, based on local average features, an alignment metric between the predicted and ground truth images is calculated, using cosine similarity as the metric.

[0086] Alignment=1-CosineSimilarity(AvgFeatures,AvgPred) (27)

[0087] Next, the absolute difference of the predicted image is calculated, and the difference is multiplied by the alignment metric to obtain the alignment distance:

[0088] Distance=Abs(AvgFeatures-AvgPred)×Alignment (28)

[0089] This distance represents the degree of alignment between the predicted image and the ground truth image within a local region. Finally, a weighted loss is calculated. This loss is obtained by multiplying the alignment distance by the edge mask and summing the results, as shown below:

[0090]

[0091] The summation is performed within regions where the edge mask is non-zero, and the loss in edge regions is given greater weight. This method aims to leverage edge information of the image to enhance the alignment process, thereby improving the local alignment between the ground truth and the predicted image.

[0092] The effects of the present invention will be described in detail below with reference to experimental data and prediction graphs.

[0093] Table 1 quantitatively compares the method of this invention with 22 state-of-the-art methods, including two traditional natural image salient object detection methods (T-NSI), three CNN-based natural image salient object detection methods (C-NSI), two hybrid optical remote sensing image salient object detection methods (H-RSI), two Transformer-based optical remote sensing image salient object detection methods (T-RSI), and thirteen CNN-based optical remote sensing image salient object detection methods (C-RSI). The best results are indicated in bold. "-" indicates no data is available. It is clear from Table 1 that the method of this invention demonstrates superior performance on both optical remote sensing image salient object detection datasets compared to other methods. Of the eight metrics listed, except for the EORSSD dataset... Furthermore, the method of this invention completely surpasses other comparative methods in all metrics.

[0094] Table 1 compares this embodiment with state-of-the-art methods on the EORSSD and ORSSD datasets.

[0095]

[0096]

[0097] Table 2 Ablation experimental results of different improved methods on ORSSD and EORSSD datasets.

[0098]

[0099] The best results in the table are shown in bold. To verify the effectiveness of the seeding improvement method in the model, six variants were designed, as shown in Table 2: 1) Using only BaseLine, with the decoder replaced by the most basic Transformer block; 2) MTP (introducing the MTP module); 3) SEALoss (introducing SEALoss); 4) MTP + task prompts (adding task prompts to MTP); 5) MTP + SEALoss (introducing SEALoss to MTP); 6) MTP + task prompts + SEALoss (the complete MTPNet). As shown in the table above, different improvement methods improved all four metrics in both datasets. Adding SEALoss to the BaseLine also improved all metrics. The improvements are particularly significant. Under the condition of MTP+task hints, a further performance improvement was achieved compared to introducing only the MTP module. Similarly, deploying MTP+SEALoss also showed a certain improvement compared to introducing only the MTP module. The complete MTPNet significantly outperforms Baseline, achieving improvements of 2.73%, 3.15%, 2.65%, and 40.47% on these three metrics, respectively. Similar trends were observed on the ORSSD dataset. Clearly, with the addition of each improvement method, the overall performance of the model continuously improves, highlighting the effectiveness of the proposed improvement methods.

[0100] like Figure 4 As shown, visualizations of the EORSSD dataset in some challenging scenarios are presented to provide a clear comparison. It can be observed that the model of this invention exhibits more complete and accurate detection of salient objects in complex scenes. Figure 4 As shown in (b), compared to other methods for detecting salient objects, the method proposed in this invention can detect salient objects more completely. This is attributed to its ability to effectively share feature representations, allowing the model to fully utilize the beneficial details of edges. For Figure 4 (a) Large target object. Figure 4 (c) Multiple small salient targets and Figure 4 (e) For salient targets not located in the center, in these challenging scenarios, the method of this invention can accurately detect salient targets, while other methods incorrectly highlight cluttered background areas. This is mainly due to the fact that the model of this invention can capture long-range dependent contextual information, while task prompts improve the technology's ability to capture targets and boundaries in the image, better balancing global features and local details. And for... Figure 4 (d) For narrow and prominent targets, since the SEALoss proposed in this invention enables precise registration of local regions, the method of this invention is better able to guarantee the integrity of the object and edge details compared to other methods.

[0101] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for salient target detection in optical remote sensing images based on multi-task prompts, comprising the following steps: Step (1): Introduce task cues from the encoder into the backbone network, and extract multi-scale feature maps from the backbone network. , , , An MTP module is added to the decoder, and task prompts from the decoder are introduced into the MTP module. The multi-scale feature map is input into the MTP module. The MTP module captures the intrinsic correlation between edge tasks and salient tasks in the multi-scale feature map by interacting between specific task tokens. The MTP module includes a cross-layer fusion block, a task learning block, a task interaction block, and a specific task prediction block. Step (2): The multi-scale feature map obtained in step (1) , , , The input is fed into the cross-layer fusion block, which fuses features at different scales to obtain a more comprehensive and richer multi-scale feature map. , , , ; Step (3): The multi-scale feature map obtained in step (2) , , , The input is fed into the task learning block, where task learning is performed through a main task-specific learning branch and another auxiliary task-specific learning branch. Then, the PRA (Personal Recognition Analyzer) is used to learn the respective task tokens and prompts, thereby adaptively distinguishing the importance of tasks, realizing the intrinsic connection between tasks and task prompts, and obtaining new task tokens. , , , and general characteristics of the task Meanwhile, the construction of joint feature representations and MHSA enable the model to share information across different tasks and learn richer feature representations with long-distance dependencies. Step (4): Use the task token obtained in step (3) , , , The input is processed in the task interaction block. The task-related token is used to perform corresponding task interaction, and finally mapped to the corresponding task features to obtain the token after task interaction. Step (5): Combine the token obtained in step (4) after the task interaction with the general task feature obtained in step (3). The input is fed into the task-specific prediction block, and multiple prediction maps are obtained by performing predictions on the salient object detection and marginal object detection in the main task-specific learning branch and another auxiliary task-specific learning branch, respectively. The prediction maps include a salient object detection prediction map and a marginal object detection prediction map, wherein the salient object detection prediction map is obtained through SEALoss supervision.

2. The salient target detection method for optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: The specific implementation steps of the task prompt in step (1) are as follows: by introducing a similarity loss function, the difference between salient tasks and marginal tasks is explicitly maximized, and the importance of each task in the detection process is adaptively distinguished.

3. The salient target detection method for optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: In step (5), SEALoss is aligned with salient region features through an edge-aware mask.

4. The salient target detection method for optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: The specific implementation process of the task prompt in step (1) is as follows: the encoder task prompt is introduced into each encoder block by introducing task-specific prompts. ,in Indicates the number of channels. This represents the number of learnable cue tokens. The task cue is concatenated with features from the encoder block to form a feature block, which exists as a spatial task cue, given the image features of a specific block in the encoder. And use window attention to focus the image features Classified into window features ,in Indicates window size. That is, the number of windows; then, copy the hint for each window. And receive a prompt after copying. After that and The patch feature token is appended to each window, and self-attention is performed within each window, as shown in the following formula: ， Here, MHSA stands for Multi-Head Self-Attention Module, MLP stands for Multiple Perceptron, W-MHSA and SW-MHSA are MHSAs with regular and shifted window configurations, respectively. Subsequently, new cues are segmented from each window. And calculate their average value to obtain Finally, output window features Reassemble into the next block ; The decoder's task cues simply involve appending learnable cues to specific blocks from the decoder. Decoder feature tags Then, apply self-attention as follows to obtain the first... Task prompts for each decoder and the Cross-layer features of each decoder As shown in the formula below: 。 5. The salient target detection method for optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: The specific implementation process of the cross-layer fusion block in step (2) is as follows: process the input features, first converting the lower-sized features... Upsampling to By using inputs of the same size, alignment and integration of features at different scales are achieved. Next, the two inputs are concatenated along the channel dimension, organically fusing the feature information from both scales. Then, a linear projection maps the channel dimension to C dimensions, ultimately yielding a multi-scale feature map. .

6. The salient target detection method for optical remote sensing images based on multi-task prompts as described in claim 5, characterized in that: The specific implementation process of the task learning block in step (3) is as follows: receiving the multi-scale feature map. Then, the tokens for the salient task and the edge task are initialized as follows: and And randomly initialize salient task cues and edge task cues. , For learnable parameters, and By concatenating along the channel dimension, a joint feature representation is obtained. As shown in the formula below: ， in This represents the connection operation along the channel dimension, which is updated and iterated in each training round. , Then, the joint feature representation Utilizing PRA to obtain the token for the desired task and the multi-scale features Make predictions to obtain new joint feature representations. The process is as follows: ， in The representation layer is normalized, and then the following processing is performed to obtain intermediate features. : ， Get a new task assistance token: and general characteristics of the task Then, using MHSA in Transformer, the task general features are used as queries, and the new task auxiliary token is used as the key and value to obtain salient features. and edge features The process is as follows: ， ， Where Q represents query, K represents key, and V represents value; Simultaneously, another auxiliary task-specific learning branch is added for supervision, and the task-specific features of the other auxiliary task-specific learning branch are obtained in the same manner as described above. , The salience feature and edge features Learn task-specific features for branches of the main task.

7. The salient target detection method for optical remote sensing images based on multi-task prompts as described in claim 6, characterized in that: The specific implementation process of the PRA is as follows: given input First, obtain the query matrix through linear transformation. The process is as follows: ， in These are learnable parameters, and then... Remodeling ,in , Indicates the number of attention heads. The number of channels for each attention head, before performing pooling operations, is determined by the input... from Remodeling H and W represent the height and width of the feature map, and then multi-scale pooling operations are performed in the spatial dimensions: ， in This represents the generated multi-scale pyramid feature map, corresponding to different pooling layers, and... This indicates adaptive average pooling. Afterwards, the multi-scale pyramid feature map is input into a deep convolutional network for relative position encoding. ， in This represents a depthwise convolution operation with a kernel size of . , The encoded multi-scale pyramid feature map , Next Reshape back to the same shape as the input: ， in Indicates the first The number of tokens remaining after each pooling layer is calculated, and then these encoded pyramid feature maps are concatenated: ， Finally obtained ,in ,therefore ; A ReLU-based linear attention mechanism is used to achieve a global receptive field and linear computational complexity. In ReLU linear attention, the similarity function is defined as: ， The PRA attention mechanism is described as follows: ， in, , , , These are learnable parameters. The output matrix represents the first... OK, By utilizing the associative properties of matrix multiplication, the computational complexity and memory usage can be reduced from quadratic to linear without changing its functionality, as shown in the equation above. The calculation is as follows: and Once, then reuse them for each query, so that only Computational overhead and One memory.

8. The method for salient target detection in optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: The specific implementation process of the task interaction block in step (4) is as follows: perform a concatenation operation on the tokens of different tasks to obtain... Subsequently, MHSA is applied to capture the global relationships and long-distance dependencies between tokens. The token, after self-attention processing, is further refined using MLP. Thus obtain The process is described as follows: ， ， ， Finally, the interactively processed features are separated along the channel dimension. The interactively processed features are... Obtain the tokens for the salient task and the marginal task. As shown in the formula below:

9. The method for detecting salient targets in optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: The implementation process of the specific task prediction block in step (5) is as follows: In the main specific task learning branch, the token after task interaction and the task-specific features are transformed by LN processing and MLP, respectively. Then, the token and the task-specific features are fused by matrix multiplication to obtain task-specific features. The task-specific features are then transformed by dimensionality and mapped to a single-channel representation for final prediction to obtain the saliency prediction map. With edge prediction map This process is represented by the following formula: ， ， in Represents matrix multiplication; The other auxiliary task-specific learning branch directly performs LN processing and MLP operations on the task-specific features, and then applies linear layer mapping to obtain the auxiliary prediction map. and It can be expressed as the following formula: ， ， The final prediction graph is obtained.

10. The method for salient target detection in optical remote sensing images based on multi-task prompts as described in claim 1, characterized in that: The specific implementation process of SEALoss in step (5) is as follows: First, by calculating the predicted image... and An edge mask is generated to highlight the edge regions of the predicted image. ， in This represents the predicted image. The edge mask is represented by the image. Next, average features of each window are extracted from the real image and the predicted image respectively through local average pooling. , : ， ， Where Label represents the truth graph, and the size of the window is determined by a parameter. The diameter of the window is determined to be equal to Then, based on local average features, the alignment metric between the predicted image and the real image is calculated. Cosine similarity is used as the metric: ， Next, the absolute difference of the predicted image is calculated, and the absolute difference is multiplied by the alignment metric to obtain the alignment distance, as shown in the following formula: ， Where Distance represents the alignment distance. The formula for calculating the absolute difference is as follows: The alignment distance, in absolute value, represents the degree of alignment between the predicted image and the ground truth image within a local region. Finally, a weighted loss is calculated, obtained by multiplying the alignment distance by the edge mask and summing the results, as shown below: ， The summation is performed within regions where the edge mask is non-zero, and the loss in the edge regions is given greater weight.

Images