Optical remote sensing image saliency target detection method based on multi-task prompt

Abstract

The invention discloses an optical remote sensing image saliency target detection method based on multi-task prompt, and aims to improve the detection precision of a saliency target and solve the problems of edge blurring and information loss in the prior art. According to the method, firstly, by designing task prompts, the difference between a significance task and an edge task is maximized, gradient conflicts between the significance task and the edge task are reduced, and cooperation between the significance task and the edge task is optimized; and then an MTP module is proposed to perform learning and feature interaction of multiple tasks, and the utilization effect of edge information is improved by fusing fine edge features. In order to better align salient region features, SEALOss is introduced, alignment of salient regions is optimized through an edge perception mask, and the detection precision is improved. Meanwhile, the PRA is introduced, the calculation amount and memory consumption are remarkably reduced, and the operation efficiency of the model is improved. The method can robustly and accurately detect the saliency target in the optical remote sensing image, is suitable for complex scenes, and has high practicability and popularization value.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, relates to an improvement on a salient target detection model, and specifically is a salient target detection method for an optical remote sensing image based on multi-task prompts. Background Art

[0002] With the development of salient object detection, salient object detection on optical remote sensing images has become a hot topic in the field of remote sensing image processing in recent years. The main goal of this technology is to quickly and accurately detect salient objects such as ships, airplanes, buildings, etc. from a large number of remote sensing images. However, due to the complexity and diversity of remote sensing images, as well as the diversity of the objects themselves, saliency is still very challenging.

[0003] Remote sensing images usually contain a lot of background information, such as clouds, oceans, forests, etc., which may interfere with target detection. In addition, due to the influence of factors such as remote sensing image resolution, lighting conditions, and observation angle, the appearance of the target may change, which further increases the difficulty of target detection. Therefore, it is necessary to develop remote sensing image detection algorithms that can handle various targets in order to better handle complex and diverse optical remote sensing images. Summary of the invention

[0004] 1. Purpose of the invention:

[0005] In view of the shortcomings of the prior art, the present invention provides a method for detecting salient targets in optical remote sensing images based on multi-task prompts, which solves the problem that the current salient target detection model for optical remote sensing images has poor detection performance in some complex scenes.

[0006] 2. Technical solution:

[0007] The present invention proposes a method for detecting salient targets in optical remote sensing images based on multi-task prompts. The specific implementation steps of the method are as follows:

[0008] Step (1): Introduce the encoder task prompt into the backbone network and extract the multi-scale feature map f in the backbone network 1 、f 2 、f 3 、f 4 ; Add an MTP module to the decoder, introduce the task prompt of the decoder into the MTP module, input the multi-scale feature map into the MTP module, and the MTP module captures the intrinsic association between the edge task and the saliency task of the multi-scale feature map by interacting between specific task tokens. The MTP module includes a cross-layer fusion block, a task learning block, a task interaction block, and a task-specific prediction block;

[0009] Step (2): The multi-scale feature map f obtained in step (1) is 1 、f 2 、f 3 、f 4 Input into the cross-layer fusion block to fuse features of different scales to obtain a more comprehensive and rich multi-scale feature map

[0010] Step (3): The multi-scale feature map obtained in step (2) The task learning block is input into the task learning block, and the task learning is performed through the main specific task learning branch and another auxiliary specific task learning branch. Then, the respective task tokens and prompts are learned through PRA, so as to adaptively distinguish the importance of the tasks, realize the intrinsic connection between the tasks and the task prompts, and obtain the new tasks. And the common characteristics of the task At the same time, the construction of joint feature representation and MHSA enable the model to share information between different tasks and learn richer feature representations with long-distance dependencies;

[0011] Step (4): The task obtained in step (3) Input into the task interaction block for interactive processing, perform corresponding task interaction on task-related tokens, and finally map to corresponding task features to obtain the token after task interaction;

[0012] Step (5): The task interaction token obtained in step (4) and the task common features obtained in step (3) are combined into The input is input into the specific task prediction block, and a plurality of prediction graphs are obtained by predicting the respective salient target detection and the marginal target detection in the main specific task learning branch and another auxiliary specific task learning branch, wherein the prediction graphs include salient target detection prediction graphs and marginal detection prediction graphs, wherein the salient prediction graphs are obtained through SEALoss supervision.

[0013] Preferably, the specific implementation steps of the task prompt are: by introducing a similarity loss function, explicitly maximizing the difference between significant tasks and marginal tasks, and adaptively distinguishing the importance of each task in the detection process, thereby improving the effectiveness of collaboration between tasks.

[0014] Preferably, the SEALoss improves detection accuracy and robustness by aligning salient region features through edge-aware masks.

[0015] Preferably, the PRA greatly reduces the model computation and memory consumption through multi-scale pooling, and then introduces a linear attention mechanism to achieve efficient and accurate salient object detection.

[0016] 3. Beneficial effects:

[0017] (1) The present invention solves the problem of gradient conflict caused by the salient object edge detection task by using task hints, and learns the intrinsic relationship between tasks through the MTP module.

[0018] (2) The model proposed in the present invention has superior performance. The algorithm of the present invention is superior to the existing advanced algorithms on the two optical remote sensing image datasets EORSSD and ORSSD. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the overall process framework of the present invention;

[0020] Figure 2 It is a flow chart of the implementation of the MTP module of the present invention;

[0021] Figure 3 A comparison diagram of the PRA complexity and self-attention complexity of the present invention;

[0022] Figure 4 This is a visualization of the challenging situation of the present invention on the EORSSD dataset. DETAILED DESCRIPTION

[0023] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. This embodiment uses a Swin-Transformer encoder, which uses a Swin-Tiny backbone network, and provides a method for detecting salient targets in optical remote sensing images based on multi-task prompts. The specific implementation steps are as follows:

[0024] (S1): Design task prompts

[0025] Extract encoder 1 / 4, 1 / 8, 1 / 16, 1 / 32 features in the Swin-Tiny backbone network and introduce task-specific cues in each encoder block The task prompt and the features in the encoder block are concatenated into a feature block, which exists as a spatial task prompt.

[0026] Given an image feature f for a specific block in the Swin-Transformer encoder i , and use window attention to convert the feature f i Divide into window features Where M represents the window size, l i / M 2is the number of windows. Then, copy the prompt for each window and obtain Where N i Indicates the number of learnable hint tokens, c i represents the number of channels. Next, they are appended to the patch feature token in each window and self-attention is performed within each window, as shown in formula (1):

[0027]

[0028] Where W-MHSA and SW-MHSA are multi-head self-attention modules with regular and shifted window configurations, respectively. Residual connections and layer normalization are omitted here. Subsequently, And calculate its average to get the encoder task prompt Finally, the output window feature f iw Reassemble into the feature f of the next block i+1 .

[0029] Add an MTP module to the decoder, introduce the task hints of the decoder in the MTP module, the MTP module includes a cross-layer fusion block, a task learning block, a task interaction block and a task-specific prediction block; Regarding the task-specific hints in the decoder, just attach the learnable hints to the specific blocks in the decoder The decoder feature label f i+1 Then, self-attention is applied as follows (2) to obtain the task hint for the i-th decoder: and the cross-layer features of the i+1th decoder

[0030]

[0031] Among them, MHSA stands for multi-head self-attention and MLP stands for multi-perceptron.

[0032] (S2) Cross-layer fusion block

[0033] like Figure 2 As shown in (cross-layer fusion block), the specific implementation process of the cross-layer fusion block is: for the input feature f i-1 and f i The first operation is to convert the lower-sized features f i Upsample to f i-1The same size is used to align and integrate features of different scales, so as to facilitate subsequent feature fusion and interaction. Then, the two inputs are connected in the channel dimension to organically integrate the feature information of the two scales and construct a richer and more diverse feature representation. Then, the channel dimension is mapped to the C dimension through linear projection. Through the mapping operation, the adjustment and unification of the feature dimension is achieved, providing a more unified feature representation space for subsequent feature processing and task adaptation. Finally, the multi-scale feature map obtained is Provide the model with a more comprehensive and rich feature representation to facilitate subsequent task processing and information transfer.

[0034] (S3) Task learning block

[0035] like Figure 2 (Task Learning Block) is shown in the figure. The specific implementation process of the task learning block is as follows: receiving a multi-scale feature map Then initialize the tokens of the saliency task and the edge task as and And randomly initialize the salient task prompts and edge task prompts is a learnable parameter, and Connect in the channel dimension to get the joint feature representation X f , as shown in the following formula:

[0036]

[0037] Where Concat represents the connection operation on the channel dimension, which is updated and iterated in each round of training. Then the joint feature representation X f Use PRA to find the tokens and features of the required tasks Make predictions to obtain a new joint feature representation X′ f , the process is as follows:

[0038] X′ f =LN(PRA(X9)) (4)

[0039] Where LN represents layer normalization. Next, in order to learn a more accurate joint feature representation and learn the intrinsic relationship between task tokens and task prompts and tasks, the following processing is performed:

[0040] X″ f =MLP(X″ f )+X″ f +X f (5)

[0041] After this process, a new task auxiliary token is obtained: And the common characteristics of the task Afterwards, in order to learn the respective feature representations for the corresponding tasks, MHSA in Transformer is used to obtain the salient features f″ using the task-general features as queries and the task-specific tokens as keys and values. s i and edge features f″ e i , the process is as follows:

[0042]

[0043] In order to prevent possible interference between task interactions, a specific task learning branch is added for supervision to increase the stability of the model, such as Figure 2 As shown in , we obtain the task-specific features of the other branch.

[0044] (S4) Cross-task interaction block

[0045] like Figure 2 As shown in the figure (Cross-task interaction block), the specific implementation process of the cross-task interaction block is as follows: First, the tokens of different tasks are connected. Then, the multi-head self-attention mechanism is applied to capture the global relationship and long-distance dependency between features. Through this process, the model can more comprehensively understand the intrinsic relationship between tasks. Finally, the MLP is used to further refine the tokens after self-attention processing to obtain Thereby enhancing its representation ability, the process can be described as follows:

[0046]

[0047] Then, the interactively processed features are separated in the channel dimension, and the tokens of the two tasks are t′ s i , t′ e i , as shown in the following formula:

[0048]

[0049] The tokens of the obtained significant tasks and marginal tasks are interactively processed. Different from other methods, this embodiment adopts corresponding task interaction for task-related tokens, and finally maps them to corresponding task features. Since the feature sequence length L is much larger than the token sequence length L′, this method can greatly reduce the amount of calculation while obtaining deep information in task interaction.

[0050] (S5) Task-specific prediction block

[0051] like Figure 2 As shown in (task-specific prediction block), the specific implementation process of the task-specific prediction block is as follows: through dual-branch task-specific prediction, the final feature map is obtained. In the first branch, the token after task interaction and the corresponding task-specific features are processed by LN and transformed by MLP respectively. Subsequently, their outputs are fused by matrix multiplication. The final task-specific features are transformed in dimension and mapped into a single-channel representation, which is used for the final prediction to obtain the saliency prediction map s i With edge prediction graph e i , the process can be expressed by the following formula:

[0052]

[0053] Where ⊙ represents matrix multiplication. This process ensures the layer-by-layer refinement and fusion of task-specific information, where LN independently normalizes each input to eliminate input offset and scale issues; MLP enhances the complexity of feature expression through nonlinear transformation; and matrix multiplication deeply fuses the processed tokens with features to further enhance information exchange between tasks. Finally, dimensionality transformation simplifies the feature map to make it suitable for prediction.

[0054] The other branch (auxiliary branch) directly performs LN and MLP operations on task-specific features, and then applies linear layer mapping to obtain auxiliary prediction graphs. and It can be expressed as the following formula:

[0055]

[0056]

[0057] Through this process, the model is able to more deeply understand and represent the complex relationships and long-distance dependencies between tasks, thereby improving the overall performance and making final predictions.

[0058] (S6)PRA

[0059] like Figure 3 As shown in Figure 2, the complexity of PRA is compared with the complexity of traditional self-attention. It can be clearly seen that PRA reduces the computational complexity to linear, as shown in Figure 2. Figure 3 As shown in (b), the specific implementation process of the PRA design is as follows: given an input First, the query matrix Q is obtained through linear transformation. The process is as follows:

[0060]

[0061] in is a learnable parameter. Then Q is reshaped into Where C = h × d, h is the number of attention heads, and d is the number of channels of each attention head. Before performing the pooling operation, the input X is transformed from Reshape Where H and W represent the height and width of the feature map, and then multi-scale pooling operations are performed in the spatial dimension:

[0062] P1 = AvgPool1(X - ),P2=AvgPool2(X - ),…,P n =AvgPool n (X - ) (17)

[0063] where P1, P2, …, P n Represents the generated pyramid feature map, corresponding to different pooling layers, and AvgPool represents adaptive average pooling. Subsequently, these multi-scale pyramid feature maps are input into the deep convolutional network for relative position encoding:

[0064]

[0065] Where DWConv represents the deep convolution operation, and the convolution kernel size is 3×3. is the encoded pyramid feature map P i .

[0066] Then Reshape back to the same shape as the input:

[0067]

[0068] Where N i Represents the number of tokens remaining after the i-th pooling layer. Then these encoded pyramid feature maps are concatenated:

[0069]

[0070] Finally get Where M = (N1 + N2 + ... + N n ),therefore

[0071] like Figure 3 As shown in (a), when the pooling window is large enough, the P sequence is significantly shorter than the input sequence. This greatly reduces the computational complexity. In addition, P captures the contextual information of the input X, so when computing MHSA, P can be used as an effective substitute for X. Given an input The PRA attention mechanism can be described as:

[0072]

[0073] Where Q = XW q ,K=PW k ,V=PW Q , is a learnable parameter, A i represents the i-th row of the output matrix, This corresponds to the standard softmax attention mechanism, Q i ,K j ,V j Represents a query, key, and value.

[0074] In this embodiment, if Figure 3 As shown in (b), the PRA uses a ReLU-based linear attention mechanism to achieve global receptive field and linear computational complexity. In ReLU linear attention, the similarity function is defined as:

[0075] Sim(Q,K) = ReLU(Q i )ReLU(K j ) T (twenty two)

[0076] At this point, the PRA attention mechanism in this embodiment is restated as:

[0077]

[0078] By taking advantage of the associative nature of matrix multiplication, we can reduce the computational complexity and memory usage from quadratic to linear without changing its functionality. As shown in the above equation, we only need to calculate: and Once, and then reuse them for each query, so you only need to The computational cost and This enables a more efficient balance between computational cost and performance when learning long-range dependencies between images.

[0079] (S7)SEALoss

[0080] The specific implementation process of SEALoss is as follows: Figure 1 As shown. First, by calculating the dilation and erosion of the predicted image, an edge mask is generated to highlight the edge area of ​​the predicted image:

[0081] Edge=Dilation(Pred)-Erosion(Pred) (24)

[0082] In order to focus on the alignment of the image boundaries in subsequent calculations. Next, through the local average pooling operation, the average features AvgFeatures and AvgPred of each window are extracted for the real image and the predicted image respectively:

[0083] AvgFeatures=AvgPool(Label) (25)

[0084] AvgPred = AvgPool(Pred) (26)

[0085] Label represents the true value image, and the size of the window is determined by a parameter r, and the diameter of the window is equal to 2×r+1. Then, based on the local average features, the alignment metric between the predicted image and the real image is calculated, and the cosine similarity is used as the metric:

[0086] Alignment=1-CosineSimilarity(AvgFeatures,AvgPred) (27)

[0087] Next, the absolute difference of the predicted image is calculated and multiplied by the alignment metric to obtain the alignment distance:

[0088] Distance=Abs(AvgFeatures-AvgPred)×Alignment (28)

[0089] This distance represents the degree of alignment between the predicted image and the true image in the local area. Finally, the weighted loss is calculated. The loss is obtained by multiplying the alignment distance with the edge mask and summing them up, as shown below:

[0090]

[0091] The summation is performed in the area where the edge mask is non-zero, and the loss in the edge area will be given a larger weight. This method aims to use the edge information of the image to strengthen the alignment process to improve the local alignment between the real image and the predicted image.

[0092] The effects of the present invention are described in detail below in combination with experimental data and prediction graphs.

[0093] Table 1 quantitatively compares the method of the present invention with 22 state-of-the-art methods, including two traditional natural image salient object detection methods (T-NSI), three CNN-based natural image salient object detection methods (C-NSI), two hybrid-based optical remote sensing image salient object detection methods (H-RSI), two Transformer-based optical remote sensing image salient object detection methods (T-RSI) and thirteen CNN-based optical remote sensing image salient object detection methods (C-RSI). The best results are in bold. “-” indicates that no data is available. It can be clearly seen from Table 1 that compared with other comparison methods, the method of the present invention shows excellent performance on both optical remote sensing image salient object detection datasets. Among the 8 indicators listed here, except for the EORSSD dataset, the In addition, the method of the present invention completely surpasses other compared methods in all indicators.

[0094] Table 1 Comparison of this embodiment with advanced methods on EORSSD and ORSSD datasets

[0095]

[0096]

[0097] Table 2 Ablation test results of different improved methods on ORSSD and EORSSD datasets

[0098]

[0099] The best results in the table are shown in bold. In order to verify the effectiveness of the seeding improvement method in the model, 6 variants were designed, as shown in Table 2: 1) Only BaseLine is used, and the decoder is replaced by the most basic Transformer block; 2) MTP (introducing the MTP module); 3) SEALoss (introducing SEALoss); 4) MTP+task prompt (on the basis of MTP, adding task prompts); 5) MTP+SEALoss (on the basis of MTP, introducing SEALoss); 6) MTP+task prompt+SEALoss (complete MTPNet). It can be seen from the above table that different improvement methods have certain improvements in the four indicators of the two data sets. When SEALoss is added on the basis of BaseLine, all indicators are also improved. The improvement is particularly obvious. Under the condition of MTP+task prompt, a certain performance improvement was achieved compared to introducing only the MTP module. When MTP+SEALoss was deployed, a certain improvement was also achieved compared to introducing only the MTP module. The complete MTPNet significantly outperformed BaseLine, achieving 2.73%, 3.15%, 2.65% and 40.47% improvements in these three indicators respectively. A similar trend was also observed on the ORSSD dataset. It is obvious that with the addition of each improvement method, the overall performance of the model continues to improve, which highlights the effectiveness of the proposed improvement method.

[0100] like Figure 4 As shown in FIG, visualization results of the EORSSD dataset in some challenging scenes are shown to provide a clear comparison. It can be observed that the model of the present invention exhibits more complete and accurate detection of salient objects in complex scenes. Figure 4 As shown in (b), compared with the salient object target method, the method proposed in the present invention can detect salient targets more completely, which is attributed to its ability to effectively share feature representations and the model can make full use of the beneficial detail information of the edge. Figure 4 (a) Large target objects, Figure 4 (c) Multiple tiny salient objects and Figure 4 (e) Salient objects that are not in the center. In these challenging scenes, the method of the present invention can accurately detect salient objects, while other methods mistakenly highlight cluttered background areas. This is mainly due to the fact that the model of the present invention can capture long-distance dependent context information. At the same time, the task prompt improves the technology's ability to capture objects and boundaries in the image, and better balances global features with local details. Figure 4 (d) For narrow and long salient objects, since the SEALoss proposed in the present invention promotes accurate registration of local areas, the method of the present invention is more capable of ensuring the integrity and edge details of the object than other methods.

[0101] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting salient objects in optical remote sensing images based on multi-task prompts, comprising the following steps: Step (1): Introduce the encoder task prompt into the backbone network and extract the multi-scale feature map f in the backbone network 1 、f 2 、f 3 、f 4 ; Add an MTP module to the decoder, introduce the task prompt of the decoder into the MTP module, input the multi-scale feature map into the MTP module, and the MTP module captures the intrinsic association between the edge task and the saliency task of the multi-scale feature map by interacting between specific task tokens. The MTP module includes a cross-layer fusion block, a task learning block, a task interaction block and a specific task prediction block; Step (2): The multi-scale feature map f obtained in step (1) is 1 、f 2 、f 3 、f 4 Input into the cross-layer fusion block to fuse features of different scales to obtain a more comprehensive and rich multi-scale feature map Step (3): The multi-scale feature map obtained in step (2) The task learning block is input into the task learning block, and the task learning is performed through the main specific task learning branch and another auxiliary specific task learning branch. Then, the respective task tokens and prompts are learned through PRA, so as to adaptively distinguish the importance of the tasks, realize the intrinsic connection between the tasks and the task prompts, and obtain the new tasks. And the common characteristics of the task At the same time, the construction of joint feature representation and MHSA enable the model to share information between different tasks and learn richer feature representations with long-distance dependencies; Step (4): The task obtained in step (3) Input into the task interaction block for interaction processing, perform corresponding task interaction on task-related tokens, and finally map to corresponding task features to obtain the token after task interaction; Step (5): The task interaction token obtained in step (4) and the task common features obtained in step (3) are combined into The input is input into the specific task prediction block, and a plurality of prediction graphs are obtained by predicting the respective salient target detection and the marginal target detection in the main specific task learning branch and another auxiliary specific task learning branch, wherein the prediction graphs include salient target detection prediction graphs and marginal detection prediction graphs, wherein the salient prediction graphs are obtained through SEALoss supervision.

2. The method for detecting salient objects in optical remote sensing images based on multi-task prompts according to claim 1, characterized in that: The specific implementation steps of the task prompt in step (1) are: by introducing a similarity loss function, explicitly maximizing the difference between the salient tasks and the marginal tasks, and adaptively distinguishing the importance of each task in the detection process.

3. The method for detecting salient objects in optical remote sensing images based on multi-task prompts according to claim 1, characterized in that: In step (5), the SEALoss is aligned with the salient region features via an edge-aware mask.

4. The method for detecting salient targets in optical remote sensing images based on multi-task prompts according to claim 1, characterized in that: The specific implementation process of the task prompt in step (1) is as follows: the task prompt of the encoder is to introduce a task-specific prompt in each encoder block. where c i Indicates the number of channels, N i represents the number of learnable hint tokens, concatenating the task hint with the features in the encoder block into a feature block, which exists as a spatial task hint, given the image feature f of the specific block in the encoder i , and use window attention to transform the image features f i Divide into window features Where M represents the window size, l i / M 2 is the number of windows; then, the prompt is copied for each window and obtain Then f iw and Appended to the patch feature token in each window, and self-attention is performed within each window, as shown in the following formula: Where MHSA stands for Multi-Head Self-Attention Module, MLP stands for Multi-Perceptron, W-MHSA and SW-MHSA are MHSA with regular and shifted window configurations, respectively. Subsequently, And calculate its average value to get Finally, the output window feature f iw Reassemble into the next block f i+1 ; The decoder task hints are simply appended to the learnable hints from a specific block in the decoder. The decoder feature label f i+1 , then, self-attention is applied as follows to obtain the task hint for the i-th decoder and the cross-layer features of the i+1th decoder As shown in the following formula:

5. The method for detecting salient targets in optical remote sensing images based on multi-task prompts according to claim 1, characterized in that: The specific implementation process of the cross-layer fusion block in step (2) is as follows: the input features are processed, and the lower-size features f i Upsample to f i-1 The same size is used to align and integrate features of different scales; then, the two inputs are connected in the channel dimension to organically integrate the feature information of the two scales, and then the channel dimension is mapped to the C dimension through linear projection. Finally, a multi-scale feature map is obtained.

6. The method for detecting salient targets in optical remote sensing images based on multi-task prompts according to claim 5, characterized in that: The specific implementation process of the task learning block in step (3) is as follows: receiving the multi-scale feature map Then initialize the tokens of the saliency task and the edge task as and And randomly initialize the salient task prompts and edge task prompts is a learnable parameter, and Connect in the channel dimension to get the joint feature representation X f , as shown in the following formula: Where Concat represents the connection operation on the channel dimension, which is updated and iterated in each round of training. Then, the joint feature representation X f Use PRA to map the tokens of the required tasks and the multi-scale features Make a prediction to get a new joint feature representation X′8. The process is as follows: X′ f =LN(PRA(X f )) (4) Where LN represents layer normalization. Then, the following processing is performed to obtain the intermediate feature X″8: X″ f =MLP(X′ f )+X′ f +X f (5) Get a new task assistance token: And the common characteristics of the task Then, MHSA in Transformer is used to obtain significant features with the common features of the task as the query and the new task auxiliary token as the key and value. and edge features The process is as follows: Among them, Q represents query, K represents key, and V represents value; At the same time, another auxiliary specific task learning branch is added for supervision, and the task specific features of the other auxiliary specific task learning branch are obtained in the same way as the above operation. The distinctive features and edge features Learn task-specific features of branches for the main specific task.

7. The method for detecting salient objects in optical remote sensing images based on multi-task prompts according to claim 6, characterized in that: The specific implementation process of the PRA is as follows: given input First, the query matrix Q is obtained through linear transformation. The process is as follows: in is a learnable parameter, and then Q is reshaped into Where C = h × d, h represents the number of attention heads, d is the number of channels of each attention head, and before performing the pooling operation, the input X is transformed from Reshape H and W represent the height and width of the feature map, and then multi-scale pooling operations are performed in the spatial dimension: P1=AvgPool1(X - ),P2=AvgPool2(X - ),…,P n =AvgPool n (X - ) (17) where P1, P2, …, P n Represents the generated multi-scale pyramid feature map, corresponding to different pooling layers, and AvgPool represents adaptive average pooling. After that, the multi-scale pyramid feature map is input into the deep convolutional network for relative position encoding: Where DWConv represents the deep convolution operation, and the convolution kernel size is 3×3. is the encoded multi-scale pyramid feature map P i , Then P i enc Reshape back to the same shape as the input: Where N i Represents the number of tokens remaining after the i-th pooling layer is processed, and then these encoded pyramid feature maps are concatenated: Finally get Where M = (N1 + N2 + ... + N n ),therefore Use the ReLU-based linear attention mechanism to achieve global receptive field and linear computational complexity. In ReLU linear attention, the similarity function is defined as: Sim(Q,K)=ReLU(Q i )ReLU(K j ) T (22) The PRA attention mechanism is expressed as: Where, Q = XW q ,K=PW k ,V=PW v , is a learnable parameter, A i represents the i-th row of the output matrix, Q i ,K j ,V j Representing query, key, and value, by taking advantage of the associative nature of matrix multiplication, it is possible to reduce the computational complexity and memory usage from quadratic to linear without changing its functionality. As shown in the above equation, only the following calculation is required: and Once, and then reuse them for each query, so you only need to The computational cost and memory.

8. The method for detecting salient objects in optical remote sensing images based on multi-task prompts according to claim 1, characterized in that: The specific implementation process of the task interaction block in step (4) is: connect the tokens of different tasks to obtain t i , then, MHSA is applied to capture the global relationship and long-distance dependency between tokens to obtain Then use MLP to further refine the token after self-attention processing to obtain The process is described as follows: Finally, the interactively processed features are separated in the channel dimension to obtain tokens for the salient task and the marginal task. As shown in the following formula:

9. The method for detecting salient objects in optical remote sensing images based on multi-task prompts according to claim 1, characterized in that: The implementation process of the specific task prediction block in step (5) is as follows: in the main specific task learning branch, the token after task interaction and the task specific feature are processed by LN and transformed by MLP respectively, and then the token and the task specific feature are fused by matrix multiplication to obtain the task specific feature, and the task specific feature is then transformed in dimension and mapped into a single channel representation for the final prediction to obtain the saliency prediction map s i With edge prediction graph e i , the process is expressed by the following formula: Where ⊙ represents matrix multiplication; The other auxiliary specific task learning branch directly performs LN processing and MLP operation on the task specific features, and then applies linear layer mapping to obtain an auxiliary prediction map and It is expressed as the following formula: Get the final prediction graph.

10. The optical remote sensing image salient object detection method based on multi-task prompting as claimed in claim 1, characterized in that: The specific implementation process of SEALoss in step (5) is as follows: First, an edge mask is generated by calculating the Dilation and Erosion of the predicted image, thereby highlighting the edge area of ​​the predicted image: Edge=Dilation(Pred)-Erosion(Pred) (24) Where Pred represents the predicted image, Edge represents the edge mask, and then, through the local average pooling operation, the average features AvgFeatures and AvgPred of each window are extracted for the real image and the predicted image respectively: AvgFeatres=AvgPool(Label) (25) AvgPred = AvgPool(Pred) (26) Where Label represents the true value image, the size of the window is determined by a parameter r, and the diameter of the window is equal to 2×r+1; then, based on the local average feature, the alignment metric Alignment between the predicted image and the real image is calculated, and the cosine similarity is used as the metric: Alignment=1-CosineSimilarity(AvgFeatures, AvgPred) (27) Next, the absolute difference of the predicted image is calculated, and the absolute difference is multiplied by the alignment metric to obtain the alignment distance, and the process is shown in the following formula: Distance=Abs(AvgFeatures-AvgPred)×Alignment (28) Wherein Distance represents the alignment distance, Abs(AvgFeatures-AvgPred) is the calculation formula of the absolute difference, Abs is the absolute value, and the alignment distance represents the degree of alignment between the predicted image and the real image in the local area; finally, the weighted loss is calculated, and the loss is obtained by multiplying the alignment distance and the edge mask and summing them, as shown below: The summation is performed over regions where the edge mask is non-zero, and the loss in the edge region is given a larger weight.

Images