Remote sensing monitoring method for river and lake violation problem based on attention guiding and anti-fact

Abstract

The invention relates to a remote sensing monitoring method for river and lake violation problems based on attention guidance and anti-facts, and belongs to the technical field of deep learning and remote sensing image processing. Guiding attention is introduced into each level of a decoder, and multi-scale feature fusion in a decoding stage is realized through jump connection; an anti-fact attention graph is constructed according to the attention graph generated by each attention guiding module and is used for calculating anti-fact learning loss, output features of each attention guiding module are spliced to obtain a final feature graph, and mask prediction is carried out by using the feature graph; and constructing a composite loss function comprising image segmentation loss and anti-factual learning supervision loss to constrain and optimize the model. According to the method, the model is guided to pay attention to the regional features with discrimination, and the anti-fact sample is constructed for comparative learning, so that the discrimination capability and generalization performance of the model are improved, and a new thought is provided for efficient and accurate monitoring of river and lake violation problems.

Description

Technical Field

[0001] The present invention relates to a remote sensing monitoring method for river and lake violation problems based on guided attention and counterfactuals, and belongs to the technical field of river and lake supervision and management and remote sensing image processing. Background Art

[0002] With the continuous advancement of ecological civilization, the protection and management of river and lake resources are receiving increasing attention from all sectors of society. Rivers and lakes not only play an irreplaceable role in regulating regional climate, conserving water resources, and maintaining ecosystem balance, but they also serve as crucial vehicles for flood control and drainage, water resource allocation, water transportation, and biodiversity conservation. Healthy and stable river and lake ecosystems are crucial for achieving green development and ecological security. However, in recent years, with the acceleration of urbanization and industrialization and the increasing frequency of human activities, various illegal activities around rivers and lakes, such as illegal occupation, construction, mining, and dumping, have persisted despite repeated prohibitions, becoming increasingly frequent, hidden, and fragmented. These problems not only undermine the natural form and ecological functions of rivers and lakes but also lead to a series of environmental problems, including increased water pollution, shrinking water bodies, and ecological degradation, seriously threatening regional water security and sustainable ecological development.

[0003] Traditional river and lake management relies primarily on manual inspections, on-site evidence collection, and routine law enforcement. However, these methods are often subject to long lead times, high costs, and susceptibility to subjective factors, making them inadequate for the large-scale, frequent, and dynamic monitoring of river and lake resources. Given the increasingly burdensome task of river and lake supervision and the relatively weak regulatory capacity, there is an urgent need to develop more intelligent and automated regulatory methods to enable early detection, rapid response, and strict enforcement of violations, effectively improving the efficiency and precision of river and lake management.

[0004] Remote sensing technology has gradually become an indispensable tool in river and lake management due to its advantages of large-scale coverage, high timeliness and multi-source data fusion. Through the analysis and processing of remote sensing images, changes in river and lake shorelines, water distribution and surrounding land use can be dynamically grasped, providing basic support for the accurate detection of violations. At present, remote sensing image processing methods based on deep learning are widely used in tasks such as target recognition and scene understanding. They have shown significant advantages in extracting deep semantic features of images. However, deep learning methods still face many challenges in actual remote sensing applications. The resolution of remote sensing images is significantly lower than that of natural images. For example, a pile of garbage randomly piled up by a river or lake may only occupy a few pixels in a remote sensing image. Therefore, how to make the model focus on small illegal targets is a major difficulty. On the one hand, mainstream models continuously downsample during feature extraction, resulting in reduced feature map resolution. This results in poor recognition of small, non-compliant objects (such as simple buildings and stockpiles) commonly found in river and lake imagery, leading to feature loss and missed detections. Furthermore, remote sensing images are often affected by imaging angle, lighting conditions, and the diversity of object types, often exhibiting geometric misalignment and spectral variations. Deep learning models are susceptible to interference from irrelevant features, reducing their recognition accuracy and robustness. Therefore, there is an urgent need to introduce more sophisticated feature enhancement and discrimination mechanisms to improve the model's ability to perceive small objects and adapt to complex backgrounds. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals. By guiding the model to focus on discriminative regional features and constructing counterfactual samples for comparative learning, the model's discriminative ability and generalization performance are improved, providing new ideas for achieving efficient and accurate monitoring of river and lake violations.

[0006] The technical solution adopted by the present invention is: The remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals includes the following steps: S1. Divide the remote sensing image into a dataset and preprocess it, then input the preprocessed image into an encoder to extract multi-layer features; S2. Guided attention is applied to the extracted features of different scales at each layer. The extracted high-level features are passed through the corresponding guided attention module, and the output features are upsampled and fused with the extracted secondary features. The fused features are input into the guided attention module corresponding to the secondary layer features. S3. Based on the attention-enhanced feature maps generated by each guided attention module, a counterfactual attention map is constructed for calculating the counterfactual learning loss to form an additional supervision signal; S4. Concatenate the output features of each attention-guiding module to obtain a final feature map, and use the feature map to perform mask prediction; S5. Construct a composite loss function that includes image segmentation loss and counterfactual learning supervision loss to constrain and optimize the model; S6. Use the training set to train the weights, use the test set to verify the model, and obtain the final remote sensing segmentation model for prediction.

[0007] In the above method, the feature extraction network used by the encoder in step S1 is ResNet-18.

[0008] The guided attention module described in step S2 first performs a standard deviation operation on the input features, and then obtains a preliminary attention weight vector after mapping through the nonlinear function Sigmoid. The attention weight is expanded and transformed to the required dimension, and then multiplied by the input features point by point. Then, it is added to the input feature elements to obtain the intermediate features; then, the intermediate features are average pooled and maximum pooled in the channel dimension respectively to generate two sets of spatial description features. These two sets of features are spliced ​​in the channel dimension and sent to a convolution operation, and a spatial attention weight map is generated through the Sigmoid activation function. The spatial attention weight map is multiplied point by point with the intermediate features to output the attention enhancement feature map.

[0009] The construction of the counterfactual attention map in step S3 is achieved by intervening in the attention enhancement feature map. The intervention measures include randomization, homogenization, and inverting the attention weight. Only one intervention measure is used for the attention enhancement feature map each time, and the intervention measures are randomly selected. Among them, randomization is to randomly sample a value from a uniform distribution as the attention weight of each position in the attention enhancement feature map. Homogenization is to set all positions in the attention enhancement feature map to the same constant, so that the segmentation network loses its prominent response to specific areas. Inverting the attention weight is to resize the original attention enhancement feature map. F t Perform the reverse operation, F t '=max ( F t )− F t , which results in low weights for areas that should be of high concern.

[0010] The segmentation loss described in step S5 uses the Binary Cross Entropy (BCE) loss function, which is expressed as: , in Pr represents the predicted change graph, G represents the corresponding true label.

[0011] For the supervised loss of counterfactual learning, we first obtain the predicted change map of each attention map, that is, the pixel-level predicted distribution. For the real attention map F t , the predicted distribution is Pr ; For the counterfactual attention map F t ’ , set the prediction distribution to Cf , after obtaining the pixel-level prediction distribution of the corresponding attention map, calculate the difference between the two ∆Y , which is expressed as follows: , For each pixel i in the prediction map, it is expressed as ∆Y ( i ); Sum all pixels and normalize them to get the overall supervision loss for counterfactual learning: , Among them, max() is the maximum value function, and the parameter 𝑚 is a design hyperparameter used to set the minimum difference value expected to be achieved in the target area. ∆Y is a macroscopic representation of the difference between the true attention prediction distribution and the counterfactual attention prediction distribution, ∆Y ( i ) represents the difference of each pixel position, that is, the microscopic representation. Here, the loss function formula is uniformly expressed in macroscopic representation.

[0012] The composite loss function is: .

[0013] The beneficial effects of the present invention are: On the one hand, this invention utilizes a guided attention module to enable the model to automatically focus on potential violation areas in remote sensing images of rivers and lakes, significantly improving the extraction and recognition accuracy of small targets. On the other hand, by introducing a counterfactual intervention mechanism, the segmentation output of the model under factual attention and counterfactual attention is compared, thereby quantifying the contribution of key areas to the final prediction, further enhancing the model's ability to suppress background noise and its robustness. This method not only reduces the high cost and low efficiency of traditional reliance on manual inspections, achieving all-weather, full-coverage dynamic monitoring, but also improves the model's adaptability and interpretability through the additional loss function established based on counterfactual supervision, making the detection process more transparent and easier to optimize later.

[0014] The guided attention used in this invention calculates the channel standard deviation of the input features and maps them into preliminary channel weights, allowing the model to rely more on the distribution characteristics of the features themselves, rather than weights derived entirely by blind learning in the subsequent network. The addition of this statistical prior improves the attention's resilience to outliers and noise, making the model more stable in complex backgrounds. Introducing guided attention at each level of the decoder allows it to capture small objects (such as simple buildings and stockpiles in remote sensing images) in detail at high resolution, while focusing on larger objects or overall structures at low resolution. The two complement each other.

[0015] The counterfactual learning loss used in this paper is calculated on the pixel-level segmentation output, which requires that the difference between the fact and the counterfactual in the offending area reaches a threshold m , while the background difference approaches 0, aiming to accurately identify and locate violations. Furthermore, compared to using counterfactual learning in other tasks (such as classification), this method directly embeds it into the backpropagation of the segmentation network. Pixel-level differences and cross-entropy loss are calculated in parallel, allowing efficient updates of the attention module and backbone network during training, achieving integrated "segmentation + counterfactual" optimization without the need for additional large-scale data annotation or complex preprocessing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the method of the present invention; Figure 2 A network structure diagram of the model of the present invention; Figure 3 This is a structural diagram of the attention-guiding module of the present invention. DETAILED DESCRIPTION

[0017] The present invention will be further described below with reference to specific embodiments.

[0018] Example 1 A remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals, comprising the steps of ( Figure 1 as follows: S1. Divide the remote sensing image into a dataset and preprocess it. Then input the preprocessed image into the encoder to extract multi-layer features: Images around rivers and lakes were cropped from existing remote sensing images, and a portion of the cropped river and lake remote sensing images was visually interpreted to form label data; the cropped river and lake remote sensing images and their labels were scaled to obtain 1024×1024 tiles; a random number of images were selected for data augmentation operations such as random horizontal and vertical flipping and random 90-degree rotation; the mean and standard deviation of the three RGB channels were calculated, and the tiles were standardized.

[0019] In order to reduce the complexity of the model, the feature extraction network used in the encoder stage of this invention is ResNet-18 (see Figure 2 ), which introduces a residual module to construct a residual function of input and output, making parameter optimization easier while increasing network depth and preventing overfitting. ResNet-18 has four layers, each containing 2, 2, 2, and 2 residual blocks, respectively. The embedding dimensions of each layer are 64, 128, 256, and 512, respectively. Taking an input image of 3×1024×1024 as an example, the feature maps obtained after each layer of ResNet-18 are: (64, 256, 256), (128, 128, 128), (256, 64, 64), and (512, 32, 32).

[0020] S2. Guided attention is applied to the extracted features of different scales at each layer. After the extracted high-level features pass through the corresponding guided attention module, the output features are upsampled and fused with the extracted secondary features element-wise. The fused features are input into the guided attention module corresponding to the secondary layer features: Attention guidance is introduced at each level of the decoder, and multi-scale feature fusion in the decoding stage is achieved through skip connections, such as Figure 2 . Figure 3 The specific structure of the guided attention module is shown. In the guided attention, the input features F First, a branch is entered, and a standard deviation operation is performed on it. Then, after applying a nonlinear sigmoid function to it, a preliminary attention weight vector is obtained. This attention weight represents the channel-level distribution and is used to indicate the importance of different channels. Because it needs to be element-by-element weighted fusion with the original features, the attention weights generally need to have the same or broadcastable shape as the input features. After expansion, the attention weights are transformed to the required dimensions so that they can be subsequently multiplied point-by-point with the input features. After expansion and transformation to the required dimensions, the attention weights are point-by-point multiplied with the input features and then element-wise added to the input features to produce the intermediate features.

[0021] After obtaining the intermediate features, the attention module further introduces a spatial attention mechanism to enhance attention to key areas such as violation areas. The module performs average pooling and maximum pooling on the intermediate features in the channel dimension, generating two sets of spatial description features. These two sets of features are concatenated in the channel dimension and fed into a convolution operation, where a sigmoid activation function is used to generate a spatial attention weight map. Finally, this spatial weight map is point-by-point multiplied with the intermediate features to output the final attention-enhanced feature map, i.e., the true attention map. F t , thereby achieving effective focusing and expression enhancement of salient areas.

[0022] S3. Based on the attention-enhanced feature maps generated by each guided attention module, a counterfactual attention map is constructed for calculating the counterfactual learning loss to form an additional supervision signal: By learning the true attention map F t Apply intervention to construct a counterfactual attention map F t ’ The present invention adopts a variety of intervention measures, including randomization, homogenization, and inverted attention weights. Among them, randomization is to randomly sample a value from a uniform distribution for each position in the true attention map as the attention weight of the position. Homogenization is to set all positions in the true attention map to the same constant, so that the segmentation network loses its prominent response to specific areas. Inverted attention weights are to randomly sample a value from a uniform distribution for each position in the true attention map. F t Perform the reverse operation, F t '=max ( F t )− F t , so that the areas that should have high attention have low weights. Only one intervention measure is used on the attention map each time, and the intervention measures are randomly selected.

[0023] After constructing the counterfactual attention map, the attention map and the counterfactual attention map will be used to calculate the counterfactual learning loss in the following steps to form an additional supervision signal.

[0024] S4. Concatenate the output features of each attention-guiding module (i.e., concatenate the features decoded at each layer at the end of the decoding phase) to obtain the final feature map, which is used for mask prediction: The decoded features of each layer are uniformly upsampled to the same size, and then the four layers of features are spliced ​​to obtain the final feature map. The feature map will be further upsampled to the original image size and the sigmoid function will be used to obtain the final prediction mask to complete the result prediction.

[0025] S5. Construct a composite loss function that includes image segmentation loss and counterfactual learning supervision loss to constrain and optimize the model: In order to incorporate counterfactual interventions into the model training process, it is necessary to design a composite loss function that includes traditional segmentation loss and supervision loss for counterfactual learning.

[0026] The traditional segmentation loss uses the Binary Cross Entropy (BCE) loss function, which is expressed as: , in Pr represents the predicted change graph, G represents the corresponding true label.

[0027] For the supervised loss of counterfactual learning, we first obtain the predicted change map of each attention map, that is, the pixel-level predicted distribution. F t , the predicted distribution is Pr ; For the counterfactual attention map F t ’ , set the prediction distribution to Cf After obtaining the pixel-level prediction distribution of the corresponding attention map, calculate the difference between the two ∆Y , which is expressed as follows: , For each pixel i in the prediction map, it is expressed as ∆Y ( i ).

[0028] The purpose of calculating the difference: If the pixel is located in the real violation area, the model will expect the original attention to focus on this area better, while the counterfactual attention will weaken the target area due to the intervention effect. ∆Y ( i ) should be significantly larger. On the contrary, for background or non-violation regions, the difference between the effects of original and counterfactual attention should be smaller, ∆Y ( i ) should be close to 0. ∆Y In fact, it quantifies the effect of the attention module on improving the importance of the target area.

[0029] Now construct the supervised loss for counterfactual learning. For the pixels in the violation area, ∆Y ( i ) is at least greater than a certain boundary value 𝑚, otherwise a loss will occur. For pixels in non-violation areas, the original and counterfactual outputs should be as close as possible, that is, ∆Y ( i ) The smaller the better. Combining these two parts, summing over all pixels and normalizing them, we get the overall supervision loss for counterfactual learning: , Among them, max() is the maximum value function, and the parameter 𝑚 is a design hyperparameter used to set the minimum difference value expected to be achieved in the target area.

[0030] The final composite loss function is the sum of these two loss functions, expressed as follows: .

[0031] S6. Use the training set to train the weights, use the test set to verify the model, and obtain the final remote sensing segmentation model for prediction.

[0032] The above is a further description of the present invention in conjunction with specific embodiments, and the protection scope of the present invention is not limited thereto.

Claims

1. A remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals, characterized by: The steps are as follows: S1. Divide the remote sensing image into a dataset and preprocess it, then input the preprocessed image into an encoder to extract multi-layer features; S2. Guided attention is applied to the extracted features of different scales at each layer. The extracted high-level features are passed through the corresponding guided attention module, and the output features are upsampled and fused with the extracted secondary features. The fused features are input into the guided attention module corresponding to the secondary layer features. S3. Based on the attention-enhanced feature maps generated by each guided attention module, a counterfactual attention map is constructed for calculating the counterfactual learning loss to form an additional supervision signal; S4. Concatenate the output features of each attention-guiding module to obtain a final feature map, and use the feature map to perform mask prediction; S5. Construct a composite loss function that includes image segmentation loss and counterfactual learning supervision loss to constrain and optimize the model; S6. Use the training set to train the weights, use the test set to verify the model, and obtain the final remote sensing segmentation model for prediction.

2. The remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals according to claim 1 is characterized in that: The feature extraction network used by the encoder in step S1 is ResNet-18.

3. The remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals according to claim 1 is characterized in that: The guided attention module described in step S2 first performs a standard deviation operation on the input features, and then obtains a preliminary attention weight vector after mapping through the nonlinear function Sigmoid. The attention weight is expanded and transformed to the required dimension, and then multiplied by the input features point by point. Then, it is added to the input feature elements to obtain the intermediate features; then, the intermediate features are average pooled and maximum pooled in the channel dimension respectively to generate two sets of spatial description features. These two sets of features are spliced ​​in the channel dimension and sent to a convolution operation, and a spatial attention weight map is generated through the Sigmoid activation function. The spatial attention weight map is multiplied point by point with the intermediate features to output the attention enhancement feature map.

4. The remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals according to claim 1 is characterized in that: The counterfactual attention map in step S3 is constructed by applying interventions to the attention enhancement feature map. The intervention measures include randomization, homogenization, and inversion of attention weights. Only one intervention measure is used on the attention enhancement feature map each time, and the intervention measures are randomly selected.

5. The remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals according to claim 4 is characterized in that: Randomization is to randomly sample a value from a uniform distribution as the attention weight of each attention-enhanced feature map position; Homogenization is to set all positions in the attention-enhanced feature map to the same constant, so that the segmentation network loses its prominent response to specific areas; Reversing the attention weights is to enhance the original attention feature map F t Perform the reverse operation, F t '=max ( F t )− F t , which results in low weights for areas that should be of high concern.

6. The remote sensing monitoring method for river and lake violations based on guided attention and counterfactuals according to claim 1 is characterized in that: The segmentation loss described in step S5 adopts a binary cross entropy loss function, which is expressed as: ， in Pr represents the predicted change graph, G Indicates the corresponding true label; For the supervised loss of counterfactual learning, we first obtain the predicted change map of each attention map, that is, the pixel-level predicted distribution. For the real attention map F t , the predicted distribution is Pr ; For the counterfactual attention map F t ’ , set the prediction distribution to Cf , after obtaining the pixel-level prediction distribution of the corresponding attention map, calculate the difference between the two ∆Y , which is expressed as follows: ， For each pixel i in the prediction map, it is expressed as ∆Y ( i ); Sum and normalize all pixels to get the overall supervision loss for counterfactual learning: ， Among them, max() is the maximum value function, and the parameter 𝑚 is a design hyperparameter used to set the minimum difference value expected to be achieved in the target area.

7. The method for remote sensing monitoring of river and lake violations based on directed attention and counterfactuals according to claim 6 is characterized in that: The composite loss function described in step S5 is: 。

Images