Remote sensing identification method and system for area suitable for layout of photovoltaic facilities in pond aquaculture

Abstract

The invention discloses a remote sensing identification method and system for an area suitable for layout of photovoltaic facilities in pond aquaculture, and the method comprises the following steps: obtaining an optical image and an SAR image of a pond aquaculture area, and carrying out the preprocessing of the optical image and the SAR image, and obtaining a processed optical image and a processed SAR image; performing manual annotation and data enhancement on the processed optical image, and performing data division on the enhanced optical image and the processed SAR image to obtain a training data set; constructing an initial recognition model by using a double-branch encoder, a cross-modal attention mechanism and a multi-task learning framework, and training and optimizing the initial recognition model by using the training data set to obtain a region recognition model; and identifying a suitable layout area in the pond culture area by using the area identification model. Technical support is provided for reasonable layout of photovoltaic facilities, scientific allocation of resources is ensured, the influence on the aquaculture ecological environment is avoided, and the space utilization rate of photovoltaic power generation is increased.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing data processing and deep learning, and specifically relates to a remote sensing identification method and system for an area suitable for the deployment of photovoltaic facilities in pond aquaculture. Background Art

[0002] With the continuous growth of global energy demand, the development and promotion of clean energy technology has become an important part of the sustainable development strategy of various countries. As a representative of green energy, photovoltaic power generation not only has the advantages of no pollution and no noise, but also can be widely deployed, especially in various types of agriculture and aquaculture. By deploying photovoltaic facilities around ponds to develop a "fish-light complementarity" model, it can not only help improve the utilization rate of land and water resources, but also significantly improve power generation efficiency and provide a reliable energy supply for aquaculture farms. Reasonable deployment of photovoltaic facilities in aquaculture ponds can help achieve both economic and ecological benefits, and can alleviate the pressure of fishery production on natural resources.

[0003] In the process of arranging photovoltaic facilities, the production characteristics of pond aquaculture should be fully considered. Since aquaculture has strict requirements on water environment, light and ventilation conditions, photovoltaic facilities should not directly occupy the aquaculture water surface, but should be arranged in areas that do not affect production. Suitable locations for the layout of photovoltaic facilities usually include areas such as pond foundations, pond embankments, green spaces and tailwater treatment pools. These areas have stable characteristics and low environmental requirements, and the layout of photovoltaic facilities will not directly interfere with the normal operation of aquaculture. By arranging photovoltaic facilities in these non-productive areas, not only can the spatial resources around the pond be fully utilized, but also the obstruction and disturbance of the aquaculture water body can be effectively avoided, so as to achieve the coordinated development of power generation and aquaculture. However, it is a technical challenge to reasonably identify the locations suitable for the layout of photovoltaic facilities inside the pond, especially to realize efficient and accurate layout plans in large aquaculture areas.

[0004] At present, remote sensing technology has been widely used in fields such as agriculture and water resources management, but in the scenario of photovoltaic facilities in ponds, existing technologies still have certain limitations. First, existing pond area identification methods mostly rely on a single data source, such as optical images or radar images, which have poor adaptability to environmental conditions. Although optical images can provide rich spectral information under conditions of sufficient light and help identify the features of objects in ponds, they are easily interfered with in cloudy weather or insufficient light. Radar images have the advantage of all-weather imaging and can obtain data in bad weather, but they lack spectral information and it is difficult to fully identify vegetation, soil and other features. The limitations of a single data source lead to low recognition accuracy in complex environments, affecting the accurate identification of suitable deployment areas.

[0005] In addition, the fusion of multimodal data has obvious advantages in improving recognition accuracy, but existing methods mostly use simple feature splicing or weighted averaging, which cannot effectively utilize the complementarity of different modal data. Optical images and SAR images each have their own advantages in feature expression. Optical images can distinguish vegetation coverage areas and soil characteristics, while SAR images can provide more subtle surface structure information. The existing simple fusion method ignores the complementary characteristics of the two types of data under different environmental conditions, resulting in poor robustness of the model in practical applications, making it difficult to accurately identify areas around ponds suitable for the deployment of photovoltaic facilities.

[0006] On the other hand, suitable layout areas in the pond area (such as pond bases, pond embankments, etc.) often belong to minority categories or small targets, with a low area share and are adjacent to the main aquaculture water areas, making them difficult to identify. Traditional remote sensing models tend to ignore minority categories or small areas when dealing with category imbalance problems, causing these areas to be misidentified or missed. Especially in the scenario of multimodal fusion, the category imbalance problem will increase the difficulty of the model in identifying small targets, resulting in poor detection of minority areas, which in turn affects the rationality of the layout of photovoltaic facilities.

[0007] In summary, the current remote sensing identification technology for the layout of photovoltaic facilities in ponds has many limitations in terms of data source dependence, multimodal fusion, class imbalance and dynamic feature extraction. In order to break through these technical bottlenecks and achieve high-precision and strong robust identification of suitable areas for the layout of photovoltaic facilities in ponds, it is necessary to innovatively design multimodal data fusion strategies and deep learning models. Summary of the invention

[0008] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0009] A remote sensing identification method for an area suitable for the deployment of photovoltaic facilities in pond aquaculture comprises the following steps:

[0010] Obtaining optical images and SAR images of the pond breeding area and performing preprocessing to obtain processed optical images and processed SAR images;

[0011] Manually annotating and data enhancing the processed optical image, and performing data segmentation on the enhanced optical image and the processed SAR image to obtain a training data set;

[0012] An initial recognition model is constructed using a dual-branch encoder, a cross-modal attention mechanism, and a multi-task learning framework, and the initial recognition model is trained and optimized using the training data set to obtain a region recognition model;

[0013] The area recognition model is used to identify suitable layout areas in the pond breeding area.

[0014] Preferably, the pretreatment method comprises:

[0015] Performing radiation correction, atmospheric correction, geometric correction and band fusion on the optical image;

[0016] performing radiation correction, speckle noise filtering and geometric correction on the SAR image;

[0017] Performing image registration on the fused optical image and the corrected SAR image to obtain a registered optical image and a registered SAR image;

[0018] Calculating a normalized vegetation index and a normalized water index based on the registered optical image;

[0019] Based on the normalized vegetation index and the normalized water index, the registered optical image and the registered SAR image are stacked, and the stacked multi-channel image is cropped to obtain the processed optical image and the processed SAR image.

[0020] Preferably, the method for obtaining the enhanced optical image comprises:

[0021] Classifying the ground object categories, and manually annotating the processed optical image based on the classification results to obtain an annotated optical image;

[0022] rasterizing the annotated optical image, and cropping the rasterized optical image to obtain a cropped optical image;

[0023] Data enhancement is performed on the cropped optical image to obtain the enhanced optical image.

[0024] Preferably, the dual-branch encoder comprises: an optical image encoder and a SAR image encoder;

[0025] The optical image encoder uses the ResNet-34 model as the basic encoder, and adjusts the number of input channels of the first convolutional layer of the basic encoder from 3 to 6. The weight of the first convolutional layer is randomly initialized using the Kaiming initialization method, and the weights of other layers are pre-trained weights loaded on the ImageNet dataset;

[0026] The SAR image encoder uses the ResNet-34 model as the basic encoder, and adjusts the number of input channels of the first convolutional layer of the basic encoder from 3 to 1, and the weights of all convolutional layers are randomly initialized using the Kaiming initialization method.

[0027] Preferably, the cross-modal attention mechanism is arranged after each main layer of the dual-branch encoder;

[0028] The workflow of the cross-modal attention mechanism includes:

[0029] In the first layer of the dual-branch encoder, an optical feature map of the enhanced optical image and a SAR feature map of the processed SAR image are respectively extracted through convolutional layer and residual block calculation;

[0030] Performing a linear transformation on the optical feature map and the SAR feature map to obtain a query, a key, and a value, calculating a similarity between the query and the key, and obtaining an attention matrix;

[0031] Using the attention matrix to perform weighted fusion on the values ​​to obtain a fusion feature, and adding the fusion feature to the optical feature map to obtain an output feature;

[0032] The output features are passed to the next layer of the dual-branch encoder or the multi-task learning framework.

[0033] Preferably, the multi-task learning framework includes: a scene recognition branch and a suitable area recognition branch;

[0034] The workflow of the scene recognition branch includes: extracting global features from the output features of the last layer of the dual-branch encoder, averaging the spatial dimensions to obtain a global feature vector, and mapping the global feature vector to the probability of scene existence through a fully connected layer to complete scene recognition;

[0035] The workflow of the suitable area identification branch includes: when the scene identification branch determines that it is a pond farming scene, the output features are decoded through five layers of upsampling, and a suitable area identification result is output.

[0036] Preferably, the method for training and optimizing the initial recognition model includes:

[0037] The scene recognition loss function, suitable area recognition loss function, modality loss function and modality weight are introduced to construct the total loss function;

[0038] Using the training data set to train the initial recognition model, and using the total loss function to calculate the total loss;

[0039] The model gradient is calculated based on the total loss, and the model parameters are updated to obtain the region recognition model.

[0040] The present invention also provides a remote sensing identification system for suitable areas for photovoltaic facilities in pond aquaculture, wherein the identification system applies any of the above-mentioned identification methods and comprises: an image preprocessing module, a data division module, a model training module and an identification module;

[0041] The image preprocessing module is used to obtain the optical image and the SAR image of the pond breeding area and perform preprocessing to obtain the processed optical image and the processed SAR image;

[0042] The data partitioning module is used to manually annotate and enhance the data of the processed optical image, and to partition the data of the enhanced optical image and the processed SAR image to obtain a training data set;

[0043] The model training module uses a dual-branch encoder, a cross-modal attention mechanism and a multi-task learning framework to build an initial recognition model, and uses the training data set to train and optimize the initial recognition model to obtain a region recognition model;

[0044] The identification module uses the area identification model to identify suitable layout areas in the pond breeding area.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention significantly improves the robustness and adaptability of the model through the coordinated application of multimodal data and the complementary characteristics of optical images and SAR images. It can still ensure the accuracy of recognition even in bad weather, provide technical support for the rational layout of photovoltaic facilities, ensure the scientific allocation of resources, avoid the impact on the ecological environment of aquaculture, and maximize the spatial utilization rate of photovoltaic power generation, achieving a win-win situation of power generation efficiency and aquaculture benefits.

[0047] The cross-modal attention mechanism of the present invention adaptively adjusts the fusion ratio of different modal features through weight calculation among query (Q), key (K), and value (V), effectively retains key features, avoids information interference, and enables the model to maintain high recognition accuracy when the internal structure of the pond changes (for example, the pond base boundary is blurred due to water level changes). It significantly enhances the model's ability to express the characteristics of different areas inside the pond, making the model still highly adaptable in diversified scenarios of pond aquaculture, ensuring the accurate identification and reasonable layout of areas suitable for the deployment of photovoltaic facilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the structure of a region recognition model according to an embodiment of the present invention;

[0051] Figure 3 Schematic diagram of the structure of a residual block in an encoder according to an embodiment of the present invention;

[0052] Figure 4 Schematic diagram of the structure of the cross-modal attention mechanism of an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Embodiment 1

[0056] In this embodiment, if Figure 1 As shown, a remote sensing identification method for an area suitable for the deployment of photovoltaic facilities in pond aquaculture comprises the following steps:

[0057] S1. Obtain optical images and SAR images of the pond breeding area and perform preprocessing to obtain processed optical images and processed SAR images.

[0058] The preprocessing methods include: performing radiation correction, atmospheric correction, geometric correction and band fusion on optical images; performing radiation correction, speckle noise filtering and geometric correction on SAR images; performing image registration on the fused optical image and the corrected SAR image to obtain the registered optical image and the registered SAR image; calculating the normalized vegetation index and the normalized water index based on the registered optical image; stacking the registered optical image and the registered SAR image based on the normalized vegetation index and the normalized water index, and cropping the stacked multi-channel image to obtain the processed optical image and the processed SAR image.

[0059] In this embodiment, the optical image of the pond breeding area is obtained by the GF-2 satellite, and the band information of the optical image includes: Panchromatic band: 450-900 nanometers, spatial resolution 1 meter; Multispectral band (MS): blue (B, 450-520 nanometers), green (G, 520-590 nanometers), red (R, 630-690 nanometers), near infrared (NIR, 770-890 nanometers), spatial resolution 4 meters. The SAR image of the pond breeding area is obtained by the GF-3 satellite, and the imaging mode of the SAR image includes: Fine Stripmap (FS): spatial resolution 1-3 meters, suitable for high resolution requirements; Polarization mode: select single polarization (VV polarization) or dual polarization (VV+VH) data, according to actual needs and data availability.

[0060] The optical image is preliminarily processed as follows: (1) Radiation correction: Use the “Radiation Correction” tool of ENVI software to perform radiation calibration on the optical image and convert the digital value (DN value) into radiation brightness; (2) Atmospheric correction: Use the FLAASH module of ENVI to eliminate the atmospheric influence based on the radiation transfer model and obtain the surface reflectance; (3) Geometric correction: Use a high-precision digital elevation model (DEM) and ground control points (GCP) to perform orthorectification on the optical image to eliminate the geometric distortion caused by terrain undulation and sensor viewing angle; (4) Band fusion: Use principal component analysis (PCA) fusion or IHS transform fusion to fuse the panchromatic band (1-meter resolution) with the multispectral band (4-meter resolution) to obtain the fused optical image. The SAR images were preliminarily processed as follows: (1) Radiation correction: the ENVI SAR Scape module was used to perform radiation correction on the SAR images and obtain the backscatter coefficient (Sigma0); (2) Speckle noise filtering: the Lee filter was applied to reduce the speckle noise and retain the edge and detail information; (3) Geometric correction: the “geocoding” tool of SAR Scape was used to perform geometric correction on the SAR images in combination with precise orbit data and DEM, and the images were projected into the geographic coordinate system to obtain the corrected SAR images. The processes that affect the registration include: (1) Checking the registration accuracy: In ENVI or ArcGIS, overlay the processed GF-2 and GF-3 images to check the alignment of feature points (such as road intersections and buildings); (3) Accurate registration: If there is displacement, use the "Image Registration" tool of ArcGIS to manually select the conjugate points on the fused optical image and the corrected SAR image (such as obvious ground feature points), apply the nearest neighbor method, bilinear interpolation or cubic convolution, etc., resample the SAR image, complete the registration, and obtain the registered optical image and registered SAR image. Calculate the normalized vegetation index and normalized water index based on the registered optical image:

[0061]

[0062]

[0063] Among them, NDVI represents the normalized difference vegetation index, R NIR Represents the surface reflectivity in the near-infrared band, R Red represents the surface reflectance of the red band, NDWI represents the normalized difference water index, R Green Represents the surface reflectance of the green band. Afterwards, based on the normalized vegetation index and the normalized water index, the four bands (blue, green, red, and near-infrared) of the registered optical image, the calculated NDVI and NDWI, a total of 6 channels are stacked, and the registered SAR image (such as the backscatter coefficient of VV polarization) is used as the 7th channel. In ENVI, the "Layer Stack" tool is used to stack the above 7 band images into a multi-channel image, and then the stacked multi-channel image is cropped to obtain a 256×256 pixel processed optical image and processed SAR image.

[0064] S2. Manually annotate and enhance the processed optical images, and divide the data of the enhanced optical images and processed SAR images to obtain the training data set.

[0065] The method for obtaining an enhanced optical image includes: classifying the ground objects, and manually annotating the processed optical image based on the classification results to obtain the annotated optical image; gridding the annotated optical image, and cropping the gridded optical image to obtain the cropped optical image; and data enhancement of the cropped optical image to obtain the enhanced optical image.

[0066] In this embodiment, the ground object categories are first divided into: 0: background or other ground objects of no concern, 1: pond base (suitable for the layout of photovoltaic facilities), 2: pond embankment (suitable for the layout of photovoltaic facilities), 3: green space (suitable for the layout of photovoltaic facilities), 4: tailwater treatment pool (suitable for the layout of photovoltaic facilities), 5: aquaculture water surface (not suitable for the layout of photovoltaic facilities); then the processed optical image is manually annotated based on the division results: on the cropped processed optical image, a vector layer (Shapefile) is created to annotate the ground objects of each category respectively, and the boundaries of the pond base, pond embankment, green space, tailwater treatment pool and aquaculture water surface are accurately outlined using polygon drawing tools to obtain the annotated optical image; then the "rasterization" tool of ArcGIS is used to convert the annotated optical image into a raster format, and the pixel size is set to be consistent with the image to ensure that the raster label The labels correspond to the image pixels one by one, the grid value field is specified as the category coding field, the gridded optical image is cropped to ensure that the cropped label grid and the corresponding image cropping block are completely consistent in spatial position and size, and the cropped label data is saved as the cropped optical image; secondly, in order to increase the diversity of samples, improve the generalization ability of the model, alleviate the category imbalance problem, and enhance the model's learning ability for minority classes and small targets, the transformation tools in TensorFlow or PyTorch are used to randomly flip, rotate, translate, spectrally transform and add noise to the samples to expand the data set and obtain the enhanced optical image; finally, the enhanced optical image and the processed SAR image are taken as a whole and divided into training set, validation set and test set in a ratio of 14:3:3 to obtain the training data set.

[0067] S3. Use the dual-branch encoder, cross-modal attention mechanism and multi-task learning framework to build an initial recognition model, and use the training dataset to train and optimize the initial recognition model to obtain a region recognition model, such as Figure 2 shown.

[0068] The dual-branch encoder includes: an optical image encoder and a SAR image encoder. The optical image encoder uses the ResNet-34 model as the basic encoder, and adjusts the number of input channels of the first convolutional layer of the basic encoder from 3 to 6. The weight of the first convolutional layer is randomly initialized using the Kaiming initialization method, and the weights of other layers are pre-trained weights loaded on the ImageNet dataset. In this embodiment, the residual network (ResNet) solves the gradient vanishing problem of the deep network by introducing residual connections, and enhances the feature extraction capability, such as Figure 3As shown, each ResidualBlock in the encoder is composed of a different number of residual blocks, and the ratio of Layer1-Layer4 residual blocks is (3:4:6:3). The SAR image encoder uses the ResNet-34 model as the basic encoder, and adjusts the number of input channels of the first convolution layer of the basic encoder from 3 to 1. The weights of all convolution layers are randomly initialized using the Kaiming initialization method. In this embodiment, due to the large difference in the characteristics of SAR images and optical images, random initialization allows the model to better learn the characteristics of SAR images.

[0069] The cross-modal attention mechanism is set after each main layer of the dual-branch encoder. The workflow of the cross-modal attention mechanism includes: in the lth layer of the dual-branch encoder, the optical feature map of the enhanced optical image and the SAR feature map of the processed SAR image are extracted respectively through the convolution layer and the residual block calculation; the optical feature map and the SAR feature map are linearly transformed to obtain the query, key and value, and the similarity between the query and the key is calculated to obtain the attention matrix; the value is weightedly fused using the attention matrix to obtain the fused feature, and the fused feature is added to the optical feature map to obtain the output feature; the output feature is passed to the next layer of the dual-branch encoder or the multi-task learning framework.

[0070] In this embodiment, the cross-modal attention mechanism is a method of dynamically fusing multimodal features, such as Figure 4 As shown in the figure, by calculating the correlation between the features of one modality (such as optical images) and the features of another modality (such as SAR images), attention weights are generated to selectively fuse multimodal features. In this way, the model can learn which modality's information should be paid attention to at different spatial positions, and dynamically adjust the weights of each modality's features to enhance the feature expression ability, highlight important information, suppress interference, and improve recognition accuracy. A cross-modal attention module is introduced after each main layer of the encoder (layer1, layer2, layer3, layer4), so that multimodal features can be fused on feature maps of different scales, making full use of the feature representation of multimodal data at different levels to enhance the expression ability of the model. The workflow of the cross-modal attention mechanism includes:

[0071] (1) Obtain the optical features and SAR features of the encoder specified layer:

[0072] In the lth layer of the encoder, the optical feature map of the enhanced optical image and the SAR feature map of the processed SAR image are extracted through the convolutional layer and residual block calculation of the ResNet-34 encoder:

[0073]

[0074] in, Represents the optical characteristic diagram, represents the SAR feature map, B represents the number of samples input into the model at one time, C represents the number of feature channels, H represents the height of the feature map, and W represents the width of the feature map.

[0075] (2) Perform linear transformation on the features to obtain the query, key, and value:

[0076] In order to calculate the correlation between the optical signature and the SAR signature, they need to be linearly transformed to obtain the query, key, and value:

[0077]

[0078] Among them, Q opt represents the query from the optical feature map, K sar Represents the key from the SAR characteristic graph, V sar represents the value from the SAR characteristic map, W Q , W K , W V represents the learnable weight matrix; in order to reduce the computational complexity, the number of channels is usually reduced to C' = C / 8. Therefore, W Q and W K The number of output channels is C', and W V The number of output channels is still C.

[0079] (3) Calculate the similarity between the query and the key to obtain the attention matrix:

[0080] Expand the query and key in the spatial dimension and transform Q opt Reshape K sar Reshape Where N = H × W, which represents the number of spatial positions in the feature graph; calculate the dot product of the query and the key to obtain the similarity matrix S:

[0081]

[0082] Among them, T represents the transpose of the matrix; in order to prevent the dot product result from being too large, a scaling operation is performed and the Softmax function is used for normalization to obtain the attention weight matrix A l :

[0083]

[0084] Among them, d k=C', which represents the scaling factor. The Softmax function is normalized on the last dimension (i.e., the spatial position dimension). Before calculating the Softmax, the maximum value can be subtracted from each row of S to prevent numerical overflow caused by exponential operations.

[0085] (4) Use the attention matrix to obtain output features:

[0086] Using attention weight A l , for the value V sar Perform weighted summation to obtain fusion features

[0087]

[0088] V sar Reshape Indicates A l Transpose in the spatial dimension; in order to retain the information of the original optical features and integrate the useful information from the SAR features, the residual connection method is used to obtain the output features:

[0089]

[0090] (5) Pass the output features to the next layer of the dual-branch encoder or to the multi-task learning framework:

[0091] The output features obtained It contains both the original optical features and the related SAR feature information. As the output of the lth layer of the encoder, it is passed to the next layer to continue feature extraction, or passed to the decoder at the end of the encoder for upsampling and classification.

[0092] The multi-task learning framework includes: scene recognition branch and suitable area recognition branch. The workflow of the scene recognition branch includes: extracting global features from the output features of the last layer of the dual-branch encoder, averaging the spatial dimension to obtain the global feature vector, mapping the global feature vector to the probability of scene existence through the fully connected layer, and completing scene recognition; the workflow of the suitable area recognition branch includes: when the scene recognition branch determines that it is a pond farming scene, the output features are decoded through five layers of upsampling, and the suitable area recognition result is output.

[0093] In this embodiment, in the remote sensing identification task of suitable areas for the layout of photovoltaic facilities for pond aquaculture, it is necessary to solve two tasks, scene recognition and pixel-level object classification, at the same time. To this end, a multi-task learning framework is constructed, which allows the model to learn these two tasks at the same time on the basis of shared features, thereby improving the overall recognition efficiency and accuracy. The core idea of ​​the multi-task learning framework is to share some parameters of the model so that the model can use the information of another task while learning one task, thereby achieving a mutually reinforcing effect. This method can not only improve the generalization ability of the model, but also reduce the number of parameters of the model and improve computational efficiency.

[0094] The multi-task learning framework includes: scene recognition branch and suitable area recognition branch.

[0095] (1) Scene recognition branch

[0096] The purpose of the scene recognition branch is to first determine whether the input remote sensing image belongs to a pond farming scene. If non-target scenes can be screened out in advance, complex pixel-level classification of irrelevant areas can be avoided, saving computing resources and improving the efficiency of the model. The workflow of the scene recognition branch includes: Extract global features from the feature map; average the spatial dimensions of the feature map to obtain a global feature vector

[0097]

[0098] Through a fully connected layer, the global feature vector is mapped to the probability p∈[0,1] of the scene existence.

[0099] p=σ(W p ·f gap +b p )

[0100] in, is the weight vector, b p is the bias term, σ is the Sigmoid activation function, defined as

[0101] (2) Suitable region identification branch

[0102] The purpose of the suitable area identification branch is to classify the input image at the pixel level and identify areas suitable for the deployment of photovoltaic facilities (such as pond foundations, pond embankments, green spaces, and tailwater treatment pools) and unsuitable areas (such as aquaculture water surfaces). In order to improve computational efficiency, the suitable area identification branch is activated only when the scene recognition branch determines that it is a pond aquaculture scene. The workflow of the suitable area identification branch includes: during the forward propagation of the model, determine whether p is greater than the threshold θ (usually set to 0.5). If p>θ, the suitable area identification branch is activated, otherwise it is skipped; a decoder with a U-Net structure is used to gradually upsample the features of the encoder, gradually double the spatial size of the feature map, and restore it to the original spatial resolution. In each upsampling stage, the features of the decoder are jump-connected to the features of the corresponding layer of the encoder to fuse multi-scale and multi-modal feature information. Specifically:

[0103] First upsampling layer: input the output features of the highest layer of the encoder Upsample the input feature map size from H / 32×W / 32 to H / 16×W / 16, and fuse the features with the corresponding layer of the encoder (such as layer4) Splicing, convolution processing is performed on the feature map after jump splicing to reduce the number of channels to 1 / 2 of the original, thereby extracting new features.

[0104] The second upsampling layer: inputs the output features of the first layer, upsamples the feature map size from H / 16×W / 16 to H / 8×W / 8, and fuses the features with the corresponding layer of the encoder (such as layer3) Splicing, convolution processing is performed on the feature map after jump splicing to reduce the number of channels to 1 / 2 of the original, thereby extracting new features.

[0105] The third upsampling layer: inputs the output features of the second layer, upsamples the feature map size from H / 8×W / 8 to H / 4×W / 4, and fuses the features with the corresponding layer of the encoder (such as layer2) Splicing: convolution processing is performed on the feature map after jump splicing to reduce the number of channels to 1 / 2 of the original, thereby extracting new features;

[0106] The fourth upsampling layer: inputs the output features of the third layer, upsamples the feature map size from H / 4×W / 4 to H / 2×W / 2, and fuses the features with the corresponding layer of the encoder (such as layer1) Splicing: convolution processing is performed on the feature map after jump splicing to reduce the number of channels to 1 / 2 of the original, thereby extracting new features;

[0107] The fifth upsampling layer: inputs the output features of the fourth layer, upsamples the feature map size from H / 2×W / 2 to H×W, obtains a feature map of size H×W, and completes decoding.

[0108] Finally, a 1×1 convolution is used to map the feature map to the number of categories to obtain the classification result of each pixel, that is, the suitable area recognition result.

[0109] The method for training and optimizing the initial recognition model includes: introducing a scene recognition loss function, a suitable region recognition loss function, a modal loss function and a modal weight to construct a total loss function; training the initial recognition model using a training data set, and calculating the total loss using the total loss function; calculating the model gradient based on the total loss, and updating the model parameters to obtain a region recognition model.

[0110] In this embodiment, in order to optimize the scene recognition and pixel-level classification tasks simultaneously, a joint loss function needs to be designed to coordinate the training of the two tasks. The joint loss function should be composed of the scene recognition loss function L 场景 , suitable area identification loss function L 适宜区域 And the modal loss function together constitute.

[0111] (1) Scene recognition loss L 场景

[0112] The scene recognition task is a binary classification problem, that is, to determine whether the input remote sensing image contains a pond farming scene. In order to measure the performance of the model on this task, the binary cross-entropy loss function (BinaryCross-EntropyLoss) is used. This loss function can measure the difference between the scene existence probability predicted by the model and the true label.

[0113] L 场景 =-[y·log(p)+(1-y)·log(1-p)]

[0114] Among them, y represents the true label, y∈{0,1}, 1 represents the pond farming scene, and 0 represents the non-pond farming scene; p represents the probability of the scene predicted by the model, p∈[0,1].

[0115] (2) Suitable region identification loss L 适宜区域

[0116] The suitable region recognition task is a multi-classification pixel-level task that faces the problems of category imbalance and difficulty in recognizing small objects. To address these challenges, a hybrid loss function is designed that combines focal loss and Dice loss.

[0117] ①Focal loss L Focal :

[0118] L Focal =-α t ·(1-p t ) γ·log(p t )

[0119] Among them, p t represents the predicted probability of the model for the true category, γ represents the modulation factor, usually γ = 2, α t represents the balancing factor, which is adjusted for different categories;

[0120] ②Dice loss L Dice :

[0121]

[0122] Among them, p i ∈[0, 1] represents the probability predicted by the model at pixel i, g i ∈{0, 1} represents the true label of pixel i;

[0123] ③Mixed loss function L 适宜区域

[0124] L 适宜区域 =λ Focal ·L Focal +λ Dice ·L Dice

[0125] Among them, λ Focal and λ Dice It represents the weight coefficient, which balances the impact of the two losses. It can be initially set to 0.5 and adjusted according to the experimental results.

[0126] (3) Modal loss and modal weight

[0127] In the multimodal data fusion model, the model may be overly dependent on a certain modality (such as optical images or SAR images). In order to dynamically balance the contribution of different modalities to model training and prevent the model from being biased towards a certain modality, it is necessary to introduce modal loss and corresponding modal weights.

[0128] The modal loss L of optical image and SAR image is introduced respectively. opt and L sar , and its modal weight γ opt and γ sar The features of the optical SAR encoder are reconstructed back to the input image through the decoder, and the reconstruction error is calculated, which is its modal loss L opt and L sar . Modal weight γ opt and γ sar Initially, the weights can be set to be equal, for example, γ opt =γ sar =0.1. During the training process, the weights are adjusted dynamically according to the model's dependence on each modality.

[0129] (4) Total loss function L 总

[0130] L 总 =L 场景 +I(p>θ)·L 适宜区域 +γ opt L opt +γ sar L sar

[0131] Wherein, I(p>θ) represents an indicator function, which takes the value of 1 when p>θ and 0 otherwise.

[0132] The model gradient is calculated based on the total loss, and the model parameters are updated to obtain the region recognition model. In this embodiment, (1) Training parameter setting: Adam optimizer is selected, which has the characteristics of adaptive learning rate. The initial learning rate is set to 0.0001. The cosine annealing strategy is adopted to dynamically adjust the learning rate to improve the training effect. According to the video memory capacity, the number of samples input to the model at one time is selected to be 16 or 32. The number of training rounds is set to 100 epochs, which can be adjusted according to the training effect; (2) Training process: multi-channel images and corresponding labels are read from the training set for batch processing. The input image passes through a dual-branch encoder to extract optical and SAR features. At each level, the cross-modal attention mechanism is used to fuse multimodal features. After passing through the scene recognition branch, the scene existence probability p is output. When p>θ, the appropriate region recognition branch is activated to output the pixel-level ground object classification result; Loss calculation: Calculate the scene recognition loss L 场景 , calculate the suitable area identification loss L 适宜区域 (When p>θ), calculate the modal loss L of the optical branch and the SAR branch opt and L sar , calculate the total loss L 总 ; Back propagation and parameter update: According to the total loss L 总 , calculate the gradient, and update the model parameters; learning rate adjustment: dynamically update the learning rate according to the learning rate adjustment strategy; model verification: evaluate the model performance on the verification set, evaluate indicators (such as accuracy, mIoU, Kappa coefficient, etc.), save the best model, and obtain the region recognition model.

[0133] S4. Use the regional recognition model to identify suitable layout areas in the pond aquaculture area.

[0134] Embodiment 2

[0135] In this embodiment, a remote sensing identification system for areas suitable for the deployment of photovoltaic facilities in pond aquaculture includes: an image preprocessing module, a data division module, a model training module and an identification module.

[0136] The image preprocessing module is used to obtain the optical image and SAR image of the pond breeding area and perform preprocessing to obtain the processed optical image and processed SAR image; the data partitioning module is used to manually annotate and enhance the processed optical image, and perform data partitioning on the enhanced optical image and processed SAR image to obtain the training data set; the model training module uses a dual-branch encoder, a cross-modal attention mechanism and a multi-task learning framework to construct an initial recognition model, and uses the training data set to train and optimize the initial recognition model to obtain a regional recognition model; the recognition module uses the regional recognition model to identify suitable layout areas in the pond breeding area.

[0137] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A remote sensing identification method for suitable areas for photovoltaic facilities in pond aquaculture, characterized in that: The following steps are involved: Obtaining optical images and SAR images of the pond breeding area and performing preprocessing to obtain processed optical images and processed SAR images; Manually annotating and data enhancing the processed optical image, and performing data segmentation on the enhanced optical image and the processed SAR image to obtain a training data set; An initial recognition model is constructed using a dual-branch encoder, a cross-modal attention mechanism, and a multi-task learning framework, and the initial recognition model is trained and optimized using the training data set to obtain a region recognition model; The area recognition model is used to identify suitable layout areas in the pond breeding area.

2. A remote sensing identification method for suitable areas for photovoltaic facilities deployment in pond aquaculture according to claim 1, characterized in that: The pretreatment method comprises: Performing radiation correction, atmospheric correction, geometric correction and band fusion on the optical image; performing radiation correction, speckle noise filtering and geometric correction on the SAR image; Performing image registration on the fused optical image and the corrected SAR image to obtain a registered optical image and a registered SAR image; Calculating a normalized vegetation index and a normalized water index based on the registered optical image; Based on the normalized vegetation index and the normalized water index, the registered optical image and the registered SAR image are stacked, and the stacked multi-channel image is cropped to obtain the processed optical image and the processed SAR image.

3. A remote sensing identification method for suitable areas for photovoltaic facilities deployment in pond aquaculture according to claim 1, characterized in that: The method for obtaining the enhanced optical image comprises: Classifying the ground object categories, and manually annotating the processed optical image based on the classification results to obtain an annotated optical image; rasterizing the annotated optical image, and cropping the rasterized optical image to obtain a cropped optical image; Data enhancement is performed on the cropped optical image to obtain the enhanced optical image.

4. A remote sensing identification method for suitable areas for photovoltaic facilities deployment in pond aquaculture according to claim 1, characterized in that: The dual-branch encoder includes: an optical image encoder and a SAR image encoder; The optical image encoder uses the ResNet-34 model as the basic encoder, and adjusts the number of input channels of the first convolutional layer of the basic encoder from 3 to 6. The weight of the first convolutional layer is randomly initialized using the Kaiming initialization method, and the weights of other layers are pre-trained weights loaded on the ImageNet dataset; The SAR image encoder uses the ResNet-34 model as the basic encoder, and adjusts the number of input channels of the first convolutional layer of the basic encoder from 3 to 1, and the weights of all convolutional layers are randomly initialized using the Kaiming initialization method.

5. A remote sensing identification method for suitable areas for photovoltaic facilities deployment in pond aquaculture according to claim 1, characterized in that: The cross-modal attention mechanism is set after each main layer of the dual-branch encoder; The workflow of the cross-modal attention mechanism includes: In the first layer of the dual-branch encoder, an optical feature map of the enhanced optical image and a SAR feature map of the processed SAR image are respectively extracted through convolutional layer and residual block calculation; Performing a linear transformation on the optical feature map and the SAR feature map to obtain a query, a key, and a value, calculating a similarity between the query and the key, and obtaining an attention matrix; Using the attention matrix to perform weighted fusion on the values ​​to obtain a fusion feature, and adding the fusion feature to the optical feature map to obtain an output feature; The output features are passed to the next layer of the dual-branch encoder or the multi-task learning framework.

6. A remote sensing identification method for suitable areas for photovoltaic facilities deployment in pond aquaculture according to claim 5, characterized in that: The multi-task learning framework includes: a scene recognition branch and a suitable area recognition branch; The workflow of the scene recognition branch includes: extracting global features from the output features of the last layer of the dual-branch encoder, averaging the spatial dimensions to obtain a global feature vector, and mapping the global feature vector to the probability of scene existence through a fully connected layer to complete scene recognition; The workflow of the suitable area identification branch includes: when the scene identification branch determines that it is a pond farming scene, the output features are decoded through five layers of upsampling, and a suitable area identification result is output.

7. A remote sensing identification method for suitable areas for photovoltaic facilities deployment in pond aquaculture according to claim 1, characterized in that: The method for training and optimizing the initial recognition model includes: The scene recognition loss function, suitable area recognition loss function, modality loss function and modality weight are introduced to construct the total loss function; Using the training data set to train the initial recognition model, and using the total loss function to calculate the total loss; The model gradient is calculated based on the total loss, and the model parameters are updated to obtain the region recognition model.

8. A remote sensing identification system for areas suitable for the deployment of photovoltaic facilities in pond aquaculture, the identification system applying the identification method according to any one of claims 1 to 7, characterized in that: include: Image preprocessing module, data partitioning module, model training module and recognition module; The image preprocessing module is used to obtain the optical image and the SAR image of the pond breeding area and perform preprocessing to obtain the processed optical image and the processed SAR image; The data partitioning module is used to manually annotate and data enhance the processed optical image, and to perform data partitioning on the enhanced optical image and the processed SAR image to obtain a training data set; The model training module uses a dual-branch encoder, a cross-modal attention mechanism and a multi-task learning framework to build an initial recognition model, and uses the training data set to train and optimize the initial recognition model to obtain a region recognition model; The identification module uses the area identification model to identify suitable layout areas in the pond breeding area.

Images