Satellite remote sensing image-based mariculture identification method and system based on deep learning

Abstract

This invention discloses a method and system for identifying mariculture facilities based on satellite remote sensing imagery using deep learning. The method includes: acquiring raw satellite remote sensing imagery and labeled data, and performing data preprocessing to obtain preprocessed image slice data and mask labeling data; constructing a mariculture facility identification model based on the UPerNet segmentation network, selecting ConvNeXt as the encoder, and introducing the SCSE attention mechanism; and identifying the preprocessed image slice data and mask labeling data based on the mariculture facility identification model to obtain the mariculture facility identification result from the satellite remote sensing imagery. This invention can accurately locate the boundaries of aquaculture facilities, improving the accuracy of mariculture facility identification. As a method and system for identifying mariculture facilities based on satellite remote sensing imagery using deep learning, this invention can be widely applied in the field of remote sensing image processing technology.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method and system for identifying mariculture based on satellite remote sensing images using deep learning. Background Technology

[0002] With the rapid development of mariculture, accurate identification and monitoring of the distribution of mariculture facilities are of great significance for aquatic resource management, environmental protection, and industrial planning. Currently, the identification and monitoring of mariculture facilities mainly rely on traditional visual interpretation methods and traditional machine learning methods.

[0003] Traditional identification of mariculture facilities primarily relies on manual visual interpretation of satellite remote sensing imagery by professionals. These professionals manually delineate and label different types of mariculture facilities based on features such as hue, texture, shape, and size in the images, combined with field survey data. This method suffers from drawbacks including low efficiency, high subjectivity, difficulty in handling large amounts of data, and high labor costs, failing to meet the needs of large-scale, rapid monitoring. To improve the efficiency and accuracy of mariculture facility identification, researchers have attempted to use traditional machine learning methods (such as support vector machines and random forests) for classification and identification of remote sensing images. These methods typically rely on manually designed features (such as texture features and spectral features) for classification decisions. However, these methods are highly dependent on manually designed features, struggle to handle complex sea surface backgrounds and diverse morphologies of aquaculture facilities, have weak generalization capabilities, and are unable to accurately distinguish between different types of aquaculture facilities with similar morphologies. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to provide a method and system for identifying marine aquaculture facilities based on deep learning satellite remote sensing images, which can accurately locate the boundaries of aquaculture facilities and improve the accuracy of marine aquaculture facility identification.

[0005] The first technical solution adopted in this invention is: a deep learning-based method for identifying mariculture in satellite remote sensing images, comprising the following steps:

[0006] The original satellite remote sensing images and annotation data are acquired and preprocessed to obtain preprocessed image slice data and mask annotation data.

[0007] Based on the UPerNet segmentation network, ConvNeXt was selected as the encoder, and the SCSE attention mechanism was introduced to construct a marine aquaculture facility recognition model.

[0008] Based on the marine aquaculture facility identification model, the preprocessed image slice data and mask annotation data are identified to obtain the marine aquaculture facility identification results from satellite remote sensing images.

[0009] Furthermore, the step of acquiring raw satellite remote sensing images and annotation data and performing data preprocessing to obtain preprocessed image tile data and mask annotation data specifically includes:

[0010] The original satellite remote sensing images are acquired and sliced ​​using a sliding window strategy to obtain the sliced ​​satellite remote sensing images.

[0011] By filling zero values ​​into the cut satellite remote sensing image using an edge-filling strategy, satellite remote sensing image slices with standard sizes are obtained.

[0012] Set up overlapping areas for the slices, perform overlapping slice processing on satellite remote sensing image slices of standard size using an overlapping slice strategy, then calculate the proportion of valid values ​​in each slice and filter them to obtain the image slice file.

[0013] The labeled data is unified and integrated to obtain the integrated labeled data.

[0014] The integrated labeled data is converted into vector data, and the spatial location and type information of the corresponding marine aquaculture facilities in the image slice file are obtained. The vector data is then reprojected onto the spatial location and type information of the corresponding marine aquaculture facilities in the image slice file and converted into a raster mask to obtain the preprocessed image slice data and mask labeled data.

[0015] Furthermore, the marine aquaculture facility identification model specifically includes a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on the SCSE attention mechanism, a hybrid layer, and a classification head. The first output of the ConvNeXt encoder is connected to the input of the pyramid pooling module, the second output of the ConvNeXt encoder is connected to the input of the feature fusion layer based on the SCSE attention mechanism, the output of the feature fusion layer based on the SCSE attention mechanism is connected to the input of the hybrid layer, and the output of the hybrid layer is connected to the input of the classification head, wherein:

[0016] The ConvNeXt encoder includes a first encoder layer, a second encoder layer, a third encoder layer, and a fourth encoder layer;

[0017] The feature fusion layer based on the SCSE attention mechanism includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer, and a fourth feature fusion layer;

[0018] The output of the first encoder layer is connected to the input of the second encoder layer and the input of the fourth feature fusion layer, respectively. The output of the second encoder layer is connected to the input of the third encoder layer and the input of the third feature fusion layer, respectively. The output of the third encoder layer is connected to the input of the fourth encoder layer and the input of the second feature fusion layer, respectively. The output of the fourth encoder layer is connected to the input of the pyramid pooling module and the input of the first feature fusion layer, respectively.

[0019] Furthermore, the pyramid pooling module includes a first branch module, a second branch module, a third branch module, a fourth branch module, a feature concatenation layer, and a convolutional layer. The first branch module includes a global pooling layer, a global average pooling layer, a first convolutional layer, and a first upsampling layer. The second branch module includes... The third branch module includes a pooling layer, a first adaptive average pooling layer, a second convolutional layer, and a second upsampling layer. The fourth branch module includes a pooling layer, a second adaptive average pooling layer, a third convolutional layer, and a third upsampling layer. Pooling layer, third adaptive average pooling layer, fourth convolutional layer and fourth upsampling layer.

[0020] Furthermore, the loss function of the marine aquaculture facility identification model is the Tversky Loss function, whose expression is:

[0021] ;

[0022] In the above formula, Indicates the number of true positive pixels. Indicates the number of false negative pixels. Indicates the number of false positive pixels. This represents a small constant used to prevent the denominator from being zero. This represents the Tversky Loss function.

[0023] Furthermore, the step of identifying the preprocessed image slice data and mask annotation data based on the marine aquaculture facility identification model to obtain the satellite remote sensing image marine aquaculture facility identification result specifically includes:

[0024] The preprocessed image slice data and mask annotation data are input into the marine aquaculture facility recognition model;

[0025] The ConvNeXt encoder based on the marine aquaculture facility identification model performs feature extraction processing on the preprocessed image slice data and mask annotation data to obtain preliminary multi-scale features.

[0026] The pyramid pooling module based on the marine aquaculture facility identification model performs contextual information fusion processing on the preliminary multi-scale features to obtain multi-scale features.

[0027] The feature fusion layer based on the SCSE attention mechanism of the marine aquaculture facility identification model performs feature fusion processing on the initial multi-scale features to obtain fused multi-scale feature information.

[0028] Based on the hybrid layer of the marine aquaculture facility identification model, semantic detail information balancing processing is performed on the fused multi-scale feature information to obtain balanced multi-scale feature information.

[0029] Based on the classification head of the marine aquaculture facility identification model, feature classification is performed on the balanced multi-scale feature information to obtain the identification results of marine aquaculture facilities from satellite remote sensing images.

[0030] Furthermore, the pyramid pooling module based on the marine aquaculture facility identification model performs contextual information fusion processing on the preliminary multi-scale features to obtain multi-scale features. This step specifically includes:

[0031] The preliminary multi-scale features are input into the pyramid pooling module of the marine aquaculture facility identification model;

[0032] The first branch module based on the pyramid pooling module performs global average pooling on the preliminary multi-scale features to obtain the first multi-scale global features.

[0033] The second branch module based on the pyramid pooling module performs adaptive average pooling on the initial multi-scale features to obtain the second multi-scale global features.

[0034] The third branch module based on the pyramid pooling module performs adaptive average pooling on the preliminary multi-scale features to obtain the third multi-scale global features.

[0035] The fourth branch module based on the pyramid pooling module performs adaptive average pooling on the preliminary multi-scale features to obtain the fourth multi-scale global features.

[0036] Based on the feature stitching layer of the pyramid pooling module, the first multi-scale global features, the second multi-scale global features, the third multi-scale global features, the fourth multi-scale global features, and the preliminary multi-scale features are stitched together to obtain the stitched multi-scale global features.

[0037] Convolutional layers based on pyramid pooling modules are used to perform convolution processing on the concatenated multi-scale global features to obtain multi-scale features.

[0038] Furthermore, the feature fusion layer based on the SCSE attention mechanism of the marine aquaculture facility identification model performs feature fusion processing on the preliminary multi-scale features to obtain fused multi-scale feature information. This step specifically includes:

[0039] The preliminary multi-scale features and multi-scale features are input into the feature fusion layer of the SCSE attention mechanism of the marine aquaculture facility identification model;

[0040] Based on the channel attention branch of the feature fusion layer with the SCSE attention mechanism, channel attention features are extracted from the initial multi-scale features to obtain channel attention weights.

[0041] Based on the spatial attention branch of the feature fusion layer with the SCSE attention mechanism, spatial attention features are extracted from the initial multi-scale features to obtain spatial attention weights.

[0042] The feature fusion layer based on the SCSE attention mechanism combines channel attention weights and spatial attention weights to perform feature fusion processing on the initial multi-scale features and multi-scale features, and obtain the fused multi-scale feature information.

[0043] Furthermore, the method includes inputting the preprocessed image slice data and mask annotation data into the marine aquaculture facility recognition model for data augmentation processing. The data augmentation includes geometric transformation processing, color space transformation processing, and random copy-paste. The training strategy of the marine aquaculture facility recognition model includes the AdamW optimizer, cosine annealing learning rate scheduling strategy, and mixed precision training strategy.

[0044] The second technical solution adopted in this invention is: a deep learning-based satellite remote sensing image mariculture identification system, comprising:

[0045] The first module is used to acquire raw satellite remote sensing images and annotation data and perform data preprocessing to obtain preprocessed image slice data and mask annotation data.

[0046] The second module is used to construct a marine aquaculture facility recognition model based on the UPerNet segmentation network, selecting ConvNeXt as the encoder and introducing the SCSE attention mechanism.

[0047] The third module is used to identify preprocessed image slice data and mask annotation data based on the marine aquaculture facility identification model, so as to obtain the identification results of marine aquaculture facilities in satellite remote sensing images.

[0048] The beneficial effects of the method and system of this invention are as follows: This invention acquires raw satellite remote sensing images and labeled data and performs data preprocessing to obtain preprocessed image slice data and mask labeled data, realizing fully automated processing from satellite remote sensing images to vector data output, significantly reducing the need for manual intervention. Furthermore, based on the UPerNet segmentation network, ConvNeXt is selected as the encoder, and the SCSE attention mechanism is introduced to construct a marine aquaculture facility identification model. Based on the network architecture design of UPerNet and ConvNeXt, combined with the SCSE attention mechanism, the accuracy of marine aquaculture facility identification is improved. Advanced semantic segmentation technology is adopted, which can accurately locate the boundaries of aquaculture facilities and improve the accuracy of subsequent area statistics and spatial analysis. Finally, based on the marine aquaculture facility identification model, the preprocessed image slice data and mask labeled data are identified to obtain the marine aquaculture facility identification results from the satellite remote sensing images. Attached Figure Description

[0049] Figure 1 This is a flowchart of the steps of the deep learning-based satellite remote sensing image marine aquaculture identification method of the present invention;

[0050] Figure 2 This is a structural block diagram of the deep learning-based satellite remote sensing image marine aquaculture identification system of the present invention;

[0051] Figure 3 This is a schematic diagram of a satellite remote sensing imagery-based marine aquaculture identification framework provided in a specific embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the data preprocessing framework provided in a specific embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of the marine aquaculture facility identification model provided in a specific embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of the pyramid pooling module provided in a specific embodiment of the present invention;

[0055] Figure 7 This is a schematic diagram of the SCSE attention mechanism provided in a specific embodiment of the present invention;

[0056] Figure 8 This is a schematic diagram of deep learning training for the marine aquaculture facility identification model provided in a specific embodiment of the present invention;

[0057] Figure 9 This is a schematic diagram illustrating the application of the marine aquaculture facility identification model provided in a specific embodiment of the present invention;

[0058] Figure 10This is a schematic diagram of the recognition results provided in a specific embodiment of the present invention. Detailed Implementation

[0059] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0060] First, it needs to be pointed out that there are some shortcomings in the relevant technologies, such as:

[0061] Existing technologies for identifying marine aquaculture facilities suffer from the following main problems: First, high-resolution satellite remote sensing images typically have dimensions of tens of thousands of pixels, resulting in a large data volume that cannot be directly input into the identification model for processing. Current technologies lack effective strategies for handling edge areas. Second, marine aquaculture facilities are diverse, and different types exhibit similarities in their appearance within remote sensing images, increasing the difficulty of identification. Third, remote sensing images of marine aquaculture areas are susceptible to environmental factors such as cloud cover, water quality, and lighting conditions, and current technologies have limited adaptability to these changing factors. Fourth, the boundaries of marine aquaculture facilities are often unclear in remote sensing images, making it difficult for current technologies to accurately locate these boundaries. Finally, existing technologies lack sufficient automation in the entire workflow from remote sensing image identification to the final generation of vector data suitable for GIS analysis, often requiring manual intervention.

[0062] Based on this, refer to Figure 1 and Figure 3 This invention provides a method for identifying marine aquaculture based on satellite remote sensing imagery using deep learning. The method includes the following steps:

[0063] S100: Acquire raw satellite remote sensing images and annotation data and perform data preprocessing to obtain preprocessed image slice data and mask annotation data;

[0064] S110. Acquire the original satellite remote sensing image and slice it using a sliding window strategy to obtain the sliced ​​satellite remote sensing image.

[0065] S120. Fill the cut satellite remote sensing image with zero values ​​using the edge filling strategy to obtain a satellite remote sensing image slice with a standard size.

[0066] S130. Set the overlapping area of ​​the slices. Perform overlapping slice processing on satellite remote sensing image slices of standard size through the overlapping slice strategy. Then calculate the proportion of effective values ​​in each slice and filter them to obtain the image slice file.

[0067] In this embodiment, as Figure 4 As shown, this invention implements a slicing algorithm for processing large satellite remote sensing images, capable of effectively handling remote sensing images of arbitrary sizes. Specifically, a sliding window strategy is used to cut large remote sensing images into fixed-size blocks. To address the problem of information loss in edge regions, an edge-filling strategy is employed. For areas where the image edges are insufficient to form a complete slice, standard-sized slices are generated by filling with zero values. Simultaneously, to address the discontinuity of slice boundaries, this invention introduces an overlapping slicing strategy, ensuring information continuity between adjacent slices by setting overlapping areas. Furthermore, this invention implements a slice selection mechanism based on effective values. By calculating the proportion of effective values ​​in each slice, slices containing sufficient effective information are selected, improving the quality of the training data.

[0068] S140. Perform data unification and integration processing on the labeled data to obtain the integrated labeled data;

[0069] In this embodiment, for data unification and integration, the present invention implements a method for unified processing and format standardization of data from different regions and time phases. Specifically, all annotation files are first merged into a unified table, including feature IDs, feature geometric information, and metadata of the original image (satellite product type, sensor type, imaging time, coordinate reference system, original image range, etc.), ensuring that the annotation data is unified and traceable. Finally, it is saved in geojson format. All images are uniformly converted to unsigned 8-bit integer geotiff format, and invalid data is uniformly filled with zero values, with the original invalid region mask stored as a boolean type.

[0070] S150. Convert the integrated annotation data into vector data, obtain the spatial location and type information of the corresponding marine aquaculture facilities in the image slice file, reproject the vector data onto the spatial location and type information of the corresponding marine aquaculture facilities in the image slice file, and then convert it into a raster mask to obtain the preprocessed image slice data and mask annotation data.

[0071] In this embodiment, for vector data rasterization, the present invention implements a method for converting vector annotation data in shapefile and geojson formats into pixel-level masks. Specifically, the image slice file and the integrated annotation file are first read to obtain the spatial location and type information of the mariculture facilities corresponding to the image slices. Then, the vector data is reprojected onto the coordinate system of the satellite remote sensing image. Next, the rasterization function of the rasterio library in Python is used to convert the vector data into a raster mask, where different types of mariculture facilities correspond to different pixel values ​​(e.g., raft aquaculture corresponds to pixel value 1, deep-sea cages correspond to pixel value 2, ordinary cages correspond to pixel value 3, and the background corresponds to pixel value 0). Finally, the generated raster mask is saved as a mask file corresponding to the original image slice.

[0072] In summary, data preprocessing can convert raw, large-size satellite remote sensing images and vector-annotated data into standard format data suitable for deep learning model training, providing high-quality training data for subsequent model training. This invention uses high-resolution true-color satellite remote sensing images with a resolution greater than 1m as training data. Images of concentrated mariculture areas were collected in coastal waters. The outlines of mariculture facilities were manually drawn using polygon tools in a GIS platform, and different mariculture types were assigned corresponding values. The annotation results were saved as shapefiles, with each annotation corresponding one-to-one with the image. To ensure the effectiveness of subsequent model training, the training sample set in this embodiment covers mariculture targets of different time phases, types, and characteristics, encompasses various sea surface backgrounds and land cover distributions, and contains a sufficient number of training samples to ensure that the model training does not suffer from overfitting.

[0073] S200, based on the UPerNet segmentation network, selects ConvNeXt as the encoder, introduces the SCSE attention mechanism, and constructs a marine aquaculture facility recognition model;

[0074] Specifically, such as Figure 5 As shown, the marine aquaculture facility identification model specifically includes a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on the SCSE attention mechanism, a hybrid layer, and a classification head. The first output of the ConvNeXt encoder is connected to the input of the pyramid pooling module, the second output of the ConvNeXt encoder is connected to the input of the feature fusion layer based on the SCSE attention mechanism, the output of the feature fusion layer based on the SCSE attention mechanism is connected to the input of the hybrid layer, and the output of the hybrid layer is connected to the input of the classification head.

[0075] The ConvNeXt encoder comprises a first encoder layer, a second encoder layer, a third encoder layer, and a fourth encoder layer; the feature fusion layer based on the SCSE attention mechanism comprises a first feature fusion layer, a second feature fusion layer, a third feature fusion layer, and a fourth feature fusion layer; the output of the first encoder layer is connected to the input of the second encoder layer and the input of the fourth feature fusion layer, respectively; the output of the second encoder layer is connected to the input of the third encoder layer and the input of the third feature fusion layer, respectively; the output of the third encoder layer is connected to the input of the fourth encoder layer and the input of the second feature fusion layer, respectively; and the output of the fourth encoder layer is connected to the input of the pyramid pooling module and the input of the first feature fusion layer, respectively.

[0076] like Figure 6 As shown, the pyramid pooling module includes a first branch module, a second branch module, a third branch module, a fourth branch module, a feature concatenation layer, and a convolutional layer. The first branch module includes a global pooling layer, a global average pooling layer, a first convolutional layer, and a first upsampling layer. The second branch module includes... The third branch module includes a pooling layer, a first adaptive average pooling layer, a second convolutional layer, and a second upsampling layer. The fourth branch module includes a pooling layer, a second adaptive average pooling layer, a third convolutional layer, and a third upsampling layer. Pooling layer, third adaptive average pooling layer, fourth convolutional layer and fourth upsampling layer.

[0077] This invention employs the UPerNet segmentation network structure, which effectively utilizes feature information at different scales and is suitable for handling semantic segmentation tasks in complex scenes. As the backbone network, ConvNeXt is selected as the encoder. This network structure combines the inductive bias of CNNs with the modeling capabilities of Transformers, enabling the extraction of rich features from satellite remote sensing images. To further improve the model's ability to distinguish between different types of aquaculture facilities, this invention introduces the SCSE (Spatial and Channel Squeeze & Excitation) attention mechanism, such as... Figure 7 As shown, this mechanism can enhance important features and suppress unimportant features in both spatial and channel dimensions, thereby improving the segmentation accuracy of the model.

[0078] The UPerNet network architecture consists of an encoder and a decoder. The encoder uses a ConvNeXt network, which comprises four stages, each containing multiple ConvNeXt blocks. Each block includes depthwise convolutions, pointwise convolutions, and a self-attention mechanism. The decoder employs a feature pyramid structure, improving segmentation accuracy through multi-scale feature fusion. During feature fusion, an SCSE attention module is introduced, which includes channel attention branches and spatial attention branches. These branches weight the channel and spatial dimensions of the feature map, respectively, to highlight important features.

[0079] The final output of the model is a recognition confidence score map, which includes three dimensions: category, width, and height. The width and height are the same as the input image. The category dimension represents the confidence score output by the model for different categories. The confidence score is between 0 and 1, representing the probability that the model believes that the pixel belongs to that category.

[0080] S300: Based on the marine aquaculture facility identification model, the preprocessed image slice data and mask annotation data are identified to obtain the marine aquaculture facility identification results from satellite remote sensing images.

[0081] Specifically, the preprocessed image slice data and mask annotation data are input into the marine aquaculture facility recognition model. Based on the ConvNeXt encoder of the marine aquaculture facility recognition model, feature extraction is performed on the preprocessed image slice data and mask annotation data to obtain preliminary multi-scale features. Based on the pyramid pooling module of the marine aquaculture facility recognition model, contextual information fusion processing is performed on the preliminary multi-scale features to obtain multi-scale features. Based on the SCSE attention mechanism of the marine aquaculture facility recognition model, feature fusion processing is performed on the preliminary multi-scale features and multi-scale features to obtain fused multi-scale feature information. Based on the hybrid layer of the marine aquaculture facility recognition model, semantic detail information balancing processing is performed on the fused multi-scale feature information to obtain balanced multi-scale feature information. Based on the classification head of the marine aquaculture facility recognition model, feature classification is performed on the balanced multi-scale feature information to obtain the marine aquaculture facility recognition result from satellite remote sensing imagery.

[0082] The pyramid pooling module further includes: a pyramid pooling module that inputs preliminary multi-scale features into the marine aquaculture facility identification model; a first branch module based on the pyramid pooling module that performs global average pooling on the preliminary multi-scale features to obtain the first multi-scale global features; a second branch module based on the pyramid pooling module that performs adaptive average pooling on the preliminary multi-scale features to obtain the second multi-scale global features; a third branch module based on the pyramid pooling module that performs adaptive average pooling on the preliminary multi-scale features to obtain the third multi-scale global features; a fourth branch module based on the pyramid pooling module that performs adaptive average pooling on the preliminary multi-scale features to obtain the fourth multi-scale global features; a feature concatenation layer based on the pyramid pooling module that concatenates the first, second, third, and fourth multi-scale global features with the preliminary multi-scale features to obtain the concatenated multi-scale global features; and a convolutional layer based on the pyramid pooling module that performs convolution on the concatenated multi-scale global features to obtain the multi-scale features.

[0083] The feature fusion layer of the SCSE attention mechanism also includes: inputting the preliminary multi-scale features and multi-scale features into the marine aquaculture facility identification model; the channel attention branch of the feature fusion layer based on the SCSE attention mechanism extracts channel attention features from the preliminary multi-scale features and multi-scale features to obtain channel attention weights; the spatial attention branch of the feature fusion layer based on the SCSE attention mechanism extracts spatial attention features from the preliminary multi-scale features and multi-scale features to obtain spatial attention weights; and the feature fusion layer based on the SCSE attention mechanism combines the channel attention weights and spatial attention weights to perform feature fusion processing on the preliminary multi-scale features and multi-scale features to obtain fused multi-scale feature information.

[0084] Regarding data augmentation strategies, to enhance the model's generalization ability and adaptability to environmental changes, this invention implements a rich set of data augmentation methods. Basic geometric transformations include random horizontal flipping (probability 0.5), random vertical flipping (probability 0.5), and random 90-degree rotation (probability 0.5). Color space transformations include brightness adjustment, contrast adjustment, saturation adjustment, and hue adjustment.

[0085] Furthermore, this invention introduces a random copy-paste technique, a specific data augmentation method for class imbalance problems. In its implementation, firstly, image regions containing the target class (especially classes with fewer samples, such as deep-sea cages) are randomly selected from the training batch; then, a target location is randomly selected (this can be another location within the same image or another image in the batch); next, the selected region is copied and pasted into the target location, while maintaining consistent mask annotations; finally, appropriate smoothing is applied to ensure the inserted region blends naturally with its surroundings. This technique effectively increases the number and diversity of samples from smaller classes, improves the model's ability to recognize these smaller classes, and also enhances the model's adaptability to targets appearing in different backgrounds.

[0086] The data augmentation is implemented using the Kornia library in Python, which provides efficient GPU-accelerated image enhancement capabilities. During training, data augmentation operations are applied after each batch of data is loaded, ensuring that the model sees some changes in the data during each training cycle, thereby enhancing the model's generalization ability.

[0087] Regarding the design of the loss function, to address the class imbalance problem, this invention employs the Tversky Loss function. Tversky Loss is a generalization of Dice Loss; by adjusting the alpha and beta parameters, it can balance the weights of false positives and false negatives, making it suitable for handling class imbalance segmentation tasks. In this invention, alpha is set to 0.3 and beta to 0.7, so the penalty for false negatives (missed detections) is greater than the penalty for false positives (false detections), which helps improve the recall rate of the target.

[0088] The formula for calculating Tversky Loss is as follows:

[0089] ;

[0090] in, Indicates the number of true positive pixels. Indicates the number of false negative pixels. Indicates the number of false positive pixels. It is a small constant used to prevent the denominator from being zero.

[0091] Furthermore, this invention also implements multi-class Tversky Loss for handling multi-class segmentation tasks. Multi-class Tversky Loss calculates the Tversky Loss for each class and then averages the results as the final loss function.

[0092] For the model training process, such as Figure 8As shown, the encoder is first initialized using a pre-trained ConvNeXt model, and transfer learning is used to accelerate the training process. Then, a low-learning-rate AdamW optimizer is used for training, with a cosine annealing learning rate scheduling strategy to gradually decrease the learning rate as training progresses. During training, mini-batch stochastic gradient descent with a batch size of 16 is employed, and mixed-precision training is used to accelerate the training process. To prevent overfitting, this invention also introduces a weight decay (set to 0.01) and an early stopping strategy. When the validation set loss does not improve within five consecutive epochs, training is stopped and the best-performing model weights are restored.

[0093] Regarding cross-validation and model ensemble, to improve the robustness and generalization ability of the models, this invention adopts a k-fold cross-validation strategy. In the specific implementation (taking k=4 as an example), the training dataset is first randomly divided into four non-overlapping subsets; then, four training iterations are performed, each time selecting three subsets as the training set and the remaining subset as the validation set; finally, the recognition results of the four models are integrated to obtain the final recognition result.

[0094] The model ensemble employs a soft-voting approach, which averages the predicted probabilities of each model and then selects the category with the highest probability as the final identification result. This method fully leverages the complementarity of multiple models, improving the accuracy and robustness of predictions.

[0095] Regarding model export and deployment, to facilitate subsequent model deployment and application, this invention converts the trained model into TorchScript format. TorchScript is an intermediate representation format that can run independently of the Python environment and is suitable for deployment in production environments. Specifically, the trained model weights are first loaded; then, the `jit.trace` function from Python's `torch` library is used to convert the model into TorchScript format; finally, the converted model file is saved.

[0096] Through the above design, the deep learning model training module can build and train a high-performance marine aquaculture facility recognition model, providing strong support for subsequent reasoning and applications.

[0097] The inference and application module is responsible for identifying mariculture facilities in new satellite remote sensing imagery and outputting vector results. The processing flow of this module is as follows: Figure 9 As shown, it mainly includes three parts: sliding window prediction, recognition result splicing and merging, and raster-to-vector conversion.

[0098] Regarding sliding window prediction, this invention implements a sliding window-based inference strategy to handle satellite remote sensing images of arbitrary sizes. Specifically, the satellite remote sensing image to be processed is first loaded; then image preprocessing is performed, including image padding (ensuring the image can be completely segmented); next, the sliding window method is used to segment the large image into multiple small blocks (e.g., 512×512 pixels), with a certain overlap between adjacent windows (e.g., 256 pixels); then prediction is performed for each small block; finally, the recognition results are stitched back to the original image size.

[0099] During the image filling process, this invention adopts a zero-fill strategy, that is, for areas where the image edges are insufficient to form a complete window, a standard-sized window is generated by filling with zero values.

[0100] Regarding the stitching and merging of recognition results, to address the overlapping region issue caused by sliding windows, this invention implements a confidence-based fusion strategy. Specifically, a recognition confidence score map of the same size as the original image with an initial value of 0 is first created. Then, for each window's recognition result, it is accumulated and added to the corresponding position in the recognition confidence score map. Finally, the category with the highest confidence score at each position is selected to obtain the final recognition result. This method can fully utilize the information in overlapping regions, improving the accuracy and continuity of prediction. Especially for regions at window edges, fusing the recognition results from multiple windows can reduce the impact of edge effects and improve prediction quality.

[0101] Regarding the conversion from raster to vector, to facilitate application in GIS systems, this invention implements a method for converting pixel-level classification results into vector data. Specifically, the classification result image is first read; then, the outline of the corresponding region is extracted for each category; next, the extracted outline is converted into polygons in a geographic coordinate system; finally, the polygon data is saved as a vector file in geojson or shapefile format.

[0102] During contour extraction, this invention utilizes the `features.shapes` function from Python's `rasterio` library, which efficiently extracts vector features from raster data. During coordinate transformation, this invention uses geographic transformation information from the original imagery to ensure the extracted polygons have correct geographic coordinates.

[0103] Through the above design, the reasoning and application module can efficiently and accurately identify marine aquaculture facilities from satellite remote sensing images of any size and output vector results that are easy for GIS applications, realizing fully automated processing from remote sensing images to vector data output.

[0104] Therefore, the embodiments of the present invention differ from the prior art in the following ways:

[0105] 1) Overall architecture design of a deep learning-based marine aquaculture facility identification system, including data preprocessing, deep learning model training, inference, and application.

[0106] 2) A tiling method for large-size satellite remote sensing images, which uses a sliding window tiling algorithm, an edge filling strategy and an overlapping area processing method to solve the problem of edge information loss.

[0107] 3) Based on the network architecture design of UPerNet and ConvNeXt, combined with the SCSE attention mechanism, the accuracy of marine aquaculture facility identification is improved.

[0108] 4) A data augmentation strategy for imbalanced marine aquaculture facility types, especially random copy-paste technology, to increase the diversity of small category samples.

[0109] 5) Based on the Tversky Loss loss function design, the recall rate is improved by adjusting the alpha and beta parameters in the Tversky Loss to balance false positives and false negatives.

[0110] 6) A large image inference strategy based on a sliding window, combined with a confidence fusion method, enables efficient processing of satellite remote sensing images of arbitrary size.

[0111] 7) An automatic conversion method from raster classification results to vector data, realizing the full-process automatic processing from remote sensing imagery to usable GIS data.

[0112] The embodiments of the present invention have the following advantages compared with the prior art:

[0113] 1) High recognition accuracy: Compared with traditional methods, the system of the present invention achieves an overall accuracy of over 87% in the task of recognizing marine aquaculture facilities, and performs particularly well in distinguishing different types of aquaculture facilities with similar shapes.

[0114] 2) High processing efficiency: The system of this invention achieves a processing speed of seconds in processing large satellite remote sensing images, which can meet the needs of large-scale and rapid monitoring.

[0115] 3) High degree of automation: This invention realizes fully automated processing from satellite remote sensing images to vector data output, which greatly reduces the need for manual intervention.

[0116] 4) High adaptability: Through rich data augmentation strategies and advanced network architecture design, the system of this invention has good adaptability to satellite remote sensing images of different time phases and regions.

[0117] 5) Precise Boundary: This invention uses advanced semantic segmentation technology to accurately locate the boundaries of aquaculture facilities, improving the accuracy of subsequent area statistics and spatial analysis.

[0118] 6) Easy deployment: By converting the model to TorchScript format, the system of this invention is easy to deploy and apply.

[0119] Finally, simulation experiments are conducted to illustrate the embodiments of the present invention in conjunction with the accompanying drawings:

[0120] In this embodiment, a sea area of ​​a certain province was selected as the example area. High-resolution satellite remote sensing images of the coastal aquaculture area of ​​the province were collected. The image resolution was 1m, covering various types of marine aquaculture facilities, including raft aquaculture, ordinary net cages, and deep-water net cages.

[0121] First, the training images were loaded into ArcGIS software. The polygon tool was used to classify and label the mariculture targets in the images, and the labeling results were saved as shapefiles. The data preprocessing module sliced ​​the original satellite remote sensing images, generating 512×512 pixel image blocks and corresponding label masks. A total of 1649 images were generated, with 13897 labeled features. The training, validation, and test set ratio was set to 7:2:1. A mariculture facility recognition model was built and trained using the deep learning model training module. During training, the network architecture, data augmentation strategies, and loss function design described earlier were adopted, and model performance was improved through cross-validation and model ensemble. Finally, the inference and application module identified mariculture facilities in the satellite remote sensing images of the test area, and the results were converted to geojson format and saved.

[0122] Figure 10 The identification results of the system of the present invention are presented. It can be seen that the system of the present invention can more accurately identify different types of marine aquaculture facilities, especially excelling in distinguishing aquaculture facilities with similar morphologies. The quantitative evaluation results shown in Table 1 indicate that the system of the present invention achieved an overall F1 score of 87.55% on the test data. In terms of processing efficiency, the system of the present invention can process a 10000×10000 pixel satellite remote sensing image in only 23 seconds.

[0123] Table 1. Identification and Evaluation Results Data Table

[0124] ;

[0125] In summary, the deep learning-based satellite remote sensing imagery-based automatic identification system and method for mariculture provided by this invention can efficiently and accurately identify different types of mariculture facilities in satellite remote sensing images, and realize fully automated processing from remote sensing images to vector data output, providing important technical support for mariculture supervision, environmental impact assessment, industrial planning and other fields.

[0126] Reference Figure 2 A deep learning-based satellite remote sensing imagery-based mariculture identification system includes:

[0127] The first module 201 is used to acquire raw satellite remote sensing images and annotation data and perform data preprocessing to obtain preprocessed image slice data and mask annotation data.

[0128] The second module 202 is used to construct a marine aquaculture facility recognition model based on the UPerNet segmentation network, selecting ConvNeXt as the encoder and introducing the SCSE attention mechanism.

[0129] The third module 203 is used to identify the preprocessed image slice data and mask annotation data based on the marine aquaculture facility identification model, and obtain the marine aquaculture facility identification results from satellite remote sensing images.

[0130] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0131] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A satellite remote sensing image-based mariculture identification method based on deep learning, characterized in that, The method comprises the following steps: obtaining original satellite remote sensing images and label data and performing data preprocessing to obtain preprocessed image slice data and mask label data; based on the UPerNet segmentation network, selecting ConvNeXt as the encoder, introducing the SCSE attention mechanism, and constructing a mariculture facility recognition model; The mariculture facility recognition model specifically comprises a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on the SCSE attention mechanism, a mixing layer, and a classification head, the first output end of the ConvNeXt encoder is connected with the input end of the pyramid pooling module, the second output end of the ConvNeXt encoder is connected with the input end of the feature fusion layer based on the SCSE attention mechanism, the output end of the feature fusion layer based on the SCSE attention mechanism is connected with the input end of the mixing layer, and the output end of the mixing layer is connected with the input end of the classification head, wherein: The ConvNeXt encoder comprises a first encoder layer, a second encoder layer, a third encoder layer, and a fourth encoder layer; The feature fusion layer based on the SCSE attention mechanism comprises a first feature fusion layer, a second feature fusion layer, a third feature fusion layer, and a fourth feature fusion layer; The output end of the first encoder layer is respectively connected with the input end of the second encoder layer and the input end of the fourth feature fusion layer, the output end of the second encoder layer is respectively connected with the input end of the third encoder layer and the input end of the third feature fusion layer, the output end of the third encoder layer is respectively connected with the input end of the fourth encoder layer and the input end of the second feature fusion layer, the output end of the fourth encoder layer is respectively connected with the input end of the pyramid pooling module and the input end of the first feature fusion layer, the output end of the pyramid pooling module is connected with the input end of the first feature fusion layer, the output end of the first feature fusion layer is connected with the input end of the second feature fusion layer, the output end of the second feature fusion layer is connected with the input end of the third feature fusion layer, the output end of the third feature fusion layer is connected with the input end of the fourth feature fusion layer, and the output end of the second feature fusion layer, the output end of the third feature fusion layer, and the output end of the fourth feature fusion layer are all connected with the input end of the mixing layer; based on the mariculture facility recognition model, the preprocessed image slice data and the mask label data are recognized to obtain a satellite remote sensing image mariculture facility recognition result.

2. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 1, characterized in that, The step of obtaining original satellite remote sensing images and label data and performing data preprocessing to obtain preprocessed image slice data and mask label data specifically comprises: obtaining original satellite remote sensing images and performing slicing through a sliding window strategy to obtain sliced satellite remote sensing images; filling zero values in the sliced satellite remote sensing images through an edge padding strategy to obtain satellite remote sensing image slices with a standard size; A slice overlap region is set, a satellite remote sensing image slice with a standard size is processed by an overlap slice strategy, a proportion of effective values in each slice is calculated and screened, and an image slice file is obtained; The labeled data is subjected to data unification and integration processing to obtain integrated labeled data; The integrated labeled data is converted into vector data, the spatial position and type information of the corresponding mariculture facilities in the image slice file are acquired, the vector data is re-projected into the spatial position and type information of the corresponding mariculture facilities in the image slice file and then converted into a raster mask to obtain preprocessed image slice data and mask labeled data.

3. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 2, characterized in that, The pyramid pooling module comprises a first branch module, a second branch module, a third branch module, a fourth branch module, a feature splicing layer and a convolution layer, the first branch module comprises a global pooling layer, a global average pooling layer, a first convolution layer and a first up-sampling layer, the second branch module comprises a pooling layer, a first adaptive average pooling layer, a second convolution layer and a second up-sampling layer, the third branch module comprises a pooling layer, a second adaptive average pooling layer, a third convolution layer and a third up-sampling layer, the fourth branch module comprises a pooling layer, a third adaptive average pooling layer, a fourth convolution layer and a fourth up-sampling layer.

4. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 3, characterized in that, The loss function of the mariculture facility recognition model is a Tversky Loss loss function, and its expression is: ; In the above formula, denotes the number of true positive pixels, denotes the number of false negative pixels, denotes the number of false positive pixels, denotes a small constant to prevent the denominator from being zero, denotes the Tversky Loss loss function.

5. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 4, characterized in that, The mariculture facility recognition model is used to recognize the preprocessed image slice data and mask labeled data to obtain a satellite remote sensing image mariculture facility recognition result, and the specific steps include: The preprocessed image slice data and mask labeled data are input into the mariculture facility recognition model; The ConvNeXt encoder of the mariculture facility recognition model is used to perform feature extraction processing on the preprocessed image slice data and mask labeled data to obtain preliminary multi-scale features; The pyramid pooling module of the mariculture facility recognition model is used to perform context information fusion processing on the preliminary multi-scale features to obtain multi-scale features; The feature fusion layer of the SCSE attention mechanism of the mariculture facility recognition model is used to perform feature fusion processing on the preliminary multi-scale features and the multi-scale features to obtain fused multi-scale feature information; The mixing layer of the mariculture facility recognition model is used to perform semantic detail information balancing processing on the fused multi-scale feature information to obtain balanced multi-scale feature information; The classification head of the mariculture facility recognition model is used to perform feature classification on the balanced multi-scale feature information to obtain a satellite remote sensing image mariculture facility recognition result.

6. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 5, characterized in that, The pyramid pooling module of the mariculture facility recognition model is used to perform context information fusion processing on the preliminary multi-scale features to obtain multi-scale features, and the specific steps include: The preliminary multi-scale features are input into the pyramid pooling module of the mariculture facility recognition model; The first branch module of the pyramid pooling module is used to perform global average pooling processing on the preliminary multi-scale features to obtain first multi-scale global features; The second branch module of the pyramid pooling module is used to perform adaptive average pooling processing on the preliminary multi-scale features to obtain second multi-scale global features; The third branch module of the pyramid pooling module is used to perform adaptive average pooling processing on the preliminary multi-scale features to obtain third multi-scale global features; The fourth branch module of the pyramid pooling module is used to perform adaptive average pooling processing on the preliminary multi-scale features to obtain fourth multi-scale global features; The feature splicing layer based on the pyramid pooling module splices the first multi-scale global feature, the second multi-scale global feature, the third multi-scale global feature, the fourth multi-scale global feature, and the preliminary multi-scale feature to obtain the spliced multi-scale global feature; The convolution layer based on the pyramid pooling module performs convolution processing on the spliced multi-scale global feature to obtain the multi-scale feature.

7. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 6, characterized in that, The feature fusion layer based on the SCSE attention mechanism of the mariculture facility recognition model performs feature fusion processing on the preliminary multi-scale feature and the multi-scale feature to obtain the fused multi-scale feature information, which specifically includes: The preliminary multi-scale feature and the multi-scale feature are input into the feature fusion layer of the SCSE attention mechanism of the mariculture facility recognition model; The channel attention branch of the feature fusion layer based on the SCSE attention mechanism extracts channel attention features from the preliminary multi-scale feature and the multi-scale feature to obtain channel attention weights; The spatial attention branch of the feature fusion layer based on the SCSE attention mechanism extracts spatial attention features from the preliminary multi-scale feature and the multi-scale feature to obtain spatial attention weights; The feature fusion layer of the feature fusion layer based on the SCSE attention mechanism combines the channel attention weights and the spatial attention weights to perform feature fusion processing on the preliminary multi-scale feature and the multi-scale feature to obtain the fused multi-scale feature information.

8. The satellite remote sensing image-based mariculture identification method based on deep learning according to claim 7, characterized in that, It also includes data enhancement processing before the preprocessed image slice data and the mask annotation data are input into the mariculture facility recognition model, and the data enhancement includes geometric transformation processing, color space transformation processing, and random copy-paste. The training strategy of the mariculture facility recognition model includes AdamW optimizer, cosine annealing learning rate scheduling strategy, and mixed precision training strategy.

9. A satellite remote sensing image-based mariculture identification system based on deep learning, characterized in that, It includes the following modules: The first module is used to obtain original satellite remote sensing images and annotation data and perform data preprocessing to obtain preprocessed image slice data and mask annotation data; The second module is used to construct a mariculture facility recognition model based on the UPerNet segmentation network, select ConvNeXt as the encoder, and introduce the SCSE attention mechanism; The mariculture facility recognition model specifically includes a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on an SCSE attention mechanism, a mixing layer, and a classification head. The first output end of the ConvNeXt encoder is connected to the input end of the pyramid pooling module. The second output end of the ConvNeXt encoder is connected to the input end of the feature fusion layer based on the SCSE attention mechanism. The output end of the feature fusion layer based on the SCSE attention mechanism is connected to the input end of the mixing layer. The output end of the mixing layer is connected to the input end of the classification head. Wherein: The ConvNeXt encoder includes a first encoder layer, a second encoder layer, a third encoder layer, and a fourth encoder layer. The SCSE attention mechanism-based feature fusion layer comprises a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer; The output ends of the first encoder layer are connected with the input ends of the second encoder layer and the input ends of the fourth feature fusion layer respectively, the output ends of the second encoder layer are connected with the input ends of the third encoder layer and the input ends of the third feature fusion layer respectively, the output ends of the third encoder layer are connected with the input ends of the fourth encoder layer and the input ends of the second feature fusion layer respectively, the output ends of the fourth encoder layer are connected with the input ends of the pyramid pooling module and the input ends of the first feature fusion layer respectively, the output end of the pyramid pooling module is connected with the input end of the first feature fusion layer, the output end of the first feature fusion layer is connected with the input end of the second feature fusion layer, the output end of the second feature fusion layer is connected with the input end of the third feature fusion layer, the output end of the third feature fusion layer is connected with the input end of the fourth feature fusion layer, and the output ends of the second feature fusion layer, the third feature fusion layer and the fourth feature fusion layer are connected with the input end of the mixed layer; The third module is configured to identify the preprocessed image slice data and the mask annotation data based on the mariculture facility identification model to obtain a satellite remote sensing image mariculture facility identification result.

Images