Satellite remote sensing image mariculture identification method and system based on deep learning

Abstract

The invention discloses a satellite remote sensing image mariculture identification method and system based on deep learning, and the method comprises the steps: obtaining an original satellite remote sensing image and annotation data, and carrying out the data preprocessing, and obtaining the preprocessed image slice data and mask annotation data; based on a UPerNet segmentation network, ConvNeXt is selected as an encoder, an SCSE attention mechanism is introduced, and a mariculture facility identification model is constructed; and based on a mariculture facility identification model, identifying the preprocessed image slice data and the mask labeling data to obtain a satellite remote sensing image mariculture facility identification result. According to the invention, the boundary of the aquaculture facility can be accurately positioned, and the identification accuracy of the mariculture facility is improved. The satellite remote sensing image mariculture identification method and system based on deep learning can be widely applied to the technical field of remote sensing image processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a satellite remote sensing image mariculture recognition method and system based on deep learning. BACKGROUND

[0002] With the rapid development of mariculture industry, accurately recognizing and monitoring the distribution of mariculture facilities is of great significance for aquatic resources management, environmental protection and industry planning. Currently, the recognition and monitoring of mariculture facilities mainly rely on traditional visual interpretation methods and traditional machine learning methods.

[0003] Traditional mariculture facility recognition mainly relies on professional personnel to manually interpret satellite remote sensing images. Professional personnel manually delineate and label different types of mariculture facilities according to the color, texture, shape, size and other characteristics in the image, combined with field survey data. This method has the disadvantages of low efficiency, strong subjectivity, difficulty in processing large amounts of data, high labor cost, etc., and cannot meet the needs of large-scale and rapid monitoring. In order to improve the efficiency and accuracy of mariculture facility recognition, researchers try to use traditional machine learning methods (such as support vector machine, random forest, etc.) to classify and recognize remote sensing images. These methods usually make classification decisions based on manually designed features (such as texture features, spectral features, etc.). However, this kind of method has strong dependence on manually designed features, is difficult to handle complex sea surface background and diversified mariculture facility shapes, has weak generalization ability, and is difficult to accurately distinguish different types of mariculture facilities with similar shapes. SUMMARY

[0004] In order to solve the above technical problems, the purpose of the present application is to provide a satellite remote sensing image mariculture recognition method and system based on deep learning, which can accurately locate the boundary of mariculture facilities and improve the accuracy of mariculture facility recognition.

[0005] The first technical solution adopted by the present application is: a satellite remote sensing image mariculture recognition method based on deep learning, comprising the following steps: Obtain original satellite remote sensing images and label data and perform data preprocessing to obtain preprocessed image slice data and mask label data; Based on the UPerNet segmentation network, select ConvNeXt as the encoder, introduce the SCSE attention mechanism, and construct a mariculture facility recognition model; Based on the mariculture facility recognition model, the preprocessed image slice data and mask label data are recognized to obtain the satellite remote sensing image mariculture facility recognition result.

[0006] Further, the step of obtaining original satellite remote sensing images and label data and performing data preprocessing to obtain preprocessed image slice data and mask label data specifically includes: Obtaining original satellite remote sensing images and slicing through a sliding window strategy to obtain cut satellite remote sensing images; Filling zero values in the cut satellite remote sensing images through an edge filling strategy to obtain satellite remote sensing image slices with a standard size; Setting a slice overlap area, overlapping and slicing the satellite remote sensing image slices with a standard size through an overlapping slice strategy, calculating the proportion of effective values in each slice and performing screening to obtain image slice files; Performing data unification and integration processing on the label data to obtain integrated label data; Converting the integrated label data into vector data, obtaining the spatial position and type information of the corresponding mariculture facilities in the image slice files, and converting the vector data into a raster mask after reprojecting the vector data to the spatial position and type information of the corresponding mariculture facilities in the image slice files, to obtain preprocessed image slice data and mask label data.

[0007] Further, the mariculture facility recognition model specifically includes a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on an SCSE attention mechanism, a mixing layer, and a classification head. The first output end of the ConvNeXt encoder is connected to the input end of the pyramid pooling module, the second output end of the ConvNeXt encoder is connected to the input end of the feature fusion layer based on the SCSE attention mechanism, the output end of the feature fusion layer based on the SCSE attention mechanism is connected to the input end of the mixing layer, and the output end of the mixing layer is connected to the input end of the classification head. Wherein: The ConvNeXt encoder includes a first encoder layer, a second encoder layer, a third encoder layer, and a fourth encoder layer; The feature fusion layer based on the SCSE attention mechanism includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer, and a fourth feature fusion layer; The output end of the first encoder layer is respectively connected to the input end of the second encoder layer and the input end of the fourth feature fusion layer, the output end of the second encoder layer is respectively connected to the input end of the third encoder layer and the input end of the third feature fusion layer, the output end of the third encoder layer is respectively connected to the input end of the fourth encoder layer and the input end of the second feature fusion layer, and the output end of the fourth encoder layer is respectively connected to the input end of the pyramid pooling module and the input end of the first feature fusion layer.

[0008] Further, the pyramid pooling module comprises a first branch module, a second branch module, a third branch module, a fourth branch module, a feature splicing layer and a convolutional layer, the first branch module comprises a global pooling layer, a global average pooling layer, a first convolutional layer and a first up-sampling layer, the second branch module comprises a pooling layer, a first adaptive average pooling layer, a second convolutional layer and a second up-sampling layer, the third branch module comprises a pooling layer, a second adaptive average pooling layer, a third convolutional layer and a third up-sampling layer, the fourth branch module comprises a pooling layer, a third adaptive average pooling layer, a fourth convolutional layer and a fourth up-sampling layer.

[0009] Further, the loss function of the mariculture facility recognition model is a Tversky Loss loss function, and its expression is: ; In the above formula, denotes the number of true positive pixels, denotes the number of false negative pixels, denotes the number of false positive pixels, denotes a small constant to prevent the denominator from being zero, denotes the Tversky Loss loss function.

[0010] Further, the mariculture facility recognition model, the step of identifying the pre-processed image slice data and the mask annotation data to obtain the satellite remote sensing image mariculture facility recognition result comprises: inputting the pre-processed image slice data and the mask annotation data into the mariculture facility recognition model; based on the ConvNeXt encoder of the mariculture facility recognition model, performing feature extraction processing on the pre-processed image slice data and the mask annotation data to obtain preliminary multi-scale features; based on the pyramid pooling module of the mariculture facility recognition model, performing context information fusion processing on the preliminary multi-scale features to obtain multi-scale features; based on the feature fusion layer of the SCSE attention mechanism of the mariculture facility recognition model, performing feature fusion processing on the preliminary multi-scale features and the multi-scale features to obtain fused multi-scale feature information; based on the mixing layer of the mariculture facility recognition model, performing semantic detail information balancing processing on the fused multi-scale feature information to obtain balanced multi-scale feature information; based on the classification head of the mariculture facility recognition model, performing feature classification on the balanced multi-scale feature information to obtain the satellite remote sensing image mariculture facility recognition result.

[0011] Further, the pyramid pooling module based on the mariculture facility recognition model fuses the context information of the preliminary multi-scale features to obtain the multi-scale features, and the step specifically includes: inputting the preliminary multi-scale features into the pyramid pooling module of the mariculture facility recognition model; based on the first branch module of the pyramid pooling module, performing global average pooling processing on the preliminary multi-scale features to obtain the first multi-scale global feature; based on the second branch module of the pyramid pooling module, performing adaptive average pooling processing on the preliminary multi-scale features to obtain the second multi-scale global feature; based on the third branch module of the pyramid pooling module, performing adaptive average pooling processing on the preliminary multi-scale features to obtain the third multi-scale global feature; based on the fourth branch module of the pyramid pooling module, performing adaptive average pooling processing on the preliminary multi-scale features to obtain the fourth multi-scale global feature; based on the feature concatenation layer of the pyramid pooling module, performing feature concatenation on the first multi-scale global feature, the second multi-scale global feature, the third multi-scale global feature, the fourth multi-scale global feature, and the preliminary multi-scale features to obtain the concatenated multi-scale global feature; based on the convolution layer of the pyramid pooling module, performing convolution processing on the concatenated multi-scale global feature to obtain the multi-scale feature.

[0012] Further, the feature fusion layer of the SCSE attention mechanism based on the mariculture facility recognition model fuses the preliminary multi-scale features and the multi-scale features to obtain the fused multi-scale feature information, and the step specifically includes: inputting the preliminary multi-scale features and the multi-scale features into the feature fusion layer of the SCSE attention mechanism of the mariculture facility recognition model; based on the channel attention branch of the feature fusion layer of the SCSE attention mechanism, performing channel attention feature extraction on the preliminary multi-scale features and the multi-scale features to obtain channel attention weights; based on the spatial attention branch of the feature fusion layer of the SCSE attention mechanism, performing spatial attention feature extraction on the preliminary multi-scale features and the multi-scale features to obtain spatial attention weights; based on the feature fusion layer of the SCSE attention mechanism, combining the channel attention weights and the spatial attention weights to fuse the preliminary multi-scale features and the multi-scale features to obtain the fused multi-scale feature information.

[0013] Further, before inputting the pre-processed image slice data and mask annotation data into the mariculture facility recognition model, data enhancement processing is performed, the data enhancement includes geometric transformation processing, color space transformation processing and random copy-paste, the training strategy of the mariculture facility recognition model includes AdamW optimizer, cosine annealing learning rate scheduling strategy and mixed precision training strategy.

[0014] The second technical solution adopted by the present application is a satellite remote sensing image mariculture recognition system based on deep learning, comprising: A first module is configured to obtain original satellite remote sensing images and annotation data and perform data preprocessing to obtain pre-processed image slice data and mask annotation data. A second module is configured to select ConvNeXt as an encoder based on the UPerNet segmentation network and introduce an SCSE attention mechanism to construct a mariculture facility recognition model. A third module is configured to recognize the pre-processed image slice data and mask annotation data based on the mariculture facility recognition model to obtain a satellite remote sensing image mariculture facility recognition result.

[0015] The present application has the following advantages: the present application obtains original satellite remote sensing images and annotation data and performs data preprocessing to obtain pre-processed image slice data and mask annotation data, realizes full-process automatic processing from satellite remote sensing images to vector data output, greatly reduces the need for manual intervention, further selects ConvNeXt as an encoder based on the UPerNet segmentation network and introduces an SCSE attention mechanism to construct a mariculture facility recognition model, improves the accuracy of mariculture facility recognition based on the network architecture design of UPerNet and ConvNeXt combined with the SCSE attention mechanism, adopts advanced semantic segmentation technology to accurately locate the boundaries of mariculture facilities and improve the accuracy of subsequent area statistics and spatial analysis, and finally recognizes the pre-processed image slice data and mask annotation data based on the mariculture facility recognition model to obtain a satellite remote sensing image mariculture facility recognition result. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a step flow chart of the satellite remote sensing image mariculture recognition method based on deep learning of the present application; Figure 2 is a structural block diagram of the satellite remote sensing image mariculture recognition system based on deep learning of the present application; Figure 3 is a satellite remote sensing image mariculture recognition framework schematic diagram provided by the specific embodiment of the present application; Figure 4is a schematic diagram of a data preprocessing framework provided by an embodiment of the present application; Figure 5 is a structural schematic diagram of a mariculture facility identification model provided by an embodiment of the present application; Figure 6 is a structural schematic diagram of a pyramid pooling module provided by an embodiment of the present application; Figure 7 is a structural schematic diagram of an SCSE attention mechanism provided by an embodiment of the present application; Figure 8 is a deep learning training schematic diagram of a mariculture facility identification model provided by an embodiment of the present application; Figure 9 is an application schematic diagram of a mariculture facility identification model provided by an embodiment of the present application; Figure 10 is a recognition result schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] The present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the purpose of facilitating the description and explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0018] First of all, it needs to be pointed out that there are some deficiencies in the related art, such as: The existing mariculture facility identification technology mainly has the following problems: first, high-resolution satellite remote sensing images usually have a size of tens of thousands of pixels, and the data volume is large, which cannot be directly input into the identification model for processing, and the existing technology lacks effective processing strategies for edge regions; second, there are many types of mariculture facilities, and there is similarity between different types in the form of remote sensing images, which increases the difficulty of identification; third, the remote sensing images of mariculture areas are easily affected by environmental factors such as clouds, water quality, and lighting conditions, and the existing technology has limited adaptability to these changing factors; fourth, the boundaries of mariculture facilities in remote sensing images are often not clear enough, and the existing technology cannot accurately locate the boundaries of mariculture facilities; finally, the existing technology has insufficient automation in the complete workflow from remote sensing image identification to the final production of vector data that can be used for GIS analysis, and often requires manual intervention.

[0019] Based on this, with reference to Figure 1 and Figure 3 , the present application provides a satellite remote sensing image mariculture identification method based on deep learning, which comprises the following steps: S100, obtaining original satellite remote sensing images and annotation data and performing data preprocessing to obtain preprocessed image slice data and mask annotation data; S110, obtaining an original satellite remote sensing image and slicing it using a sliding window strategy to obtain a sliced ​​satellite remote sensing image; S120, filling the sliced ​​satellite remote sensing image with zero values ​​using an edge filling strategy to obtain a satellite remote sensing image slice with a standard size; S130, setting a slice overlap area, performing overlapping slicing processing on satellite remote sensing image slices of standard size using an overlapping slicing strategy, then calculating the ratio of valid values ​​in each slice and screening the slices to obtain an image slice file; In this embodiment, if Figure 4 As shown, for large-scale satellite remote sensing image processing, an embodiment of the present invention implements a slicing algorithm that can effectively process remote sensing images of any size. The specific implementation method is to use a sliding window strategy to cut large-scale remote sensing images into small blocks of fixed size. In order to solve the problem of information loss in edge areas, the present invention adopts an edge filling strategy. For areas where the image edge is not sufficient to form a complete slice, standard-sized slices are generated by filling zero values. At the same time, in order to solve the problem of discontinuity of slice boundaries, an embodiment of the present invention introduces an overlapping slicing strategy. By setting the slice overlapping area, the information continuity between adjacent slices is ensured. In addition, the present invention also implements a slice screening mechanism based on valid values. By calculating the proportion of valid values ​​in each slice, slices containing sufficient valid information are screened out, thereby improving the quality of training data.

[0020] S140, unifying and integrating the labeled data to obtain integrated labeled data; In this embodiment, the present invention implements a method for data unification and integration, uniformly processing and formatting data from different regions and time periods. This implementation involves first merging all annotated files into a unified table, including feature IDs, feature geometry, and metadata about the original image (such as satellite product type, sensor type, imaging time, coordinate reference system, and original image extent), ensuring uniform and traceable annotated data. Finally, the data is saved in GeoJSON format. All images are then converted to an unsigned 8-bit integer GeoTIFF format, with invalid data padded with zeros. The original invalid region mask is stored as a Boolean type.

[0021] S150. Convert the integrated annotation data into vector data, obtain the spatial position and type information of the corresponding marine aquaculture facilities in the image slice file, reproject the vector data to the spatial position and type information of the corresponding marine aquaculture facilities in the image slice file, and then convert it into a raster mask to obtain the preprocessed image slice data and mask annotation data.

[0022] In this embodiment, for rasterization of vector data, an embodiment of the present invention implements a method for converting vector annotation data in shapefile and geojson formats into pixel-level masks. In the specific implementation process, the image slice file and the integrated annotation file are first read to obtain the spatial location and type information of the marine aquaculture facilities corresponding to the image slice; the vector data is then reprojected to the coordinate system of the satellite remote sensing image; the rasterization function of the rasterio library in Python is then used to convert the vector data into a raster mask, where different types of marine aquaculture facilities correspond to different pixel values ​​(e.g., raft aquaculture corresponds to pixel value 1, deep-water cages correspond to pixel value 2, ordinary cages correspond to pixel value 3, and background corresponds to pixel value 0); finally, the generated raster mask is saved as a mask file corresponding to the original image slice.

[0023] In summary, data preprocessing can convert the original large-scale satellite remote sensing images and vector annotation data into standard format data suitable for deep learning model training, providing high-quality training data for subsequent model training. The present invention uses high-resolution satellite remote sensing true color images with a resolution greater than 1m as training data. In the coastal waters, concentrated areas of marine aquaculture are found for image acquisition, and the outline of marine aquaculture facilities is manually outlined using polygon tools in the GIS platform. Different types of marine aquaculture are assigned corresponding values, and the annotation results are saved as shapefile files. The annotation results correspond one-to-one with the images. In order to ensure the effect of subsequent model training, the training sample set in the embodiment of the present invention covers marine aquaculture targets of different phases, types, and characteristics, covers a variety of sea surface backgrounds and land feature distributions, and contains a sufficient number of training samples to ensure that the model training will not have overfitting problems.

[0024] S200, based on the UPerNet segmentation network, select ConvNeXt as the encoder, introduce the SCSE attention mechanism, and build a marine aquaculture facility recognition model; Specifically, if Figure 5As shown, the marine aquaculture facility identification model specifically includes a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on the SCSE attention mechanism, a mixing layer and a classification head. The first output end of the ConvNeXt encoder is connected to the input end of the pyramid pooling module, the second output end of the ConvNeXt encoder is connected to the input end of the feature fusion layer based on the SCSE attention mechanism, the output end of the feature fusion layer based on the SCSE attention mechanism is connected to the input end of the mixing layer, and the output end of the mixing layer is connected to the input end of the classification head.

[0025] Among them, the ConvNeXt encoder includes a first encoder layer, a second encoder layer, a third encoder layer and a fourth encoder layer; the feature fusion layer based on the SCSE attention mechanism includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer; the output end of the first encoder layer is respectively connected to the input end of the second encoder layer and the input end of the fourth feature fusion layer, the output end of the second encoder layer is respectively connected to the input end of the third encoder layer and the input end of the third feature fusion layer, the output end of the third encoder layer is respectively connected to the input end of the fourth encoder layer and the input end of the second feature fusion layer, and the output end of the fourth encoder layer is respectively connected to the input end of the pyramid pooling module and the input end of the first feature fusion layer.

[0026] like Figure 6 As shown, the pyramid pooling module includes a first branch module, a second branch module, a third branch module, a fourth branch module, a feature splicing layer and a convolution layer. The first branch module includes a global pooling layer, a global average pooling layer, a first convolution layer and a first upsampling layer. The second branch module includes Pooling layer, first adaptive average pooling layer, second convolution layer and second upsampling layer, the third branch module includes Pooling layer, second adaptive average pooling layer, third convolution layer and third upsampling layer, the fourth branch module includes Pooling layer, third adaptive average pooling layer, fourth convolutional layer and fourth upsampling layer.

[0027] The embodiment of the present application adopts a UPerNet segmentation network structure, which can effectively utilize feature information of different scales and is suitable for processing semantic segmentation tasks in complex scenes. As the backbone network, the present application selects ConvNeXt as the encoder, which has both the inductive bias of CNN and the modeling ability of Transformer, and can extract rich features in satellite remote sensing images. In order to further improve the model's ability to distinguish different types of breeding facilities, the present application introduces the SCSE (Spatial and Channel Squeeze & Excitation) attention mechanism, as shown in Figure 7 which can enhance important features and suppress unimportant features in the spatial and channel dimensions, thereby improving the segmentation accuracy of the model.

[0028] The UPerNet network architecture consists of an encoder and a decoder. The encoder part uses the ConvNeXt network, which contains four stages, each stage contains multiple ConvNeXt blocks, and each block contains depth convolution, pointwise convolution and self-attention mechanism. The decoder part adopts a feature pyramid structure to improve the segmentation accuracy through multi-scale feature fusion. In the feature fusion process, the SCSE attention module is introduced, which contains a channel attention branch and a spatial attention branch, which respectively weight the channel dimension and the spatial dimension of the feature map, highlighting important features.

[0029] The final output of the model is a recognition confidence score map, which includes three dimensions of class, width and height. The width and height are the same as the input image, and the class dimension represents the confidence score output by the model for different categories. The confidence score is between 0 and 1, representing the probability that the model considers the pixel to belong to the category.

[0030] S300, based on the marine aquaculture facility recognition model, the preprocessed image slice data and the mask label data are recognized to obtain the satellite remote sensing image marine aquaculture facility recognition result.

[0031] Specifically, the pre-processed image slice data and the mask annotation data are input into the mariculture facility recognition model; based on the ConvNeXt encoder of the mariculture facility recognition model, the pre-processed image slice data and the mask annotation data are subjected to feature extraction processing to obtain preliminary multi-scale features; based on the pyramid pooling module of the mariculture facility recognition model, the preliminary multi-scale features are subjected to context information fusion processing to obtain multi-scale features; based on the feature fusion layer of the SCSE attention mechanism of the mariculture facility recognition model, the preliminary multi-scale features and the multi-scale features are subjected to feature fusion processing to obtain fused multi-scale feature information; based on the mixing layer of the mariculture facility recognition model, the fused multi-scale feature information is subjected to semantic detail information balancing processing to obtain balanced multi-scale feature information; based on the classification head of the mariculture facility recognition model, the balanced multi-scale feature information is subjected to feature classification to obtain a satellite remote sensing image mariculture facility recognition result.

[0032] In the pyramid pooling module, the preliminary multi-scale features are input into the pyramid pooling module of the mariculture facility recognition model; based on the first branch module of the pyramid pooling module, the preliminary multi-scale features are subjected to global average pooling processing to obtain first multi-scale global features; based on the second branch module of the pyramid pooling module, the preliminary multi-scale features are subjected to adaptive average pooling processing to obtain second multi-scale global features; based on the third branch module of the pyramid pooling module, the preliminary multi-scale features are subjected to adaptive average pooling processing to obtain third multi-scale global features; based on the fourth branch module of the pyramid pooling module, the preliminary multi-scale features are subjected to adaptive average pooling processing to obtain fourth multi-scale global features; based on the feature concatenation layer of the pyramid pooling module, the first multi-scale global features, the second multi-scale global features, the third multi-scale global features, the fourth multi-scale global features, and the preliminary multi-scale features are subjected to feature concatenation to obtain concatenated multi-scale global features; based on the convolution layer of the pyramid pooling module, the concatenated multi-scale global features are subjected to convolution processing to obtain the multi-scale features.

[0033] For the feature fusion layer of the SCSE attention mechanism, the preliminary multi-scale feature and the multi-scale feature are input to the feature fusion layer of the SCSE attention mechanism of the mariculture facility identification model; the channel attention branch of the feature fusion layer based on the SCSE attention mechanism performs channel attention feature extraction on the preliminary multi-scale feature and the multi-scale feature to obtain channel attention weight; the spatial attention branch of the feature fusion layer based on the SCSE attention mechanism performs spatial attention feature extraction on the preliminary multi-scale feature and the multi-scale feature to obtain spatial attention weight; and the feature fusion layer of the feature fusion layer based on the SCSE attention mechanism combines the channel attention weight and the spatial attention weight to perform feature fusion processing on the preliminary multi-scale feature and the multi-scale feature to obtain fused multi-scale feature information.

[0034] Regarding the data augmentation strategy, in order to enhance the generalization ability and adaptability to environmental changes of the model, the present application implements a rich data augmentation method. The basic geometric transformation includes random horizontal flipping (probability 0.5), random vertical flipping (probability 0.5) and random 90-degree rotation (probability 0.5). Color space transformation includes brightness adjustment, contrast adjustment, saturation adjustment and hue adjustment.

[0035] In addition, the present application also introduces a random copy-paste technique, which is a special data augmentation method for class imbalance problems. In the specific implementation process, first, an image area containing the target class (especially the class with fewer samples, such as deep water net cage) is randomly selected from the training batch; then a target position (which can be other positions in the same image or other images in the batch) is randomly selected; then the selected area is copied and pasted to the target position while keeping the corresponding mask annotation consistent; finally, appropriate smoothing transition processing is applied to ensure that the inserted area is naturally integrated with the surrounding environment. This technique can effectively increase the number and diversity of small class samples, improve the model's recognition ability for small classes, and also enhance the model's adaptability to the appearance of targets in different backgrounds.

[0036] The specific implementation of data augmentation is based on the kornia library in python, which provides efficient GPU-accelerated image enhancement functions. In the training process, data augmentation operations are applied after each batch of data is loaded, ensuring that the model sees some changes in data during each training cycle, thereby enhancing the model's generalization ability.

[0037] Regarding the design of the loss function, in order to solve the problem of class imbalance, the present application adopts the Tversky Loss loss function. Tversky Loss is a generalization of Dice Loss, by adjusting the alpha and beta parameters, the weights of false positives and false negatives can be balanced, which is suitable for processing class imbalance segmentation tasks. In the present application, alpha is set to 0.3 and beta is set to 0.7, so that the penalty for false negatives (missed detection) is greater than the penalty for false positives (false detection), which helps to improve the recall rate of the target.

[0038] The calculation formula of Tversky Loss is as follows: ; Wherein, TP represents the number of true positive pixels, FN represents the number of false negative pixels, FP represents the number of false positive pixels, is a small constant to prevent the denominator from being zero.

[0039] In addition, the present application also realizes multi-class Tversky Loss, which is used to process multi-class segmentation tasks. Multi-class Tversky Loss calculates the Tversky Loss of each class, and then takes the average value as the final loss function.

[0040] For the model training process, as shown in Figure 8 , first, use the pre-trained ConvNeXt model to initialize the encoder, use transfer learning to speed up the training process; then use the AdamW optimizer with a low learning rate (initial learning rate set to 0.0001) for training, while using the cosine annealing learning rate scheduling strategy to gradually reduce the learning rate as the training process progresses. During the training process, the batch size is 16, and the mixed precision training technique is used to speed up the training process. To prevent overfitting, the present application also introduces weight decay (set to 0.01) and early stopping strategy, when the validation set loss does not improve for 5 consecutive periods, stop training and restore the best model weight.

[0041] Regarding cross-validation and model integration, in order to improve the robustness and generalization ability of the model, the present application adopts the k-fold cross-validation strategy. In the specific implementation process (taking k=4 as an example), first, randomly divide the training data set into 4 non-overlapping subsets; then perform 4 training, each time select 3 subsets as the training set, and the remaining 1 subset as the validation set; finally, integrate the recognition results of the 4 models to obtain the final recognition result.

[0042] The model ensemble adopts a soft voting manner, that is, the prediction probabilities of each model are averaged, and then the class with the highest probability is selected as the final recognition result. This manner can fully utilize the complementarity of multiple models, and improve the prediction accuracy and robustness.

[0043] Regarding model export and deployment, in order to facilitate subsequent model deployment and application, the present application converts the trained model into TorchScript format. TorchScript is an intermediate representation format that can run independently of the Python environment and is suitable for deployment in production environments. In the specific implementation process, first, the trained model weight is loaded; then the jit.trace function in the torch library of python is used to convert the model into TorchScript format; finally, the converted model file is saved.

[0044] Through the above design, the deep learning model training module can construct and train a high-performance mariculture facility recognition model, providing strong support for subsequent inference and application.

[0045] The inference and application module is responsible for mariculture facility recognition on new satellite remote sensing images and outputs vector results. The processing flow of this module is shown in Figure 9 , mainly including three main parts of sliding window prediction, recognition result splicing and merging, and raster to vector conversion.

[0046] Regarding sliding window prediction, in order to process satellite remote sensing images of any size, the present application implements an inference strategy based on sliding window. In the specific implementation process, first, the satellite remote sensing image to be processed is loaded; then image preprocessing is performed, including image padding (to ensure that the image can be completely divided); then the sliding window method is used to divide the large size image into multiple small blocks (such as 512x512 pixels), with a certain overlap (such as 256 pixels) between adjacent windows; then each small block is predicted; finally, the recognition results are spliced back to the original image size.

[0047] In the image padding process, the present application adopts a zero padding strategy, that is, for the areas on the edge of the image that are not enough to form a complete window, a standard size window is generated by padding zero values.

[0048] Regarding the splicing and merging of recognition results, the present invention implements a confidence-based fusion strategy to address the overlapping areas caused by sliding windows. In this implementation, a recognition confidence score map is first created, the same size as the original image and with an initial value of 0. The recognition results for each window are then accumulated to the corresponding position in the recognition confidence score map. Finally, the category with the highest confidence score for each position is selected to obtain the final recognition result. This approach fully utilizes information from overlapping areas, improving prediction accuracy and continuity. In particular, for areas at the edges of windows, the fusion of recognition results from multiple windows can reduce the impact of edge effects and improve prediction quality.

[0049] Regarding raster-to-vector conversion, this paper implements a method for converting pixel-level classification results into vector data for ease of application in GIS systems. The implementation involves first reading the classification result map; then extracting the outlines of the corresponding areas for each category; then converting the extracted outlines into polygons in a geographic coordinate system; and finally saving the polygon data as a vector file in GeoJSON or Shapefile format.

[0050] During contour extraction, the present invention uses the features.shapes function in Python's rasterio library, which can efficiently extract vector features from raster data. During coordinate conversion, the present invention uses the geographic transformation information of the original image to ensure that the extracted polygons have the correct geographic coordinates.

[0051] Through the above design, the inference and application module can efficiently and accurately identify marine aquaculture facilities in satellite remote sensing images of any size and output vector results that are convenient for GIS application, realizing the full process automation from remote sensing imagery to vector data output.

[0052] Therefore, the embodiments of the present invention have the following differences compared with the prior art: 1) Overall architecture design of a deep learning-based marine aquaculture facility identification system, including data preprocessing, deep learning model training, inference, and application.

[0053] 2) A slicing method for large-scale satellite remote sensing images, which uses a sliding window slicing algorithm, an edge filling strategy, and an overlapping area processing method to solve the problem of edge information loss.

[0054] 3) A network architecture design based on UPerNet and ConvNeXt, combined with the SCSE attention mechanism, improves the accuracy of marine aquaculture facility recognition.

[0055] 4) A data augmentation strategy for the imbalance of mariculture facility types, especially the random copy-paste technique, to increase the diversity of small category samples.

[0056] 5) A loss function design based on Tversky Loss, which balances false positives and false negatives by adjusting the alpha and beta parameters in Tversky Loss, to improve recall.

[0057] 6) A large image inference strategy based on sliding window, combined with confidence fusion method, to realize efficient processing of satellite remote sensing images of any size.

[0058] 7) An automatic conversion method from raster classification results to vector data, realizing the whole process of automatic processing from remote sensing images to GIS available data.

[0059] Compared with the prior art, the embodiments of the present application have the following advantages: 1) High recognition accuracy: Compared with traditional methods, the system of the present application has an overall accuracy of more than 87% in the task of mariculture facility recognition, especially in distinguishing different types of mariculture facilities with similar shapes.

[0060] 2) High processing efficiency: The system of the present application can process large satellite remote sensing images at a speed of seconds, meeting the needs of large-scale and rapid monitoring.

[0061] 3) High automation: The present application realizes the whole process of automatic processing from satellite remote sensing images to vector data output, greatly reducing the need for manual intervention.

[0062] 4) Strong adaptability: Through rich data augmentation strategies and advanced network architecture design, the system of the present application has good adaptability to satellite remote sensing images of different time phases and different regions.

[0063] 5) Accurate boundary: The present application uses advanced semantic segmentation technology to accurately locate the boundary of mariculture facilities, improving the accuracy of subsequent area statistics and spatial analysis.

[0064] 6) Easy deployment: By converting the model to TorchScript format, the system of the present application is easy to deploy and apply.

[0065] Finally, the simulation experiment of the embodiments of the present application is described in combination with the drawings: In this embodiment, a certain province of China is selected as an example area. High-resolution satellite remote sensing images of a certain coastal aquaculture area in a certain province are collected, with an image resolution of 1m, covering various types of mariculture facilities, including raft culture, ordinary net cages and deep water net cages.

[0066] Firstly, the training images are loaded into ArcGIS software, and the polygon tool is used to classify and label the mariculture targets in the images, and the labeling results are saved as shapefile files. Through the data preprocessing module, the original satellite remote sensing images are sliced to generate 512x512 pixel image blocks and corresponding label masks. A total of 1649 images are generated, with 13897 labeled elements. The training, validation, and test set ratio is set as 7:2:1. Through the deep learning model training module, the mariculture facility recognition model is constructed and trained. During the training process, the network architecture, data augmentation strategy, and loss function design described above are used to improve the model performance through cross-validation and model integration. Finally, the satellite remote sensing images of the test area are identified by the inference and application module, and the results are converted to geojson format and saved.

[0067] Figure 10 The recognition results of the system of the application are shown. It can be seen that the system of the application can more accurately identify different types of mariculture facilities, especially in distinguishing similar mariculture facilities. The quantitative evaluation results shown in Table 1 show that the overall f1 score of the system of the application on the test data reaches 87.55%. In terms of processing efficiency, the system of the application only takes 23 seconds to process a satellite remote sensing image with a size of 10000x10000 pixels.

[0068] Table 1 Recognition evaluation result data table ; In summary, the satellite remote sensing image mariculture automatic recognition system and method based on deep learning provided by the application can efficiently and accurately identify different types of mariculture facilities in satellite remote sensing images, realize the full-process automatic processing from remote sensing images to vector data output, and provide important technical support for mariculture supervision, environmental impact assessment, industry planning and other fields.

[0069] Reference Figure 2 The satellite remote sensing image mariculture recognition system based on deep learning comprises: The first module 201 is used for acquiring original satellite remote sensing images and label data and performing data preprocessing to obtain preprocessed image slice data and mask label data; The second module 202 is used for selecting ConvNeXt as an encoder based on the UPerNet segmentation network, introducing the SCSE attention mechanism, and constructing a mariculture facility recognition model; The third module 203 is used for identifying the preprocessed image slice data and mask label data based on the mariculture facility recognition model to obtain the satellite remote sensing image mariculture facility recognition result.

[0070] The content in the method embodiments is applicable to the system embodiments. The system embodiments specifically implement the same functions as the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0071] The above is a specific description of the preferred embodiments of the application. However, the application is not limited to the embodiments described above. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the application. These equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A method for identifying marine aquaculture using satellite remote sensing images based on deep learning, characterized in that: The following steps are involved: Obtain original satellite remote sensing images and annotation data and perform data preprocessing to obtain preprocessed image slice data and mask annotation data; Based on the UPerNet segmentation network, ConvNeXt was selected as the encoder, and the SCSE attention mechanism was introduced to build a marine aquaculture facility recognition model. Based on the marine aquaculture facility recognition model, the pre-processed image slice data and mask annotation data are identified to obtain the marine aquaculture facility recognition results of satellite remote sensing images.

2. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 1, characterized in that: The step of obtaining the original satellite remote sensing image and the annotation data and performing data preprocessing to obtain the preprocessed image slice data and the mask annotation data specifically includes: Obtain the original satellite remote sensing image and slice it using a sliding window strategy to obtain the sliced ​​satellite remote sensing image; The cut satellite remote sensing image is filled with zero values ​​by using the edge filling strategy to obtain satellite remote sensing image slices with standard size; Set the slice overlap area, perform overlapping slicing on satellite remote sensing image slices of standard size using the overlapping slicing strategy, calculate the ratio of valid values ​​in each slice and filter them to obtain the image slice file; Perform data unification and integration processing on the labeled data to obtain integrated labeled data; The integrated annotation data is converted into vector data to obtain the spatial position and type information of the corresponding marine aquaculture facilities in the image slice file. The vector data is reprojected to the spatial position and type information of the corresponding marine aquaculture facilities in the image slice file and then converted into a raster mask to obtain the preprocessed image slice data and mask annotation data.

3. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 2, characterized in that: The marine aquaculture facility identification model specifically includes a ConvNeXt encoder, a pyramid pooling module, a feature fusion layer based on the SCSE attention mechanism, a mixing layer and a classification head. The first output end of the ConvNeXt encoder is connected to the input end of the pyramid pooling module, the second output end of the ConvNeXt encoder is connected to the input end of the feature fusion layer based on the SCSE attention mechanism, the output end of the feature fusion layer based on the SCSE attention mechanism is connected to the input end of the mixing layer, and the output end of the mixing layer is connected to the input end of the classification head, wherein: The ConvNeXt encoder includes a first encoder layer, a second encoder layer, a third encoder layer and a fourth encoder layer; The feature fusion layer based on the SCSE attention mechanism includes a first feature fusion layer, a second feature fusion layer, a third feature fusion layer and a fourth feature fusion layer; The output end of the first encoder layer is connected to the input end of the second encoder layer and the input end of the fourth feature fusion layer, respectively. The output end of the second encoder layer is connected to the input end of the third encoder layer and the input end of the third feature fusion layer, respectively. The output end of the third encoder layer is connected to the input end of the fourth encoder layer and the input end of the second feature fusion layer, and the output end of the fourth encoder layer is connected to the input end of the pyramid pooling module and the input end of the first feature fusion layer.

4. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 3, characterized in that: The pyramid pooling module includes a first branch module, a second branch module, a third branch module, a fourth branch module, a feature splicing layer and a convolution layer. The first branch module includes a global pooling layer, a global average pooling layer, a first convolution layer and a first upsampling layer. The second branch module includes Pooling layer, first adaptive average pooling layer, second convolution layer and second upsampling layer, the third branch module includes Pooling layer, second adaptive average pooling layer, third convolution layer and third upsampling layer, the fourth branch module includes Pooling layer, third adaptive average pooling layer, fourth convolutional layer and fourth upsampling layer.

5. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 4, characterized in that: The loss function of the marine aquaculture facility identification model is the Tversky Loss loss function, which is expressed as follows: In the above formula, represents the number of true positive pixels, represents the number of false negative pixels, represents the number of false positive pixels, represents a small constant to prevent the denominator from being zero, Represents the Tversky Loss loss function.

6. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 5, characterized in that: The step of identifying the pre-processed image slice data and the mask annotation data based on the seawater aquaculture facility recognition model to obtain the satellite remote sensing image seawater aquaculture facility recognition result specifically includes: Inputting the pre-processed image slice data and mask annotation data into the marine aquaculture facility recognition model; Based on the ConvNeXt encoder of the marine aquaculture facility recognition model, feature extraction is performed on the pre-processed image slice data and mask annotation data to obtain preliminary multi-scale features; Based on the pyramid pooling module of the marine aquaculture facility recognition model, the preliminary multi-scale features are fused with contextual information to obtain multi-scale features; The feature fusion layer of the SCSE attention mechanism based on the marine aquaculture facility recognition model performs feature fusion processing on the preliminary multi-scale features and multi-scale features to obtain the fused multi-scale feature information; Based on the hybrid layer of the marine aquaculture facility recognition model, the semantic detail information is balanced on the fused multi-scale feature information to obtain the balanced multi-scale feature information. Based on the classification head of the marine aquaculture facility recognition model, the balanced multi-scale feature information is classified to obtain the marine aquaculture facility recognition results from satellite remote sensing images.

7. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 6, characterized in that: The pyramid pooling module based on the marine aquaculture facility recognition model performs context information fusion processing on the preliminary multi-scale features to obtain the multi-scale features, which specifically includes: The preliminary multi-scale features are input into the pyramid pooling module of the marine aquaculture facility recognition model; Based on the first branch module of the pyramid pooling module, global average pooling is performed on the preliminary multi-scale features to obtain the first multi-scale global features; The second branch module based on the pyramid pooling module performs adaptive average pooling on the preliminary multi-scale features to obtain the second multi-scale global features; The third branch module based on the pyramid pooling module performs adaptive average pooling on the preliminary multi-scale features to obtain the third multi-scale global features; The fourth branch module based on the pyramid pooling module performs adaptive average pooling on the preliminary multi-scale features to obtain the fourth multi-scale global features; Based on the feature splicing layer of the pyramid pooling module, the first multi-scale global feature, the second multi-scale global feature, the third multi-scale global feature, the fourth multi-scale global feature and the preliminary multi-scale feature are spliced ​​to obtain the spliced ​​multi-scale global feature; Based on the convolution layer of the pyramid pooling module, the spliced ​​multi-scale global features are convolved to obtain multi-scale features.

8. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 7, characterized in that: The feature fusion layer of the SCSE attention mechanism based on the marine aquaculture facility recognition model performs feature fusion processing on the preliminary multi-scale features and the multi-scale features to obtain the fused multi-scale feature information, which specifically includes: The preliminary multi-scale features and multi-scale features are input into the feature fusion layer of the SCSE attention mechanism of the marine aquaculture facility recognition model; Based on the channel attention branch of the feature fusion layer of the SCSE attention mechanism, channel attention features are extracted from the preliminary multi-scale features and multi-scale features to obtain the channel attention weights; Based on the spatial attention branch of the feature fusion layer of the SCSE attention mechanism, spatial attention features are extracted from the preliminary multi-scale features and multi-scale features to obtain the spatial attention weights; The feature fusion layer based on the SCSE attention mechanism combines the channel attention weights and the spatial attention weights to fuse the preliminary multi-scale features with the multi-scale features to obtain the fused multi-scale feature information.

9. The method for identifying marine aquaculture using satellite remote sensing images based on deep learning according to claim 8, characterized in that: It also includes performing data enhancement processing before inputting the preprocessed image slice data and mask annotation data into the marine aquaculture facility recognition model. The data enhancement includes geometric transformation processing, color space transformation processing and random copy-paste. The training strategy of the marine aquaculture facility recognition model includes AdamW optimizer, cosine annealing learning rate scheduling strategy and mixed precision training strategy.

10. A deep learning-based satellite remote sensing image marine aquaculture identification system, characterized by: Includes the following modules: The first module is used to obtain original satellite remote sensing images and annotation data and perform data preprocessing to obtain preprocessed image slice data and mask annotation data; The second module is used to construct a marine aquaculture facility recognition model based on the UPerNet segmentation network, select ConvNeXt as the encoder, introduce the SCSE attention mechanism; The third module is used to identify the pre-processed image slice data and mask annotation data based on the seawater aquaculture facility recognition model to obtain the seawater aquaculture facility recognition results of the satellite remote sensing image.

Images