An improved swin-transformer-based coastline automatic extraction method

Abstract

The application discloses a kind of based on improved Swin-Transformer's coastline automatic extraction method, the method includes: collecting four kinds of coastline data such as bedrock coast, sandy coast, silt coast and biological coast, constructs data set, and is divided into training set, test set;Establish improved Swin Transformer network model, optimize original Swin Transformer network model;Pre-training improved Swin Transforme network model, data enhancement each kind of training set coastline data, form expansion training set;Improved Swin Transformer network model is trained using expansion training set, obtains the coastline detection model after training;Using the detection model after training to the coastline picture to be detected is detected.The method is used to solve the problem that existing data set is less, and the imbalance between the number of different categories of coastline samples exists, and the difficulty of model training is influenced, and each type of coastline such as bedrock coast, sandy coast, silt coast and biological coast is extracted quickly and effectively.

Description

Technical Field

[0001] This invention provides an automatic coastline extraction method, belonging to the fields of deep learning and computer vision technology, and particularly relates to an automatic coastline extraction method based on an improved Swin-Transformer. Background Technology

[0002] A coastline is a dynamic boundary between the ocean and the land. There are many types of coastlines, including bedrock coasts, sandy coasts, silty coasts, and biogenic coasts. Bedrock coasts are composed of hard rocks, sandy coasts are mainly composed of gravel and sand, silty coasts are fertile, and biogenic coasts mainly include mangrove coasts and coral reef coasts.

[0003] Traditional methods for coastline extraction mainly involve field surveys and photogrammetry, but these methods are labor-intensive and time-consuming, lacking advantages in terms of timeliness, dynamism, and convenience in coastline monitoring. With the development of remote sensing technology and the abundance of image data, coastline extraction research has evolved from primitive visual interpretation methods to computer-automated interpretation methods. Previous researchers have actively studied and explored various methods for coastline extraction based on remote sensing images, broadly categorized as threshold-based methods, methods using edge operators to extract the edges of land and sea areas, and object-oriented segmentation methods. However, these methods are susceptible to subjective biases and factors such as noise and complex backgrounds. If traditional single-feature segmentation is used, the segmentation results are not ideal, leading to unclear land-sea boundaries and hindering high-resolution remote sensing image analysis.

[0004] With the rise of big data and the improvement of computing resources, deep learning has been widely applied in the field of computer vision due to its powerful feature extraction capabilities and fitting ability for complex problems. Research on automatic coastline extraction using deep learning is also ongoing. In optical remote sensing images, the spectral contrast between water bodies and vegetation is not obvious, and the constraint of "different objects with the same spectrum" leads to a large number of misclassified pixels. The influence of atmospheric factors such as clouds, sea surface echoes, and wind makes the instantaneous state of land and sea differ from the remote sensing image, accompanied by complex noise interference, resulting in discontinuous boundary phenomena. When existing deep learning network models perform land-sea segmentation on high-resolution remote sensing images rich in spatial information, publicly available datasets are scarce and difficult to obtain, and misclassification problems exist. Existing coastline extraction methods often cannot quickly and effectively extract various types of coastlines, and it is difficult to guarantee the accuracy of the coastline extraction results, resulting in poor performance. Therefore, coastline detection methods based on deep learning require further research. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this application provides an automatic coastline extraction method based on an improved Swing-Transformer. This method solves the problem that existing datasets are limited and the imbalance between the number of coastline samples of different categories affects model training. It can quickly and effectively extract various types of coastlines, such as bedrock coastlines, sandy coastlines, silty coastlines, and biogenic coastlines.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: an automatic coastline extraction method based on an improved Swin-Transformer, characterized by comprising the following steps:

[0007] S1. Collect monitoring video data of four types of coastlines: bedrock coast, sandy coast, silty coast, and biological coast, and preprocess the data to construct a dataset.

[0008] S2. Optimize the original Swing Transformer network model to obtain the improved Swing Transformer network model.

[0009] S3. Pre-training improves the Swin Transformer network model, and data augmentation is performed on various training set coastline data to form an expanded training set;

[0010] S4. The improved Swin Transformer network model is trained using an expanded training set to obtain the coastline detection model;

[0011] S5. Deploy the trained model, input the image to be detected, and detect the coastline.

[0012] Preferably, the preprocessing in step S1 includes, but is not limited to, rotation, cropping, and adjusting image brightness and contrast to improve the quality of the initial dataset; the constructed dataset should be constructed by manually labeling each image according to a pre-set labeling tool and dividing the labeled images into training set and test set according to proportion.

[0013] Preferably, the improved Swing Transformer network model includes the following steps:

[0014] a. The backbone network is changed to Swin-S with moderate structural complexity, and the feature downsampling is performed in four stages using Swin Transformer blocks. The first stage uses 2 Swin Transformer blocks, the second stage uses 2 Swin Transformer blocks, the third stage uses 18 Swin Transformer blocks, and the fourth stage uses 2 Swin Transformer blocks.

[0015] b. The neck network replaces global average pooling with a multilayer perceptron consisting of two fully connected FC layers with ReLU activation function;

[0016] c. The dimension of the output features of the fully connected (FC) layer in the head network is converted to the embedding dimension, mapping the input features to a higher dimensional space, and then more non-linear features are obtained by attaching the Swish activation function.

[0017] d. The hidden layer features are then mapped to the output categories through a fully connected (FC) layer.

[0018] Preferably, the pre-trained improved Swing Transformer network model includes the following steps:

[0019] S31. Train the improved Swin Transformer network model without parameter initialization on the training set and calculate the F1-score for each class;

[0020] S32. Using an F1-score-sensitive sampling number, calculate the number of samples required for each class based on the initial recognition accuracy results;

[0021] S33. Adaptive Simple Cope-Paste method is used to augment the training sample data of each type of coastline to form an expanded training set.

[0022] Preferably, in step S31, the F1-score is calculated according to the formula and then assigned equal weights to recall and precision, where recall represents the proportion of correctly identified positive samples to actual positive samples, and precision represents the proportion of correctly identified positive samples to predicted positive samples.

[0023] Preferably, in S32, each class F1-score is defined as class i, and the number of samples required for class i is the ratio of the error rate of class i to the sum of the error rates of all classes multiplied by the total number of samples in the dataset.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] This invention collects images and video frames from monitoring videos of bedrock coasts, sandy coasts, silty coasts, and biogenic coasts, constructs a dataset, and enables rapid and effective detection of various types of coastlines, including bedrock coasts, sandy coasts, silty coasts, and biogenic coasts. It solves the problem of imbalance in the number of samples of different types of coastlines and improves the recognition effect of each class while ensuring the overall recognition accuracy.

[0026] Compared to traditional convolutional neural networks (CNNs), the Swing Transformer structure employed in this invention can capture global dependencies in images at different scales, fusing global image features and local detail features to better understand and process image content. This invention replaces the global average pooling layer in the neck network with a two-layer fully connected layer plus a ReLU activation function, thus protecting transfer learning. Adding fully connected layers and a Swing activation function to the head network allows for the combination of more non-linear features.

[0027] In the data augmentation stage of this invention, the test set data is only used to calculate the F1-score in the sampling quantity formula, without any data augmentation. The model's recognition performance for each class is evaluated based on the F1-score, thereby adjusting the data distribution of the training samples and indirectly affecting the model training process. This ensures that the distribution of the test set data is consistent with the distribution of the real data, thus better evaluating the model's generalization ability.

[0028] The Simple Copy-Paste method used in this invention is similar to MixUp and CutMix. This method can ensure that no incorrect labels are generated while increasing the number of samples, thereby effectively improving the generalization performance of the model and enabling the model to adapt to the complex environment in real-world scenarios.

[0029] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from an examination of the following, or may be learned from the practice of the invention. Attached Figure Description

[0030] Figure 1 This is a flowchart of an automatic coastline extraction method based on an improved Swin-Transformer, according to the present invention.

[0031] Figure 2 This invention provides a network structure for an automatic coastline extraction method based on an improved Swing-Transformer.

[0032] Figure 3 This is a schematic diagram of the Swintransformer block for an automatic coastline extraction method based on an improved Swin-Transformer according to the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0036] like Figure 1 , Figure 2 , Figure 3 As shown, this invention discloses an automatic coastline extraction method based on an improved Swin-Transformer, comprising the following steps:

[0037] Step S1 involves collecting various types of coastline images and video data, preprocessing this data, constructing a dataset, and continuously expanding it during subsequent runs.

[0038] Specifically, step S1 above includes:

[0039] Step S11 involves capturing images or videos of various coastline targets to be identified using surveillance cameras, drones, etc., then extracting video frames containing coastlines from dynamic videos and extracting images containing various coastlines for training, as well as selecting suitable datasets from existing open-source datasets to construct the dataset.

[0040] Step S12 involves preprocessing all images in the dataset, including rotating and cropping images, and adjusting image brightness and contrast.

[0041] Step S13: Use the LabelImg annotation tool, set to YOLO format, to manually annotate all images.

[0042] Step S14: Divide the labeled coastline target images into training set and test set according to the proportion.

[0043] For example, the partitioning ratio used in this embodiment is 8:2, where 80% of the data in the dataset is divided into a training set for model training and 20% of the data is divided into a test set for model testing. The specific partitioning ratio can be adjusted and optimized according to the actual training.

[0044] Step S2: Optimize the original Swin Transformer network model to obtain the improved SwinTransformer network model.

[0045] Specifically, the improvements to the network model in step S2 above include:

[0046] S21, the backbone network adopts the Swin-S with moderate structural complexity, and uses Swin Transformer blocks for feature downsampling in four stages. The first stage uses 2 Swin Transformer blocks, the second stage uses 2 Swin Transformer blocks, the third stage uses 18 Swin Transformer blocks, and the fourth stage uses 2 Swin Transformer blocks.

[0047] S22, the neck network replaces global average pooling with a multilayer perceptron consisting of two fully connected FC layers with ReLU activation function.

[0048] In step S23, the dimension of the output features from the fully connected (FC) layers in the head network is transformed into the embedding dimension, mapping the input features to a higher-dimensional space. Then, more non-linear features are obtained by applying the Swish activation function. Next, the hidden layer features are mapped to the output category through another fully connected (FC) layer.

[0049] S3, pre-trained and improved Swin Transformer network model, data augmentation of various training set coastline data to form an expanded training set;

[0050] Specifically, step S3 above includes:

[0051] S31, train the improved Swin Transformer network model without parameter initialization on the training set and calculate the F1-score for each class;

[0052] The formulas for calculating the F1 score for various coastlines are as follows:

[0053]

[0054] The F1-score assigns equal weight to recall and precision, where recall represents the proportion of correctly identified positive samples out of the actual positive samples, and precision represents the proportion of correctly identified positive samples out of the predicted positive samples.

[0055] S32 utilizes an F1-score-sensitive sampling number to "tailor" the number of additional samples needed for each class based on the initial recognition accuracy results;

[0056] The number of samples required for class i is the ratio of the error rate of class i to the sum of the error rates of all classes, multiplied by the total number of samples in the dataset. The formula is as follows:

[0057]

[0058] Where num represents the number of samples to be taken from each class, i represents the class, n is the number of classes in the dataset, and f 1i For each class's F1 score, 1-f 1i This represents the error rate of class i, where N is the total number of samples in the dataset (original dataset).

[0059] S33 uses the adaptive Simple Cope-Paste method to augment the training sample data of each category of coastline, forming an expanded training set.

[0060] Based on the S32 calculation of the increased number of samples, the adaptive Simple Copy-Paste determines whether to paste by calculating the mask overlap area, ensuring that the coastline is pasted in the background area of ​​the target image without covering the original coastline target area.

[0061] S4. An improved Swing Transformer network model is trained using an expanded training set to obtain a coastline detection model.

[0062] S5 deploys the trained model, takes the image to be detected as input, performs data preprocessing such as contrast and brightness, and outputs the results of detecting the coastline.

[0063] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the claims.

Claims

1. An automatic coastline extraction method based on an improved Swin-Transformer, characterized in that, Includes the following steps: S1. Collect monitoring video data of four types of coastlines: bedrock coast, sandy coast, silty coast, and biological coast, and preprocess the data to construct a dataset; S2. Optimize the original Swing Transformer network model to obtain the improved Swing Transformer network model. S3. Pre-training improves the Swin Transformer network model, and data augmentation is performed on various training set coastline data to form an expanded training set; S4. The improved Swin Transformer network model is trained using an expanded training set to obtain the coastline detection model; S5. Deploy the trained model, input the image to be detected, and detect the coastline; The improved Swin Transformer network model includes: a. The backbone network is changed to Swin-S with moderate structural complexity, and the feature downsampling is performed in four stages using Swin Transformer blocks. The first stage uses 2 Swin Transformer blocks, the second stage uses 2 Swin Transformer blocks, the third stage uses 18 Swin Transformer blocks, and the fourth stage uses 2 Swin Transformer blocks. b. The neck network replaces global average pooling with a multilayer perceptron consisting of two fully connected FC layers with ReLU activation function; c. The dimension of the output features of the fully connected FC layer in the head network is converted into the embedding dimension, which maps the input features to a higher dimensional space. Then, more non-linear features are obtained by attaching the Swish activation function. d. The hidden layer features are then mapped to the output categories through a fully connected (FC) layer.

2. The automatic coastline extraction method based on the improved Swin-Transformer according to claim 1, characterized in that: The preprocessing in step S1 includes rotation, cropping, and adjusting image brightness and contrast to improve the quality of the initial dataset. The constructed dataset should be constructed by manually labeling each image according to a pre-set labeling tool and dividing the labeled images into training and test sets according to a ratio.

3. The automatic coastline extraction method based on the improved Swin-Transformer according to claim 1, characterized in that: The pre-trained improved Swin Transformer network model includes the following steps: S31. Train the improved Swin Transformer network model without parameter initialization on the training set and calculate the F1-score for each class; S32. Using an F1-score-sensitive sampling number, calculate the number of samples required for each class based on the initial recognition accuracy results; S33. Adaptive Simple Cope-Paste method is used to augment the training sample data of each type of coastline to form an expanded training set.

4. The automatic coastline extraction method based on the improved Swin-Transformer according to claim 3, characterized in that: In S31, the F1-score is calculated according to the formula and then assigned equal weights to recall and precision. Recall represents the proportion of correctly identified positive samples out of the actual positive samples, and precision represents the proportion of correctly identified positive samples out of the predicted positive samples.

5. The automatic coastline extraction method based on the improved Swing-Transformer according to claim 3, characterized in that: In S32, each class's F1-score is defined as class i, and the number of samples required for class i is the ratio of the error rate of class i to the sum of the error rates of all classes multiplied by the total number of samples in the dataset.

Images