Coastline automatic extraction method based on improved Swin-Transform

Abstract

The invention discloses a coastline automatic extraction method based on an improved Swin-Transform, and the method comprises the steps: collecting four types of coastline data of a bedrock coast, a sandy coast, a sludge coast and a biological coast, constructing a data set, and dividing the data set into a training set and a test set; the method comprises the following steps: establishing an improved Swin Transform network model, and optimizing an original Swin Transform network model; pre-training an improved Swin Transforme network model, and performing data enhancement on coastline data of various training sets to form an expanded training set; training the improved Swin Transform network model by using the extended training set to obtain a trained coastline detection model; and detecting a to-be-detected coastline picture by using the trained detection model. The method is used for solving the problems that existing data sets are few, and model training is affected by imbalance existing between the number of different types of coastline samples, and extraction of various types of coastlines such as bedrock coast, sandy coast, sludge coast and biological coast is rapidly and effectively achieved.

Description

Technical Field

[0001] The present invention provides a coastline automatic extraction method, which belongs to the technical field of deep learning and computer vision, and particularly relates to a coastline automatic extraction method based on an improved Swin-Transformer. Background Art

[0002] The coastline is the dynamic boundary between the sea and the land. There are many types of coastlines, including bedrock coasts, sandy coasts, silt coasts and biological coasts. Bedrock coasts are composed of hard rocks, sandy coasts are mainly composed of gravel and sand, silt coasts are fertile, and biological coasts mainly include mangrove coasts and coral reef coasts.

[0003] Traditional shoreline extraction methods are mainly field survey and photogrammetry, but their field survey work is large and time-consuming, and they do not have advantages in the timeliness, dynamics and convenience of shoreline monitoring. With the development of remote sensing technology and the enrichment of image data, the research on shoreline extraction has developed from the original visual interpretation method to the computer automatic interpretation method. Predecessors have actively studied and explored a variety of methods for extracting shorelines based on remote sensing images, which can be roughly divided into threshold-based methods, edge operator-based methods for extracting the edges of sea and land areas, and object-oriented segmentation methods. These methods are easily affected by subjectivity, noise, complex background and other conditions. If traditional single features are used for sea and land segmentation, the segmentation effect is not ideal, which will lead to unclear sea and land boundaries, which is not conducive to high-resolution remote sensing image analysis.

[0004] With the rise of big data and the improvement of computing resources, deep learning has been widely used in the field of computer vision due to its powerful feature extraction capabilities and complex problem fitting capabilities. Research on the automatic extraction of coastlines using deep learning is also ongoing. In optical remote sensing images, the spectral contrast between water bodies and vegetation is not obvious, and there is a constraint of "same spectrum but different objects", which will lead to a large number of misclassified pixels. The influence of atmospheric factors such as clouds, sea surface echoes, and wind makes the instantaneous state of sea and land different from remote sensing images, accompanied by complex noise interference, resulting in boundary discontinuity. When existing deep learning network models perform sea and land segmentation on high-resolution remote sensing images with rich spatial information, there are few public data sets, which are difficult to obtain and have misclassification problems. Existing coastline extraction methods often cannot quickly and effectively extract various types of coastlines, and it is difficult to ensure the accuracy of coastline extraction results, and the use effect is poor. Therefore, coastline detection methods based on deep learning need further research. Summary of the invention

[0005] In order to make up for the shortcomings of the existing technology, the embodiments of the present application provide an automatic coastline extraction method based on an improved Swin-Transformer, which solves the problem that the existing data set is small and the imbalance between the number of samples of different categories of coastlines affects the model training. Various types of coastlines, such as bedrock coasts, sandy coasts, silt coasts and biological coasts, can be extracted quickly and effectively.

[0006] In order to solve the above technical problems, the present invention provides the following technical solution: a coastline automatic extraction method based on improved Swin-Transformer, characterized in that it comprises the following steps:

[0007] S1, collect monitoring video data of four types of coastlines, including bedrock coast, sandy coast, silty coast and biological coast, and pre-process these data to construct a data set;

[0008] S2, optimizing the original Swin Transformer network model to obtain an improved Swin Transformer network model;

[0009] S3, pre-training improves the Swin Transforme network model, and data enhances various types of training set coastline data to form an expanded training set;

[0010] S4, using the expanded training set to train and improve the Swin Transformer network model to obtain the coastline detection model;

[0011] S5. Deploy the trained model, input the image to be detected, and detect the coastline.

[0012] Preferably: the preprocessing in step S1 includes but is not limited to rotation, cropping, and adjusting image brightness and contrast to improve the quality of the initial data set; the constructed data set should be manually labeled for each image according to a pre-set labeling tool and the labeled images should be divided into a training set and a test set in proportion.

[0013] Preferably: the improved Swin Transformer network model comprises the following steps:

[0014] a. The backbone network is changed to Swin-S with moderate structural complexity, and four stages of feature downsampling are performed using Swin Transformer blocks, of which two Swin Transformer blocks are used in the first stage, two Swin Transformer blocks are used in the second stage, 18 Swin Transformer blocks are used in the third stage, and two Swin Transformer blocks are used in the fourth stage;

[0015] b. The neck network replaces the global average pooling with two fully connected FC layers and a multi-layer perceptron with a ReLU activation function;

[0016] c. The dimension of the output feature of the fully connected FC layer in the head network is converted to the embedding dimension, the input feature is mapped to a higher dimensional space, and then more nonlinear features are obtained by adding the Swish activation function.

[0017] d. Then the hidden layer features are mapped to the output categories through the fully connected FC layer.

[0018] Preferably: the pre-training improved Swin Transforme network model comprises the following steps:

[0019] S31. Train the improved Swin Transforme network model without parameter initialization on the training set and calculate the F1-score of each class;

[0020] S32. Using an F1-score sensitive sampling number, calculate the number of samples required for each class according to the initial recognition accuracy result;

[0021] S33. The adaptive Simple Cope-Paste method is used to enhance the training sample data of each category of coastline to form an expanded training set.

[0022] Preferably: the F1-score in S31 is calculated according to the formula and gives equal weights to the recall rate (Recall) and the precision rate (Precision), wherein Recall represents the proportion of correctly identified positive samples to actual positive samples, and Precision represents the proportion of correctly identified positive samples to predicted positive samples.

[0023] Preferably, in S32, each class of F1-score is defined as class i, and the number of samples required for class i is the ratio of the error rate of class i to the sum of the error rates of all classes multiplied by the total number of samples in the data set.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0025] The present invention collects pictures of bedrock coasts, sandy coasts, silty coasts, biological coasts, and video frames from surveillance videos, constructs a data set, and realizes fast and effective detection of various types of coastlines, such as bedrock coasts, sandy coasts, silty coasts, and biological coasts, solves the problem of imbalance in the number of samples of different categories of coastlines, and improves the recognition effect of each class while ensuring the overall recognition accuracy.

[0026] Compared with the traditional convolutional neural network (CNN), the Swin Transformer structure adopted in the present invention can capture the global dependencies in the image at different scales, integrate the global image features and local detail features, so as to better understand and process the image content. The present invention replaces the global average pooling layer of the neck network with a structure of two fully connected layers plus a ReLU activation function to protect the transfer learning, and adds a fully connected layer and a Swish activation function in the head network to combine more nonlinear features.

[0027] In the data enhancement stage, the data of the test set is only used to calculate the F1-score in the sampling quantity formula, and no data enhancement is performed. The model's recognition of each class is evaluated based on the F1-score, so as to adjust the data distribution of the training samples, thereby indirectly affecting the model training process. This ensures that the distribution of the test set data is consistent with the distribution of the real data, thereby better evaluating the generalization ability of the model.

[0028] The Simple Copy-Paste method adopted in the present invention is similar to MixUp and CutMix. This method can ensure that no wrong labels are generated while increasing the number of samples, thereby effectively improving the generalization performance of the model and enabling the model to adapt to complex environments in actual scenarios.

[0029] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flow chart of a coastline detection method based on an improved Swin Transformer for an automatic coastline extraction method based on an improved Swin-Transformer of the present invention;

[0031] Figure 2 A network structure of a coastline automatic extraction method based on an improved Swin-Transformer of the present invention;

[0032] Figure 3 This is a schematic diagram of a Swintransformer block of an automatic coastline extraction method based on an improved Swin-Transformer according to the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] It should be noted that the terms “vertical”, “horizontal”, “up”, “down”, “left”, “right” and similar expressions used in this document are only for illustrative purposes and do not represent the only implementation method.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0036] like Figure 1 , Figure 2 , Figure 3 As shown, the present invention discloses a coastline automatic extraction method based on an improved Swin-Transformer, comprising the following steps:

[0037] Step S1, collect pictures and video data of various coastlines, pre-process these data, build a data set, and continuously expand it in subsequent operations.

[0038] Specifically, the above step S1 includes:

[0039] Step S11, use surveillance cameras, drones, etc. to capture pictures or videos of various coastline targets to be identified, then capture video frames containing coastlines from dynamic videos and intercept pictures containing various coastlines to be trained, and select data sets that meet the requirements from existing open source data sets to construct a data set.

[0040] Step S12, preprocessing all images in the data set, including rotating and cropping images, and adjusting image brightness and contrast.

[0041] Step S13, using the LabelImg annotation tool, set to YOLO format, to manually annotate all images.

[0042] Step S14, dividing the labeled coastline target images into a training set and a test set according to the ratio.

[0043] Exemplarily, the division ratio adopted in this embodiment is 8:2, that is, 80% of the data in the data set is divided into a training set for model training, and 20% of the data is divided into a test set for model testing. The specific division ratio can be adjusted and optimized according to actual training.

[0044] Step S2, optimizing the original Swin transformer network model to obtain an improved Swin Transformer network model.

[0045] Specifically, the improvement of the network model in step S2 above includes:

[0046] S21, the backbone network uses Swin-S with moderate structural complexity, and uses Swin Transformer blocks for 4 stages of feature downsampling. The first stage uses 2 Swin Transformer blocks, the second stage uses 2 Swin Transformer blocks, the third stage uses 18 Swin Transformer blocks, and the fourth stage uses 2 Swin Transformer blocks.

[0047] S22, the neck network replaces the global average pooling with a multi-layer perceptron with two fully connected FC layers and a ReLU activation function.

[0048] S23, the dimension of the output feature of the fully connected FC layer in the head network is converted to the embedding dimension, the input feature is mapped to a higher dimensional space, and then more nonlinear features are obtained by adding the Swish activation function. Again, the hidden layer features are mapped to the output category through the fully connected FC layer.

[0049] S3, pre-training improved Swin Transforme network model, data enhancement of various training set coastline data to form an expanded training set;

[0050] Specifically, the above step S3 includes:

[0051] S31, train the improved Swin Transforme network model without parameter initialization on the training set and calculate the F1-score of each class;

[0052] The F1-score calculation formula for various coastlines is as follows

[0053]

[0054] F1-score gives equal weight to recall and precision, where Recall represents the proportion of correctly identified positive samples to actual positive samples, and Precision represents the proportion of correctly identified positive samples to predicted positive samples.

[0055] S32 uses an F1-score-sensitive sampling quantity to "tailor" the number of samples that need to be added for each class based on the initial recognition accuracy results;

[0056] The number of samples required for class i is the ratio of the error rate of class i to the sum of the error rates of all classes multiplied by the total number of samples in the data set. The calculation formula is:

[0057]

[0058] Among them, num represents the number of samples required for each category, i represents the category, n is the number of categories in the data set, and f 1i is the F1-score of each category, 1-f 1i It represents the error rate of class i, and N is the total number of samples in the data set (original data set).

[0059] S33, the adaptive Simple Cope-Paste method is used to enhance the training sample data of each category of coastline to form an expanded training set.

[0060] According to the number of samples added by S32, the adaptive Simple Copy-Paste determines whether to paste by calculating the mask overlapping area, ensuring that the coastline is pasted in the background area of ​​the target image without covering the original coastline target area.

[0061] S4, using the expanded training set to train the improved Swin Transformer network model to obtain the coastline detection model;

[0062] S5 deploys the trained model, inputs the image to be detected, preprocesses the contrast, brightness and other data, and outputs the result of coastline detection.

[0063] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the claims.

Claims

1. An automatic coastline extraction method based on improved Swin-Transformer, characterized in that: The following steps are involved: S1, collect monitoring video data of four types of coastlines, including bedrock coast, sandy coast, silty coast and biological coast, and pre-process these data to construct a data set; S2, optimizing the original Swin Transformer network model to obtain an improved Swin Transformer network model; S3, pre-training improves the Swin Transforme network model, and data enhances various types of training set coastline data to form an expanded training set; S4, using the expanded training set to train and improve the Swin Transformer network model to obtain the coastline detection model; S5. Deploy the trained model, input the image to be detected, and detect the coastline.

2. The method for automatically extracting coastlines based on improved Swin-Transformer according to claim 1, characterized in that: The preprocessing in step S1 includes but is not limited to rotation, cropping, and adjusting image brightness and contrast to improve the quality of the initial data set; the constructed data set should manually mark each image according to a pre-set marking tool and divide the marked images into a training set and a test set in proportion.

3. The method for automatically extracting coastlines based on improved Swin-Transformer according to claim 1, characterized in that: The improved Swin Transformer network model includes: a. The backbone network is changed to Swin-S with moderate structural complexity, and four stages of feature downsampling are performed using Swin Transformer blocks, of which two Swin Transformer blocks are used in the first stage, two Swin Transformer blocks are used in the second stage, 18 Swin Transformer blocks are used in the third stage, and two Swin Transformer blocks are used in the fourth stage; b. The neck network replaces the global average pooling with two fully connected FC layers and a multi-layer perceptron with a ReLU activation function; c. The dimension of the output feature of the fully connected FC layer in the head network is converted to the embedding dimension, the input feature is mapped to a higher dimensional space, and then more nonlinear features are obtained by adding the Swish activation function. d. Then the hidden layer features are mapped to the output categories through the fully connected FC layer.

4. The method for automatically extracting coastlines based on improved Swin-Transformer according to claim 1, characterized in that: The pre-training improved Swin Transforme network model includes the following steps: S31. Train the improved Swin Transforme network model without parameter initialization on the training set and calculate the F1-score of each class; S32. Using an F1-score sensitive sampling number, calculate the number of samples required for each class according to the initial recognition accuracy result; S33. The adaptive Simple Cope-Paste method is used to enhance the training sample data of each category of coastline to form an expanded training set.

5. The method for automatically extracting coastlines based on improved Swin-Transformer according to claim 4, characterized in that: The F1-score in S31 is calculated according to the formula and gives equal weights to the recall rate (Recall) and the precision rate (Precision), where Recall represents the proportion of correctly identified positive samples to actual positive samples, and Precision represents the proportion of correctly identified positive samples to predicted positive samples.

6. The method for automatically extracting coastlines based on improved Swin-Transformer according to claim 4, characterized in that: In S32, each class F1-score is defined as class i, and the number of samples required for class i is the ratio of the error rate of class i to the sum of the error rates of all classes multiplied by the total number of samples in the data set.

Images