River bank line extraction method, device and equipment

Abstract

The invention discloses a riparian line extraction method, device and equipment, and relates to the technical field of remote sensing data processing, and the method comprises the steps: obtaining a remote sensing image containing a specified river region in a preset time length; processing the remote sensing image to obtain a target data set corresponding to the remote sensing image; training an initial riparian line monitoring model based on the target data set to obtain a target riparian line monitoring model; the target riparian line monitoring model at least comprises a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used for feature enhancement; inputting the processed to-be-monitored river image into the target riparian line monitoring model for processing to obtain an initial riparian line; and performing target strategy extraction processing on the initial river bank line to extract a target river bank line. Therefore, the clarity of the river bank line can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing data processing, and in particular to a river bank extraction method, device and equipment. Background Art

[0002] Rivers are an important part of the Earth's ecosystem and play a vital role in regulating climate, providing freshwater resources, and maintaining biodiversity. Therefore, monitoring changes in river banks has become an important topic in water resources management and environmental science research.

[0003] Remote sensing technology can quickly and efficiently acquire geographic spatial information over large areas. It has the characteristics of wide coverage, high frequency of data acquisition, and no terrain restrictions. It is very suitable for large-scale monitoring and change analysis of riverbanks.

[0004] Traditional methods for extracting riverbanks mainly include single-band analysis, water body index and other methods. However, in complex terrain conditions, such as where rivers meet or where multiple tributaries converge, river channels intersect with each other and paddy fields are enriched, forming a complex water network structure. In this case, the boundary between the river and the land is often unclear, which poses a great challenge to the clear extraction of the riverbank. Summary of the invention

[0005] The object of the present invention is to provide a river bank extraction method, device and equipment for improving the clarity of river bank extraction.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a riverbank extraction method, comprising: Get remote sensing images of a specified river area with a preset duration; Processing the remote sensing image to obtain a target data set corresponding to the remote sensing image; The initial riverbank monitoring model is trained based on the target data set to obtain a target riverbank monitoring model; the target riverbank monitoring model at least includes a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used to perform feature enhancement on the data in the target data set; Inputting the processed river image to be monitored into the target river bank monitoring model for processing to obtain an initial river bank; The initial river bank line is subjected to target strategy extraction processing to extract the target river bank line.

[0007] In an optional implementation, the processing the remote sensing image to obtain a target data set corresponding to the remote sensing image includes: Preprocessing the remote sensing image to obtain a full-scene remote sensing image corresponding to the designated river area; Extracting a first image corresponding to a first preset band, a second image corresponding to a second preset band, and a third image corresponding to a third preset band from the entire remote sensing image to form an RGB image; Performing type labeling and rasterization processing on the RGB image to obtain a binary label corresponding to the RGB image; Segmenting the RGB image based on a preset pixel size to obtain a first data set; Segmenting the binary labels based on the preset pixel size to obtain a second data set; Data expansion is performed on both the first data set and the second data set to obtain the target data set.

[0008] In an optional implementation, before performing data expansion on the first data set, the method further includes: The float32 TIFF data type corresponding to the data in the first data set is converted to a unit8 PNG data type.

[0009] In an optional implementation, converting the float32 TIFF data type corresponding to the data in the first data set into a unit8 PNG data type includes: Calculate the pixel value corresponding to the low percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the low percentile as the lower limit of data scaling; Calculate the pixel value corresponding to the high percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the high percentile as the upper limit of data scaling; Based on the upper bound and the lower bound, data in the first data set corresponding to each preset band is truncated and scaled; Map the truncated and scaled data to the range [0, 255] through linear transformation, assign it to an array of type unit8, and write it to a PNG file.

[0010] In an optional implementation, the target strategy extraction process for the initial riverbank includes: The initial river bank line is subjected to projection coordinate addition, splicing and vectorization processing.

[0011] In an optional implementation, the target riverbank monitoring model further includes an encoder and a decoder; The encoder includes at least a CNN module and a Transformer module; The CNN module is used to extract input features from the target data set and output initial features; the Transformer module is used to process the initial features based on a multi-head self-attention mechanism; The dynamic channel graph convolutional network module includes a global average pooling layer, a two-layer fully connected network, a matrix construction submodule, a feature vector extraction submodule, a node feature extraction submodule, a fusion submodule and a tensor format conversion submodule; Among them, the global average pooling layer is used to calculate the global average of the features output by the encoder; the two-layer fully connected network is used to determine the channel weight of each channel based on the output result of the global average pooling layer; the matrix construction submodule is used to construct a dynamic adjacency matrix according to the channel weight; the feature vector extraction submodule is used to convert the multi-dimensional data corresponding to the features output by the encoder into a one-dimensional feature vector; the node feature extraction submodule is used to extract node features from the one-dimensional feature vector; the fusion submodule is used to fuse the node features with the dynamic adjacency matrix to obtain a channel graph; the tensor format conversion submodule is used to convert the channel graph into a tensor format and output a feature graph; The decoder includes three upsampling layers and a matching submodule; Among them, each upsampling layer is used to fuse the feature map with the initial features output by the encoder through jump connection and element-by-element addition; the matching submodule is used to use a 1×1 convolution layer to convert the feature map that has undergone three rounds of upsampling processing into a number of channels that matches the number of categories, and output the result.

[0012] In an optional implementation, preprocessing the remote sensing image to obtain a full-scene remote sensing image corresponding to the designated river area includes: Performing radiation correction on the remote sensing image; Perform geometric correction on the remote sensing images after radiation correction; The geometrically corrected remote sensing images are spliced ​​and cropped to obtain the entire remote sensing image corresponding to the designated river area.

[0013] In an optional embodiment, the riverbank extraction method further includes: Using the test set in the target data set to evaluate the trained initial riverbank monitoring model, and obtain the prediction accuracy; When the prediction accuracy is greater than or equal to the preset accuracy, the trained initial riverbank monitoring model with the prediction accuracy greater than or equal to the preset accuracy is determined as the target riverbank monitoring model.

[0014] Compared with the prior art, the present invention provides a riverbank extraction method, which obtains remote sensing images of a specified river area of ​​a preset duration and produces a target data set for training an initial riverbank monitoring model to obtain a target riverbank monitoring model, wherein the target riverbank monitoring model includes at least a dynamic channel graph convolutional network module, which can be used to enhance the image features of the data in the target data set, so that the processed river image to be monitored is input into the target riverbank monitoring model, and the obtained riverbank is clearer.

[0015] In a second aspect, the present invention further provides a riverbank extraction device, comprising: Acquisition and construction module, used to acquire remote sensing images of a preset duration containing a designated river area; A first processing module is used to process the remote sensing image to obtain a target data set corresponding to the remote sensing image; A training module, used to train the initial riverbank monitoring model based on the target data set to obtain a target riverbank monitoring model; the target riverbank monitoring model at least includes a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used to perform feature enhancement on the data in the target data set; The second processing module is used to input the processed river image to be monitored into the target river bank monitoring model for processing to obtain an initial river bank; The extraction module is used to perform target strategy extraction processing on the initial river bank line to extract the target river bank line.

[0016] In a third aspect, the present invention also provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate via the communication bus; the memory stores a computer program that can be run by the processor; when the processor runs the computer program, it executes any one of the riverbank extraction methods described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 One of the flow diagrams of a riverbank extraction method provided by an embodiment of the present invention; Figure 2 An image corresponding to the B8 band extracted from a full-scene remote sensing image provided by an embodiment of the present invention; Figure 3An image corresponding to the B4 band extracted from a full-scene remote sensing image provided by an embodiment of the present invention; Figure 4 An image corresponding to the B3 band extracted from a full-scene remote sensing image provided by an embodiment of the present invention; Figure 5 An image of a water body before being marked provided by an embodiment of the present invention; Figure 6 For Figure 5 The corresponding water body labeled image; Figure 7 A schematic diagram of the structure of a target river bank monitoring model provided by an embodiment of the present invention; Figure 8 A schematic diagram of the results of running the target river bank monitoring model provided by one embodiment of the present invention; Fig. 9 A schematic diagram of a river bank obtained by vectorizing the result of running a target river bank monitoring model provided by an embodiment of the present invention; Fig.10 A schematic diagram of a final river bank line and an original image superimposed and displayed according to an embodiment of the present invention; Fig.11 A second flow chart of a riverbank extraction method provided by an embodiment of the present invention; Fig.12 A schematic diagram of the structure of a riverbank extraction device provided by an embodiment of the present invention; Fig.13 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0019] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0020] In the present invention, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the association relationship of the associated objects, indicating that three types of relationships may exist.

[0021] like Figure 1 As shown, the embodiment of the present invention provides a river bank extraction method, which may include: Step 110: Acquire a remote sensing image of a predetermined time period including a designated river area; The remote sensing image is acquired through a satellite. Optionally, the remote sensing image can be acquired from the European Space Agency's Sentinel-2 image, and the resolution can be selected to be 10m.

[0022] Optionally, the preset time period is generally in years, for example, 8 years of Sentinel-2 images from the European Space Agency may be selected.

[0023] Step 120: Process the remote sensing image to obtain a target data set corresponding to the remote sensing image; Specifically, step 120 may include: 1) Preprocess the remote sensing image to obtain the entire remote sensing image corresponding to the designated river area; Step 1) may specifically include: Perform radiation correction on remote sensing images; radiation correction can eliminate the adverse effects of factors such as atmospheric influence and sensor inconsistency, and ensure the radiation accuracy of remote sensing images.

[0024] The remote sensing image after radiation correction is geometrically corrected; specifically, geometric correction refers to the geographic registration of the acquired remote sensing image so that it accurately corresponds to the actual geographic location to facilitate subsequent processing and observation.

[0025] The geometrically corrected remote sensing images are stitched and cropped to obtain a full-scene remote sensing image corresponding to the specified river area. It is understandable that the remote sensing image may include multiple sub-images at different time points and different locations. For multiple sub-images at a time point and different locations, each sub-image may only contain images corresponding to a part of the specified river area. Therefore, in this case, after the remote sensing image is acquired, it is necessary to stitch the multiple sub-images in the remote sensing image into a large-area image, which includes the specified river area. Finally, it is necessary to crop this large-area image according to the edge of the specified river area according to the actual size of the specified river area to obtain a full-scene remote sensing image that completely corresponds to the specified river area.

[0026] 2) extracting a first image corresponding to a first preset band, a second image corresponding to a second preset band, and a third image corresponding to a third preset band from the entire remote sensing image to form an RGB image; Regarding some of the bands, there are the following introductions: B1 is the blue band (0.45-0.52um). This band is located at the part with the smallest attenuation coefficient of the water body and has the greatest penetration into the water body. It is used to determine the water depth, study the underwater topography of shallow seas, water turbidity, etc., and to map water systems and shallow sea areas.

[0027] B2 is the green band (0.52-0.60um), which is located near the reflection peak of green plants. It is sensitive to plant reflection and can identify plant types and evaluate plant productivity. It has a certain penetration into water bodies and can reflect features such as underwater terrain, sandbars and coastal sandbars.

[0028] B3 is the red band (0.63-0.69um), which is located in the main absorption band of chlorophyll. It can be used to distinguish plant types, coverage, and judge plant growth conditions. In addition, this band can provide rich plant information for exposed surfaces, vegetation, rock types, structures, landforms, hydrology and other features.

[0029] B4 is the near-infrared band (0.76-0.90um). This band is located in the high reflectivity area of ​​plants and reflects a large amount of plant information. It is mostly used for plant identification and classification. At the same time, it is also located in the strong absorption area of ​​water bodies and can be used to outline water body boundaries and identify geological structures related to water bodies.

[0030] B5 is a short infrared band (1.55-1.75um), which is located between two water absorption bands. It is sensitive to the moisture content of plants and soil, thereby improving the ability to distinguish crops. In addition, the reflectivity of snow and clouds in this band is low, and the two are easy to distinguish. It has a large amount of information and a high application rate.

[0031] B6 is the thermal infrared band (10.4-12.5um), which is sensitive to the thermal radiation of ground objects. It can be used to distinguish between crops and forests, and identify surface features such as water bodies and rocks based on the difference in radiation heat.

[0032] B7 is the shortwave outer band (2.08-2.35um), which is a band added specifically for geological tracing. This band is sensitive to rocks and specific minerals. It is used to distinguish major rock types, rock hydrothermal alteration, and detect clay minerals related to replacement rocks.

[0033] B8 is the panchromatic band (0.50-0.90), which covers a wider spectral range and has a higher spatial resolution than other bands. Therefore, it is mostly used to obtain the geometric features of the ground.

[0034] Since an RGB image is to be formed, it is necessary to select images corresponding to three bands from the above eight bands. In order to better highlight the water body characteristics, a false color image is used, so the B8, B4, and B3 bands are selected. That is to say, the first preset band can be the B8 band, the second preset band can be the B4 band, and the third preset band can be the B3 band. For example Figure 2 The image corresponding to the B8 band extracted from the entire remote sensing image is shown; Figure 3 The image corresponding to the B4 band extracted from the entire remote sensing image is shown; Figure 4 The image corresponding to the B3 band extracted from the entire remote sensing image is shown.

[0035] It is understandable that the binary label is a single-band binary image, so there is no need to perform the band selection step.

[0036] 3) Perform type labeling and rasterization on the RGB image to obtain the binary label corresponding to the RGB image; Type labeling is to label which areas in the RGB image are water bodies. Specifically, the land-water boundary is regarded as the river bank, which is to label which areas in the RGB image are water bodies and which areas are non-water bodies. Figure 5 and Figure 6 .

[0037] In step 3), data labeling is based on the pre-processed Sentinel-2 image. A new surface vector is created in the user desktop component ArcMap to label the entire riverbank. The entire scene labeling can ensure the consistency of the labeling. After the labeling is completed, the labeled surface is converted into a raster to obtain the binary label of the RGB image.

[0038] 4) Segmenting the RGB image based on a preset pixel size to obtain a first data set; Specifically, the preset pixel size may be selected as 256*256.

[0039] In the user desktop component ArcMap, use the "Split Raster" tool to split the RGB image into 256*256 sizes to obtain the first data set.

[0040] 5) Segment the binary labels based on a preset pixel size to obtain a second data set; In the user desktop component ArcMap, use the "Split Raster" tool to split the binary labels into 256*256 sizes to obtain the second data set.

[0041] 6) Perform data expansion on both the first data set and the second data set to obtain the target data set.

[0042] In step 6), the first data set is expanded by rotation, color enhancement, contrast enhancement, etc.; the second data set is expanded by rotation, color enhancement, contrast enhancement, etc.; the expanded first data set is written to, for example, an Imges folder, and the expanded second data set is written to, for example, a Masks folder. It is understandable that the data in the Imges folder and the data in the Masks folder constitute the target data set.

[0043] Step 130: training the initial riverbank monitoring model based on the target data set to obtain a target riverbank monitoring model; the target riverbank monitoring model at least includes a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used for feature enhancement; It should be noted that the initial riverbank monitoring model is improved based on the TransUNet model. The overall structure of the traditional TransUNet model consists of two parts, the image encoder and the decoder. The image encoder is mainly responsible for converting the input image into some smaller feature maps and combining these feature maps through the self-attention mechanism. The decoder then decodes the combined feature maps into the output image for subsequent processing.

[0044] In the specific implementation, it was found that the clarity of the riverbank could not meet the expectations when only the traditional TransUNet model was used to extract the riverbank. Therefore, one of the concepts of the present invention is to improve the traditional TransUNet model. The specific concept is to integrate the dynamic channel graph convolutional network module (Densely Connected Graph Convolutional Network, DCGCN) into the TransUNet model to form a DCGCN-TransUNet model, which is the riverbank extraction model defined in the embodiment of the present invention.

[0045] The initial riverbank monitoring model (initial DCGCN-TransUNet model) is an untrained model.

[0046] From the target data set obtained in the previous steps, 20% is extracted as the test set and 80% is extracted as the training set according to a preset ratio (for example, 2 / 8).

[0047] The initial riverbank monitoring model is trained using the training set. In a specific implementation, the configuration of the experimental environment is that the CPU is 15vCPUIntel(R)Xeon(R)Platinum8358PCPU@2.60GHz, the GPU is RTX3090(24GB), the number is 1, and the Python version is 3.8; the main parameters and numerical settings involved in the model training are as follows: the basic learning rate (base_lr) is 0.01, the batch size (batch_size) is 24, the input image size (img_size) is 256, the maximum training round (max_epochs) is 101, the maximum number of iterations (max_iterations) is 40000, the number of target categories (num_classes) is 2, and the size of the partitioned patches (vit_patches_size) is 16.

[0048] After the training is completed, the test set is used to evaluate the trained riverbank monitoring model. If the prediction accuracy requirement is met, the trained riverbank monitoring model that meets the prediction accuracy requirement will be determined as the target riverbank monitoring model (target DCGCN-TransUNet model).

[0049] Step 140: inputting the processed river image to be monitored into the target river bank monitoring model for processing to obtain an initial river bank; The dynamic channel graph convolutional network module is used for feature enhancement. Then, the dynamic channel graph convolutional network module can perform feature enhancement on the processed data in the river image to be monitored, and output an initial river bank line with higher clarity and higher resolution.

[0050] Step 150: Perform target strategy extraction processing on the initial river bank line to extract the target river bank line.

[0051] From the above content, it can be seen that the present invention provides a riverbank extraction method, which obtains remote sensing images containing a specified river area of ​​a preset time length, and produces a target data set for training an initial riverbank monitoring model to obtain a target riverbank monitoring model, wherein the target riverbank monitoring model includes at least a dynamic channel graph convolutional network module, which can be used to enhance the image features of the data in the target data set, so that the processed river image to be monitored is input into the target riverbank monitoring model, and a riverbank with higher clarity can be output.

[0052] In an optional implementation, before performing data expansion on the first data set, the method further includes: The float32 TIFF data type corresponding to the data in the first data set is converted to a unit8 PNG data type.

[0053] Float32, also known as FP32 (floatpoint32), is a 32-bit floating-point number with 1 sign bit, 8 bits for the exponent, and 23 bits for the mantissa. It is a standard data type.

[0054] Tag Image File Format (TIFF) is a flexible bitmap format used primarily to store images, including photographs and artwork.

[0055] uint8 is a data type that represents an 8-bit unsigned integer. In computers, data is usually divided into different types, which have different ranges and storage methods. uint8 is usually used to represent integers from 0 to 255.

[0056] PNG (Portable Network Graphics) is a lossless compressed bitmap image format.

[0057] If you directly convert float32 TIFF data to unit8 PNG data, the resulting PNG data will be completely black. This is because the value range of float32 is [-3.4028235E38, 3.4028235E38], and the value range of unit8 is [0, 255]. Direct conversion results in the TIFF value being mapped to the PNG value being too small to perform color division. The conventional method is to multiply the TIFF by a larger value, such as 100, and then convert it to PNG. However, the resulting PNG image color is too different from the TIFF image color, and for remote sensing images, many obvious features of land objects are missing. Therefore, in order to ensure that the processed image is consistent with the color of the original image, the following data conversion method is used: Calculate the pixel value corresponding to the low percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the low percentile as the lower limit of data scaling; wherein each preset band refers to any one of the first preset band, the second preset band and the third preset band; Calculate the pixel value corresponding to the high percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the high percentile as the upper limit of data scaling; Based on the upper bound and the lower bound, data in the first data set corresponding to each preset band is truncated and scaled to determine that all pixel values ​​fall between the upper bound and the lower bound; Map the truncated and scaled data to the range [0, 255] through linear transformation, assign it to an array of type unit8, and write it to a new PNG file.

[0058] In an optional embodiment, in combination Figure 7As shown in the figure, the model structure of the target river bank monitoring model is introduced: The target riverbank monitoring model includes an encoder, a dynamic channel graph convolutional network module, and a decoder; The encoder includes at least a CNN module and a Transformer module; wherein CNN is a Convolutional Neural Network, a convolutional neural network.

[0059] Among them, the CNN module is used to extract the input features from the target data set and output the initial features; the Transformer module is used to process the initial features based on the multi-head self-attention mechanism; The encoder also includes a first hidden layer, a linear processing layer, and a second hidden layer. The input image size is , input encoder, first the CNN module performs three convolutions on it for preliminary feature extraction, after processing by the first hidden layer, and then by linear processing by the linear processing layer, the initial features output by the linear processing layer are input to the Transformer module; the Transformer module includes at least 12 Transformer layers, each of which includes a multi-head attention layer and a feedforward network layer. The Transformer layer implements multi-head attention parallel processing, which can capture contextual information of different scales; the second hidden layer processes the feature map output by the Transformer layer, and can compress the feature map output by the Transformer layer into a smaller dimension, such as , and then use the Reshape function to adjust the array shape to Before the array enters the dynamic channel graph convolutional network module, an extra dimension is added using the Unsqueeze operation, and the resulting tensor format is .in, It represents the number of small blocks into which the feature map is divided or the length of the sequence. B, D, H, and C are introduced below.

[0060] The dynamic channel graph convolutional network module includes a global average pooling layer, a two-layer fully connected network, a matrix construction submodule, a feature vector extraction submodule, a node feature extraction submodule, a fusion submodule, and a tensor format conversion submodule; Among them, the global average pooling layer is used to calculate the global average of the features output by the encoder; the two-layer fully connected network is used to determine the channel weight of each channel based on the output result of the global average pooling layer; the matrix construction submodule is used to construct a dynamic adjacency matrix according to the channel weight; the feature vector extraction submodule is used to convert the multi-dimensional data corresponding to the features output by the encoder into a one-dimensional feature vector; the node feature extraction submodule is used to extract node features from the one-dimensional feature vector; the fusion submodule is used to fuse the node features and the dynamic adjacency matrix to obtain a channel graph; the tensor format conversion submodule is used to convert the channel graph into a tensor format and output a feature graph; In the specific implementation, it is assumed that the input feature tensor input to the global average pooling layer is , where B is the batch size, D is the number of channels, H and C represent the height and width of the input image respectively; the processing process of the global average pooling layer can be represented by formula (1): (1); in, represents the global average pooling result of the i-th channel, Represents the i-th channel at position The eigenvalue at .

[0061] Substitute the global average pooling result into formula (2) to calculate the weight of each channel. The channel weight is calculated through a two-layer fully connected network: (2) in, represents the weight of the i-th channel, is the weight matrix of the first fully connected layer, r is the scaling factor, is the ReLU activation function, The weight matrix of the second fully connected layer is is the Sigmoid activation function.

[0062] The dynamic adjacency matrix is ​​constructed according to the channel weights of formula (3): (3); in, represents the correlation strength between the i-th channel and the j-th channel, represents an update function, represents the weight of the j-th channel.

[0063] The feature vector extraction submodule can be specifically used to convert the feature corresponding multi-dimensional data output by the encoder into a one-dimensional feature vector; The node feature extraction submodule can be used to extract node features from a one-dimensional feature vector. The node feature extraction submodule includes multiple layers. Assume that the node feature of the i-th node is , after each layer of graph convolution, the node feature update formula of the node is as follows: (4) in, represents the set of neighbor nodes of the ith node, and are the degrees of node i and node j respectively, Characterize the node features of the jth node in the lth layer, Characterize the node features of the i-th node in the l+1th layer; The weight matrix representing the lth layer; The dynamic adjacency matrix is ​​added to the node features extracted from the input features to perform feature enhancement and obtain a channel graph.

[0064] Assume that the node feature matrix in batch processing is , the relationship matrix between channels is , the calculation formula of channel graph convolution is: (5) in, ; K=[ k 1 , k 2 ,⋯, k i ] ; in, is the feature after channel convolution, F is the relationship matrix between channels, A is the dynamic adjacency matrix, It is a static channel relationship matrix, which is used to model the basic channel relationship. is the node feature matrix, represents the feature vector of the node, and N represents the number of nodes.

[0065] The decoder consists of three upsampling layers and a matching submodule; Among them, each upsampling layer is used to fuse the feature map with the initial features output by the encoder through skip connection and element-by-element addition; the matching submodule is used to convert the feature map that has undergone three rounds of upsampling into a channel number that matches the number of categories using a 1×1 convolutional layer, and output the result.

[0066] From the above content, we can see that the dynamic channel graph convolutional network module, when extracting local features through this module, uses the topological structure of the graph and captures the complex relationship between nodes by dynamically adjusting the weights of the channels.

[0067] In an optional implementation, step 150 may specifically include: adding projection coordinates, splicing, and vectorizing the initial river bank line.

[0068] The steps of adding projection coordinates specifically include: Obtain the segmented original image and the binary result map output by the target river bank monitoring model; The original image and the binary result Figure 1 One to one correspondence; Read geographic information from the segmented original image; Assign the read geographic information to the binary result map; Output the corresponding results. For example, the output results refer to Figure 8 , Fig. 9 and Fig.10 .

[0069] From the above content, it can be seen that by adding projection coordinates, splicing and vectorizing the initial river bank line, a river bank line with better clarity can be obtained.

[0070] Combination Fig.11 The flow chart of FIG. 1 summarizes the riverbank extraction method provided by the embodiment of the present invention: The first stage is to preprocess and extract information from remote sensing images acquired through satellites.

[0071] The specific process of the first stage is: collect remote sensing image data through satellites, pre-process remote sensing images, choose to manually extract riverbanks as annotation samples, perform sample cropping and format conversion, and finally form a data set, which includes a training set and a test set.

[0072] The second stage is to build and train the model based on the improved TransUNet model.

[0073] The specific process of the second stage is: build the TransUNet model, improve the model by adding a dynamic channel graph convolutional network module, add training samples (also known as training sets) to start training, output the training model, and obtain the target version of the improved TransUNet model.

[0074] The third stage is to extract and vectorize the river bank line.

[0075] The specific process of the third stage is: call the training model (the target version of the improved TransUNet model), input the riverbank image to be monitored, the model outputs the riverbank binary map, adds the corresponding projection coordinates, splices the vectorized riverbank, and obtains the target riverbank.

[0076] The riverbank extraction device provided by the present invention is described below. The riverbank extraction device described below and the riverbank extraction method described above can be referenced to each other.

[0077] like Fig.12 As shown, an embodiment of the present invention further provides a riverbank extraction method device, which is used to implement the riverbank extraction method in any of the above embodiments. The riverbank extraction device may include: An acquisition module 1210 is used to acquire a remote sensing image of a preset time period including a designated river area; The first processing module 1220 is used to process the remote sensing image to obtain a target data set corresponding to the remote sensing image; A training module 1230 is used to train the initial riverbank monitoring model based on the target data set to obtain a target riverbank monitoring model; the target riverbank monitoring model at least includes a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used to enhance the features of the data in the target data set; The second processing module 1240 is used to input the processed river image to be monitored into the target river bank monitoring model for processing to obtain an initial river bank; The extraction module 1250 is used to perform target strategy extraction processing on the initial river bank line to extract the target river bank line.

[0078] In an optional implementation, the first processing module 1220 may be specifically used to: pre-process the remote sensing image to obtain a full-scene remote sensing image corresponding to the designated river area; Extracting a first image corresponding to a first preset band, a second image corresponding to a second preset band, and a third image corresponding to a third preset band from the entire remote sensing image to form an RGB image; Perform type labeling and rasterization on the RGB image to obtain the binary label corresponding to the RGB image; Segmenting the RGB image based on a preset pixel size to obtain a first data set; Segment the binary labels based on a preset pixel size to obtain a second data set; Data expansion is performed on both the first data set and the second data set to obtain a target data set.

[0079] The riverbank extraction device may further include a data conversion module for converting the float32 TIFF data type corresponding to the data in the first data set into a unit8 PNG data type.

[0080] The data conversion module can be specifically used to: calculate the pixel value corresponding to the low percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the low percentile as the lower limit of data scaling; Calculate the pixel value corresponding to the high percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the high percentile as the upper limit of data scaling; Based on the upper bound and the lower bound, data in the first data set corresponding to each preset band is truncated and scaled; Map the truncated and scaled data to the range [0, 255] through linear transformation, assign it to an array of type unit8, and write it to a PNG file.

[0081] The extraction module 1250 can be specifically used to add projection coordinates, splice and vectorize the initial river bank line.

[0082] The target riverbank monitoring model also includes an encoder and a decoder; The encoder includes at least a CNN module and a Transformer module; Among them, the CNN module is used to extract the input features from the target data set and output the initial features; the Transformer module is used to process the initial features based on the multi-head self-attention mechanism; The dynamic channel graph convolutional network module includes a global average pooling layer, a two-layer fully connected network, a matrix construction submodule, a feature vector extraction submodule, a node feature extraction submodule, a fusion submodule, and a tensor format conversion submodule; Among them, the global average pooling layer is used to calculate the global average of the features output by the encoder; the two-layer fully connected network is used to determine the channel weight of each channel based on the output result of the global average pooling layer; the matrix construction submodule is used to construct a dynamic adjacency matrix according to the channel weight; the feature vector extraction submodule is used to convert the multi-dimensional data corresponding to the features output by the encoder into a one-dimensional feature vector; the node feature extraction submodule is used to extract node features from the one-dimensional feature vector; the fusion submodule is used to fuse the node features and the dynamic adjacency matrix to obtain a channel graph; the tensor format conversion submodule is used to convert the channel graph into a tensor format and output a feature graph; The decoder consists of three upsampling layers and a matching submodule; Among them, each upsampling layer is used to fuse the feature map with the initial features output by the encoder through skip connection and element-by-element addition; the matching submodule is used to convert the feature map that has undergone three rounds of upsampling into a channel number that matches the number of categories using a 1×1 convolutional layer, and output the result.

[0083] The first processing module 1220 can be specifically used to: perform radiation correction on the remote sensing image; Perform geometric correction on the remote sensing images after radiation correction; The geometrically corrected remote sensing images are stitched and cropped to obtain the entire remote sensing image corresponding to the designated river area.

[0084] The riverbank extraction device also includes a model testing module, which is used to use a test set in the target data set to perform model evaluation on the trained initial riverbank monitoring model to obtain prediction accuracy; When the prediction accuracy is greater than or equal to the preset accuracy, the trained initial riverbank monitoring model with the prediction accuracy greater than or equal to the preset accuracy is determined as the target riverbank monitoring model.

[0085] like Fig.13 As shown, an embodiment of the present invention further provides an electronic device, which may include: a processor 1310, a communications interface 1320, a memory 1330 and a communication bus, wherein the processor 1310, the communications interface 1320 and the memory 1330 communicate with each other through the communication bus. The memory 1330 stores a computer program that can be run by the processor 1310; when the processor 1310 runs the computer program, it can execute the riverbank extraction method in any of the above embodiments.

[0086] In addition, the logic instructions in the above-mentioned memory 1330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0087] On the other hand, the present invention further provides a non-transitory computer-readable storage medium, in which instructions are stored. When the instructions are executed, the riverbank extraction method in any of the above embodiments is implemented.

[0088] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0089] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the claims of the present invention and their equivalents.

Claims

1. A river bank extraction method, characterized in that: include: Get remote sensing images of a specified river area with a preset duration; Processing the remote sensing image to obtain a target data set corresponding to the remote sensing image; The initial river bank monitoring model is trained based on the target data set to obtain a target river bank monitoring model; the target river bank monitoring model at least includes a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used for feature enhancement; Inputting the processed river image to be monitored into the target river bank monitoring model for processing to obtain an initial river bank; The initial river bank line is subjected to target strategy extraction processing to extract the target river bank line.

2. The riverbank extraction method according to claim 1, characterized in that: The step of processing the remote sensing image to obtain a target data set corresponding to the remote sensing image includes: Preprocessing the remote sensing image to obtain a full-scene remote sensing image corresponding to the designated river area; Extracting a first image corresponding to a first preset band, a second image corresponding to a second preset band, and a third image corresponding to a third preset band from the entire remote sensing image to form an RGB image; Performing type labeling and rasterization processing on the RGB image to obtain a binary label corresponding to the RGB image; Segmenting the RGB image based on a preset pixel size to obtain a first data set; Segmenting the binary labels based on the preset pixel size to obtain a second data set; Data expansion is performed on both the first data set and the second data set to obtain the target data set.

3. The riverbank extraction method according to claim 2, characterized in that: Before performing data expansion on the first data set, the method further includes: The float32 TIFF data type corresponding to the data in the first data set is converted to a unit8 PNG data type.

4. The riverbank extraction method according to claim 3, characterized in that: The converting the float32 TIFF data type corresponding to the data in the first data set to the unit8 PNG data type includes: Calculate the pixel value corresponding to the low percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the low percentile as the lower limit of data scaling; Calculate the pixel value corresponding to the high percentile in the first data set corresponding to each preset band, and determine the pixel value corresponding to the high percentile as the upper limit of data scaling; Based on the upper bound and the lower bound, data in the first data set corresponding to each preset band is truncated and scaled; Map the truncated and scaled data to the range [0, 255] through linear transformation, assign it to an array of type unit8, and write it to a PNG file.

5. The riverbank extraction method according to claim 1, characterized in that: The target strategy extraction process for the initial river bank line includes: The initial river bank line is subjected to projection coordinate addition, splicing and vectorization processing.

6. The riverbank extraction method according to claim 1, characterized in that: The target river bank monitoring model also includes an encoder and a decoder; The encoder includes at least a CNN module and a Transformer module; The CNN module is used to extract input features from the target data set and output initial features; the Transformer module is used to process the initial features based on a multi-head self-attention mechanism; The dynamic channel graph convolutional network module includes a global average pooling layer, a two-layer fully connected network, a matrix construction submodule, a feature vector extraction submodule, a node feature extraction submodule, a fusion submodule and a tensor format conversion submodule; Among them, the global average pooling layer is used to calculate the global average of the features output by the encoder; the two-layer fully connected network is used to determine the channel weight of each channel based on the output result of the global average pooling layer; the matrix construction submodule is used to construct a dynamic adjacency matrix according to the channel weight; the feature vector extraction submodule is used to convert the multi-dimensional data corresponding to the features output by the encoder into a one-dimensional feature vector; the node feature extraction submodule is used to extract node features from the one-dimensional feature vector; the fusion submodule is used to fuse the node features with the dynamic adjacency matrix to obtain a channel graph; the tensor format conversion submodule is used to convert the channel graph into a tensor format and output a feature graph; The decoder includes three upsampling layers and a matching submodule; Among them, each upsampling layer is used to fuse the feature map with the initial features output by the encoder through jump connection and element-by-element addition; the matching submodule is used to use a 1×1 convolution layer to convert the feature map that has undergone three rounds of upsampling processing into a number of channels that matches the number of categories, and output the result.

7. The riverbank extraction method according to claim 2, characterized in that: Preprocessing the remote sensing image to obtain a full-scene remote sensing image corresponding to the designated river area includes: Performing radiation correction on the remote sensing image; Perform geometric correction on the remote sensing images after radiation correction; The geometrically corrected remote sensing images are spliced ​​and cropped to obtain the entire remote sensing image corresponding to the designated river area.

8. The riverbank extraction method according to claim 1, characterized in that: Also includes: Using the test set in the target data set to evaluate the trained initial riverbank monitoring model, and obtain the prediction accuracy; When the prediction accuracy is greater than or equal to the preset accuracy, the trained initial riverbank monitoring model with the prediction accuracy greater than or equal to the preset accuracy is determined as the target riverbank monitoring model.

9. A river bank extraction device, characterized in that: include: An acquisition module is used to acquire remote sensing images of a preset time period including a designated river area; A first processing module is used to process the remote sensing image to obtain a target data set corresponding to the remote sensing image; A training module, used to train the initial riverbank monitoring model based on the target data set to obtain a target riverbank monitoring model; the target riverbank monitoring model at least includes a dynamic channel graph convolutional network module; the dynamic channel graph convolutional network module is used to perform feature enhancement on the data in the target data set; The second processing module is used to input the processed river image to be monitored into the target river bank monitoring model for processing to obtain an initial river bank; The extraction module is used to perform target strategy extraction processing on the initial river bank line to extract the target river bank line.

10. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate via the communication bus; The memory stores a computer program that can be executed by the processor; when the processor runs the computer program, it executes the riverbank extraction method as described in any one of claims 1-8.

Images