Automatic fine water body extraction method for high-spatial-resolution optical remote sensing image

Through deep convolutional neural networks and pseudo-twin neural networks, multi-feature parallel self-learning in high-spatial resolution optical remote sensing images, combined with building shadow removal methods, the problem of missed division and missed in the extraction of fine water bodies in urban areas is solved. Improved extraction accuracy and efficiency.

CN118968303APending Publication Date: 2024-11-15ANHUI UNIV +2

Patent Information

Application Number
CN202411063990.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the prior art, in high-spatial resolution optical remote sensing images, there are problems of missed division and missed extraction of fine water bodies in urban areas, and insufficient data leads to insufficient model performance, especially under the influence of building shadows.

Method used

Deep convolutional neural network is used to combine pseudo-twin neural network and Densenets model to perform multi-feature parallel self-learning through space-perceptual subnet and context semantic information-perceptual subnet, and combine building shadow removal method to achieve accurate extraction of small water bodies.

Benefits of technology

It improves the accuracy and efficiency of water extraction in urban areas, reduces the impact of building shadows on extraction accuracy, and realizes the complete extraction of fine water bodies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968303A_ABST
    Figure CN118968303A_ABST
Patent Text Reader

Abstract

The invention discloses a fine water body automatic extraction method of a high-spatial-resolution optical remote sensing image, relates to the technical field of image processing and target identification, and aims to solve the problems that fine water bodies are lost and are seriously influenced by building shadows in current urban area water body intelligent extraction. A human eye visual cognition related theory is combined with a deep learning model, a pseudo twinning neural network and a Densents deep learning model are constructed, and the method comprises the steps of high-spatial-resolution optical remote sensing image preprocessing, water body sample set making, model construction and building shadow removal based on context information. Remote sensing image multi-feature parallel self-learning and multi-feature integration are realized, and the influence of building shadows on water body extraction precision is reduced. A high-spatial-resolution optical remote sensing image urban area water body full-automatic classification technology with practical value is obtained, the precision of urban area small water body extraction is improved, complete extraction of urban area water bodies is achieved, and the efficiency of urban area water body extraction is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image processing and target recognition, in particular to a method for automatically extracting small water bodies from optical remote sensing images with high spatial resolution. Background Art

[0002] Remote sensing technology has been widely used in large-scale topographic mapping, land resource management, ecological environment monitoring, water resource evaluation, etc. due to its advantages such as fast processing speed, high degree of automation, real-time dynamic monitoring, and wide detection range. The degree of automation of remote sensing image information extraction directly affects large-scale data production, wide industry applications, and full utilization of data. At present, remote sensing image data can quickly, efficiently, and fully automatically obtain regional multi-scale long-term spatial information through the steps of "field collection, image preprocessing, and product production", providing an effective method for the dynamic observation of surface water. High spatial resolution optical remote sensing images can distinguish more detailed ground targets and provide richer information for urban surface water monitoring, effectively overcoming the problem that medium and low resolution optical remote sensing images cannot obtain small urban surface water bodies.

[0003] At present, the traditional methods for extracting water bodies using optical remote sensing images mainly include single-band or multi-band threshold methods and water index methods. Due to the great differences in water and land characteristics in different regions, single-band or multi-band threshold methods and water index methods have only achieved high water extraction accuracy in remote sensing images with relatively simple backgrounds and rich bands. For high-spatial-resolution optical remote sensing images with less band information, there are still a lot of mis-extraction and erroneous extraction phenomena in the extraction of surface water bodies in urban areas with complex backgrounds. Deep learning can perform unsupervised feature learning from the original image data set, which can effectively avoid the limitations of manual feature extraction and has been widely used for information extraction from high-spatial-resolution remote sensing images. However, the existing deep learning extraction methods for surface water bodies mainly focus on extracting surface water bodies based on image block convolutional neural networks. Since the classification of images is performed according to the entire image block cut during classification, the block is only associated with one label, and surface water body extraction based on image blocks only produces a rough map. Fully convolutional neural networks can capture a large amount of contextual information, obtain abstract features with higher discrimination, and improve the classification accuracy of images. Therefore, it is urgent and necessary to study the automated extraction technology of small water bodies from high spatial resolution optical remote sensing images.

[0004] Under current technical conditions, the urban area water body extraction technology based on deep learning mainly faces the following challenges: 1) Due to the limitations of remote sensing image process conditions, the spectral and texture information of water bodies are highly similar to the shadows of buildings. The extraction of water bodies in urban high-rise building dense areas is affected by the shadows of buildings, and there are a lot of misclassifications in the results of water body extraction; 2) Urban area water bodies are composed of a large number of small rivers and lakes interwoven with larger water bodies. Deep convolutional neural network is prone to cause a large number of high-frequency detail features to be lost, and there are a lot of problems of missing small water bodies in urban areas; 3) There are few studies on existing deep learning in urban area water body extraction, and there is a lack of available public urban water body image data sets. To train an excellent deep learning model for a specific research field, the amount of data is usually insufficient. Therefore, how to improve the model performance as much as possible under the premise of limited data is the primary problem to be solved in this field. In order to better utilize the pseudo-twin network to reduce the feature loss caused by the downsampling of the deep convolutional neural network and improve the extraction accuracy of small urban water bodies, the present invention designs a high spatial resolution optical remote sensing image small water body automatic extraction technology to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects in the prior art, the present invention provides a method for automatically extracting small water bodies from optical remote sensing images with high spatial resolution.

[0006] To achieve the above object, the present invention adopts the following technical solutions, including:

[0007] The method for automatically extracting small water bodies from high spatial resolution optical remote sensing images comprises the following steps:

[0008] Step 1: Preprocess the remote sensing image;

[0009] Step 2: Based on the preprocessed remote sensing images, a water body sample set is produced, and the sample data consists of remote sensing images and corresponding label images;

[0010] Step 3: Construct a deep convolutional neural network and use the water body sample set to train the network to obtain a deep learning model; the deep learning model is used to perform initial water body extraction in remote sensing images to obtain initial water body extraction results;

[0011] Step 4: Based on the building shadow removal method, the building shadows in the initial water body extraction results are eliminated to complete the accurate extraction of the water body and obtain the accurate water body extraction results.

[0012] Preferably, in step 2,

[0013] First, the target water bodies in the remote sensing image are drawn. The drawing of water bodies follows three standards: first, the water bodies that can be identified under visual conditions are directly drawn; second, the slender water bodies with a width of less than 2 pixels are drawn using line features; third, the water bodies with a width of not less than 2 pixels are drawn using surface features;

[0014] Then, define the pixel values ​​of water bodies and non-water bodies as 1 and 0 respectively in the attribute table, convert the vector file into a raster file, and fill the pixel RGB with white for water bodies and black for non-water bodies, thus generating a water body binary image of the same size as the remote sensing image, i.e., the label image;

[0015] Finally, all remote sensing images and corresponding label images are sliced ​​in batches. The remote sensing image slices obtained are consistent in size with the label image slices and correspond to each other. The sliced ​​sample data are screened and invalid sample data are eliminated to form a water body sample set.

[0016] Preferably, in step three, the deep convolutional neural network is composed of a parallel self-learning module, a multi-feature fusion module and a decoding module.

[0017] Preferably, a spatial perception subnetwork for water body extraction and a contextual semantic information perception subnetwork are established respectively, and then the pseudo-twin neural network idea is used to construct a parallel self-learning network structure of these two features, forming a multi-feature and multi-level parallel self-learning module that takes into account spatial and contextual information.

[0018] Preferably, it is characterized in that the spatial perception subnetwork consists of 2 densely connected blocks and 1 conversion layer, and the contextual semantic information perception subnetwork consists of 5 densely connected blocks and 4 conversion layers; two adjacent densely connected blocks are connected through a conversion layer, and the conversion layer is used to perform downsampling and pooling operations to extract feature information from remote sensing images;

[0019] Within a single densely connected block, the combination function F is used between layers L For nonlinear transformation, the definition formula of densely connected blocks is as follows:

[0020] X L =F L (.)=F L ([X0,X1,…,X L-1 ])

[0021] Among them, X L is the output feature map of layer L, [X0,X1,…,X L-1 ] is the concatenation of the feature maps of all layers before layer L.

[0022] Preferably, the multi-feature fusion module adopts the DenseNets network to fuse the detail features with semantic information and spatial information into features that take into account both spatial position information and semantic information, providing effective features for the decoding module.

[0023] Preferably, the decoding module performs cumulative decoding through a local attention multi-feature fusion convolution layer, and combines the comprehensive feature description of the corresponding level to achieve upsampling decoding;

[0024] Finally, the deep convolutional neural network passes through a convolution layer containing a Softmax classifier, and after a convolution operation, the output result is restored to a binary image of the water body with the same length and width as the input remote sensing image, thereby extracting the water body in the remote sensing image to be identified and obtaining the initial water body extraction result.

[0025] Preferably, in step four, the shadows of buildings in the initial water body extraction results are detected and deleted through proximity relationship analysis and buffer zone analysis, and then the water body after deleting the building shadows is transformed using adhesion to eliminate the fragmented patches of the buildings. Finally, the precise water body extraction is completed to obtain a precise water body extraction result.

[0026] Preferably, in step one, a remote sensing image of the urban area is obtained, and the remote sensing image is preprocessed, including radiation calibration, atmospheric correction, geometric correction, image fusion and image mosaicking.

[0027] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the above-mentioned method for automatically extracting small water bodies from optical remote sensing images with high spatial resolution.

[0028] The advantages of the present invention are:

[0029] The present invention discloses a technology for automatic extraction of small water bodies from high spatial resolution optical remote sensing images. Aiming at the problem that small water bodies are lost in the current intelligent extraction of water bodies in urban areas and are seriously affected by the shadows of buildings, the present invention combines the relevant theories of human visual cognition with the deep learning model, constructs a pseudo-twin neural network and a Densenet deep learning model, realizes multi-feature parallel self-learning and multi-feature integration of high spatial resolution optical remote sensing images, and reduces the influence of building shadows on the accuracy of water body extraction. In terms of application, the present invention obtains a practical technology for fully automatic classification of water bodies in urban areas of high spatial resolution optical remote sensing images, improves the accuracy of small water body extraction in urban areas, realizes complete extraction of water bodies in urban areas, and greatly improves the efficiency of water body extraction in urban areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The figure is a flow chart of the method of the present invention.

[0031] Figure 2 This is a model architecture diagram of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Example 1

[0034] Depend on Figure 1 As shown, the method for automatically extracting small water bodies from high spatial resolution optical remote sensing images includes the following steps:

[0035] Step 1: Preprocessing of high spatial resolution optical remote sensing images

[0036] Acquire high spatial resolution optical remote sensing images, preprocess the remote sensing images, mainly including radiation calibration, atmospheric correction, geometric correction, image fusion, image mosaicking, and obtain remote sensing image data sets.

[0037] Step 2: Preparation of water sample set

[0038] Referring to the image features of the ground truth data, the target water bodies in the remote sensing images are manually interpreted visually. Small water bodies usually only cover a small number of pixel values ​​in remote sensing images, and because small water bodies are usually shallow, they also have certain differences in optical features from large rivers. Therefore, when making water samples, in order to better extract small water bodies, the samples should include small water bodies such as ponds, small rivers, and canals. In the specific sample library production, in order to better include small water bodies. The drawing of water bodies follows three standards: first, if the water bodies can be identified under visual conditions, they are drawn; second, line features are used to draw slender water bodies with a width of less than 2 pixels; third, water bodies with a width of not less than 2 pixels are drawn using surface features.

[0039] Finally, the water body and non-water body pixel values ​​are defined as 1 and 0 in the attribute table, and the vector file is converted into a raster file. The pixel size is consistent with the remote sensing image, and the pixel RGB is filled, where the water body is white and the non-water body is black, to generate a water body binary image of the same size as the remote sensing image, i.e., the label image. Limited by the computer operation efficiency problem, in order to increase the training efficiency and accuracy of the neural network, it is necessary to batch slice all remote sensing images and label images, and the slices are one-fourth the size of the original image. The size of the obtained remote sensing image slices and label image slices is consistent and corresponds to each other. The sliced ​​sample data (remote sensing image slices and label image slices) are screened, and invalid sample data are eliminated. All sample data are divided into training sets and test sets in proportion to produce a water body sample set.

[0040] Step 3: Model construction

[0041] A deep convolutional neural network is constructed, and the water sample set is used for network training. The model input data is remote sensing images and label images. The deep convolutional neural network designed by the present invention is composed of a multi-feature multi-level parallel self-learning module that takes into account spatial and contextual information, a multi-feature fusion module, and an upsampling decoding module for local feature fusion.

[0042] Furthermore, a spatial perception subnetwork (SptialDenseCNNs) and a contextual semantic information perception subnetwork (ContextDenseCNNs) for water body extraction are established respectively, and then the idea of ​​pseudo-twin neural network is used to construct a parallel self-learning network structure of these two features, forming a multi-feature multi-level parallel self-learning module that takes into account spatial and contextual information. Among them, the spatial perception subnetwork consists of 2 densely connected blocks and 1 conversion layer, and the contextual semantic information perception subnetwork consists of 5 densely connected blocks and 4 conversion layers. A conversion layer is designed between two adjacent densely connected blocks for connection, and the conversion layer is used for downsampling and pooling operations to extract feature information from remote sensing images. The conversion layer designed by the present invention includes a normalization layer and a pooling layer.

[0043] Specifically, within a single densely connected block, the combination function F is used between layers L For nonlinear transformation, the definition formula of densely connected blocks is as follows:

[0044] X L =F L (.)=F L ([X0,X1,…,X L-1 ])

[0045] Among them, X L is the output feature map of layer L, [X0,X1,…,X L-1] is the concatenation of feature maps of all previous layers of layer L. Each densely connected block contains 3 layers, and the increment rate is set to 32.

[0046] The multi-feature fusion module uses the DenseNets network concept to integrate highly abstract semantic information and high-frequency detail features with spatial information into features that take into account both spatial location information and highly abstract semantic information, providing effective features for the decoding part.

[0047] The decoding module performs cumulative decoding through the local attention multi-feature fusion convolution layer, and combines the comprehensive feature description of the corresponding level to achieve upsampling decoding.

[0048] Finally, through a convolution layer containing a Softmax classifier, the convolution operation is restored to a single-channel water body binary image with the same length and width as the input remote sensing image, so as to realize semantic target recognition, extract the water body in the urban area of ​​the remote sensing image to be identified, and obtain the initial water body extraction result.

[0049] The error cost function of the network is constructed by combining the labeled images in the water sample set, and the network is iteratively solved through the error feedback mechanism. The network error cost function used is:

[0050]

[0051] Among them, {W, B} represents the weights and biases of each node in the convolutional neural network, X Ci is the i-th sample data in the input data, i.e., the remote sensing image, Y Hi Represents the label of the i-th sample data, that is, the label image, j is the total number of twin sub-networks, and j is set to 2, indicating that multi-feature parallel self-learning is performed from two aspects of the two sub-networks.

[0052] The model optimizer selects the Adam optimization algorithm.

[0053] Step 4: Building shadow removal based on contextual information

[0054] The building shadow removal method based on context information eliminates the building shadow information in the initial water body results extracted by the pseudo-twin network. The water body and building results extracted by the pseudo-twin network are respectively used to detect the building shadow information in the initially extracted water body through proximity relationship analysis, buffer zone analysis and other means, and then delete it. Then, the water body where the building shadow is removed uses adhesion transformation to eliminate the broken patches of the building, and finally completes the extraction of the accurate water body to obtain the accurate water body extraction result.

[0055] The building shadow removal method based on contextual information can eliminate the building shadow information in the initial water body extraction results extracted by the pseudo-twin network.

[0056] Example 2

[0057] In addition to the above method, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for automated extraction of small water bodies from high spatial resolution optical remote sensing images described in the above embodiment 1 of this specification.

[0058] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0059] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically extracting small water bodies from high spatial resolution optical remote sensing images, characterized in that: The following steps are involved: Step 1: Preprocess the remote sensing image; Step 2: Based on the preprocessed remote sensing images, a water body sample set is produced, and the sample data consists of remote sensing images and corresponding label images; Step 3: Construct a deep convolutional neural network and use the water body sample set to train the network to obtain a deep learning model; the deep learning model is used to perform initial water body extraction in remote sensing images to obtain initial water body extraction results; Step 4: Based on the building shadow removal method, the building shadows in the initial water body extraction results are eliminated to complete the accurate extraction of the water body and obtain the accurate water body extraction results.

2. The method for automatically extracting small water bodies from high spatial resolution optical remote sensing images according to claim 1, characterized in that: In step 2, First, the target water bodies in the remote sensing image are drawn. The drawing of water bodies follows three standards: first, the water bodies that can be identified under visual conditions are directly drawn; second, the slender water bodies with a width of less than 2 pixels are drawn using line features; third, the water bodies with a width of not less than 2 pixels are drawn using surface features; Then, define the pixel values ​​of water bodies and non-water bodies as 1 and 0 respectively in the attribute table, convert the vector file into a raster file, and fill the pixel RGB with white for water bodies and black for non-water bodies, thus generating a water body binary image of the same size as the remote sensing image, i.e., the label image; Finally, all remote sensing images and corresponding label images are sliced ​​in batches. The remote sensing image slices obtained are consistent in size with the label image slices and correspond to each other. The sliced ​​sample data are screened and invalid sample data are eliminated to form a water body sample set.

3. The method for automatically extracting small water bodies from high spatial resolution optical remote sensing images according to claim 1, characterized in that: In step three, the deep convolutional neural network consists of a parallel self-learning module, a multi-feature fusion module and a decoding module.

4. The method for automatically extracting small water bodies from high spatial resolution optical remote sensing images according to claim 3, characterized in that: A spatial perception subnetwork for water body extraction and a contextual semantic information perception subnetwork are established respectively, and then the idea of ​​pseudo-twin neural network is used to construct a parallel self-learning network structure of these two features, forming a multi-feature and multi-level parallel self-learning module that takes into account spatial and contextual information.

5. The method for automatically extracting small water bodies from high spatial resolution optical remote sensing images according to claim 4, characterized in that: The spatial perception subnetwork consists of 2 densely connected blocks and 1 conversion layer, and the contextual semantic information perception subnetwork consists of 5 densely connected blocks and 4 conversion layers; two adjacent densely connected blocks are connected through a conversion layer, and the conversion layer is used to perform downsampling and pooling operations to extract feature information from remote sensing images; In a single densely connected block, the combination function F is used between layers L For nonlinear transformation, the definition formula of densely connected blocks is as follows: X L =F L (.)=F L ([X0, X1,..., X L-1 ]) Among them, X L is the output feature map of layer L, [X0, X1, ..., X L-1 ] is the concatenation of the feature maps of all layers before layer L.

6. According to the method for automatic extraction of small water bodies from high spatial resolution optical remote sensing images of claim 4, the multi-feature fusion module adopts the DenseNets network to fuse the detail features with semantic information and spatial information into features that take into account both spatial position information and semantic information, providing effective features for the decoding module.

7. According to the method for automatic extraction of small water bodies from high spatial resolution optical remote sensing images of claim 6, the decoding module performs cumulative decoding through a local attention multi-feature fusion convolution layer, and at the same time combines the comprehensive feature description of the corresponding level to achieve upsampling decoding; Finally, the deep convolutional neural network passes through a convolution layer containing a Softmax classifier, and after a convolution operation, the output result is restored to a binary image of the water body with the same length and width as the input remote sensing image, thereby extracting the water body in the remote sensing image to be identified and obtaining the initial water body extraction result.

8. According to the method for automatically extracting small water bodies from high spatial resolution optical remote sensing images of claim 1, in step 4, the shadows of buildings in the initial water body extraction results are detected and deleted through proximity relationship analysis and buffer zone analysis, and then the water body after deleting the shadows of the buildings is subjected to adhesion transformation to eliminate the fragmented patches of the buildings, and finally the accurate water body extraction is completed to obtain the accurate water body extraction result.

9. The method for automatically extracting small water bodies from high spatial resolution optical remote sensing images according to claim 1, characterized in that: In step 1, remote sensing images of urban areas are obtained and preprocessed, including radiometric calibration, atmospheric correction, geometric correction, image fusion and image mosaicking.

10. A computer program product, characterized in that The invention comprises a computer program / instruction, which, when executed by a processor, realizes the method for automatically extracting small water bodies from optical remote sensing images with high spatial resolution as claimed in any one of claims 1 to 9.

Citation Information

Patent Citations

  • Remote-sensing image water body patch extraction method based on fully convolutional network and building shadow geometric model

    CN108830896A

  • Unmanned aerial vehicle remote sensing building extraction method based on multi-feature fusion deep learning

    CN114494868A

  • Water body remote sensing extraction method considering optical characteristic multi-scale fusion

    CN117274834A

Cited By

  • Urban river network water system polluted water body identification method and device, storage medium and equipment

    CN120689744A

  • Engineering facility target identification method and system based on multi-source fusion data

    CN120766077A