A flood extraction and location method based on deep learning and spatial information fusion

Abstract

The present invention provides a flood extraction and positioning method based on deep learning and spatial information fusion, which relates to the field of deep learning and spatial information technology, including: constructing a flood water body area recognition network model; the flood water body area recognition network model includes a tensor conversion module, a water body preliminary feature extraction unit, an SE attention mechanism module, a dilated convolution module and a water body segmentation decision module; after being photographed by an unmanned aerial vehicle and preprocessed, an image of the target area is obtained, which is input into the trained flood water body area recognition network model, and a flood mask image is extracted and generated from the target area image, and flood area positioning is realized. The present invention solves the multi-scale and multi-feature problems of water body objects, enhances the feature representation of water systems, and realizes accurate extraction of floods; automatically recognizes the similarity correspondence between the flood mask image and the geographic reference image, and realizes accurate extraction and positioning of floods.

Description

Technical Field

[0001] The present invention belongs to the technical field of flood area identification, and specifically relates to a flood extraction and positioning method based on deep learning and spatial information fusion. Background Art

[0002] With climate change and the frequent occurrence of extreme weather events, flood disasters pose a severe challenge to human society and the natural environment. Therefore, efficient and accurate flood monitoring and analysis are crucial for disaster warning, emergency response and recovery. Traditional flood monitoring methods, such as manual field surveys and rule-based remote sensing image analysis, are inefficient when facing large-scale disasters and cannot meet the needs of real-time monitoring and rapid response. The complexity of the terrain and the diversity of remote sensing data further increase the difficulty of flood identification and location.

[0003] In recent years, the rapid development of deep learning technology has brought new solutions to flood monitoring. However, flood extraction and positioning are affected by complex terrain, complex spatial information and semantic information contained in high-resolution remote sensing images, resulting in water body features extracted based on local information that are easily deviated from global information, and have problems such as low accuracy and efficiency in flood area identification, which cannot meet usage requirements. Summary of the invention

[0004] In view of the defects of the prior art, the present invention provides a flood extraction and positioning method based on deep learning and spatial information fusion, which can effectively solve the above problems.

[0005] The technical solution adopted by the present invention is as follows:

[0006] The present invention provides a flood extraction and positioning method based on deep learning and spatial information fusion, comprising the following steps:

[0007] Step S1, obtaining the original flood image;

[0008] Step S2, preprocessing the original flood image to obtain flood image samples, thereby forming a flood image training sample set;

[0009] Step S3, constructing a flood water area identification network model; the flood water area identification network model includes a tensor conversion module, a water body preliminary feature extraction unit, an SE attention mechanism module, a dilated convolution module and a water body segmentation decision module;

[0010] Step S4, using the flood image training sample set to train the flood water body area recognition network model to obtain a trained flood water body area recognition network model, the specific training method is:

[0011] Step S4.1, the flood image sample is in RGB format, with a height of H, a width of W, and a number of channels of C;

[0012] Step S4.2, using formula (1), convert the flood image sample into a tensor I through the tensor conversion module:

[0013] I∈R H×W×C (1)

[0014] Step S4.3, input tensor I into the water body preliminary feature extraction unit to extract the water body feature map Among them, water body characteristic map The height is H', the width is W', and the number of channels is C';

[0015] Step S4.4, convert the water body characteristic map Input the SE attention mechanism module; using formula (2), the SE attention mechanism module converts the water body feature map Each channel k of is compressed into the description vector z of this channel k k :

[0016]

[0017] Where: k = 1, 2, ..., C', represents the water body characteristic map The kth channel of Representative water body characteristics map The pixel value of the pixel at the i-th row and j-th column in channel k of ;

[0018] Step S4.5, using formula (3), the SE attention mechanism module dynamically adjusts the description vector z of channel k according to the global features k , the description vector z of channel k k Dynamically adjusted to describe vector s k , description vector s k That is the self-attention weight of channel k:

[0019] s k =σ(W2δ(W1z k )) (3)

[0020] Where: W1 represents the first fully connected layer; δ() represents the ReLU activation function; W2 represents the second fully connected layer; σ() represents the Sigmoid activation function;

[0021] Step S4.6, using formula (4), the self-attention weight s k Effect on water body characteristics map Adjust water feature map The self-attention weights of each channel are used to obtain the water feature map adjusted by the SE attention mechanism.

[0022]

[0023] Step S4.7, using formula (5), adjust the water feature map after the SE attention mechanism Perform dilation convolution operation to obtain the water body feature map F dilated :

[0024]

[0025] Where: d represents the expansion rate; Γ dilated () represents the dilated convolution operation;

[0026] Step S4.8, the water body feature map F dilated Input to the multi-scale fusion module, the multi-scale fusion module outputs the multi-scale fused water body feature map F miniUNet :

[0027] Step S4.9: The multi-scale fused water feature map F miniUNet Input to the decision convolution layer, the decision convolution layer converts the water body feature map F through 1×1 convolution miniUNet Convert to single-channel output and get the flood mask image F mask :

[0028] Step S4.10, setting the loss function and the self-decreasing learning rate;

[0029] The loss value is designed through the loss function; if the training termination condition is not reached, the learning rate is dynamically reduced and adjusted by using the learning rate decay method, and the training process of the flood water body area recognition network model is repeated until the training termination condition is reached, and the trained flood water body area recognition network model is obtained;

[0030] Step S5: In actual use, the target area image is obtained by taking pictures with a drone and preprocessing it, and then input into the flood water area recognition network model trained in step S4 to extract and generate a flood mask image F from the target area image. mask ;

[0031] Step S6: Using geocoding and image matching algorithms, the flood mask image F mask Matching with spatial information makes the flood mask image F mask Each pixel point has precise geographic coordinates, which enables spatial information fusion and obtains flood area location information.

[0032] Preferably, step S4.8 is specifically:

[0033] Step S4.8.1, the multi-scale fusion module includes an encoder, a skip connection layer and a decoder; the encoder includes the first convolution layer, the 1-1 activation function, the second convolution layer and the 1-2 activation function; the decoder includes the first transposed convolution layer, the 2-1 activation function, the second transposed convolution layer and the 2-2 activation function;

[0034] Step S4.8.2, the water body characteristic map F dilated The first convolutional layer is used to extract water features and output a water feature map F with reduced scale. dilated (1), water body characteristic map F dilated (1) After being activated by the 1-1 activation function, the image is input to the second convolutional layer for water body feature extraction, and the output is a water body feature map F with a further reduced scale. dilated (2); Water body characteristic map F dilated (2) after being activated by the 1-2 activation function, the result is input into the decoder;

[0035] The first transposed convolutional layer of the decoder performs an upsampling operation on the input water body feature map to increase the scale of the water body feature map to obtain the water body feature map F dilated (3), water body characteristic map F dilated (3) After activation by the 2-1 activation function, it is compared with the water body feature map F dilated (1) Splicing is performed to obtain the first spliced ​​water body feature map; the first spliced ​​water body feature map is upsampled by the second transposed convolution layer to increase the scale of the water body feature map to obtain the water body feature map F dilated (4), water body characteristic map F dilated (4) After activation by the 2-2 activation function, it is compared with the water body feature map F dilated After splicing, we can get the multi-scale fused water feature map F miniUNet .

[0036] Preferably, step S4.9 is specifically as follows:

[0037] Step S4.9.1, using a convolution kernel with a height and width of M and N respectively, and using formula (6), the water body feature map F miniUNet Perform convolution operation to obtain the water body feature map F after convolution operation. conv :

[0038]

[0039] Where: w(m,n) represents the weight of the convolution kernel at the mth row and nth column position; F miniUNet (xm,yn) represents the water body feature map F miniUNetThe pixel value at position (xm,yn) after offset by position (m,n); F conv (x,y) represents the water body characteristic map F miniUNet After the convolution operation is performed at the (x, y) position in the middle, the pixel value output; the pixel values ​​at each position form the water body feature map F conv ;

[0040] Step S4.9.2, using formula (7), the water body characteristic map F conv Perform a variant operation of the maximum value pooling to obtain the water body feature map F after the variant operation max :

[0041]

[0042] Meaning: From the water body characteristic map F conv The (x, y) position of the water body is offset to the (m, n) position, and the area formed takes the maximum pixel value as the water body feature map F max The pixel value at the (x,y) position;

[0043] Step S4.9.3, water body feature map F max Perform average weighting and maximum weighting processing, and output the average weighted value g and the maximum weighted value h. The method is:

[0044] Make water feature map F max The pixel value of each (x, y) position is multiplied by the weight to obtain the weighted pixel value; then the weighted pixel values ​​of all positions are averaged to obtain the average weighted value g;

[0045] Compare the weighted pixel values ​​at each position, take the maximum weighted pixel value, and then multiply it by the preset magnification coefficient K to obtain the maximum weighted value h;

[0046] Step S4.9.4, using formula (8), obtain the combined feature map F 组合 :

[0047] F 组合 =g*F max +h*F max (8)

[0048] Step S4.9.5, calculate the combined feature map F 组合 The histogram is normalized to a probability distribution. By traversing all possible thresholds T, the intra-class variance is calculated and the threshold T that minimizes the intra-class variance is found as the optimal threshold.

[0049] Use the best threshold to combine feature maps F 组合 Perform binarization to obtain a binary mask image, which is the flood mask image F mask; Wherein, in the flood mask image F mask In the figure, the flood range is marked, 1 represents the flood area and 0 represents the non-flood area.

[0050] Preferably, step S6 specifically comprises:

[0051] Step S6.1, control point coordinate correction:

[0052] In the flood mask image F mask In the example, the center point of the image is selected as the control point P0; the geographic coordinates (E0, W0) and image coordinates (x0, y0) of the control point P0 are read; where E0 and W0 represent the longitude and latitude coordinates of the control point P0; x0 and y0 represent the control point P0 in the flood mask image F. mask The pixel position coordinates in ;

[0053] Obtain a georeferenced image, locate a reference point C0 corresponding to the control point P0 in the georeferenced image, and read the geographic coordinates of the reference point C0;

[0054] The geographic coordinates of the control point P0 are corrected using the geographic coordinates of the reference point C0 (E0, W0), thereby obtaining the corrected geographic coordinates of the control point P0, expressed as (E [0] ,W [0] );

[0055] Step S6.2, after the geographic coordinates of the control point P0 are corrected, a feature extraction algorithm is used to extract the flood mask image F mask Extract multiple feature points Q, and describe the feature point Q in the form of a feature vector Q(m,θ); where m represents the magnitude of the gradient of the feature point Q; θ represents the direction of the gradient of the feature point Q;

[0056] Step S6.3, for the flood mask image F mask For each feature point Q extracted, the associated area is preliminarily determined in the georeferenced image; the distance between the feature point Q and each pixel in the associated area in the georeferenced image is calculated using a distance matching algorithm, and the pixel with the smallest distance is selected, denoted as R, as the pixel point that is located and matches the feature point Q. Therefore, the feature point Q and the pixel point R form a feature point pair;

[0057] Therefore, for the flood mask image F mask For each feature point Q in , the corresponding pixel point R is determined in the georeferenced image to form a feature point pair;

[0058] Step S6.4, based on the matched feature point pairs, an affine transformation model is used to transform the flood mask image F by maintaining the linearity and scale invariance of the straight line. maskPerform geometric correction to obtain the corrected flood mask image F mask ;

[0059] Step S6.5, corrected flood mask image F mask Each pixel point except the control point P0 is represented as pixel point P1, and its geographic coordinates (E1, W1) are determined by the following method, where E1 and W1 represent the longitude and latitude coordinates of the control point P1;

[0060] Step S6.5.1, determine the pixel point P1 in the corrected flood mask image F mask The image coordinates in are (x1, y1); using formula (9), the geographic offset (ΔE, ΔW) of pixel point P1 relative to control point P0 is obtained:

[0061]

[0062] Where: pixelSize represents the OGD value, that is, the actual geographical distance represented by each pixel value;

[0063] Step S6.5.2, using formula (10), obtain the geographic longitude and latitude coordinates (E1, W1) of pixel point P1:

[0064]

[0065] Preferably, in step S6.2, a feature extraction algorithm is used to extract the flood mask image F mask Extract multiple feature points Q, specifically:

[0066] Step S6.2.1, setting the Gaussian kernel Among them, σ represents the scale of the Gaussian kernel; k represents the scale magnification factor;

[0067] Step S6.2.2, set k to 0, 1, 2, ..., q in sequence; use formula (11) to obtain the flood mask image F mask Gaussian blurred images at different scales

[0068]

[0069] Among them: F mask (x,y) represents the flood mask image F mask The pixel value at position (x,y);

[0070] Step S6.2.3, therefore, when k is equal to 0, 1, 2, ..., q in sequence, the corresponding Gaussian blurred image is obtained

[0071] make and Subtract and get the first difference image F difference1 ;

[0072] make and Subtract and get the second difference image F difference2 ;

[0073] And so on

[0074] make and Subtract and get the qth difference image F difference q ;

[0075] For the first differential image F difference1 , the second difference image F difference2 ,…,qth difference image F difference q Pixel comparison is performed, and significantly identical pixels are identified as feature points Q.

[0076] Preferably, in the feature vector Q(m,θ) of the feature point Q, the gradient magnitude m and the gradient direction θ are calculated as follows:

[0077] When k is equal to 0, a Gaussian blurred image is obtained. Gaussian Blur Image and the flood mask image F mask The scale is the same, so for the feature point Q, let it be in the flood mask image F mask The coordinates in (x Q ,y Q ), using formula (12), we can get m and θ of feature point Q:

[0078]

[0079] Thus, the feature vector Q(m,θ) of the feature point Q is obtained.

[0080] The flood extraction and positioning method based on deep learning and spatial information fusion provided by the present invention has the following advantages:

[0081] The present invention solves the multi-scale and multi-feature problems of water objects, enhances the feature representation of water systems, and realizes accurate extraction of floods; it automatically identifies the similar correspondence between flood mask images and geographic reference images, and realizes accurate extraction and positioning of floods. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0083] Figure 1 A schematic diagram of a flow chart of a flood extraction and positioning method based on deep learning and spatial information fusion provided by the present invention;

[0084] Figure 2 A structural diagram of a flood water area identification network model provided by the present invention;

[0085] Figure 3 The main network structure diagram of the coupled deep dilated convolution provided by the present invention;

[0086] Figure 4 A network structure diagram of a custom structure layer provided by the present invention;

[0087] Figure 5 A schematic diagram of data enhancement for an original image provided by the present invention;

[0088] Figure 6 A comparison diagram of the extraction results of the flood area in Fankou area of ​​Ezhou City provided by the present invention;

[0089] Figure 7 This is the segmentation result diagram provided by the present invention. DETAILED DESCRIPTION

[0090] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0091] The invention discloses a flood extraction and positioning method based on deep learning and spatial information fusion, and relates to the field of deep learning and spatial information technology. The main innovation of the invention is that the invention optimizes the feature extraction framework of the Mask R-CNN network model, introduces the SE (Squeeze-and-Excitation) attention mechanism and dilated convolution, and designs an innovative custom layer for water body feature enhancement (WaterBodyEnhanceLayer), effectively enhances the feature representation of the water system, and further optimizes the loss function and sets the self-decreasing learning rate to achieve the precision optimization of the Mask R-CNN network model; solves the multi-scale and multi-feature problems of water objects, enhances the feature representation of the water system, and achieves accurate extraction of floods; improves the prediction segmentation of the model network, matches and fuses the predicted flood mask image with the spatial information through the geocoding and image matching algorithm, automatically identifies the similar correspondence between the flood mask image and the georeferenced image, and assigns specific geographic coordinates to each pixel point, thereby generating a Shapefile vector file reflecting the flood inundation range. This invention not only realizes the accurate extraction and positioning of floods, but also provides strong technical support for the study of water environment problems and water resource security under complex backgrounds.

[0092] See also Figure 1 The present invention provides a flood extraction and positioning method based on deep learning and spatial information fusion, comprising the following steps:

[0093] Step S1, obtaining original flood images; for example, using a drone to obtain a variety of high-resolution flood images;

[0094] Step S2, preprocessing the original flood image to obtain flood image samples, thereby forming a flood image training sample set;

[0095] Specifically, in order to improve the accuracy of the model in identifying and locating flood areas in images, enhance the robustness of the model, and adapt to the needs of network input, high-resolution flood images are preprocessed, including:

[0096] Step S2.1, orthorectify the distortion of the high-resolution flood image caused by the shooting angle (tilt angle) and height change;

[0097] Step S2.2, based on the spatial position information of the DEM data of the study area, each pixel in the image is adjusted according to the terrain changes to compensate for the distortion caused by the terrain;

[0098] Step S2.3, after distortion correction, perform data enhancement operations such as cropping and rotation on the image;

[0099] The specific process is as follows: select multiple easily identifiable points (intersections, building corners, etc.) in the image as landmarks, and record the image coordinates and corresponding geographic coordinates of the landmarks, and use this data to estimate the parameters of the calibration model; based on the spatial position relationship of the DEM data of the study area, adjust each pixel in the image according to the terrain change data provided by the DEM data to compensate for the distortion caused by the terrain;

[0100] A mathematical model is established to describe the relationship between image coordinates and ground coordinates, and the model parameters are estimated through mathematical optimization methods to ensure the highest correction accuracy. The specific formula is as follows:

[0101] X=a0+a1x'+a2y'+a3x' 2 +a4x'y'+a5y'2 (13)

[0102] Y=b0+b1x'+b2y'+b3x' 2 +b4x'y'+b5y' 2 (14)

[0103] Among them, X and Y represent the latitude and longitude coordinates of the ground, a i and b i are model parameters, x′ and y′ represent the image pixel coordinates, and are estimated by minimizing the difference between the geographical locations of the GCPs and the model predicted locations.

[0104]

[0105]

[0106] Where Δx and Δy represent the differences between the geographical locations of the GCPs and the model-predicted locations that are minimized, f is the focal length of the camera, (X0, Y0, Z0) is the position of the camera, (x0, y0) is the optical center of the image, and Z is the ground elevation of the (X, Y) coordinates provided by the digital elevation image.

[0107] After the image is distorted, data enhancement operations such as cropping and rotation are performed on the image data to expand the data set capacity.

[0108] The Labelme tool is used to annotate the image dataset. The resulting dataset includes original images in JPG format and one-to-one corresponding label files in JSON format.

[0109] Based on the above preprocessing results, a high-resolution flood image dataset suitable for network model training and testing is constructed, and divided into training dataset, validation dataset, and test dataset. For example, the flood sample dataset is divided into training dataset and test dataset in a ratio of 9:1, and an independent test dataset is prepared to evaluate the performance and segmentation ability of the model.

[0110] Step S3, constructing a flood water area identification network model; the flood water area identification network model includes a tensor conversion module, a water body preliminary feature extraction unit, an SE attention mechanism module, a dilated convolution module and a water body segmentation decision module;

[0111] In the present invention, the feature extraction framework of the Mask R-CNN model is improved, and deep dilated convolution is introduced and a custom layer specially designed for water feature enhancement is designed to construct a Mask R-CNN network model for pixel-level segmentation of water areas in high-resolution flood images. In the present invention, the residual between input and output is learned through jump connections between structural layers, which can solve the problems of gradient vanishing and gradient exploding in the model training process.

[0112] Step S4, using the flood image training sample set to train the flood water body area recognition network model to obtain a trained flood water body area recognition network model, the specific training method is:

[0113] Step S4.1, the flood image sample is in RGB format, with a height of H, a width of W, and a number of channels of C;

[0114] Step S4.2, using formula (1), the flood image sample is converted into a tensor I through a tensor conversion module:

[0115] I∈R H×W×C (1)

[0116] Step S4.3, input tensor I into the water body preliminary feature extraction unit to extract the water body feature map Among them, water body characteristic map The height is H', the width is W', and the number of channels is C';

[0117] The preliminary water feature extraction unit may include multiple convolution layers and pooling layers. Each convolution operation is calculated by the convolution kernel and the input water feature map. The output water feature map is nonlinearly operated through the ReLU activation function and downsampled through the pooling operation when necessary. Through the above steps, the high-level semantic features of the high-resolution flood image are effectively extracted to ensure the stability and effectiveness of the model during the training process.

[0118] Step S4.4, convert the water body characteristic map Input the SE attention mechanism module; using formula (2), the SE attention mechanism module converts the water body feature map Each channel k of is compressed into the description vector z of the channel k k :

[0119]

[0120] Where: k = 1, 2, ..., C', represents the water body characteristic map The kth channel of Representative water body characteristics map The pixel value of the pixel at the i-th row and j-th column in channel k of ;

[0121] Specifically, the SE attention mechanism module uses formula (2) to analyze the characteristic channels of flood water bodies through global average pooling (GAP) and compresses its feature map into a channel-level description vector z k , thereby creating a spatial dimension compressed vector. The global average pooling operation averages each channel of the flood feature map to obtain a description vector z that describes the global information of each channel k .

[0122] Step S4.5, using formula (3), the SE attention mechanism module dynamically adjusts the description vector z of channel k according to the global features k , the description vector z of channel k k Dynamically adjusted to describe vector s k , description vector s k That is the self-attention weight of channel k:

[0123] s k =σ(W2δ(W1z k )) (3)

[0124] Among them: W1 represents the first fully connected layer; δ() represents the ReLU activation function; W2 represents the second fully connected layer; σ() represents the Sigmoid activation function; through the operation of these two fully connected layers, the weight of each feature channel can be dynamically adjusted according to the global features, thereby realizing the attention mechanism.

[0125] Specifically, through formula (3), the weights of each feature channel in the flood feature map are dynamically adjusted through two fully connected layers W1 and W2 and the activation function to realize the attention mechanism. In the SE attention mechanism, the fully connected layers W1 and W2 play a core role. By transforming and reshaping the channel weights of the feature map, the network can focus on the feature channels with more information. First, through a compression (dimensionality reduction) process, the original channel description vector z k The fully connected layer W1 has a weight matrix whose size is Where C' is the number of input channels and r is the reduction ratio, which is used to control the degree of dimensionality reduction. This dimensionality reduction process can be regarded as information compression of feature channels, reducing the parameters and computational complexity of the model while retaining key information. After the first fully connected layer and ReLu activation function, δ(W1z c) contains the channel information after dimensionality reduction. The second fully connected layer W2 is used to map this dimensionality reduction matrix back to the original number of channels C', and its weight matrix size is The importance score of each channel is obtained through the mapping of this layer.

[0126] Step S4.6, using formula (4), the self-attention weight s k Effect on water body characteristics map Adjust water feature map The self-attention weights of each channel are used to obtain the water feature map adjusted by the SE attention mechanism. This step is implemented by element-wise multiplication, that is, the self-attention weight s k and water body characteristics map Each channel corresponds to element-wise multiplication.

[0127]

[0128] Through this step, the introduction of the SE attention mechanism can effectively enhance the model's ability to identify the importance of different feature channels, improve the accuracy and effectiveness of feature extraction, and ultimately improve the overall performance of flood extraction and positioning.

[0129] Step S4.7, dilated convolution expands the receptive field of the convolution kernel by introducing intervals, which can capture more contextual information, while maintaining the depth of the network and reducing the number of parameters and calculations. Specifically, by introducing dilated convolution, the receptive field can be expanded without increasing the convolution kernel parameters, enhancing the network's ability to extract global and local features, thereby improving the effect of flood extraction and positioning.

[0130] Therefore, formula (5) is used to adjust the water body feature map after the SE attention mechanism is adjusted. Perform dilation convolution operation to obtain the water body feature map F dilated :

[0131]

[0132] Where: d represents the expansion rate; Γ dilated () represents the dilated convolution operation;

[0133] Specifically, by introducing dilated convolution, the receptive field is expanded without increasing the number of parameters and calculations by introducing intervals between the elements in the convolution kernel. Specifically, the role of dilated convolution is to introduce a dilation rate d in the standard convolution operation and insert holes between the convolution kernel elements, so that the convolution kernel can cover a wider range of input features and capture more contextual information. By dilated convolution, the network's ability to extract multi-scale features is effectively enhanced, which not only maintains the network depth, but also reduces the number of parameters and calculations, realizes multi-scale feature extraction, and enables the network to better adapt to flood feature recognition tasks in complex scenarios.

[0134] Step S4.8, the water body feature map F dilated The multi-scale fusion module is an innovative custom layer of WaterBodyEnhanceLayer in the present invention, which makes more accurate segmentation decisions at the pixel level. The multi-scale fusion module outputs a multi-scale fused water body feature map F miniUNet :

[0135] Step S4.8 is specifically:

[0136] Step S4.8.1, the multi-scale fusion module includes an encoder, a skip connection layer and a decoder; the encoder includes the first convolution layer, the 1-1 activation function, the second convolution layer and the 1-2 activation function; the decoder includes the first transposed convolution layer, the 2-1 activation function, the second transposed convolution layer and the 2-2 activation function;

[0137] Step S4.8.2, the water body characteristic map F dilated The first convolutional layer is used to extract water features and output a water feature map F with reduced scale. dilated (1), water body characteristic map F dilated (1) After being activated by the 1-1 activation function, the image is input to the second convolutional layer for water body feature extraction, and the output is a water body feature map F with a further reduced scale. dilated (2); Water body characteristic map F dilated (2) after being activated by the 1-2 activation function, the result is input into the decoder;

[0138] The first transposed convolutional layer of the decoder performs an upsampling operation on the input water body feature map to increase the scale of the water body feature map to obtain the water body feature map F dilated (3), water body characteristic map F dilated (3) After activation by the 2-1 activation function, it is compared with the water body feature map F dilated (1) Splicing is performed to obtain the first spliced ​​water body feature map; the first spliced ​​water body feature map is upsampled by the second transposed convolution layer to increase the scale of the water body feature map to obtain the water body feature map F dilated(4), water body characteristic map F dilated (4) After activation by the 2-2 activation function, it is compared with the water body feature map F dilated After splicing, we can get the multi-scale fused water feature map F miniUNet .

[0139] For example, the encoder consists of two consecutive convolutional layers, each followed by a ReLU activation function, using a 3×3 convolution kernel to keep the input and output sizes consistent. Specifically, the encoder first performs a convolution operation on the input feature map, introduces nonlinearity through the ReLU activation function, and then performs a second convolution operation to extract deep features. The decoder upsamples through a transposed convolution layer to increase the size of the feature map. The transposed convolution layer can not only restore the spatial resolution of the feature map, but also retain high-level semantic information through deconvolution operations. The output of each layer of the decoder is spliced ​​(skip connection) with the output feature map of the corresponding layer of the encoder to synthesize the original and learned features. This splicing operation can combine high-resolution low-level features with low-resolution high-level features to enhance the multi-scale fusion capability of features.

[0140] Through the design of this step, the features are multi-scale fused and refined. The specific process is as follows: First, the feature map passes through the two-layer convolution of the encoder to extract the encoded feature map; then, the encoded feature map is upsampled through the transposed convolution layer of the decoder, and the output of each upsampling layer is spliced ​​with the corresponding encoder layer output. Finally, through the multi-layer upsampling and splicing operation of the decoder, the refined water body feature map F is generated. miniUNet .

[0141] Step S4.9: The multi-scale fused water feature map F miniUNet The input is sent to the decision convolution layer, which is used as the terminal layer of the network. The decision convolution layer transforms the water body feature map F through 1×1 convolution. miniUNet Convert to single-channel output and get the flood mask image F mask , by compressing the high-dimensional feature map into a single dimension, it provides an accurate segmentation decision for each pixel to determine whether it belongs to the flood range.

[0142] Step S4.9 is specifically as follows:

[0143] Step S4.9.1, using a convolution kernel with a height and width of M and N respectively, and using formula (6), the water body feature map F miniUNet Perform convolution operation to obtain the water body feature map F after convolution operation. conv :

[0144]

[0145] Where: w(m,n) represents the weight of the convolution kernel at the mth row and nth column position; F miniUNet (xm,yn) represents the water body feature map F miniUNet The pixel value at position (xm,yn) after offset by position (m,n); F conv (x,y) represents the water body characteristic map F miniUNet After the convolution operation is performed at the (x, y) position in the middle, the pixel value output; the pixel values ​​at each position form the water body feature map F conv ;

[0146] Step S4.9.2, using formula (7), the water body characteristic map F conv Perform a variant operation of the maximum value pooling to obtain the water body feature map F after the variant operation max :

[0147]

[0148] Meaning: From the water body characteristic map F conv The (x, y) position of the water body is offset to the (m, n) position, and the area formed takes the maximum pixel value as the water body feature map F max The pixel value at the (x,y) position;

[0149] Step S4.9.3, water body feature map F max Perform average weighting and maximum weighting processing, and output the average weighted value g and the maximum weighted value h. The method is:

[0150] Make water feature map F max The pixel value of each (x, y) position is multiplied by the weight to obtain the weighted pixel value; then the weighted pixel values ​​of all positions are averaged to obtain the average weighted value g;

[0151] Compare the weighted pixel values ​​at each position, take the maximum weighted pixel value, and then multiply it by the preset magnification coefficient K to obtain the maximum weighted value h;

[0152] The average weighted value g is used to represent the average weighted value of all pixels and is used to evaluate the probability that the pixel belongs to the flood area. The maximum weighted value h is used to emphasize the features that are most likely to belong to the flood area. Formula (8) combines the average weighted value g and the maximum weighted value h to generate the combined feature map F. 组合 .

[0153] Step S4.9.4, using formula (8), obtain the combined feature map F 组合 :

[0154] F 组合 =g*F max +h*F max (8)

[0155] Step S4.9.5, calculate the combined feature map F 组合 The histogram is normalized to a probability distribution. By traversing all possible thresholds T, the intra-class variance is calculated and the threshold T that minimizes the intra-class variance is found as the optimal threshold.

[0156] Use the best threshold to combine feature maps F 组合 Perform binarization to obtain a binary mask image, which is the flood mask image F mask ; Wherein, in the flood mask image F mask In the , the flood range is marked, 1 represents the flood area, and 0 represents the non-flood area. The flood mask image clearly marks the flood range, providing a solid foundation for subsequent flood positioning.

[0157] Step S4.10, setting the loss function and the self-decreasing learning rate;

[0158] The loss value is designed through the loss function; if the training termination condition is not reached, the learning rate is dynamically reduced and adjusted by using the learning rate decay method, and the training process of the flood water body area recognition network model is repeated until the training termination condition is reached, and the trained flood water body area recognition network model is obtained;

[0159] In the present invention, the loss function is optimized and the self-decreasing learning rate is set. First, the loss function is optimized to enhance the performance of the model. The loss function consists of classification loss, bounding box regression loss and mask loss. By weighted combination of loss terms, the influence of each part on model training can be balanced and the overall segmentation accuracy can be improved. Specifically, the classification loss is used to evaluate the accuracy of target detection, the bounding box regression loss is used to evaluate the accuracy of target positioning, and the mask loss is used to evaluate the quality of instance segmentation.

[0160] Set the self-decreasing learning rate strategy and use the learning rate decay method to dynamically adjust the learning rate during the training process, so as to converge quickly in the early stage of training and stabilize the model parameters in the later stage of training. In the specific implementation, the learning rate decay method based on performance indicators is adopted, and when the loss on the validation set no longer decreases, the learning rate is automatically reduced to avoid the model falling into the local optimal solution.

[0161] The flood water area recognition network model was trained based on the image sample dataset and its parameters were tuned. During the training process, the enhanced dataset (including data augmentation techniques such as flipping, rotation, and cropping) was used to improve the generalization ability of the model. The model performance was evaluated through methods such as cross-validation, and model parameters such as batch size, optimizer type, and hyperparameter settings were adjusted based on the validation results.

[0162] Finally, after multiple rounds of training and tuning, the optimal flood water area recognition network model was obtained. This model can efficiently perform instance segmentation in various complex scenarios, has high accuracy and robustness, and meets the practical application needs of flood extraction and positioning.

[0163] Step S5: In actual use, the target area image is obtained by taking pictures with a drone and preprocessing it, and then input into the flood water area recognition network model trained in step S4 to extract and generate a flood mask image F from the target area image. mask ;

[0164] In this step, during preprocessing, in order to achieve accurate alignment of pixel points and geographic coordinates in the subsequent steps, corrections for photographic objective lens distortion and atmospheric refraction are performed.

[0165] Step S6: Using geocoding and image matching algorithms, the flood mask image F mask Matching with spatial information makes the flood mask image F mask Each pixel point has precise geographic coordinates, which enables spatial information fusion and obtains flood area location information.

[0166] Step S6 is specifically as follows:

[0167] Step S6.1, control point coordinate correction:

[0168] In the flood mask image F mask In the example, the center point of the image is selected as the control point P0; the geographic coordinates (E0, W0) and image coordinates (x0, y0) of the control point P0 are read; where E0 and W0 represent the longitude and latitude coordinates of the control point P0; x0 and y0 represent the control point P0 in the flood mask image F. mask Specifically, the flood mask image F mask In the image, only the center point has geographic coordinates, and other pixel points do not have geographic coordinates.

[0169] Obtain a georeferenced image, locate a reference point C0 corresponding to the control point P0 in the georeferenced image, and read the geographic coordinates of the reference point C0;

[0170] Using the geographic coordinates of the reference point C0, the geographic coordinates of the control point P0 are corrected using the radiometric variation (E0, W0), and the control point P0 in the flood mask image is aligned with the corresponding point in the geographic reference image to obtain the corrected geographic coordinates of the control point P0, expressed as (E [0] ,W [0] );

[0171] Step S6.2, after the geographic coordinates of the control point P0 are corrected, a feature extraction algorithm is used to extract the flood mask image Fmask Extract multiple feature points Q, and describe the feature point Q in the form of a feature vector Q(m,θ); where m represents the magnitude of the gradient of the feature point Q; θ represents the direction of the gradient of the feature point Q;

[0172] In the feature vector Q(m,θ) of feature point Q, the calculation method of gradient magnitude m and gradient direction θ is:

[0173] When k is equal to 0, a Gaussian blurred image is obtained. Gaussian Blur Image and the flood mask image F mask The scale is the same, so for the feature point Q, let it be in the flood mask image F mask The coordinates in (x Q ,y Q ), using formula (12), we can get m and θ of feature point Q:

[0174]

[0175] Thus, the feature vector Q(m,θ) of the feature point Q is obtained.

[0176] In step S6.2, a feature extraction algorithm is used to extract the flood mask image F mask Extract multiple feature points Q, specifically:

[0177] Step S6.2.1, setting the Gaussian kernel Among them, σ represents the scale of the Gaussian kernel; k represents the scale magnification factor;

[0178] Step S6.2.2, set k to 0, 1, 2, ..., q in sequence; use formula (11) to obtain the flood mask image F mask Gaussian blurred images at different scales

[0179]

[0180] Among them: F mask (x,y) represents the flood mask image F mask The pixel value at position (x,y);

[0181] Step S6.2.3, therefore, when k is equal to 0, 1, 2, ..., q in sequence, the corresponding Gaussian blurred image is obtained

[0182] make and Subtract and get the first difference image F difference1 ;

[0183] make and Subtract and get the second difference image F difference2 ;

[0184] And so on

[0185] make and Subtract and get the qth difference image F difference q ;

[0186] For the first differential image F difference1 , the second difference image F difference2 ,…,qth difference image F difference q Pixel comparison is performed, and significantly identical pixels are identified as feature points Q.

[0187] Step S6.3, for the flood mask image F mask For each feature point Q extracted, the associated area is preliminarily determined in the georeferenced image; the distance between the feature point Q and each pixel in the associated area in the georeferenced image is calculated using a distance matching algorithm, and the pixel with the smallest distance is selected, denoted as R, as the pixel point that is located and matches the feature point Q. Therefore, the feature point Q and the pixel point R form a feature point pair;

[0188] Therefore, for the flood mask image F mask For each feature point Q in , the corresponding pixel point R is determined in the georeferenced image to form a feature point pair;

[0189] Step S6.4, based on the matched feature point pairs, an affine transformation model is used to transform the flood mask image F by maintaining the linearity and scale invariance of the straight line. mask Perform geometric correction to eliminate perspective differences or other distortions and obtain the corrected flood mask image F mask ;

[0190] Step S6.5, corrected flood mask image F mask Each pixel point except the control point P0 is represented as pixel point P1, and its geographic coordinates (E1, W1) are determined by the following method, where E1 and W1 represent the longitude and latitude coordinates of the control point P1;

[0191] Step S6.5.1, determine the pixel point P1 in the corrected flood mask image F mask The image coordinates in are (x1, y1); using formula (9), the geographic offset (ΔE, ΔW) of pixel point P1 relative to control point P0 is obtained:

[0192]

[0193] Where: pixelSize represents the OGD value, that is, the actual geographical distance represented by each pixel value;

[0194] Step S6.5.2, using formula (10), obtain the geographic longitude and latitude coordinates (E1, W1) of pixel point P1:

[0195]

[0196] Through the above steps, the fusion of flood mask image and spatial information is realized, and the corresponding geographic coordinates are matched for each pixel point to ensure that the segmentation result has geographic reference significance, providing accurate spatial data support for flood extraction and positioning.

[0197] Therefore, in the present invention, after the training of the optimal flood water body area recognition network model is completed, the target area image to be identified is input to generate a segmented flood mask image. Then, through the image matching algorithm, the similarity and correspondence between the mask image and the geographic reference image are automatically identified, and specific geographic coordinates are assigned to each pixel point of the mask image. According to the above steps, a Shapefile vector file reflecting the flood inundation range is generated, and finally the accurate extraction and positioning of the flood is achieved. Through this method, the accurate extraction and positioning of the flood is achieved, providing accurate and reliable spatial data support for flood disaster management and emergency response. The generated Shapefile file can be further analyzed and visualized in various GIS software to help decision makers formulate and implement effective countermeasures in a timely manner.

[0198] A specific embodiment is described below:

[0199] Reference Figure 1-Figure 7 The present invention provides a flood extraction and positioning method based on deep learning and spatial information fusion, comprising the following steps:

[0200] Step S1, using a drone to acquire a high-resolution image.

[0201] The research area of ​​the embodiment of the present invention is focused on the Fankou area of ​​Ezhou City, Hubei Province. Aerial images are taken by a Pegasus D2000S drone equipped with a D-OP3000 oblique focusing lens. The captured images are in JPG format with a pixel size of 6000×4000.

[0202] Step S2, preprocessing the high-resolution image.

[0203] Select multiple easily identifiable points in the image (intersections, building corners, etc.), record the image coordinates and corresponding geographic coordinates of these points, and use this data to estimate the parameters of the correction model; based on the spatial position relationship of the DEM data of the study area, adjust each pixel in the image according to the terrain change data provided by the DEM data to compensate for the distortion caused by the terrain; establish a mathematical model to describe the relationship between image coordinates and ground coordinates, and estimate the model parameters through mathematical optimization methods to ensure the highest correction accuracy.

[0204] Based on the above preprocessing results, a model image dataset is constructed and divided into training dataset, validation dataset and test dataset.

[0205] The present invention constructs an image dataset RiverDataset taken by a Pegasus drone. The original image size is 6000×4000 pixels. Taking into account the memory limitations and Batch_size setting issues during GPU training under the Pytorch deep learning framework, the experiment performs data enhancement processing on the image, such as cropping, rotation and other preprocessing operations, and finally adjusts the image size to 1200×800 pixels. Since a large amount of data is usually required for training a flood water area recognition network model, insufficient data may reduce the robustness and segmentation accuracy of the network, and even lead to overfitting. Therefore, in order to enhance the generalization ability of the model, data enhancement is performed on the original image. The specific operations are as follows: Figure 5 As shown. Figure 5 In the figure, a is the original image taken by drone; b is the bright preprocessing operation image; c is the rotation preprocessing operation image; d is the Gaussian preprocessing operation image;

[0206] After processing, the dataset contains 3656 images of 1200×800 pixels. 1000 images were randomly selected as the dataset and divided into training set and validation set in a ratio of 9:1 to ensure no overlap between the two. In addition, an independent test set was prepared to evaluate the model performance and segmentation ability.

[0207] Step 3: construct a flood water area recognition network model, which is an improved Mask R-CNN network model.

[0208] Step S4, training the flood water area recognition network model;

[0209] The main architecture of the flood water area recognition network model consists of two main parts: first, the Region Proposal Network (RPN) is used as a model to generate candidate object regions, extract the possible locations of objects on the feature map through a sliding window, and generate a series of rectangular proposals; second, a classification and boundary regression network structure processes the candidate object regions and can perform pixel mask prediction in parallel. This branch shares the feature map of the fully convolutional network (FCN) with the detection branch. The network structure is shown in Figure 2. Figure 2 shown.

[0210] The network model introduces the RoAligen technology, which uses the bilinear interpolation method to accurately calculate the features of each region of interest (RoI), effectively solving the quantization error problem introduced by the previous RoIPooling method in object localization and mask prediction, thereby improving the overall performance of the model while maintaining spatial accuracy.

[0211] The present invention improves the feature extraction framework of the Mask R-CNN network model, uses the ResNet50 network as its feature extraction framework, and introduces the SE (Squeeze-and-Excitation) attention mechanism into the residual module to solve the problem of information loss in deep networks. The SE attention mechanism module is placed after the convolution layer, and the feature channels are analyzed through global average pooling (GAP) to create a spatial dimension compression vector, and then the weights of each feature channel are dynamically adjusted through the fully connected layer and the sigmoid activation function.

[0212] When extracting the flood range contour, the spectral information of water bodies may be similar to environmental factors such as wetlands and shadows, and the diversity of floods in width, shape, and size poses a challenge to model training. Therefore, deep dilated convolution is added to enhance the receptive field and resolution of the model. By introducing a spatial dilation rate between standard convolution kernel elements, the network depth is maintained while the number of parameters and computation is reduced.

[0213] The present invention innovatively designs a custom layer called WaterBodyEnhanceLayer specifically for enhancing the expression of water body features, which includes an encoder and a decoder, simplifying the complexity of the model. The encoder consists of two consecutive convolutional layers, each of which is followed by a ReLU activation function, and a 3×3 convolution kernel is used to keep the input and output sizes consistent. Downsampling is achieved through the maximum pooling layer, which not only reduces the size but also expands the receptive field of the feature. The decoder upsamples through a transposed convolutional layer to increase the size of the feature map, and concatenates the encoder output with the decoder result to synthesize the original and learned features.

[0214] The encoder-decoder architecture and skip connections are used to effectively capture contextual information in the image, especially the features of specific categories such as water bodies. On this basis, a decision convolution layer is further constructed as the terminal layer of the network. This layer converts the enhanced multi-channel feature map into a single-channel output through 1×1 convolution to form a decision map. This enables the model to provide accurate segmentation decisions for each pixel to determine whether it belongs to a water body.

[0215] To achieve this process, use formula (6) through the weight w(m,n) and the input image F miniUNet (xm,yn) to calculate the output feature map F conv (x, y). The key features in the image are captured and refined through continuous convolution operations. Using formula (7), the maximum value operation is performed during the feature extraction process, which helps to extract the most significant features in different areas of the image. Especially when dealing with complex water features, this operation can highlight the key features of the water body. Through the processing of the average weighted value g and the maximum weighted value h, the weighted average helps to balance and integrate the features, while the weighted maximum is used to identify the most critical part of the extracted features, which is crucial for ultimately determining whether the pixel point belongs to the water area.

[0216]

[0217]

[0218] The flood water area recognition network model is trained according to the image sample data set, and its parameters are optimized to obtain the optimal flood water area recognition network model.

[0219] The present invention uses the cross entropy loss function shown in formula (17) to optimize the classification task to accurately distinguish objects of different categories, including the background. At the same time, the Smooth L1 loss function shown in formula (18) is used to calculate the bounding box regression loss, which helps the model to more accurately determine the location of the object in the image, including the center point coordinates and size of the bounding box. In addition, the model also calculates the mask loss through the pixel-level binary cross entropy loss function shown in formula (19). The key to this step is to train the model to distinguish the foreground and background of the object at the pixel level, ensuring that the generated object mask has a high degree of accuracy. The specific formula is:

[0220]

[0221]

[0222]

[0223] Where C is the number of categories, y i is the true label (1 if category i is correct, 0 otherwise), is the probability of belonging to category i predicted by the model, which measures the difference g between the probability distribution predicted by the model and the distribution of the true label; v i and are the true and predicted bounding box parameters (center point coordinates x, y, width w, height h), where H×W is the size of the mask, y i is the true value of pixel i (1 for foreground, 0 for background), is the probability of belonging to the foreground predicted by the model, which encourages the model to accurately distinguish the shapes of objects at the pixel level.

[0224] In the model of the present invention, the classification loss L cls , bounding box regression loss L bbox and the mask loss L mask By combining the words together in a weighted summation, the total loss L can be expressed as formula (20):

[0225] L=λ cls L cls +λ bbox L bbox +λ mask L mask (20)

[0226] Among them, λ cls ,λ bbox and λ mask is the weight coefficient of each loss. By combining multiple loss functions, the model can be optimized in multiple tasks, performing classification, localization, and segmentation at the same time.

[0227] In the present invention, accuracy evaluation indicators are used to evaluate network performance. Multiple evaluation indicators such as average precision (AP) and mean average precision (mAP) under different intersection-over-union (IoU) thresholds are used to comprehensively evaluate the performance of the Mask R-CNN model enhanced by SE attention mechanism and dilated convolution for target detection and instance segmentation of river and lake systems. The expressions are shown in formulas (21) and (22).

[0228]

[0229] Among them, AP tis the average precision at a specific IoU threshold t, and N is the number of IoU thresholds. This study pays special attention to the following forms of AP: AP@loU = 0.50, AP@loU = 0.75, and the average precision when the IoU is taken every 0.05 from 0.50 to 0.95 (AP@loU = 0.50:0.95).

[0230]

[0231] Where N represents the number of remote sensing images, TN represents the number of pixels classified as background, TP represents the number of pixels belonging to the lake category in the predicted image, FN represents the number of pixels misclassified as background, and FP represents the number of pixels misclassified as lake category.

[0232] Step 5: Input the high-resolution image taken by the drone, extract the flood information through the optimal flood water area recognition network model and generate a flood mask image;

[0233] Water body extraction is performed on the improved flood water area recognition network model of the present invention, namely the Mask R-CNN network model, the unimproved Mask R-CNN network model, the U-Net model based on the VGG16 network, and the U-Net model based on the ResNet50 network, and the results are compared.

[0234] Figure 6 The comparison of the extraction results of the flood area in Fankou area of ​​Ezhou City is shown. Among them, a is the original aerial image taken by a drone; b is the mask image extracted by the improved Mask R-CNN model; c is the mask image extracted by the unimproved Mask R-CNN model; it can be seen that the four methods are all able to extract the flood range. However, the depth of the standard residual network structure is relatively shallow, resulting in serious loss of feature information in the river edge area, which affects the segmentation accuracy, especially the information hole problem when dealing with interferences such as tree shade and algae. The improved model of the present invention enhances the perception ability of local object information of water bodies by introducing SE attention mechanism and custom layer (WBE-Layer), effectively avoids the occurrence of information holes, and protects the integrity of objects during the segmentation process. Further analysis Figure 7 The segmentation results show that Figure 7In the figure, a is the original image taken by drone; b is the mask image extracted by the improved Mask R-CNN model; c is the mask image extracted by the U-Net model based on the VGG16 network; d is the mask image extracted by the U-Net model based on the ResNet50 network. The U-Net model based on the VGG16 and ResNet networks has high accuracy in processing river and lake water segmentation. However, when faced with complex water environments, such as the presence of interference factors such as tree shade, bridge shadows, algae or water ripples, the model has a more obvious information hole phenomenon. On the contrary, the improved flood water area recognition network model of the present invention does not have similar problems when processing these complex situations. Through the deep expansion residual convolution technology, the model solves the problem of narrow receptive field of ordinary convolution kernels, so that it can better understand a wider range of contextual information in the image. The application of this technology not only protects the integrity of the outline of the river and lake system, but also significantly improves the processing ability of complex water environments. In addition, the U-Net model fails to effectively eliminate the negative impact of the grid effect on the feature information, resulting in the extracted feature information destroying the consistency of local information. The lack of a structure similar to a local area information perception module makes the model's perception of image context information insufficient. In complex water environments, especially when precise boundary information extraction is required, this makes it impossible to effectively eliminate the influence of interference.

[0235] The accuracy of flood range extraction of the present invention is compared with other methods, and the results are shown in Tables 1 and 2.

[0236] Table 1 Accuracy comparison between the improved Mask R-CNN model and the unimproved Mask R-CNN model

[0237]

[0238] Table 2 Accuracy comparison between the improved Mask R-CNN model and the U-Net model

[0239]

[0240] Step 6: Match the flood mask image with the spatial information through geocoding and image matching algorithms, assign specific geographic coordinates to each pixel point, and realize spatial information fusion.

[0241] According to the output flood mask image and geographic reference image, the coordinates of the center point of the mask image (114.843223704, 30.4269277479) are selected as the control point, and the coordinates of the control point are corrected using affine transformation; key feature points are extracted from the mask image based on the image matching algorithm, and the feature extraction algorithm is used to identify the feature points in the image and generate feature vectors describing this point; special descriptors are used to find corresponding feature point pairs in the mask image and the geographic reference image based on the distance matching algorithm to achieve accurate correspondence of the coordinate points of the mask image; based on the matched feature points, the mask image is geometrically corrected using the affine transformation model to eliminate perspective differences or other distortions;

[0242] According to the geographical location information of the research area, the coordinate system used by the mask image is determined to be the projection coordinate system CGCS20003 Degree GK CM114E. The transformation relationship obtained by image matching and geometric correction is used to assign specific geographical coordinates to each pixel point of the mask image. Based on the coordinates of the center point of the mask image, for any pixel point (x, y) in the image, the offset of the pixel point relative to the center point of the mask image is calculated.

[0243] Calculate the geographic coordinates of the pixel point based on the offset and the center point geographic coordinates, and match the corresponding geographic coordinates for each pixel point;

[0244] Based on the above algorithm, after the training of the flood water area recognition network model of the present invention is completed, by inputting the image to be predicted, the similarity and correspondence between the mask image and the geographic reference image are automatically identified, and specific geographic coordinates are assigned to each pixel point of the mask image, thereby realizing the fusion of spatial information.

[0245] Step 7: Generate a shapefile vector file reflecting the flood inundation range according to the above steps to achieve accurate extraction and positioning of the flood. This achievement can be directly applied to the GIS system to provide strong data support for the formulation of river and lake management policies, urban safety and sustainable development of water resources.

[0246] The present invention proposes a flood extraction and positioning method based on deep learning and spatial information fusion. Through deep learning, flood features in images are automatically identified, which greatly improves the accuracy and processing speed of flood monitoring. The feature extraction framework of the flood water area recognition network model is optimized, and the generalization ability of the model and the accuracy of prediction segmentation are improved by further optimizing the loss function and setting the self-decreasing learning rate, making it more suitable for the special needs of flood monitoring. Based on the application of geocoding and image matching algorithms, the present invention can accurately align the flood mask image with the geographic reference image, assign exact geographic coordinates to each pixel point, and realize the precise geographic positioning of the flood area. On this basis, a Shapefile vector file reflecting the flood inundation range is generated. This achievement can be directly applied to the GIS system, providing strong data support for the formulation of river and lake management policies, urban security and sustainable development of water resources. Compared with traditional methods, the present invention is more adaptable to environments with complex terrain and rich remote sensing image information, can effectively process and identify flood information under complex backgrounds, and provide more accurate flood monitoring data.

[0247] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention.

Claims

1. A flood extraction and positioning method based on deep learning and spatial information fusion, characterized in that: The following steps are involved: Step S1, obtaining the original flood image; Step S2, preprocessing the original flood image to obtain flood image samples, thereby forming a flood image training sample set; Step S3, constructing a flood water area identification network model; the flood water area identification network model includes a tensor conversion module, a water body preliminary feature extraction unit, an SE attention mechanism module, a dilated convolution module and a water body segmentation decision module; Step S4, using the flood image training sample set to train the flood water body area recognition network model to obtain a trained flood water body area recognition network model, the specific training method is: Step S4.1, the flood image sample is in RGB format, with a height of H, a width of W, and a number of channels of C; Step S4.2, using formula (1), convert the flood image sample into a tensor I through the tensor conversion module: I∈R H×W×C (1) Step S4.3: Input tensor I into the preliminary water feature extraction unit to extract the water feature map Among them, water body characteristic map The height is H', the width is W', and the number of channels is C'; Step S4.4, convert the water body characteristic map Input the SE attention mechanism module; using formula (2), the SE attention mechanism module converts the water body feature map Each channel k of is compressed into the description vector z of this channel k k : Where: k = 1, 2, ..., C', represents the water body characteristic map The kth channel of Representative water body characteristics map The pixel value of the pixel at the i-th row and j-th column in channel k of ; Step S4.5, using formula (3), the SE attention mechanism module dynamically adjusts the description vector z of channel k according to the global features k , the description vector z of channel k k Dynamically adjusted to describe vector s k , description vector s k That is the self-attention weight of channel k: s k =σ(W2δ(W1z k )) (3) Where: W1 represents the first fully connected layer; δ() represents the ReLU activation function; W2 represents the second fully connected layer; σ() represents the Sigmoid activation function; Step S4.6, using formula (4), the self-attention weight s k Effect on water body characteristics map Adjust water feature map The self-attention weights of each channel are used to obtain the water feature map adjusted by the SE attention mechanism. Step S4.7, using formula (5), adjust the water feature map after the SE attention mechanism Perform dilation convolution operation to obtain the water body feature map F dilated : Where: d represents the expansion rate; Γ dilated () represents the dilated convolution operation; Step S4.8, the water body feature map F dilated Input to the multi-scale fusion module, the multi-scale fusion module outputs the multi-scale fused water body feature map F miniUNet : Step S4.9: The multi-scale fused water feature map F miniUNet Input to the decision convolution layer, the decision convolution layer converts the water body feature map F through 1×1 convolution miniUNet Convert to single-channel output and get the flood mask image F mask : Step S4.10, setting the loss function and the self-decreasing learning rate; The loss value is designed through the loss function; if the training termination condition is not reached, the learning rate is dynamically reduced and adjusted by using the learning rate decay method, and the training process of the flood water body area recognition network model is cyclically carried out until the training termination condition is reached, and the trained flood water body area recognition network model is obtained; Step S5: In actual use, the target area image is obtained by taking pictures with a drone and preprocessing it, and then input into the flood water area recognition network model trained in step S4 to extract and generate a flood mask image F from the target area image. mask ; Step S6: Using geocoding and image matching algorithms, the flood mask image F mask Matching with spatial information makes the flood mask image F mask Each pixel point has precise geographic coordinates, which enables spatial information fusion and obtains flood area location information.

2. The flood extraction and positioning method based on deep learning and spatial information fusion according to claim 1 is characterized in that: Step S4.8 is specifically: Step S4.8.1, the multi-scale fusion module includes an encoder, a skip connection layer and a decoder; the encoder includes the first convolution layer, the 1-1 activation function, the second convolution layer and the 1-2 activation function; the decoder includes the first transposed convolution layer, the 2-1 activation function, the second transposed convolution layer and the 2-2 activation function; Step S4.8.2, the water body characteristic map F dilated The first convolutional layer is used to extract water features and output a water feature map F with reduced scale. dilated (1), water body characteristic map F dilated (1) After being activated by the 1-1 activation function, the image is input to the second convolutional layer for water body feature extraction, and the output is a water body feature map F with a further reduced scale. dilated (2); Water body characteristic map F dilated (2) after being activated by the 1-2 activation function, the result is input into the decoder; The first transposed convolutional layer of the decoder performs an upsampling operation on the input water body feature map to increase the scale of the water body feature map to obtain the water body feature map F dilated (3), water body characteristic map F dilated (3) After activation by the 2-1 activation function, it is compared with the water body feature map F dilated (1) Splicing is performed to obtain the first spliced ​​water body feature map; the first spliced ​​water body feature map is upsampled by the second transposed convolution layer to increase the scale of the water body feature map to obtain the water body feature map F dilated (4), water body characteristic map F dilated (4) After activation by the 2-2 activation function, it is compared with the water body feature map F dilated After splicing, we can get the multi-scale fused water feature map F miniUNet .

3. The flood extraction and positioning method based on deep learning and spatial information fusion according to claim 1 is characterized in that: Step S4.9 is specifically as follows: Step S4.9.1, using a convolution kernel with a height and width of M and N respectively, and using formula (6), the water body feature map F miniUNet Perform convolution operation to obtain the water body feature map F after convolution operation. conv : Where: w(m,n) represents the weight of the convolution kernel at the mth row and nth column position; F miniUNet (xm,yn) represents the water body feature map F miniUNet The pixel value at position (xm,yn) after offset by position (m,n); F conv (x,y) represents the water body characteristic map F miniUNet After the convolution operation is performed at the (x, y) position in the middle, the pixel value output; the pixel values ​​at each position form the water body feature map F conv ; Step S4.9.2, using formula (7), the water body characteristic map F conv Perform a variant operation of the maximum value pooling to obtain the water body feature map F after the variant operation max : Meaning: From the water body characteristic map F conv The (x, y) position of the water body is offset to the (m, n) position, and the area formed takes the maximum pixel value as the water body feature map F max The pixel value at the (x,y) position; Step S4.9.3, water body feature map F max Perform average weighting and maximum weighting processing, and output the average weighted value g and the maximum weighted value h. The method is: Make water feature map F max The pixel value of each (x, y) position is multiplied by the weight to obtain the weighted pixel value; then the weighted pixel values ​​of all positions are averaged to obtain the average weighted value g; Compare the weighted pixel values ​​at each position, take the maximum weighted pixel value, and then multiply it by the preset magnification coefficient K to obtain the maximum weighted value h; Step S4.9.4, using formula (8), obtain the combined feature map F 组合 : F 组合 =g*F max +h*F max (8) Step S4.9.5, calculate the combined feature map F 组合 The histogram is normalized to a probability distribution. By traversing all possible thresholds T, the intra-class variance is calculated and the threshold T that minimizes the intra-class variance is found as the optimal threshold. Use the best threshold to combine feature maps F 组合 Perform binarization to obtain a binary mask image, which is the flood mask image F mask ; Wherein, in the flood mask image F mask In the figure, the flood range is marked, 1 represents the flood area and 0 represents the non-flood area.

4. The flood extraction and positioning method based on deep learning and spatial information fusion according to claim 1 is characterized in that: Step S6 is specifically as follows: Step S6.1, control point coordinate correction: In the flood mask image F mask In the example, the center point of the image is selected as the control point P0; the geographic coordinates (E0, W0) and image coordinates (x0, y0) of the control point P0 are read; where E0 and W0 represent the longitude and latitude coordinates of the control point P0; x0 and y0 represent the control point P0 in the flood mask image F. mask The pixel position coordinates in ; Obtain a georeferenced image, locate a reference point C0 corresponding to the control point P0 in the georeferenced image, and read the geographic coordinates of the reference point C0; The geographic coordinates of the control point P0 are corrected using the geographic coordinates of the reference point C0 (E0, W0), thereby obtaining the corrected geographic coordinates of the control point P0, expressed as (E [0] ,W [0] ); Step S6.2, after the geographic coordinates of the control point P0 are corrected, a feature extraction algorithm is used to extract the flood mask image F mask Extract multiple feature points Q, and describe the feature point Q in the form of a feature vector Q(m,θ); where m represents the magnitude of the gradient of the feature point Q; θ represents the direction of the gradient of the feature point Q; Step S6.3, for the flood mask image F mask For each feature point Q extracted, the associated area is preliminarily determined in the georeferenced image; the distance between the feature point Q and each pixel in the associated area in the georeferenced image is calculated using a distance matching algorithm, and the pixel with the smallest distance is selected, denoted as R, as the pixel point that is located and matches the feature point Q. Therefore, the feature point Q and the pixel point R form a feature point pair; Therefore, for the flood mask image F mask For each feature point Q in the image, the corresponding pixel point R is determined in the georeferenced image to form a feature point pair; Step S6.4, based on the matched feature point pairs, an affine transformation model is used to transform the flood mask image F by maintaining the linearity and scale invariance of the straight line. mask Perform geometric correction to obtain the corrected flood mask image F mask ; Step S6.5, corrected flood mask image F mask Each pixel point except the control point P0 is represented as pixel point P1, and its geographic coordinates (E1, W1) are determined by the following method, where E1 and W1 represent the longitude and latitude coordinates of the control point P1; Step S6.5.1, determine the pixel point P1 in the corrected flood mask image F mask The image coordinates in are (x1, y1); using formula (9), the geographic offset (ΔE, ΔW) of pixel point P1 relative to control point P0 is obtained: Where: pixelSize represents the OGD value, that is, the actual geographical distance represented by each pixel value; Step S6.5.2, using formula (10), obtain the geographic longitude and latitude coordinates (E1, W1) of pixel point P1:

5. The flood extraction and positioning method based on deep learning and spatial information fusion according to claim 4 is characterized in that: In step S6.2, a feature extraction algorithm is used to extract the flood mask image F mask Extract multiple feature points Q, specifically: Step S6.2.1, setting the Gaussian kernel Among them, σ represents the scale of the Gaussian kernel; k represents the scale magnification factor; Step S6.2.2, set k to 0, 1, 2, ..., q in sequence; use formula (11) to obtain the flood mask image F mask Gaussian blurred images at different scales Among them: F mask (x,y) represents the flood mask image F mask The pixel value at position (x,y); Step S6.2.3, therefore, when k is equal to 0, 1, 2, ..., q in sequence, the corresponding Gaussian blurred image is obtained make and Subtract and get the first difference image F difference1 ; make and Subtract and get the second difference image F difference2 ; And so on make and Subtract and get the qth difference image F differenceq ; For the first differential image F difference1 , the second difference image F difference2 ,…,qth difference image F differenceq Pixel comparison is performed, and significantly identical pixels are identified as feature points Q.

6. The flood extraction and positioning method based on deep learning and spatial information fusion according to claim 4 is characterized in that: In the feature vector Q(m,θ) of feature point Q, the calculation method of gradient magnitude m and gradient direction θ is: When k is equal to 0, a Gaussian blurred image is obtained. Gaussian Blur Image and the flood mask image F mask The scale is the same, so for the feature point Q, let it be in the flood mask image F mask The coordinates in (x Q ,y Q ), using formula (12), we can get m and θ of feature point Q: Thus, the feature vector Q(m,θ) of the feature point Q is obtained.

Images