Detection method of water bridges in remote sensing images based on deep learning semantic segmentation

Through the combination of deep learning semantic segmentation and morphological processing, water and roads are extracted, and the generalization of water segmentation and high image data requirements in water bridge detection are solved, achieving high accuracy and low complexity of bridge detection.

CN118732987BActive Publication Date: 2024-11-19HUANTIAN SMART TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411225521.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-19
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

The prior art requires multi-spectral coordination for water area segmentation in water bridge detection, high imaging data requirements, and weak generalization of segmentation, resulting in the accuracy of bridge detection being affected by disturbances.

Method used

Using a deep learning-based semantic segmentation method, the LSKNet model is combined with morphological processing, waters and roads are extracted through expansion and corrosion operations, reducing the impact of disturbances, and finally extracting the bridge into a vector diagram.

Benefits of technology

It improves the accuracy and generalization ability of bridge detection, reduces the processing complexity, and avoids the impact of interferers on the segmentation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118732987B_ABST
    Figure CN118732987B_ABST
Patent Text Reader

Abstract

The present invention discloses a remote sensing image water bridge detection method based on deep learning semantic segmentation, comprising the following steps: step S1, image data collection, using the collected image data as a training set, and segmenting the collected image data through a semantic segmentation model; step S2, morphological processing of water area segmentation patches; after the image data is segmented by the semantic segmentation model, a binary extraction result of the water area is obtained, step S3, double semantic segmentation patches are combined; the extraction results of the road and the water area are intersected to obtain the binary extraction area of ​​the bridge, and then the bridge is finally extracted as a vector by processing its data. In the present invention, the deep learning model LSKNet is used for segmentation, which can adapt to remote sensing images of different regions and different periods and has strong generalization ability. The water area is expanded and eroded by morphological processing, and complex knowledge rule processing is not required, which reduces the processing complexity.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and specifically relates to a remote sensing image water bridge detection method based on deep learning semantic segmentation. Background Art

[0002] Bridges are important transportation facilities on rivers. Their structure and location information are of great significance for urban planning, traffic management, disaster assessment and infrastructure maintenance. With the development of remote sensing technology, the use of remote sensing images for bridge detection and analysis has become an effective means. Remote sensing images can provide large-scale, high-resolution surface information, making it possible to identify and monitor bridges from the air. However, due to the complexity of bridges and the surrounding environment, how to accurately and efficiently segment bridges from remote sensing images has always been a challenge in the field of remote sensing.

[0003] The main method for existing water bridge segmentation is to segment the river and land in the image, binarize the segmentation results, connect the rivers through morphological processing, and finally obtain the bridge detection results by intersecting the river and the land. However, the current scheme has the following disadvantages: 1. Limitations of traditional methods for segmenting water areas. The above scheme uses traditional methods such as FCM clustering, water area index, and spectral characteristics for water area segmentation. These methods require multi-spectral collaboration and have high requirements for image data. At the same time, due to the deviation of spectra in different regions or different periods, the generalization of water area segmentation scheme is weak.

[0004] 2. Accuracy of land segmentation. Patent CN115661660A directly treats the background after water segmentation as land. This processing method will be subject to many interferences. For example, clouds and ship hulls across the water will be treated as land. At the same time, because they cross the water, they will be treated as bridges in subsequent processing. Summary of the invention

[0005] The purpose of the present invention is to provide a method for detecting bridges on water in remote sensing images based on deep learning semantic segmentation, so as to solve the problem in the prior art proposed in the background technology that the segmentation of water areas requires multi-spectral collaboration and has high requirements for image data.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] The remote sensing image water bridge detection method based on deep learning semantic segmentation includes the following steps:

[0008] Step S1, image data collection, using the collected image data as a training set, and segmenting the collected image data through a semantic segmentation model; the collected image data includes water areas and roads;

[0009] Step S2, morphological processing of the water area segmentation pattern; after the image data is segmented by the semantic segmentation model, a binary extraction result of the water area is obtained, and the water area is first expanded to make the split water area connected, and then the erosion operation is used to restore the expanded water body to its initial size;

[0010] Step S3, combining the dual semantic segmentation patches; taking the intersection of the road and water extraction results to obtain the binary extraction area of ​​the bridge, and then finally extracting the bridge as a vector by processing its data.

[0011] According to the above technical solution, in step S1, the image data of the water area includes rivers and reservoirs, and the image data of the road includes hardened roads.

[0012] According to the above technical solution, in step S1, the semantic segmentation model adopts the LSKNet semantic segmentation model, and through the Adam optimizer, the collected image data is iterated for 12 epochs to obtain the water segmentation model and the road segmentation model.

[0013] According to the above technical solution, the expansion operation of step S2 is calculated by the following formula:

[0014]

[0015] In the formula, B represents the structural element, A represents the image to be expanded, and Z is the position vector. Z represents the translation of the structural element B in the Z position; as long as the elements contained in B overlap with the elements at the corresponding positions in A, the elements at the overlapping positions in A are marked as 1, where Represents an expansion operation.

[0016] According to the above technical solution, the corrosion operation in step S2 is specifically calculated by the following formula:

[0017]

[0018] Where B represents the structural element, A represents the image to be expanded, and Z is the position vector. represents the translation of the structural element B in the Z position; as long as the elements contained in B are completely contained in the elements at the corresponding positions in A, the elements at the overlapping positions in A are marked as 1. Surface corrosion operation.

[0019] According to the above technical solution, in step S3, the bridge is extracted as a vector, and the specific steps include opening operation to remove noise points, edge extraction and vectorization processing.

[0020] According to the above technical solution, the specific steps of opening operation to remove noise are as follows:

[0021]

[0022] In the formula, B represents the structural element, A represents the image data, represents the opening operation, Indicates the corrosion operation, Represents an expansion operation.

[0023] According to the above technical solution, edge extraction is specifically as follows: using the Canny operator to extract edges from the segmentation results to obtain edge information of the bridge; the principles of the Canny operator include Gaussian filtering, calculation of gradient amplitude and direction, non-maximum suppression, dual threshold detection and edge connection.

[0024] According to the above technical solution, the vectorization process is specifically as follows: vectorizing the result of edge extraction to obtain a final bridge vector diagram.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The deep learning model LSKNet is used for segmentation in the present invention, which can adapt to remote sensing images of different regions and different periods and has strong generalization ability. The water area is expanded and eroded by morphological processing, which does not require complex knowledge rule processing and reduces the processing complexity. The segmentation targets include water areas and hardened roads, avoiding the influence of interference on the segmentation results and improving the accuracy of bridge detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of bridge detection according to the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Embodiment 1

[0030] like Figure 1 As shown, the remote sensing image water bridge detection method based on deep learning semantic segmentation includes the following steps:

[0031] Step S1, image data collection, using the collected image data as a training set, and segmenting the collected image data through a semantic segmentation model; the collected image data includes water areas and roads;

[0032] Step S2, morphological processing of the water area segmentation pattern; after the image data is segmented by the semantic segmentation model, a binary extraction result of the water area is obtained, and the water area is first expanded to make the split water area connected, and then the erosion operation is used to restore the expanded water body to its initial size;

[0033] Step S3, combining the dual semantic segmentation patches; taking the intersection of the road and water extraction results to obtain the binary extraction area of ​​the bridge, and then finally extracting the bridge as a vector by processing its data.

[0034] The deep learning model LSKNet is used for segmentation in the present invention, which can adapt to remote sensing images of different regions and different periods and has strong generalization ability. The water area is expanded and eroded by morphological processing, which does not require complex knowledge rule processing and reduces the processing complexity. The segmentation targets include water areas and hardened roads, avoiding the influence of interference on the segmentation results and improving the accuracy of bridge detection.

[0035] Embodiment 2

[0036] This embodiment is a further refinement of the first embodiment.

[0037] In step S1 , the image data of water areas include rivers and reservoirs, and the image data of roads include hardened roads.

[0038] In step S1, the semantic segmentation model adopts the LSKNet semantic segmentation model, and uses the collected image data through the Adam optimizer to iterate 12 epochs to obtain the water segmentation model and the road segmentation model.

[0039] The expansion operation of step S2 is calculated by the following formula:

[0040]

[0041] Where B represents the structural element, A represents the image to be expanded, and Z is the position vector. represents the translation of the structural element B in the Z position; as long as the elements contained in B overlap with the elements at the corresponding positions in A, the elements at the overlapping positions in A are marked as 1, where Represents an expansion operation.

[0042] This formula means that when the structural element B is at a certain position Z, if the intersection of the structural element B and the image A is not empty (that is, at least one pixel overlaps), then the position Z will be included in the expanded result image.

[0043] The corrosion operation of step S2 is specifically calculated by the following formula:

[0044]

[0045] Where B represents the structural element, A represents the image to be expanded, and Z is the position vector. represents the translation of the structural element B in the Z position; as long as the elements contained in B are completely contained in the elements at the corresponding positions in A, the elements at the overlapping positions in A are marked as 1. Represents an erosion operation.

[0046] The result of the corrosion operation is satisfied by all This means that at these positions Z, the structural element B is completely contained in the foreground area of ​​the dilated image A.

[0047] In step S3, the bridge is extracted as a vector. The specific steps include opening operation to remove noise points, edge extraction and vectorization processing.

[0048] The specific steps of opening operation to remove noise are as follows:

[0049]

[0050] In the formula, B represents the structural element, A represents the image data, represents the opening operation, represents the corrosion operation, Represents an expansion operation.

[0051] The edge extraction is specifically as follows: using the Canny operator to extract the edge of the segmentation result to obtain the edge information of the bridge; the principles of the Canny operator include Gaussian filtering, calculating the gradient amplitude and direction, non-maximum suppression, dual threshold detection and edge connection.

[0052] 1. Gaussian filtering:

[0053] Purpose: Smooth the image and reduce the impact of noise on edge detection.

[0054] Operation: Use a Gaussian filter to perform convolution on the image. The Gaussian filter is a filter with Gaussian distribution characteristics, which can effectively reduce high-frequency noise while maintaining the overall structure of the image. Its formula is:

[0055]

[0056] Where x and y are coordinates relative to the center of the filter. x represents the horizontal distance in pixels. y represents the vertical distance in pixels. Indicates at location The weight matrix generated by the above formula is a Gaussian kernel, which is used to convolve the image to achieve smoothing or blurring effects. Represents the standard deviation of the filter, controlling the degree of smoothing.

[0057] 2. Calculate the gradient magnitude and direction:

[0058] Purpose: Detect the direction and magnitude of grayscale changes in an image, which usually correspond to edges in the image.

[0059] Operation: Calculate the image in the horizontal direction by applying the Sobel operator and vertical direction Then, calculate the gradient magnitude and direction :

[0060]

[0061] In the formula, G represents the gradient amplitude, Indicates direction.

[0062] 3. Non-maximum suppression:

[0063] Purpose: To precisely locate edges and reduce blurring effects.

[0064] Operation: Scan the image along the gradient direction, retain the pixels with the largest gradient magnitude, and suppress the pixels with non-local maximum values. This ensures that only true edge pixels are marked as edges.

[0065] 4. Dual threshold detection:

[0066] Purpose: To distinguish strong edges, weak edges and non-edge areas.

[0067] Operation: Set two thresholds, a low threshold and a high threshold. Pixels with gradient magnitudes higher than the high threshold are considered "strong edges"; pixels lower than the low threshold are ignored; pixels between the two are considered "weak edges".

[0068] 5. Edge Connection:

[0069] Purpose: Connect weak edge pixels to strong edges to form a continuous edge.

[0070] Operation: Connect weak edges to strong edges by hysteresis thresholding. If a weak edge pixel is connected to a strong edge pixel, it is retained as an edge; otherwise, it is discarded.

[0071] Through the above processing, a binary image representing the edge is obtained, that is, the value of the edge pixel is set to 1, and the value of the non-edge pixel is set to 0.

[0072] The vectorization process is as follows: vectorize the edge extraction result to obtain the final bridge vector map.

[0073] The binary image obtained above is vectorized. The detailed steps are as follows:

[0074] 1. Raster to vector:

[0075] Use contour detection algorithm (findContours function in OpenCV) to extract contours in binary images. Each contour is a series of connected pixels representing the path of an edge.

[0076] 2. Outline simplification:

[0077] The polygon approximation algorithm (Douglas-Peucker algorithm) is applied to simplify the contour. The algorithm sets a tolerance and approximates the contour to a polygon consisting of fewer vertices, thereby reducing the number of contour points and simplifying the path.

[0078] 3. Vector format conversion:

[0079] Operation: Convert the simplified polygonal contour points into vector polygons and save them using the SHP file format, which is a common vector file format in the remote sensing field.

[0080] Embodiment 3

[0081] The inventive concept of the present invention is:

[0082] Train water and road segmentation models:

[0083] Data collection: This paper uses two semantic segmentation models, so training sets of water areas and roads are required. Water areas include rivers and reservoirs, and roads are hardened roads. In order to increase the generalization ability of the model, the training set needs to use remote sensing images from different sources and regions as much as possible. In addition to annotating data yourself, it is recommended to integrate public data sets into the training set, such as: DeepGlobe Road Detection Challenge.

[0084] Model selection: Target recognition in remote sensing images often requires a wide range of background information. This paper selects LSKNet as a semantic segmentation model. Compared with general semantic segmentation models, the LSKNet model uses a decoupled convolution method and uses multiple small convolution kernels to replace large convolution kernels. This allows LSKNet to have sufficient field of view when segmenting remote sensing images and better meet the needs of remote sensing segmentation.

[0085] Training method: The model uses the Adam optimizer and iterates 12 epochs using the collected data set to obtain the water segmentation model and the road segmentation model.

[0086] Morphological processing of water segmentation patches:

[0087] After the image passes through the water area segmentation model, the binary extraction result of the water area can be obtained. The water area is first expanded to make the split water area connected, and then the erosion operation is used to restore the expanded water body to its initial size. The specific steps are as follows:

[0088] Dilation operation: Dilation operation is a type of morphological processing. By expanding each pixel in the image, it can fill small holes in the image and connect broken parts. The principle is to use the structural element B to move on the image A. As long as the elements contained in B overlap with the elements at the corresponding position in A, the elements at that position in A are marked as 1. The formula for the dilation operation is as follows:

[0089]

[0090] Erosion operation: In the formula, B represents the structural element, A represents the image to be expanded, and Z is the position vector. represents the translation of the structural element B in the Z position; as long as the elements contained in B overlap with the elements at the corresponding positions in A, the elements at the overlapping positions in A are marked as 1, where Represents an expansion operation.

[0091] The erosion operation is the inverse operation of the dilation operation. By shrinking each pixel in the image, small objects in the image can disappear and broken parts can be separated. The principle is to use the structural element B to move on the image A. As long as the elements contained in B are completely contained in the elements at the corresponding position in A, the elements at that position in A are marked as 1. The formula for the erosion operation is as follows:

[0092]

[0093] Dual semantic segmentation patch combination and post-processing:

[0094] Where B represents the structural element, A represents the image to be expanded, and Z is the position vector. represents the translation of the structural element B in the Z position; as long as the elements contained in B are completely contained in the elements at the corresponding positions in A, the elements at the overlapping positions in A are marked as 1. Represents an erosion operation.

[0095] The intersection of the road and water extraction results is taken to obtain the binary extraction area of ​​the bridge. This result is post-processed, including opening operation to remove noise points, and edge extraction is used to finally extract the bridge as a vector. The specific steps are as follows:

[0096] Opening operation to remove noise: Use opening operation to remove small noise points in the segmentation result. The opening operation formula is as follows:

[0097]

[0098] In the above figure, B represents the structural element, A represents the image data, represents the opening operation, represents the corrosion operation, Represents an expansion operation.

[0099] Edge extraction: Use the Canny operator to extract the edge of the segmentation result to obtain the edge information of the bridge. The principles of the Canny operator include Gaussian filtering, calculating gradient amplitude and direction, non-maximum suppression, dual threshold detection and edge connection.

[0100] Vectorization processing: The edge extraction results are vectorized to obtain the final bridge vector map.

[0101] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0102] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Remote sensing image water bridge detection method based on deep learning semantic segmentation, Features: The following steps are involved: Step S1, image data collection, using the collected image data as a training set, and segmenting the collected image data through a semantic segmentation model; the collected image data includes water areas and roads; the semantic segmentation model uses the Adam optimizer, and uses the collected data set to iterate 12 epochs to obtain a water area segmentation model and a road segmentation model; Step S2, morphological processing of the water area segmentation pattern; after the image data is segmented by the semantic segmentation model, a binary extraction result of the water area is obtained, and the water area is first expanded to make the split water area connected, and then the erosion operation is used to restore the expanded water body to its initial size; Step S3, combining the dual semantic segmentation patches; taking the intersection of the road and water extraction results to obtain the binary extraction area of ​​the bridge, and then finally extracting the bridge as a vector by processing its data; the expansion operation of step S2 is calculated by the following formula: In the formula, B represents the structural element, and A represents the image data; as long as the element contained in B overlaps with the element at the corresponding position in A, the element at that position in A is marked as 1; The corrosion operation of step S2 is specifically calculated by the following formula: In the formula, B represents the structural element and A represents the image data; as long as the element contained in B is completely contained in the element at the corresponding position in A, the element at that position in A is marked as 1.

2. According to the method for detecting bridges on water using remote sensing images based on deep learning semantic segmentation according to claim 1, Features: In step S1 , the image data of water areas include rivers and reservoirs, and the image data of roads include hardened roads.

3. According to the remote sensing image water bridge detection method based on deep learning semantic segmentation according to claim 1, Features: In step S1, the semantic segmentation model adopts the LSKNet semantic segmentation model, and uses the collected image data through the Adam optimizer to iterate 12 epochs to obtain the water segmentation model and the road segmentation model.

4. According to claim 1, the remote sensing image water bridge detection method based on deep learning semantic segmentation, Features: In step S3, the bridge is extracted as a vector. The specific steps include opening operation to remove noise points, edge extraction and vectorization processing.

5. According to claim 4, the remote sensing image water bridge detection method based on deep learning semantic segmentation, Features: The specific steps of opening operation to remove noise are as follows: In the formula, B represents the structural element and A represents the image data.

6. The method for detecting bridges on water using remote sensing images based on deep learning semantic segmentation according to claim 4, Features: The edge extraction is specifically as follows: using the Canny operator to extract the edge of the segmentation result to obtain the edge information of the bridge; the principles of the Canny operator include Gaussian filtering, calculating the gradient amplitude and direction, non-maximum suppression, dual threshold detection and edge connection.

7. The method for detecting bridges on water using remote sensing images based on deep learning semantic segmentation according to claim 4, Features: The vectorization process is as follows: vectorize the edge extraction result to obtain the final bridge vector map.