Remote sensing image road breakpoint repairing method, equipment and medium

Abstract

The invention discloses a remote sensing image road breakpoint repairing method and device and a medium. The method comprises the steps that a road image is converted into a binary image, a connected region is calculated, a threshold value is set to remove part of the connected region, skeletonization operation is conducted on the optimized binary image, and a road is converted into a road skeleton with the single-pixel width. Extracting breakpoints of all roads, and obtaining breakpoint parameter values; on the basis of the matching value between each group of breakpoint pairs, connecting the breakpoint pairs of which the matching values exceed a threshold value, and obtaining a perfect road skeleton; road width repairing is conducted on the perfected road skeleton, and a repaired road is obtained. According to the method, the repaired road is regularized, so that the width of the repaired road is ensured to conform to a physical rule, the condition of road boundary distortion or excessive connection caused by mismatching is avoided, and the repaired road boundary is more accurate and natural by optimizing the smoothness and regularity of the repaired path; and the risk of misjudgment is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image road break point repair, and particularly to a method, device and medium for repairing remote sensing image road break points. Background Art

[0002] With the rapid development of remote sensing technology and the wide application of high-resolution satellite images, remote sensing images play an important role in fields such as road extraction, urban planning, and resource monitoring. However, due to factors such as sensor imaging conditions, ground object occlusion (such as trees and building shadows), and complex road structures during the image acquisition process, road information in high-resolution remote sensing images is prone to problems such as breaks and discontinuities, posing great challenges to road extraction and repair. In recent years, remote sensing image road extraction technology based on deep learning has gradually become a research hotspot and achieved certain results, but there are still the following technical difficulties:

[0003] (1) The problem of road breaks has not been effectively solved: Traditional deep learning methods mainly focus on the overall road extraction results and do not effectively repair the broken areas caused by occlusion, noise, etc., resulting in discontinuous road networks extracted, affecting the integrity and practicality of the roads. (2) There is a lack of effective modeling of road boundary and break point features: When dealing with road boundaries, existing methods often ignore the direction information and spatial features of the roads and fail to effectively capture the direction vectors and distance relationships of the break points, resulting in low matching accuracy during the road repair process and prone to introducing misjudgments and incorrect connections. (3) Insufficient regularization of roads: The extracted road networks have problems such as irregular boundaries and inconsistent widths, affecting the geometric accuracy and visual effects of the roads. Especially in high-resolution images, this problem is more prominent, restricting practical applications.

[0004] For example: The invention application with the application number 202310274763.4 discloses a road extraction method and device applied to remote sensing digital images. The road extraction method of this application can not only make full use of the road detail information presented in high-resolution images but also remove the interference caused by occlusion and shadows brought by high resolution. However, this application also has the following problems: This application uses a deep learning method, mainly focusing on the overall road extraction results and not effectively repairing the broken areas caused by occlusion, noise, etc., resulting in discontinuous road networks extracted, affecting the integrity and practicality of the roads.

[0005] Therefore, an intelligent repair method combining break point direction vectors, distance constraints, and road width information is needed. By precisely matching and optimizing the break points, the problem of road breaks is solved, and the extraction results are regularized, improving the continuity and accuracy of the road extraction results. Summary of the Invention

[0006] In view of the above problems, the purpose of the present invention is to provide a method, device and medium for repairing road breakpoints in remote sensing images. By combining the direction vector of breakpoints, the distance between breakpoints, and road width information, the broken roads in remote sensing images can be effectively repaired and the road boundaries can be optimized.

[0007] An embodiment of the present invention provides a method, device and medium for repairing road breakpoints in remote sensing images.

[0008] First aspect: A method for repairing road breakpoints in remote sensing images, comprising:

[0009] S1. Convert the road image into a binary image, calculate the size of the connected regions of each road, set a threshold to remove some connected regions, and obtain an optimized binary image;

[0010] S2. Perform a skeletonization operation on the optimized binary image to convert the road into a road skeleton with a single-pixel width;

[0011] S3. Perform breakpoint recognition processing on the road skeleton, extract the breakpoints of all roads, and obtain breakpoint parameter values;

[0012] S4. Based on the matching values between each pair of breakpoints, connect the pairs of breakpoints whose matching values exceed the threshold to obtain a refined road skeleton;

[0013] S5. Perform road width repair on the refined road skeleton to obtain the repaired road.

[0014] Further, the S1 includes the steps of:

[0015] S11. Convert the road grayscale image into a road binary image, with the formula:

[0016]

[0017] where road_image(x, y) is the road grayscale image, 1 is the pixel value of the road area, and 0 is the pixel value of the background area.

[0018] S12. Use the connected component analysis algorithm to calculate the size of the connected regions of each connected road region in the binary image.

[0019] S13. Set a connected region size threshold min_size according to prior knowledge, remove the connected regions with an area smaller than min_size, and obtain an optimized binary image.

[0020] Further, the S2 includes:

[0021] The optimized binary image is processed by an iterative method based on pixel neighborhoods to eliminate non-skeleton pixel points until only the center line of the road area remains, and all parts of the road area are reduced to a single-pixel-width line structure to obtain the road skeleton.

[0022] Further, in step S3, the breakpoint recognition process for the road skeleton adopts an eight-neighborhood search method, and the formula is expressed as:

[0023] The formula is expressed as:

[0024]

[0025] Among them, (0, 0) is the pixel position of the road skeleton.

[0026] Further, the breakpoint parameter values in step S3 include: a backtracking coordinate group, a direction vector group, and a vertical vector group, where:

[0027] Use the depth-first search algorithm to trace along the skeleton from the breakpoint until a bifurcation point or a condition is met. During this backtracking process, record the coordinate points of the backtracking of the road skeleton pixels and form the backtracking coordinate group corresponding to the breakpoint; perform fitting normalization on the backtracking coordinate group to obtain the direction vector group, and orthogonalize the direction vector group to obtain the vertical vector group.

[0028] Further, the matching value is calculated based on the distance and angle difference between each pair of breakpoints, and the formula is:

[0029]

[0030] Among them, d is the Euclidean distance between the two breakpoints of the breakpoint pair, θ is the angle of the direction vector between the two breakpoints of the breakpoint pair, w d and w θ are the weight values of the distance and the angle respectively.

[0031] Further, step S4 also includes: checking whether each breakpoint meets the threshold to form a breakpoint pair. If not, it is marked as an isolated point; for the isolated point, determine the angle and distance of the extension of the isolated point, and extend the isolated point along the road skeleton until the maximum extension distance is reached or a road pixel is encountered.

[0032] Further, step S5 includes: for each pixel point on the road skeleton segment, calculate the width of the pixel point on the vertical vector of the grayscale road image, and take the average value of the road widths corresponding to all pixel points on the skeleton segment as the repair width corresponding to the breakpoint. According to the repair width, perform width repair on the improved road skeleton to obtain the repaired road.

[0033] Second aspect: An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the steps of the method provided in the first aspect are implemented.

[0034] Third aspect: A non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0035] Advantages of the present invention:

[0036] 1. Based on the road repair method using breakpoint direction vectors and distance information, the present invention can maintain the continuity of the road skeleton by accurately calculating the direction and distance between breakpoints, significantly improving the accuracy of road extraction.

[0037] 2. By introducing road width information and regularizing the repaired road, the present invention not only ensures that the width of the repaired road conforms to physical laws, but also avoids road boundary distortion or over-connection caused by mis-matching. By optimizing the smoothness and regularity of the repair path, the boundary of the repaired road is more accurate and natural, reducing the risk of misjudgment.

[0038] 3. When dealing with roads in high-resolution remote sensing images, the method of the present invention can cope with the interference caused by occlusion, noise, and complex terrain to road extraction, ensuring good repair results even in complex environments. Compared with traditional methods, the present invention can complete road repair in a shorter time, especially when dealing with large-scale high-resolution images, showing higher efficiency. This makes the method very suitable for automated road extraction systems and large-scale remote sensing data processing tasks, and high-quality road repair results can be obtained in a short time.

[0039] 4. The repaired road of the present invention not only has enhanced continuity, smoother and more regular boundaries, but also has a high consistency with real road data. By comparing with labeled data, the repair method of the present invention can accurately restore the true shape of the road, effectively improving the overall accuracy of the road extraction system. Description of the drawings

[0040] Figure 1 is a schematic flow chart of the repair method of the present invention;

[0041] Figure 2 is a schematic principle flow chart of the repair method of the present invention;

[0042] Figure 3 is a schematic structural diagram of the repair device of the present invention;

[0043] Figure 4This is the binary image of the road before repair according to the present invention;

[0044] Figure 5 This is the binary image of the road after repair according to the present invention;

[0045] Figure 6 This is the code diagram of the eight-neighborhood search method implemented according to the present invention;

[0046] Figure 7 This is the schematic structural diagram of the electronic device according to the present invention. Detailed implementation manners

[0047] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, in which the same or similar symbols represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0048] Traditional methods based on deep learning mainly focus on the overall road extraction results and do not effectively repair the fractured areas caused by occlusion, noise, etc., resulting in discontinuous road networks extracted, which affects the integrity and usability of the roads; the matching accuracy is relatively low during the road repair process, and misjudgment and incorrect connection are easily introduced.

[0049] In view of the above problems, the present invention provides a method for repairing road breakpoints in remote sensing images, Figure 1 This is the flowchart of the method for repairing road breakpoints in remote sensing images provided by the embodiment of the present invention, Figure 1 This is the principle flowchart of the method for repairing road breakpoints in remote sensing images provided by the embodiment of the present invention. The method includes:

[0050] S1. Convert the road image into a binary image, calculate the size of the connected regions of each road, set a threshold to remove some connected regions, and obtain an optimized binary image.

[0051] As Figure 3 shown, convert the road image into a binary image. The white area is the road area with a pixel value of 1, and the black area is the background area with a pixel value of 0.

[0052] By setting a fixed or adaptive threshold, convert the grayscale image road_image(x, y) into a binary image, where the pixel value of the road area is set to 1 and the pixel value of the background area is set to 0. The formula is expressed as:

[0053]

[0054] After binarization, the connected components analysis algorithm is used to calculate the connected regions in the image, mark each connected road region, and calculate the size of each connected region. Then, according to prior knowledge, a threshold min_size for the size of the connected region is set, and the connected regions with an area smaller than min_size are removed.

[0055] For example, if the threshold is set to 64 pixels, the regions with a connected region area smaller than 64 pixels will be removed. Through threshold screening, interference pixel regions can be removed, the road break point data can be reduced, the computational amount can be decreased, and redundant break points can be prevented from being generated in subsequent break point detection.

[0056] S2. Perform skeletonization on the optimized binary image to convert the road into a road skeleton with a single-pixel width.

[0057] Based on the iterative method of pixel neighborhood, the optimized binary image is processed to remove non-skeleton pixel points until only the center line of the road region remains. The output image after skeletonization is a binary image, and all road region parts are reduced to line structures with a single-pixel width. These lines retain the topological characteristics and connectivity of the original road, and at the same time, the road skeleton is obtained.

[0058] S3. Perform break point recognition processing on the road skeleton, extract all break points of the road, and obtain the break point parameter values.

[0059] As Figure 6 shown, the eight-neighborhood search method is used for break point recognition processing of the road skeleton. The eight-neighborhood refers to the neighborhood composed of eight pixels in the up, down, left, right, and four diagonal directions centered on a certain road pixel point. For each pixel point in the skeleton, the eight surrounding neighborhoods are subjected to an AND calculation using the eight-neighborhood convolution kernel kernel. Traverse each pixel with a value of 1 in the image (skeleton pixel), and then, centered on this pixel, count the number of pixels with a value of 1 in the adjacent eight neighborhoods. If the number is 1, it indicates a break point.

[0060] The center of the kernel matrix is the road skeleton pixel with a pixel value of 1, and there may be 1, 2, or 3 pixels with a value of 1 in the surrounding eight regions. If the number is 1, it indicates a break point.

[0061] The formula is expressed as:

[0062]

[0063] Among them, (0, 0) is the position of the road skeleton pixel.

[0064] Extract all break points of the road and obtain the break point parameter values. The break point parameter values include: the backtracking coordinate group, the direction vector group, the vertical vector group, etc. Among them:

[0065] The backtracking coordinate group is obtained by using the depth-first search algorithm to trace along the road skeleton from the breakpoint until a bifurcation point or a condition is met. During this backtracking process, the coordinate points of the road skeleton pixels traced back are recorded to form the coordinate group corresponding to the breakpoint.

[0066] Then, this coordinate group is used for fitting to extract its direction vector representation, obtaining a normalized direction vector group. Finally, this direction vector group is orthonormalized to obtain a perpendicular vector group.

[0067] S4. Based on the matching values between each pair of breakpoints, connect the pairs of breakpoints whose matching values exceed the threshold to obtain a refined road skeleton.

[0068] Two breakpoints form a pair of breakpoints. First, calculate the Euclidean distance d between the two breakpoints in the pair of breakpoints. Then, calculate the included angle θ between the direction vectors of the two breakpoints in the pair of breakpoints. Finally, use w d and w θ which represent the weight values of distance and angle respectively, to regularize the distance d and the included angle θ. The calculation formula for the matching value of the pair of breakpoints can be expressed as:

[0069]

[0070] where d is the Euclidean distance between the two breakpoints in the pair of breakpoints, θ is the angle between the direction vectors of the two breakpoints in the pair of breakpoints, w d and w θ are the weight values of distance and angle respectively.

[0071] Calculate the matching values of the pairs of breakpoints through the above formula, traverse all pairs of breakpoints, then retain the pairs of breakpoints whose matching values exceed the threshold and connect these pairs of breakpoints to obtain a refined road skeleton.

[0072] For each breakpoint in the breakpoint list, check whether each breakpoint meets the threshold to form a pair of breakpoints. If not, mark it as an isolated point. For each isolated point, use the pre-calculated direction vector, then normalize it, and determine the direction and distance (step size) for the extension of the isolated point. Finally, gradually extend the road skeleton until the maximum extension step size is reached or road pixels are encountered.

[0073] S5. Repair the width of the refined road skeleton to obtain the repaired road.

[0074] Through the above method, a refined road skeleton is obtained. At this time, the road skeleton connected between the pairs of breakpoints is still a single pixel. According to the original road width information, the width of the road skeleton can be repaired to make it closer to the shape of the real road.

[0075] Specifically, the backtracking method is used to track from the breakpoint to the specified position to obtain the skeleton segment. Then, for each pixel on the skeleton segment, calculate the width of the pixel on the vertical vector of the grayscale road image. Finally, take the average value of the road widths corresponding to all pixels on the skeleton segment as the repair width corresponding to the breakpoint, and repair the width of the improved road skeleton according to the repair width to obtain the repaired road area; finally, smooth the repaired road area to eliminate possible sharp corners or protrusions, and ensure that the shape of the road after width repair is smooth and natural.

[0076] Based on the above method, the present invention also provides a remote sensing image road breakpoint repair device, as Figure 3 shown, the device includes:

[0077] An image conversion module for converting a road image into a binary image, calculating the sizes of each road connected region, and removing some connected regions by setting a threshold to obtain an optimized binary image;

[0078] A skeleton extraction module for performing a skeletonization operation on the optimized binary image to convert the road into a road skeleton with a single-pixel width;

[0079] A breakpoint acquisition module for performing breakpoint recognition processing on the road skeleton, extracting all breakpoints of the road, and obtaining breakpoint parameter values;

[0080] A skeleton repair module for calculating the matching values between each pair of breakpoints, connecting the pairs of breakpoints whose matching values exceed the threshold, and obtaining an improved road skeleton;

[0081] A width repair module for repairing the road width of the improved road skeleton to obtain the repaired road.

[0082] The method and device of the present invention can maintain the continuity of the road skeleton after repair by accurately calculating the direction and distance between breakpoints, as shown in the repaired binary image Figure 5 shown, which significantly improves the accuracy of road extraction; at the same time, by introducing road width information and regularizing the repaired road, it not only ensures that the width of the repaired road conforms to physical laws, but also avoids the situation of road boundary distortion or over-connection caused by mis-matching, optimizes the smoothness and regularity of the repair path, makes the repaired road boundary more accurate and natural, and reduces the risk of misjudgment.

[0083] The present invention also provides an electronic device, Figure 7 which is the structural schematic diagram of the electronic device provided by the embodiment of the present invention, as Figure 7As shown in the figure, the electronic device may include: a processor, a communications interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus. The processor can call the logical instructions in the memory to execute the following methods, for example:

[0084] S1. Convert the road image into a binary image, calculate the size of the connected regions of each road, set a threshold to remove some connected regions, and obtain an optimized binary image;

[0085] S2. Perform a skeletonization operation on the optimized binary image to convert the road into a road skeleton with a single-pixel width;

[0086] S3. Perform breakpoint recognition processing on the road skeleton, extract the breakpoints of all roads, and obtain breakpoint parameter values;

[0087] S4. Based on the matching values between each pair of breakpoints, connect the pairs of breakpoints whose matching values exceed the threshold to obtain a refined road skeleton;

[0088] S5. Perform road width repair on the refined road skeleton to obtain a repaired road.

[0089] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0090] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is used to execute the methods provided in the above embodiments, for example, including:

[0091] S1. Convert the road image into a binary image, calculate the size of the connected regions of each road, set a threshold to remove some connected regions, and obtain an optimized binary image;

[0092] S2. Perform a skeletonization operation on the optimized binary image to convert the road into a road skeleton with a single-pixel width.

[0093] S3. Perform breakpoint recognition processing on the road skeleton, extract the breakpoints of all roads, and obtain breakpoint parameter values.

[0094] S4. Based on the matching values between each pair of breakpoints, connect the pairs of breakpoints whose matching values exceed the threshold to obtain a refined road skeleton.

[0095] S5. Repair the road width of the refined road skeleton to obtain the repaired road.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0097] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, also by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for repairing road breakpoints in remote sensing images, characterized in that, Including: S1. Convert the road image into a binary image, calculate the size of the connected regions of each road, set a threshold to remove some connected regions, and obtain an optimized binary image; S2. Perform a skeletonization operation on the optimized binary image to convert the road into a road skeleton with a single-pixel width; S3. Perform breakpoint recognition processing on the road skeleton, extract the breakpoints of all roads, and obtain breakpoint parameter values; S4. Based on the matching values between each pair of breakpoints, connect the pairs of breakpoints whose matching values exceed the threshold to obtain a refined road skeleton; S5. Repair the road width of the refined road skeleton to obtain a repaired road.

2. The repair method according to claim 1, characterized in that, The S1 includes the steps of: S11. Convert the road grayscale image into a road binary image, with the formula: where road_image(x,y) is the road grayscale image, 1 is the pixel value of the road area, and 0 is the pixel value of the background area; S12. Use the connected component analysis algorithm to calculate the size of the connected regions of each connected road area in the binary image; S13. Set a connected region size threshold min_size according to prior knowledge, remove the connected regions with an area smaller than min_size, and obtain an optimized binary image.

3. The repair method according to claim 1, wherein The S2 includes: Process the optimized binary image based on an iterative method of pixel neighborhoods, remove non-skeleton pixel points until only the center line of the road area remains, and reduce all road area parts to a single-pixel width line structure to obtain the road skeleton.

4. The repair method according to claim 3, wherein The S3 uses the eight-neighborhood search method for breakpoint recognition processing on the road skeleton, with the formula expressed as: where (0,0) is the pixel position of the road skeleton.

5. The repair method according to claim 1, characterized in that, The breakpoint parameter values in the S3 include: a backtracking coordinate group, a direction vector group, and a vertical vector group, where: Use the depth-first search algorithm to trace along the skeleton from the breakpoint until a bifurcation point or a condition is met. During this backtracking process, record the coordinate points of the road skeleton pixel points during backtracking and form the backtracking coordinate group corresponding to the breakpoint; perform fitting normalization on the backtracking coordinate group to obtain the direction vector group, and orthogonalize the direction vector group to obtain the vertical vector group.

6. The repair method according to claim 1, wherein The matching value is calculated based on the distance and angle difference between each pair of breakpoints, with the formula: Among them, d is the Euclidean distance between the two breakpoints of the breakpoint pair, θ is the angle of the direction vector between the two breakpoints of the breakpoint pair, w d and w θ are the weight values of the distance and the angle respectively.

7. The repair method according to claim 1, wherein The S4 also includes: Check whether each breakpoint meets the threshold to form a pair of breakpoints. If not, mark it as an isolated point; for the isolated point, determine the extension angle and distance of the isolated point, and extend the isolated point along the road skeleton until the maximum extension distance is reached or a road pixel is encountered.

8. The repair method according to claim 1, wherein The S5 includes: For each pixel point on the road skeleton segment, calculate the width of the pixel point on the vertical vector of the grayscale road image, take the average value of the road widths corresponding to all pixel points on the skeleton segment as the repair width corresponding to the breakpoint, and perform width repair on the refined road skeleton according to the repair width to obtain the repaired road.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the repair method as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the repair method as described in any one of claims 1 to 8.

Images