An unmanned aerial vehicle aerial photography-based road water accumulation automatic detection and grading early warning system

Abstract

The present application relates to the field of image recognition, and particularly relates to a road water accumulation automatic detection and grading early warning system based on unmanned aerial vehicle aerial photography. The system comprises: a data acquisition module, which is used for acquiring a gray image group corresponding to an image group of each road in a to-be-detected road; a data processing module, which is used for determining a target image group from the gray image group of each road according to the information entropy of each image in the gray image group; acquiring a tile region corresponding to each target image group, determining a double-view angle difference region from the tile region, and calculating the water accumulation possibility of each double-view angle difference region; determining the double-view angle difference region with a water accumulation possibility greater than a preset possibility threshold as a water accumulation region; and a grading early warning module, which is used for acquiring a water accumulation region image corresponding to the water accumulation region from the image group of the road, and constructing a water accumulation region grading early warning model using the image features of the water accumulation region image. The present application can improve the timeliness of recognizing the road water accumulation condition.

Description

Technical Field

[0001] This invention relates to the field of image recognition, specifically to an automatic detection and graded early warning system for road flooding based on drone aerial photography. Background Technology

[0002] In existing technologies, road flooding is mainly caused by rainfall and other factors. If it cannot be dealt with in the short term, it will threaten traffic safety and vehicle equipment, while long-term flooding may erode road infrastructure and damage the roadbed. Therefore, early detection and early treatment of road flooding are key aspects of road disaster prevention.

[0003] Currently, traditional monitoring methods mainly rely on manual patrols and fixed ground sensors. Manual patrols suffer from low efficiency, limited coverage, and difficulty in continuous operation under adverse weather conditions; while fixed sensors can provide real-time data, their monitoring range is limited to the installation point, and they cannot achieve rapid, panoramic scanning and coverage of long-distance roads. Summary of the Invention

[0004] This invention provides an automatic detection and graded early warning system for road water accumulation based on drone aerial photography, in order to solve existing problems.

[0005] The present invention provides an automatic detection and graded early warning system for road flooding based on drone aerial photography, which adopts the following technical solution:

[0006] One embodiment of the present invention provides an automatic detection and graded early warning system for road flooding based on drone aerial photography. The system includes:

[0007] The data acquisition module is used to acquire grayscale image groups corresponding to the image groups of each road segment in the road to be detected. The image groups are images of the same road segment taken by different image acquisition devices from different perspectives.

[0008] The data processing module is used to determine the target image group from the grayscale image group of each road segment based on the information entropy of each image in the grayscale image group.

[0009] Obtain the tile region corresponding to each target image group, determine the dual-view difference region from the tile region, and calculate the water accumulation probability of each dual-view difference region;

[0010] Areas with a greater than preset probability threshold of water accumulation in the two-view difference are identified as water accumulation areas.

[0011] The graded early warning module is used to obtain images of waterlogged areas from the image set of the road, and to construct a graded early warning model for waterlogged areas using the image features of the waterlogged area images. The target road image is input into the graded early warning model for waterlogged areas, and the model outputs the warning level of the target road.

[0012] Optionally, in the data processing module, based on the information entropy of each image in the grayscale image group, the target image group is determined from the grayscale image group of each road segment, specifically including:

[0013] The information entropy of each image in the grayscale image group is obtained and the average value is calculated to obtain the grayscale complexity of each road segment;

[0014] The roads are clustered based on their gray-level complexity to obtain the clustering results. The mean gray-level complexity of each cluster in the clustering results is obtained, and the cluster corresponding to the largest mean is determined as the target cluster. The gray-level image group corresponding to each road in the target cluster is determined as the target image group.

[0015] Optionally, in the data processing module, roads are clustered based on grayscale complexity to obtain clustering results, specifically including:

[0016] The gray-scale complexity is defined as the clustering parameter of the K-means clustering algorithm, and the preset number of categories is defined as the number of clusters in the K-means clustering algorithm. The K-means clustering algorithm is used to cluster all roads to obtain the clustering results, where the preset number of categories is 2.

[0017] Optionally, in the data processing module, the tile region corresponding to each target image group is obtained, specifically including:

[0018] For each target image group, image registration is performed, and the images in the registered target image group are subtracted to obtain the gray-level difference matrix corresponding to each target image group.

[0019] Superpixel segmentation is used to perform tile segmentation on each gray-level difference matrix, resulting in tile regions for each gray-level difference matrix.

[0020] The tile regions of the grayscale difference matrix are determined as the tile regions corresponding to the target image group, and the tile regions corresponding to each target image group are obtained.

[0021] Optionally, in the data processing module, image registration is performed on each target image group, specifically including:

[0022] For each group of target images, the SIFT algorithm is used to extract key points from each target image and perform matching to obtain matching points;

[0023] The transformation matrix is ​​calculated based on the matching points, and the target image group is registered according to the transformation matrix.

[0024] Optionally, in the data processing module, the dual-view difference region is determined from the tile region, specifically including:

[0025] For each tile region, the average pixel value of all pixels is calculated to obtain the dual-view grayscale difference of the tile region;

[0026] Tile regions whose dual-view grayscale difference is greater than the average dual-view grayscale difference of all tile regions are identified as dual-view difference regions.

[0027] Optionally, the data processing module calculates the probability of water accumulation in each dual-view difference area, specifically including:

[0028] Calculate the edge similarity parameters and associated grayscale differences for each dual-view difference region;

[0029] The ratio of the normalized edge similarity parameter to the normalized accompanying grayscale difference is used to determine the probability of water accumulation in the dual-view difference region.

[0030] Optionally, in the data processing module, the edge similarity parameters and accompanying grayscale differences of each dual-view difference region are calculated, specifically including:

[0031] For each dual-view difference region, the corresponding region of each target image in the target image group is determined as the first difference region and the second difference region, respectively.

[0032] Using the same edge detection algorithm with the same parameters, edge detection is performed on the first difference region and the second difference region respectively to obtain the first edge image and the second edge image;

[0033] The Dice coefficients of the first and second edge images are used to determine the edge similarity parameters for each dual-view difference region.

[0034] Calculate the accompanying grayscale difference for each dual-view difference region.

[0035] Optionally, in the data processing module, the accompanying grayscale difference of each dual-view difference region is calculated, specifically including:

[0036] The variance between the pixel value of the first pixel in the first difference region and the pixel values ​​of its eight neighboring pixels is defined as the first variance. The variance between the pixel value of the second pixel in the second difference region and the pixel values ​​of its eight neighboring pixels is defined as the second variance. The positions of the first pixel and the second pixel are the same.

[0037] The absolute value of the difference between the first variance and the second variance is determined as the first grayscale contrast value of the first pixel in the first difference region;

[0038] Each pixel in the first difference region is designated as the first pixel, and each pixel in the second difference region is designated as the second pixel. The first grayscale contrast value of each first pixel in the first difference region is obtained and summed to obtain the accompanying grayscale difference of the dual-view difference region.

[0039] Optionally, in the graded early warning module, images of the waterlogged areas corresponding to the waterlogged areas are obtained from the road image set, and the image features of the waterlogged area images are used to construct a graded early warning model for the waterlogged areas, specifically including:

[0040] The trained backbone network was used to extract image features from the waterlogged areas.

[0041] By using image features as input to a neural network and the corresponding warning level of the image features as output, the neural network is trained to obtain a graded warning model for waterlogged areas.

[0042] The beneficial effects of the technical solution of the present invention are:

[0043] In this embodiment of the invention, drones are used to periodically collect aerial images of road surfaces. Image processing technology is used to analyze the differences between road images of the same location at different times, automatically detecting areas of road flooding. Subsequently, features of the flooded areas are extracted, and based on model training, the degree of flooding is classified and risk warnings are issued, thereby achieving the goal of "early detection, early warning, and early handling" of road flooding. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a structural diagram of an automatic detection and graded early warning system for road water accumulation based on drone aerial photography, provided in one embodiment of the present invention. Detailed Implementation

[0046] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an automatic detection and graded early warning system for road flooding based on UAV aerial photography proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0048] The following description, in conjunction with the accompanying drawings, details a specific scheme for an automatic detection and graded early warning system for road flooding based on UAV aerial photography provided by the present invention.

[0049] This invention provides an automatic detection and graded early warning system for road flooding based on drone aerial photography. Please refer to [link to relevant documentation]. Figure 1 The diagram illustrates a structural representation of an automatic detection and graded early warning system for road flooding based on drone aerial photography, according to an embodiment of the present invention. The system includes:

[0050] The data acquisition module 101 is used to acquire grayscale image groups corresponding to the image groups of each road segment in the road to be detected. The image groups are images of the same road segment taken by different image acquisition devices from different perspectives.

[0051] For example, to ensure the accuracy of road water accumulation detection, this embodiment uses a drone to conduct periodic and continuous aerial photography of the target road section. The specific shooting process is as follows:

[0052] Aerial photography route and parameter presets: Before takeoff, the drone's flight path and aerial photography parameters are pre-planned. The planned route must cover all target road sections with water accumulation to be detected, and the start and end points of the route must be clearly marked. A key requirement in the aerial photography parameters is that the drone maintains a constant speed to ensure that the image information collected at the same relative position on the road section within different detection cycles has spatiotemporal consistency.

[0053] Overlapping Aerial Photography Design: Develop an aerial photography strategy with overlapping road segments. Specifically, a certain percentage of the road segments covered by two adjacent aerial photography missions should overlap. That is, the road image at the end of one aerial photography mission should have a small overlap with the image at the beginning of the next aerial photography mission. The purpose of this design is to ensure seamless image coverage across the entire continuous road area to be inspected, avoiding any missed road segments.

[0054] Using the above aerial photography method, the first cycle of aerial photography is carried out on the target road (flying from the preset starting point to the destination) to obtain the first cycle of image sequence containing N images.

[0055] The drone was then recalled for necessary post-processing, including image data transmission, drone status checks, and equipment maintenance (such as battery replacement).

[0056] Next, the drone is controlled to fly along a completely opposite route (from the original endpoint to the starting point), and while maintaining the same aerial photography parameters, a second cycle of aerial photography is performed on the same road to obtain a second cycle of image sequence containing N images.

[0057] Next, the images captured in the second cycle are arranged in reverse chronological order and then matched one-to-one with the image sequence from the first cycle to form N image groups. Each image group contains two images, one from each of two flight directions, but depicting the same specific road segment.

[0058] Subsequently, preprocessing is performed on the two images in each image group, mainly including the following two steps:

[0059] Image segmentation aims to accurately extract road areas from the original image and remove non-road background elements such as sky, vegetation, and buildings to eliminate interference in subsequent processing.

[0060] Grayscale conversion: The segmented road region image is converted into a grayscale image to reduce the number of image channels, thereby reducing the data complexity and computational load of subsequent processing, and obtaining the grayscale image group corresponding to each segment of the road to be detected.

[0061] The data processing module 102 is used to determine the target image group from the grayscale image group of each road segment based on the information entropy of each image in the grayscale image group; obtain the tile region corresponding to each target image group; determine the dual-view difference region from the tile region; and calculate the water accumulation probability of each dual-view difference region; and determine the dual-view difference region with the water accumulation probability greater than the preset probability threshold as the water accumulation region.

[0062] In this embodiment, the data processing module determines the target image group from the grayscale image group of each road segment based on the information entropy of each image in the grayscale image group, specifically including:

[0063] The information entropy of each image in the grayscale image group is obtained and the average value is calculated to obtain the grayscale complexity of each road segment;

[0064] The roads are clustered based on their gray-level complexity to obtain the clustering results. The mean gray-level complexity of each cluster in the clustering results is obtained, and the cluster corresponding to the largest mean is determined as the target cluster. The gray-level image group corresponding to each road in the target cluster is determined as the target image group.

[0065] In the data processing module, roads are clustered based on grayscale complexity to obtain clustering results, specifically including:

[0066] The gray-scale complexity is defined as the clustering parameter of the K-means clustering algorithm, and the preset number of categories is defined as the number of clusters in the K-means clustering algorithm. The K-means clustering algorithm is used to cluster all roads to obtain the clustering results, where the preset number of categories is 2.

[0067] In the data processing module, the tile region corresponding to each target image group is obtained, specifically including:

[0068] For each target image group, image registration is performed, and the images in the registered target image group are subtracted to obtain the gray-level difference matrix corresponding to each target image group.

[0069] Superpixel segmentation is used to perform tile segmentation on each gray-level difference matrix, resulting in tile regions for each gray-level difference matrix.

[0070] The tile regions of the grayscale difference matrix are determined as the tile regions corresponding to the target image group, and the tile regions corresponding to each target image group are obtained.

[0071] In the data processing module, image registration is performed for each target image group, specifically including:

[0072] For each group of target images, the SIFT algorithm is used to extract key points from each target image and perform matching to obtain matching points;

[0073] The transformation matrix is ​​calculated based on the matching points, and the target image group is registered according to the transformation matrix.

[0074] In the data processing module, the dual-view difference region is determined from the tile area, specifically including:

[0075] For each tile region, the average pixel value of all pixels is calculated to obtain the dual-view grayscale difference of the tile region;

[0076] Tile regions whose dual-view grayscale difference is greater than the average dual-view grayscale difference of all tile regions are identified as dual-view difference regions.

[0077] In the data processing module, the probability of water accumulation in each dual-view difference area is calculated, specifically including:

[0078] Calculate the edge similarity parameters and associated grayscale differences for each dual-view difference region;

[0079] The ratio of the normalized edge similarity parameter to the normalized accompanying grayscale difference is used to determine the probability of water accumulation in the dual-view difference region.

[0080] In the data processing module, the edge similarity parameters and accompanying grayscale differences of each dual-view difference region are calculated, specifically including:

[0081] For each dual-view difference region, the corresponding region of each target image in the target image group is determined as the first difference region and the second difference region, respectively.

[0082] Using the same edge detection algorithm with the same parameters, edge detection is performed on the first difference region and the second difference region respectively to obtain the first edge image and the second edge image;

[0083] The Dice coefficients of the first and second edge images are used to determine the edge similarity parameters for each dual-view difference region.

[0084] Calculate the accompanying grayscale difference for each dual-view difference region.

[0085] In the data processing module, the accompanying grayscale difference in each dual-view difference region is calculated, specifically including:

[0086] The variance between the pixel value of the first pixel in the first difference region and the pixel values ​​of its eight neighboring pixels is defined as the first variance. The variance between the pixel value of the second pixel in the second difference region and the pixel values ​​of its eight neighboring pixels is defined as the second variance. The positions of the first pixel and the second pixel are the same.

[0087] The absolute value of the difference between the first variance and the second variance is determined as the first grayscale contrast value of the first pixel in the first difference region;

[0088] Each pixel in the first difference region is designated as the first pixel, and each pixel in the second difference region is designated as the second pixel. The first grayscale contrast value of each first pixel in the first difference region is obtained and summed to obtain the accompanying grayscale difference of the dual-view difference region.

[0089] For example, waterlogged areas on roads typically have a certain coverage area and depth. This physical characteristic causes images of the same waterlogged area captured by drones from different aerial angles to show visual differences. At the same time, the specular reflections produced by road surfaces, road obstacles, or debris under strong light also exhibit significantly different image characteristics from different observation angles.

[0090] In the steps described above, image segmentation techniques were used to process the image groups (including dual-view images) corresponding to each road segment, retaining only the images containing the road area. Under ideal conditions (dry road surface, no obstacles), the texture and color distribution of the road surface are relatively uniform and simple. However, when there are water accumulations or wet areas on the road surface, the water surface produces specular reflections of light, reflecting the sky, surrounding scenery, or strong light sources (such as the sun). This reflection phenomenon significantly increases the visual complexity of the image content in that local area.

[0091] Based on these physical optical properties, the likelihood of water accumulation can be preliminarily determined by calculating the grayscale complexity of the image group (i.e., the segmented road area image) corresponding to each road segment. Specifically, grayscale complexity analysis of the image group reveals that the more complex the grayscale distribution and the more irregular the texture features, the greater the likelihood of water accumulation (or large areas of slippery surfaces) in that road segment; conversely, if the grayscale distribution is uniform and simple, it indicates that the road surface condition is normal and there is a higher probability of no water accumulation.

[0092] Therefore, based on the above principles, the first... Taking a road segment as an example, its corresponding grayscale complexity (denoted as ) The calculation can be performed using the following steps:

[0093]

[0094] in, and They represent the first The information entropy of the grayscale values ​​of the middle pixels in two images of a grayscale image group corresponding to a road segment.

[0095] In the formula, the information entropy of an image reflects the complexity and randomness of its gray-level distribution. Specifically, in an image containing only a simple road surface, all pixels are used to describe the road surface. Because the road color is relatively similar and uniform, its gray-level differences are small, and its statistical distribution is concentrated, resulting in a low calculated information entropy value. Conversely, when there is water accumulation, obstacles, or debris on the road surface, the water accumulation area will reflect the surrounding environment or produce strong light spots due to specular reflection, thus introducing color differences and gray-level changes in the image that are significantly different from those of a normal road surface. This change leads to a wider and more dispersed gray-level distribution range in the image, thus significantly increasing the calculated information entropy value. By comparison... The magnitude of the value can determine the complexity of the grayscale distribution in the image of that road segment, and thus assess the likelihood of abnormal conditions such as water accumulation. The higher the value, the more complex the grayscale, and the greater the possibility of water accumulation.

[0096] By processing the drone aerial images of all N road segments using the above method, the grayscale complexity of each road segment can be obtained (the N image groups can be considered to correspond to N road segments respectively).

[0097] Subsequently, the grayscale complexity value of each road segment is used as the clustering parameter for that segment. The K-means clustering algorithm is then used to perform cluster analysis on all N road segments (i.e., N label parameters). When the number of clusters K=2, the algorithm will output two clusters, thus achieving a preliminary binary partitioning of the road segments.

[0098] Subsequently, the mean values ​​of the two clusters generated by clustering were calculated to obtain the mean value of all road segment label parameters in the first cluster. And the mean of the label parameters of all road segments in the second cluster. .

[0099] Select and The cluster corresponding to the larger value, that is, the target cluster, is used to determine all road segments contained in it as road segments with higher gray-level complexity.

[0100] Judgment logic: Since road sections with water accumulation usually exhibit higher grayscale complexity than normal road surfaces due to factors such as water reflection, color depth changes and environmental interference, the road sections corresponding to the above-selected clusters have significantly higher overall grayscale complexity than other clusters, and are correspondingly more likely to have water accumulation.

[0101] Through grayscale complexity analysis, the initial screening of aerial images of all N road segments was completed, and M candidate road segments with high grayscale complexity were obtained for further judgment.

[0102] Based on the above analysis, the initial screening of all road segments to be detected has been completed. However, not every one of the M initially selected road segments actually contains waterlogged areas, therefore, a more refined secondary identification of these candidate road segments is required. The specific identification method is based on the following physical principles:

[0103] For each image group (including dual-view images) of the initially selected M road segment, the scene contains the following three types of elements: 1) normal road surface, 2) other obstacles or debris on the road surface, and 3) possible water accumulation areas.

[0104] Since road surfaces and other obstacle surfaces are usually non-reflective, the resulting images, although varying in perspective and lighting when taken from different angles, essentially describe the same physical entity as it appears in different viewing directions, and their image features have spatial coherence and consistency.

[0105] In contrast, the surface of the waterlogged area is approximately a specular reflector. When a drone takes pictures from different angles, due to the strict correspondence between the angle of incidence and the angle of reflection, the same waterlogged area may reflect completely different surrounding scenery (such as the sky, trees, buildings, etc.) in different images. Therefore, this area will exhibit significant and discontinuous feature differences in images from different perspectives, which can manifest as "the same image location may correspond to completely different visual content".

[0106] Therefore, by utilizing the physical nature of the consistency of non-reflective features and the significant difference in features between specular reflective objects, a thorough analysis of the target image group can be conducted to accurately distinguish and identify the real water accumulation areas.

[0107] The drone aerial image corresponding to the m-th candidate road segment and For example, the specific detection process for determining whether there are areas of road flooding in this section is as follows:

[0108] Image registration based on SIFT key points (using the first) Aerial images of the road section taken by drone and (For example)

[0109] First, the SIFT algorithm is used to process the images respectively. and Keypoint detection and feature extraction are performed. Then, based on these keypoints, the two images are located and registered to ensure... and The described road segments are consistent in spatial direction and are geometrically aligned.

[0110] Step-by-step instructions: Subsequent analysis requires... and Joint comparison is performed to identify waterlogged areas. Keypoint registration unifies the two images to the same coordinate system, ensuring that the same physical object (such as road markings or fixed obstacles) has a consistent spatial position in both images. This step aims to eliminate analytical errors caused by differences in the shooting perspective, leading to different positions of the same object in the images, thus laying the foundation for accurate subsequent comparison.

[0111] Then, the drone aerial images of the m-th road segment that have been registered are... and The method involves comparing pixel-by-pixel grayscale values ​​through image subtraction. Specifically, the grayscale values ​​of pixels at the same location in two images are subtracted, and the absolute value is taken to generate a grayscale difference matrix. The value of each element in this matrix reflects the degree of grayscale difference of the corresponding pixel in the two images.

[0112] After key point registration, the image and Since the images are now in the same coordinate system, pixels at the same location theoretically correspond to the same physical object. Under ideal conditions—a dry road surface, no standing water, and no other interfering objects—the grayscale values ​​of the image at that location should be essentially consistent regardless of whether the image is taken from a forward or reverse perspective. Therefore, the grayscale difference matrix obtained by subtracting each pixel from the two images... In this context, all element values ​​should be close to 0.

[0113] Then use the first Gray-scale difference matrix of each road segment Combined with drone aerial images and Proceed to the first The waterlogged areas of each road section were inspected, as detailed below:

[0114] First, a superpixel segmentation algorithm is used to analyze the gray-level difference matrix of the m-th road segment. Divide it into regions and segment it into A local tile area.

[0115] Gray-scale difference matrix The difference matrix is ​​obtained by subtracting two registered images from different perspectives of the m-th road segment. Therefore, if there is no water accumulation in this segment, most corresponding objects in the two images (such as road surface, road markings, and fixed obstacles) should be largely identical, and the resulting grayscale difference matrix will contain a large number of areas with low grayscale values ​​(close to 0). Conversely, if there is water accumulation in this segment, the specular reflection from the water surface will cause the two images to reflect different content in that area, resulting in areas with higher grayscale values ​​after subtraction. The difference matrix is ​​divided into multiple local tiles (superpixel blocks) using a superpixel segmentation algorithm, and the grayscale distribution characteristics within each tile can be used to further determine whether water accumulation exists in that local area.

[0116] With the first Taking a single tile area as an example, its dual-view grayscale difference The calculation method is as follows:

[0117]

[0118] in, Indicates the first The total number of all pixels in each tile region Indicates the first The first tile area The grayscale value of each pixel; .

[0119] For the For each tile region, the mean grayscale value of all elements within it is calculated. The higher the mean value, the greater the sum of grayscale differences among all pixels within that tile region, meaning the more significant the grayscale differences in that region are in the original dual-view image; conversely, the lower the mean value, the more consistent the region's appearance is in the two images.

[0120] Using the above method, for the first All of the road sections The dual-view grayscale difference (i.e., the average grayscale difference within each tile) is calculated for each tile region. After calculation, the overall average value of the dual-view grayscale difference values ​​for all tile regions is taken. The difference value of each tile is compared with this overall average value. If the difference value of a tile is greater than the average value, the tile region is marked as a "dual-view difference region"; otherwise, it is considered as a region with no significant difference.

[0121] Through the above steps, regions with significant dual-view grayscale differences have been identified in the m-th road segment. These regions may correspond to water accumulation, but they could also simply be ordinary road obstacles. For more precise differentiation, the original aerial images can be analyzed in a single step. and and grayscale difference matrix The specific process is as follows:

[0122] First, The first in Each tile region is projected onto the original image according to its position coordinates. and Up, thus in and Extract the corresponding region blocks from each, and denote them as follows: and These can be denoted as the first difference region and the second difference region, respectively. Next, edge detection algorithms with consistent parameters are used to respectively... and The edges are processed to obtain their corresponding edge images. Next, calculations are performed. and The similarity between edge images is specifically calculated using the Dice coefficient. and The Dice coefficient of the edge image is used as a metric, and this value is denoted as the first value. Edge similarity parameters of each tile region .

[0123] If the tile area corresponds to a non-waterlogged area (such as a fixed obstacle), then even if the grayscale difference is large due to different viewing angles, and The two edge images describe the same object, and their texture structures are basically the same, so the similarity between the two images is high. The value is relatively large. If the area is a waterlogged area, due to specular reflection, and They may present completely different reflective content, or different reflective parts of the same object, with significantly different texture structures, resulting in low edge similarity. The value is relatively small. Therefore, by comparison... The size of the area can be used to further determine whether the difference is due to water accumulation or a regular obstacle.

[0124] Next, regarding and Perform accompanying grayscale differences The specific method for obtaining it is as follows:

[0125]

[0126] in, Indicates in The first in The variance of the grayscale values ​​of a pixel and its 8 neighboring pixels; Indicates in The first in The variance of the grayscale values ​​of one pixel and its eight neighboring pixels. Indicates the first The total number of all pixels in each tile region.

[0127] In this formula, due to the angular differences when the drone captures images from two different perspectives, the grayscale values ​​of the same object may differ. However, the object's texture structure and local grayscale contrast generally remain consistent across different perspectives. In other words, although... and Two regions may have different grayscale values ​​at corresponding pixels, but the local grayscale contrast patterns they reflect should be similar. To characterize this feature, the grayscale contrast relationship between each pixel and its surrounding neighboring pixels is calculated to describe the... The internal structure of each region should be consistent. Specifically, if the region corresponds to the same object, then even with an overall grayscale shift, the local contrast patterns within that region in both images should be highly similar. In this case, the calculated accompanying grayscale difference... A smaller value indicates a larger value; conversely, if the area is a waterlogged area, the two images may present completely different content, resulting in significantly different local contrast patterns. In this case, accompanied by grayscale differences... The value is relatively large.

[0128] Finally, the edge similarity parameters are combined. With accompanying grayscale differences Calculate the first The first section of the road The probability that a given area is a flooded area, i.e., the likelihood of flooding. The specific calculation formula is as follows:

[0129]

[0130] in, and They are respectively and Normalized parameters.

[0131] In the formula, The larger, The smaller the value, the better. The first section of the road The lower the edge similarity of a region under dual-view conditions and the greater the accompanying grayscale difference, the higher the probability that the region describes different objects in the dual-view image, and thus the greater the probability that the region is a waterlogged area.

[0132] Using the steps described above, calculate the probability that all tile areas in the m-th road segment are waterlogged areas. The samples are then filtered using a preset probability threshold. This preset probability threshold is determined by manually labeled samples combined with supervised machine learning methods, and in this embodiment, it is set to 0.71.

[0133] Optionally, the preset probability threshold can be adjusted based on actual conditions and historical experience, and no specific restrictions are imposed here. The above 0.71 is a specific value in a preferred embodiment.

[0134] By processing all M candidate road segments initially selected using the above method, areas with actual water accumulation can be identified. Thus, water accumulation detection has been completed for all road segments of the target road, ultimately yielding A confirmed water accumulation areas.

[0135] The graded early warning module 103 is used to obtain the water accumulation area image corresponding to the water accumulation area from the road image group, and to construct the water accumulation area graded early warning model using the image features of the water accumulation area image. The target road image is input into the water accumulation area graded early warning model, and the warning level of the target road is output through the water accumulation area graded early warning model.

[0136] In this embodiment, the graded early warning module obtains images of waterlogged areas corresponding to the waterlogged areas from the road image group, and constructs a graded early warning model for waterlogged areas using the image features of the waterlogged area images. Specifically, this includes:

[0137] The trained backbone network was used to extract image features from the waterlogged areas.

[0138] By using image features as input to a neural network and the corresponding warning level of the image features as output, the neural network is trained to obtain a graded warning model for waterlogged areas.

[0139] For example, the specific steps of the graded early warning model for waterlogged areas are as follows:

[0140] 1. Feature extraction of waterlogged areas:

[0141] Data preparation: The identified waterlogged areas in all road segment images were manually verified and marked. Images containing waterlogged areas were then normalized and uniformly scaled to 640×640 pixels.

[0142] Feature extraction network: The normalized image of the waterlogged area is input into a pre-trained backbone network (such as ResNet, VGG, etc.). The network performs multi-level feature extraction through its convolutional layers, pooling layers, and residual connections.

[0143] Shallow features: capture basic visual information such as edges, textures, and colors.

[0144] Mid-layer features: Extracting local shape features such as the irregular contours of the water accumulation and whether it is distributed along low-lying areas of the road.

[0145] Deep features: Obtain abstract semantic and spatial relationship features related to the water accumulation area.

[0146] Feature fusion: By utilizing the decoder structure (or feature pyramid network) of the segmentation model, the features at different levels mentioned above are fused to take into account both detailed information and high-level semantics, and finally output a comprehensive image feature vector representing the waterlogged area.

[0147] 2. Construction of a tiered early warning model:

[0148] A two-stage neural network structure of "feature adaptation + hierarchical classification" is adopted:

[0149] Feature adaptation layer: The extracted feature vectors are input into the fully connected layer and processed by the ReLU activation function and Dropout regularization to realize the mapping of high-dimensional features to low-dimensional discrimination space, providing adapted input for the classifier.

[0150] Hierarchical Classifier: A fully connected neural network is used as the classifier, taking the adapted features as input and outputting the corresponding warning level. Model training uses the cross-entropy loss function and a cosine annealing strategy to dynamically adjust the learning rate, improving model performance and preventing overfitting.

[0151] 3. Warning level setting:

[0152] This embodiment classifies the severity of water accumulation into the following four warning levels, with the core criterion being the percentage of the waterlogged area relative to the total road surface area:

[0153] Level 0 (No standing water): The road surface is dry or has only a small amount of dampness, with no visible standing water areas.

[0154] Level 1 (Slight Flooding): Flooded area < 10% of road surface area.

[0155] Level 2 (Moderate flooding): The flooded area accounts for 10% to 30% of the road surface area.

[0156] Level 3 (Severe flooding): Flooded area > 30% of road surface area.

[0157] Through the above steps, the system can use the trained hierarchical early warning model to automatically assess the level of the identified waterlogged areas and trigger corresponding early warning prompts.

[0158] After identifying the target road waterlogging areas and building a tiered early warning model, automatic tiered early warnings can be issued for the areas requiring monitoring. The specific process is as follows:

[0159] Waterlogged area identification: Aerial images of the road to be inspected are processed to identify and locate waterlogged areas.

[0160] Region feature extraction: For each identified waterlogged area, extract its multi-level image features (including texture, shape and semantic features).

[0161] Model Input and Inference: The extracted features of the waterlogged area are input into the pre-trained hierarchical early warning model.

[0162] Warning level determination: Based on the model output, determine the warning level (e.g., level 0-3) corresponding to the waterlogged area.

[0163] Warning Issuance: Based on the determined warning level, the corresponding warning mechanism will be activated to prompt management personnel to take action.

[0164] Through the above steps, the system achieves fully automated processing from "water accumulation identification" to "level assessment" and then to "early warning issuance".

[0165] In summary, in this embodiment of the invention, drones are used to periodically collect aerial images of road surfaces. Image processing techniques are then used to analyze the differences between road images of the same location at different times, automatically detecting areas of road flooding. Subsequently, features of the flooded areas are extracted, and a model is trained to classify the degree of flooding and provide risk warnings, thereby achieving the goal of "early detection, early warning, and early handling" of road flooding.

[0166] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0167] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0168] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An unmanned aerial vehicle (UAV) aerial photography-based automatic detection and grading early warning system for road waterlogging, characterized in that, The method comprises the following steps: A data acquisition module is configured to acquire a set of gray-scale images corresponding to a set of images of each road segment in a road to be detected, wherein the set of images are images of the same road segment taken by different image acquisition devices at different angles; A data processing module is configured to acquire the information entropy of each image in the set of gray-scale images and calculate the average value, thereby obtaining the gray-scale complexity of each road segment; The roads are clustered according to the gray-scale complexity, thereby obtaining a clustering result, acquiring the average value of the gray-scale complexity of each cluster in the clustering result, determining the cluster corresponding to the maximum average value as a target cluster, and determining the set of gray-scale images corresponding to each road in the target cluster as a target image set; Image registration is performed on each target image set, and the images in the target image set after image registration are subjected to difference processing, thereby obtaining a gray-scale difference matrix corresponding to each target image set; Each gray-scale difference matrix is subjected to tile segmentation using superpixel segmentation, thereby obtaining a tile region of each gray-scale difference matrix; The tile region of the gray-scale difference matrix is determined as a tile region corresponding to the target image set, and the tile region corresponding to each target image set is acquired; For each tile region, the average value of the pixel values of all pixel points is calculated, thereby obtaining the dual-view gray-scale difference of the tile region; The tile region whose dual-view gray-scale difference is greater than the average value of the dual-view gray-scale differences of all tile regions is determined as a dual-view difference region; For each dual-view difference region, the corresponding region of the dual-view difference region in each target image of the target image set is determined as a first difference region and a second difference region, respectively; An edge similarity parameter of each dual-view difference region is calculated; The variance of the pixel value of a first pixel point in the first difference region and the pixel values of eight neighboring pixel points is determined as a first variance, and the variance of the pixel value of a second pixel point in the second difference region and the pixel values of eight neighboring pixel points is determined as a second variance, wherein the first pixel point and the second pixel point are located at the same position; The absolute value of the difference between the first variance and the second variance is determined as a first gray-scale contrast value of the first pixel point in the first difference region; Each pixel point in the first difference region is determined as the first pixel point, each pixel point in the second difference region is determined as the second pixel point, the first gray-scale contrast value of each first pixel point in the first difference region is acquired and summed, thereby obtaining the accompanying gray-scale difference of the dual-view difference region; The ratio of the normalized edge similarity parameter and the normalized accompanying gray-scale difference is determined as the water accumulation possibility of the dual-view difference region; The dual-view difference region whose water accumulation possibility is greater than a preset possibility threshold is determined as a water accumulation region; A hierarchical early warning module is configured to acquire a water accumulation region image corresponding to the water accumulation region from the set of images of the road, construct a water accumulation region hierarchical early warning model using the image features of the water accumulation region image, input the images of the target road into the water accumulation region hierarchical early warning model, and output the warning level of the target road through the water accumulation region hierarchical early warning model. 2.The road waterlogging automatic detection and grading pre-warning system based on UAV aerial photography according to claim 1, characterized in that, In the data processing module, the roads are clustered according to the gray-scale complexity, thereby obtaining a clustering result, and specifically comprising the following steps: The gray complexity is determined as a clustering parameter of the K-means clustering algorithm, the preset category number is determined as a cluster number of the K-means clustering algorithm, and the K-means clustering algorithm is used to cluster all roads to obtain a clustering result, wherein the preset category number is 2. 3.The road waterlogging automatic detection and grading pre-warning system based on UAV aerial photography according to claim 1, characterized in that, In the data processing module, image registration is performed on each target image group, specifically including: For each target image group, the SIFT algorithm is used to extract key points in each target image and perform matching, to obtain matching points; Based on the matching points, a transformation matrix is calculated, and the target image group is registered according to the transformation matrix. 4.The road waterlogging automatic detection and grading pre-warning system based on UAV aerial photography according to claim 1, characterized in that, In the data processing module, the edge similarity parameter of each double-view difference region is calculated, specifically including: The same parameter edge detection algorithm is used to perform edge detection on the first difference region and the second difference region respectively, to obtain a first edge image and a second edge image; The Dice coefficient of the first edge image and the second edge image is determined as the edge similarity parameter of each double-view difference region. 5.The unmanned aerial vehicle (UAV) aerial photography-based road waterlogging automatic detection and grading pre-warning system according to claim 1, characterized in that, In the hierarchical early warning module, the waterlogging area image corresponding to the waterlogging area is obtained from the image group of the road, and the image feature of the waterlogging area image is used to construct a waterlogging area hierarchical early warning model, specifically including: The trained backbone network is used to extract the image features of the waterlogging area image; The image features are used as the input of the neural network, and the early warning level corresponding to the image features is used as the output of the neural network, and the neural network is trained to obtain the waterlogging area hierarchical early warning model.

Images