Roof water accumulation area identification method and system based on unmanned aerial vehicle vision

Abstract

The present application relates to the technical field of image processing, and proposes a roof water accumulation area recognition method and system based on unmanned aerial vehicle vision, which comprises: collecting roof images at multiple time points by an unmanned aerial vehicle; obtaining a plurality of roof areas to obtain a plurality of local roof areas; clustering based on the distribution and gray level of each local roof area to obtain a plurality of clusters; obtaining a plurality of object areas and suspected water accumulation areas; obtaining the degree of attention of each suspected water accumulation area according to the texture difference and gradient performance of the same suspected water accumulation area in different roof images; obtaining the credibility of each suspected water accumulation area based on the distribution of the suspected water accumulation area and the object area near the drain, and then obtaining the saliency weight of each suspected water accumulation area; performing saliency detection on the roof image to construct a roof water accumulation recognition model. The present application aims to solve the problem of misjudgment caused by the interference of other objects and insufficient clarity of the roof water accumulation area under the unmanned aerial vehicle aerial photography.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for identifying rooftop water accumulation areas based on UAV vision. Background Technology

[0002] Water accumulation on rooftops is often overlooked due to its high location, but it provides an ideal breeding environment for Aedes mosquitoes. As a major vector for infectious diseases, it is necessary to reduce the breeding of Aedes mosquitoes in a timely manner, which requires regular inspection and location of water accumulation on rooftops. Drone aerial photography technology can collect images and conduct visual monitoring of rooftops over a wide area, solving the problem of difficulty in inspecting each rooftop when water accumulation is distributed across multiple buildings. By using drones to visually identify and locate water accumulation on rooftops, suitable breeding environments for Aedes mosquitoes can be quickly eliminated, reducing the spread of mosquito-borne viruses.

[0003] In the process of identifying rooftop water accumulation areas using existing CA saliency monitoring algorithms, the saliency features of these areas are not obvious due to factors such as lighting, image contrast, and color representation. Furthermore, the presence of other objects in complex backgrounds, such as debris piled on the rooftop, can interfere with the water accumulation area, making both difficult to identify accurately. Additionally, the limited distance of drone aerial photography often results in insufficient clarity and low resolution when acquiring images of rooftops, further affecting the efficiency and accuracy of identifying rooftop water accumulation areas. Summary of the Invention

[0004] This invention provides a method and system for identifying rooftop water accumulation areas based on UAV vision, to solve the problem of misjudgment caused by interference from other objects and insufficient clarity in existing UAV aerial photography of rooftop water accumulation areas. The specific technical solution adopted is as follows:

[0005] This invention proposes a method for identifying rooftop water accumulation areas based on UAV vision. The method includes the following steps:

[0006] The drones were used to conduct inspections along a fixed route, collecting rooftop images at multiple times.

[0007] Edge detection and connected component extraction are performed on the rooftop image, and several rooftop regions are obtained through semantic segmentation, which further segments several local rooftop regions. Based on the distribution and grayscale representation of each local rooftop region, several clusters are obtained. Based on the clusters, the local rooftop regions are combined to obtain several regions to be analyzed. Based on the contour regularity of the regions to be analyzed, several object regions and suspected water accumulation regions are identified.

[0008] The system matches suspected water accumulation areas in rooftop images at different times. Based on the texture differences and gradient performance of the same suspected water accumulation area in different rooftop images, the system obtains the attention level of each suspected water accumulation area. Based on the distance between the suspected water accumulation area and the drain outlet in the rooftop area, and the distribution of objects near the drain outlet in the rooftop area, the system obtains the credibility of each suspected water accumulation area. Combined with the attention level, the system obtains the significant weight of each suspected water accumulation area.

[0009] Based on the saliency weight of suspected water accumulation areas in rooftop images, saliency detection is performed on rooftop images to construct a rooftop water accumulation recognition model.

[0010] Optionally, the specific methods for performing edge detection and connected component extraction on the rooftop image, and obtaining several rooftop regions through semantic segmentation, include:

[0011] For any rooftop image at any given time, several edges of the rooftop image are obtained through Canny edge detection. Connected components are extracted based on closed edges to obtain several connected components of the rooftop image. Several connected components that are not contained in other connected components are taken as several maximum contour regions of the rooftop image.

[0012] A large number of rooftop images are used as the training dataset, and several maximum contour regions are obtained from each rooftop image. The maximum contour regions belonging to the rooftop are manually labeled to construct a semantic segmentation network. The rooftop image and several maximum contour regions are input into the trained semantic segmentation network to obtain several rooftop regions of the rooftop image.

[0013] Optionally, the segmentation to obtain several local rooftop areas includes the following specific methods:

[0014] For any rooftop area, construct a sliding window and slide it from the top left corner to the next column without overlapping, resulting in several areas framed by the sliding window, which serve as several partial rooftop areas of that rooftop area.

[0015] Optionally, the specific methods for obtaining several clusters include:

[0016] For any rooftop region in any rooftop image, obtain the center of each rooftop local region and the average gray value of the pixels in each rooftop local region; perform density clustering on all rooftop local regions in the rooftop region, and use the product of the absolute value of the difference between the Euclidean distance between the centers of the rooftop local regions and the average gray value of their respective pixels to obtain several clusters of the rooftop region.

[0017] Optionally, the specific method for obtaining the plurality of regions to be analyzed is as follows:

[0018] For several rooftop local areas within the same cluster, adjacent rooftop local areas are combined to obtain several combined areas of the same cluster, which are then used as several regions to be analyzed.

[0019] Optionally, the specific method for obtaining several object regions and suspected water accumulation regions includes:

[0020] For any region to be analyzed in any rooftop image, obtain all edge pixels and their tangent directions of the region to be analyzed; obtain the minimum bounding rectangle of the region to be analyzed, obtain the proportion of the area of ​​the region to be analyzed in the area of ​​its minimum bounding rectangle, and use the ratio of the proportion to the variance of the tangent directions of all edge pixels of the region to be analyzed as the contour regularity of the region to be analyzed.

[0021] Obtain the contour regularity of each region to be analyzed in the rooftop image. Arrange all regions to be analyzed in ascending order of contour regularity. Obtain the difference between the contour regularity of sequentially adjacent regions to be analyzed. Use the contour regularity of the right region to be analyzed in the two regions corresponding to the maximum value of all differences as the rule threshold.

[0022] The regions in the rooftop image whose contour regularity is less than the rule threshold are identified as suspected water accumulation areas in the rooftop image; the regions in the rooftop image whose contour regularity is greater than or equal to the rule threshold are identified as object regions in the rooftop image.

[0023] Optionally, the specific method for matching suspected water accumulation areas in rooftop images at different times includes:

[0024] Obtain several suspected water accumulation areas and object areas from rooftop images at various times; for any two rooftop images at different times, match the respective rooftop areas to obtain the matched rooftop areas in the two rooftop images, and match the suspected water accumulation areas and object areas in the matched rooftop areas based on their positions.

[0025] Optionally, the degree of attention given to each suspected waterlogged area is obtained using the following method:

[0026] For any suspected waterlogged area, obtain several rooftop images of the suspected waterlogged area, and denote them as the judged rooftop images of the suspected waterlogged area; obtain the gray-level co-occurrence matrix of the suspected waterlogged area in any two judged rooftop images, and obtain the energy value corresponding to the gray value of the suspected waterlogged area in the two judged rooftop images based on the gray-level co-occurrence matrix;

[0027] For any suspected waterlogged area in the rooftop image, obtain the gradient value of each pixel and record the average gradient value of all pixels in the suspected waterlogged area in the rooftop image as the gradient performance value of the suspected waterlogged area in the rooftop image.

[0028] The absolute value of the difference between the energy values ​​corresponding to the gray values ​​of the suspected water accumulation area in the two rooftop images is multiplied by the absolute value of the difference between the gradient performance values ​​of the suspected water accumulation area in the two rooftop images, and the texture difference factor of the suspected water accumulation area in the two rooftop images is used as the texture difference factor of the suspected water accumulation area in the two rooftop images.

[0029] The mean and normalized result of the texture difference factors between all pairs of rooftop images corresponding to the suspected water accumulation area is taken as the degree of attention of the suspected water accumulation area.

[0030] Optionally, the specific method for obtaining the confidence level of each suspected waterlogged area includes:

[0031] For any rooftop area, based on the construction drawings of the building corresponding to the rooftop area, the spatial location of the drainage outlet corresponding to the rooftop area is obtained, and the spatial location of the suspected water accumulation area and object area in the rooftop area is marked. The spatial location corresponding to the center of each suspected water accumulation area and the spatial location corresponding to the center of each object area in the rooftop area are obtained, and the shortest distance between them and the spatial location of the drainage outlet is used as the drainage distance of each suspected water accumulation area and each object area.

[0032] For any suspected water accumulation area in the rooftop area, the drain outlet corresponding to the drainage distance of the suspected water accumulation area is recorded as the reference drain outlet of the suspected water accumulation area, and the drainage distance of the suspected water accumulation area is used as the distance threshold. The object area in the rooftop area whose drainage distance corresponding to the reference drain outlet is less than the distance threshold is recorded as the obstruction area of ​​the suspected water accumulation area.

[0033] Obtain the mean of the reciprocals of the drainage distances of each blocked area, and multiply the mean of the reciprocals by the drainage distance of the suspected waterlogged area to determine the confidence level of the suspected waterlogged area.

[0034] The present invention also proposes a rooftop water accumulation area identification system based on UAV vision. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the above method.

[0035] The beneficial effects of this invention are as follows: This invention uses drones to take aerial photos of rooftops, thereby acquiring rooftop images and extracting the top floor area. By combining and segmenting small areas within the top floor area, suspected water accumulation areas and object areas are initially screened, reflecting the irregular shape of suspected water accumulation areas relative to fixed objects. Furthermore, considering that water accumulation under different viewing angles causes significant texture changes in the reflected sky, buildings, and other surfaces in the image, this further determines the attention level of suspected water accumulation areas. The greater the spatial distance between suspected water accumulation areas on the rooftop and the drain outlet, and the greater the potential obstruction from objects near the drain outlet, the more likely the suspected water accumulation area is actually a water accumulation area due to drainage difficulties. This increases the credibility and constructs a saliency weight. By adjusting the general contrast weight in the traditional CA saliency detection algorithm, the saliency factor of water accumulation areas in the rooftop image is highlighted. Thus, by constructing a rooftop water accumulation recognition model, the water accumulation information can be accurately learned without being affected by other interfering occlusion factors, thereby improving the recognition accuracy and efficiency of rooftop water accumulation. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of a method for identifying rooftop water accumulation areas based on UAV vision, provided in one embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figure 1 The diagram illustrates a flowchart of a method for identifying rooftop water accumulation areas based on UAV vision, according to an embodiment of the present invention. The method includes the following steps:

[0040] Step S001: Conduct inspections along a fixed route using drones and collect rooftop images at multiple times.

[0041] The purpose of this embodiment is to use drones for inspection, take aerial photos of the rooftop area, and use computer vision to detect water accumulation areas. Therefore, it is necessary to first collect rooftop images from multiple perspectives through regular drone inspections.

[0042] Specifically, a fixed flight path for the UAV is set in parallel grid format through the ground station. Under uniform lighting conditions, the UAV conducts inspections along the fixed flight path. The camera takes vertical downward shots and acquires images every 5 seconds. The acquired images are preprocessed by filtering and denoising, and then grayscale processing is performed to obtain grayscale images, which serve as rooftop images at various times.

[0043] It should be noted that the system quickly locates and identifies the top floor area in the rooftop image based on the edge contour of the rooftop, divides the top floor area, and clusters it according to the grayscale performance and spatial Euclidean distance of the area; it filters out the areas to be detected (water accumulation and easily confused irregular areas) based on whether they have regular shapes; based on the characteristic that water accumulation areas have a specular reflection function and the reflection of the sky or surrounding buildings changes under different viewing angles, resulting in different texture changes in the image, the attention level of the area can be filtered; the distance between the suspected water accumulation area and the rooftop drain outlet and whether there are obstructions around the drain outlet are analyzed to calculate the credibility of the water accumulation, and the suspected water accumulation areas with high credibility are assigned higher weights.

[0044] Step S002: Perform edge detection and connected component extraction on the rooftop image, and obtain several rooftop regions through semantic segmentation, and segment several local rooftop regions; perform clustering based on the distribution and grayscale performance of each local rooftop region to obtain several clusters; combine the local rooftop regions based on the clusters to obtain several regions to be analyzed; and determine several object regions and suspected water accumulation regions based on the contour rule performance of the regions to be analyzed.

[0045] Preferably, in one embodiment of the present invention, edge detection and connected component extraction are performed on the rooftop image, and several rooftop regions are obtained through semantic segmentation. The specific method includes:

[0046] For any rooftop image at any given time, several edges of the rooftop image are obtained through Canny edge detection. Connected components are extracted based on closed edges to obtain several connected components of the rooftop image. Several connected components that are not contained in other connected components, i.e., those not inside other connected components, are taken as several maximum contour regions of the rooftop image.

[0047] Furthermore, a large number of rooftop images are used as the training dataset, and several maximum contour regions are obtained from each rooftop image. The maximum contour regions belonging to the rooftop are manually labeled, with the pixel value of the labeled region being 1, and the pixel values ​​of other regions being labeled as 0. A semantic segmentation network is constructed using a CNN network, with the cross-entropy loss function as the loss function. By inputting the rooftop images in the training dataset and the labeled maximum contour regions belonging to the rooftop into the semantic segmentation network, the semantic segmentation network is trained and outputs the pixels belonging to the rooftop in the rooftop image, that is, the rooftop region is labeled in all maximum contour regions, and the trained semantic segmentation network is obtained. The rooftop image and several maximum contour regions are then input into the trained semantic segmentation network to obtain several rooftop regions of the rooftop image.

[0048] Furthermore, for any rooftop area, a sliding window is constructed. In this embodiment, the size of the sliding window is adopted. The process involves a sliding step of 5 pixels, starting from the top left corner and moving sequentially to the next column of the rooftop area without overlapping, resulting in several regions outlined by the sliding window. These regions serve as several local rooftop areas within the rooftop area. It's worth noting that if the number of pixels is insufficient to form a local rooftop area due to proximity to the rooftop boundary, the pixels within the rooftop area outlined by the sliding window are merged with the nearest local rooftop area. This prevents the sliding window from affecting the acquisition of local rooftop areas due to proximity to the boundary.

[0049] It should be further explained that by extracting the top floor region from the rooftop image, and then performing local segmentation on each top floor region, if the grayscale values ​​of the local rooftop regions selected by the window are similar, the distance is small or they are completely adjacent, they are more likely to correspond to the same actual object or areas such as water accumulation or sand piles. This is used to cluster the local rooftop regions and further obtain the regions to be analyzed.

[0050] Preferably, in one embodiment of the present invention, clustering is performed based on the distribution and grayscale representation of local areas on each rooftop to obtain several clusters, including the following specific methods:

[0051] For any rooftop region in any rooftop image, obtain the center of each rooftop local region and the average gray value of the pixels in each rooftop local region; perform density clustering on all rooftop local regions in the rooftop region, specifically using DBSCAN clustering, and use the product of the absolute value of the difference between the Euclidean distance between the centers of the rooftop local regions and the average gray value of their respective pixels to obtain several clusters of the rooftop region.

[0052] Preferably, in one embodiment of the present invention, the method of combining local areas of the rooftop based on clusters to obtain several areas to be analyzed includes:

[0053] For several rooftop local areas within the same cluster, adjacent rooftop local areas are combined, i.e., they belong to the same cluster and are adjacent and continuous in position, to obtain several combined areas of that cluster (if a rooftop local area in a cluster is not adjacent to any other rooftop local areas in the cluster, then that rooftop local area is directly taken as a combined area), which are used as several regions to be analyzed, thereby obtaining several regions to be analyzed for each rooftop area in the rooftop image.

[0054] It should be further explained that by combining local areas of the rooftop within the same cluster, a larger area to be analyzed can be obtained. Since the distances within the same cluster are close and the grayscale representations are similar, the combined area to be analyzed usually corresponds to the same object or water accumulation, sand pile, etc. Therefore, it is necessary to further divide the area to be analyzed into regular object areas and irregular suspected water accumulation areas based on the outline regularity of the area to be analyzed.

[0055] Preferably, in one embodiment of the present invention, based on the contour rule representation of the region to be analyzed, several object regions and suspected water accumulation regions are determined, including the following specific method:

[0056] For any region to be analyzed in any rooftop image, obtain all edge pixels of the region and their tangent directions (if the edge is a straight line, then the tangent is the edge). The tangent direction is represented by the clockwise direction of the tangent, specifically in the form of an angle. At the same time, obtain the minimum bounding rectangle of the region to be analyzed, and obtain the proportion of the area of ​​the region to be analyzed in the area of ​​its minimum bounding rectangle. The ratio of the proportion to the variance of the tangent directions of all edge pixels of the region to be analyzed is used as the contour regularity of the region to be analyzed.

[0057] Furthermore, following the method described above, the contour regularity of each region to be analyzed in the rooftop image is obtained. All regions to be analyzed are arranged in ascending order of contour regularity. The difference between the contour regularity of sequentially adjacent regions to be analyzed is obtained (the difference is obtained by decreasing the larger value). The contour regularity of the right region to be analyzed in the two regions corresponding to the largest difference is taken as the rule threshold. Regions to be analyzed in the rooftop image whose contour regularity is less than the rule threshold are identified as several suspected water accumulation areas in the rooftop image. Regions to be analyzed in the rooftop image whose contour regularity is greater than or equal to the rule threshold are identified as several object regions in the rooftop image.

[0058] It should be noted that the areas to be analyzed are usually different local areas on the rooftop. These may be areas of fixed objects with regular shapes, such as tables, chairs, bricks, and discarded cardboard boxes, which have clear geometric shapes; or areas of irregular shapes, such as puddles of water and other non-fixed shapes, such as sand piles. The areas are filtered based on their regularity. The smaller the variance of the tangent direction of the edge pixels, the more regular the edge. Since regular shapes are usually close to rectangles, the area ratio within the smallest bounding rectangle is used for analysis. The larger the ratio, the closer it is to a rectangle. This is used to distinguish between regular object areas and irregular areas suspected of being puddles of water.

[0059] At this point, several areas suspected of being flooded were obtained from the rooftop images.

[0060] Step S003: Match suspected water accumulation areas in rooftop images at different times. Based on the texture differences and gradient performance of the same suspected water accumulation area in different rooftop images, obtain the attention level of each suspected water accumulation area. Based on the distance between the suspected water accumulation area and the drain outlet in the rooftop area, and the distribution of objects near the drain outlet in the rooftop area, obtain the credibility of each suspected water accumulation area. Combine the attention level to obtain the significant weight of each suspected water accumulation area.

[0061] It should be noted that the water accumulation area on the roof has a mirror-like reflective function. The water surface in the water accumulation area will produce reflections of the sky or surrounding buildings. The reflections seen in the water accumulation area are different from different viewing angles, which leads to significant changes in the texture of the water accumulation area. On the other hand, the texture of irregular areas that are not water accumulation changes very little or not at all from different viewing angles. Therefore, by analyzing the magnitude of the texture changes in the same area, the degree of attention of that area can be determined.

[0062] Preferably, in one embodiment of the present invention, suspected water accumulation areas in rooftop images at different times are matched, and the degree of attention of each suspected water accumulation area is obtained based on the texture differences and gradient performance of the same suspected water accumulation area in different rooftop images. The specific method includes:

[0063] Acquire several suspected water accumulation areas and object areas from rooftop images at various times. For any two rooftop images at different times, match the respective rooftop areas. The specific matching process is based on corner detection and SIFT feature point matching, thereby obtaining the matched rooftop areas in the two rooftop images. Match the suspected water accumulation areas and object areas in the matched rooftop areas based on their positions (the closest position to the center point, combined with corner detection). The matching of areas from different perspectives is an existing technology in computer vision, which will not be elaborated in this embodiment. The matching of suspected water accumulation areas in the rooftop images at the two times is then completed. It should be noted that due to the change in the perspective of the UAV aerial flight, the spatial range included in the monitored rooftop images will change, which may result in a certain rooftop area not existing in both rooftop images, or a certain object area or suspected water accumulation area not existing in both rooftop images, leading to matching failure. In this case, it indicates that the suspected water accumulation area or object area does not exist in one of the rooftop images.

[0064] Furthermore, for any suspected waterlogged area, several rooftop images of the suspected waterlogged area are acquired and denoted as the judged rooftop images of the suspected waterlogged area; for any two judged rooftop images, the gray-level co-occurrence matrix of the suspected waterlogged area is acquired, and the energy value corresponding to the gray level of the suspected waterlogged area in the two judged rooftop images is obtained based on the gray-level co-occurrence matrix. The specific construction process of the gray-level co-occurrence matrix is ​​not described in this embodiment, as it is existing technology in the field of image processing and is not specifically limited in this embodiment, but it is necessary to ensure that the gray level division and the direction selection of adjacent pixels corresponding to the two gray-level co-occurrence matrices are consistent; the gradient value of each pixel in the suspected waterlogged area in any judged rooftop image is acquired, and the gradient value of the suspected waterlogged area in the judged rooftop image is obtained. The average gradient value of all pixels is recorded as the gradient performance value of the suspected water accumulation area in the rooftop image. The absolute difference between the energy values ​​corresponding to the gray values ​​of the suspected water accumulation area in the two rooftop images is multiplied by the absolute difference between the gradient performance values ​​of the suspected water accumulation area in the two rooftop images, and this product is used as the texture difference factor of the suspected water accumulation area in the two rooftop images. The average and normalized result of the texture difference factors between all pairs of rooftop images corresponding to the suspected water accumulation area is used as the attention level of the suspected water accumulation area. The normalization is performed using linear normalization, and the normalization object is all suspected water accumulation areas in all rooftop images (a successfully matched suspected water accumulation area is considered as one suspected water accumulation area).

[0065] Preferably, in one embodiment of the present invention, the credibility of each suspected water accumulation area is obtained based on the distance between the suspected water accumulation area and the drain outlet in the rooftop area, and the distribution of objects near the drain outlet in the rooftop area. Combined with the degree of attention, a significant weight for each suspected water accumulation area is obtained. The specific method includes:

[0066] For any rooftop area (the same rooftop area in multiple matched rooftop images), based on the construction drawings of the building corresponding to the rooftop area, obtain the spatial location (3D actual coordinates, not image coordinates) of the drainage outlet corresponding to the rooftop area. Then, mark the spatial locations of suspected water accumulation areas and object areas in the rooftop area, obtain the spatial location corresponding to the center of each suspected water accumulation area and the spatial location corresponding to the center of each object area in the rooftop area, and the shortest distance between each of them and the spatial location of the drainage outlet (there may be more than one rooftop drainage outlet, obtain the shortest distance and its corresponding drainage outlet), and use it as the drainage distance for each suspected water accumulation area and each object area.

[0067] Furthermore, for any suspected water accumulation area in the rooftop area, the drain outlet corresponding to the drainage distance of the suspected water accumulation area is recorded as the reference drain outlet of the suspected water accumulation area, and the drainage distance of the suspected water accumulation area is used as a distance threshold. Object areas in the rooftop area whose drainage distance to the reference drain outlet is less than the distance threshold are recorded as the obstruction areas of the suspected water accumulation area. The mean of the reciprocals of the drainage distances of each obstruction area is obtained, and the product of the mean of the reciprocals and the drainage distance of the suspected water accumulation area is used as the confidence level of the suspected water accumulation area.

[0068] Furthermore, the product of the credibility and the degree of attention of the suspected waterlogged area and the result of normalization are used as the significant weight of the suspected waterlogged area. The normalization adopts the linear normalization method, and the normalization object is the product corresponding to all suspected waterlogged areas.

[0069] It should be noted that the farther the suspected water accumulation area in the rooftop area is from the drain outlet, and the more objects obstruct the area near the drain outlet, the greater the likelihood that the water accumulation in the suspected water accumulation area is caused by the drain outlet not changing, and the higher its credibility. The significance weight is adjusted according to the degree of attention. The greater the degree of attention and credibility, the higher the weight should be assigned in the subsequent significance test.

[0070] Thus, the significant weights of each suspected waterlogged area were obtained.

[0071] Step S004: Based on the saliency weight of the suspected water accumulation area in the rooftop image, perform saliency detection on the rooftop image, and then construct a rooftop water accumulation recognition model.

[0072] It should be noted that after re-acquiring the saliency weights, by adjusting the saliency weights of the suspected water accumulation areas in the rooftop area, the saliency factors obtained in the saliency detection can more intuitively reflect the pixel information distribution of the water accumulation area. This can then be used as an attention map for the learning and training of the rooftop water accumulation recognition model, thereby achieving rapid recognition of rooftop water accumulation areas.

[0073] Specifically, for each suspected water accumulation area in any rooftop image, a saliency weight is obtained. In the traditional CA saliency detection process, the contrast weight of each suspected water accumulation area is adjusted to the corresponding saliency weight, and then saliency detection is performed to obtain the saliency factor of each pixel in the rooftop image. Based on the saliency factor of each pixel in the rooftop image, an attention map of the rooftop image is constructed.

[0074] Furthermore, a large number of rooftop images collected by drone aerial photography are used as training datasets. A CNN network is used to construct a rooftop water accumulation recognition model. Each rooftop image and its attention map are input into the rooftop water accumulation recognition model, which outputs the marked water accumulation areas. The model is then compared with the water accumulation areas marked manually on each rooftop image (water accumulation areas are marked as 1, and others as 0). The cross-entropy loss function is used to train the rooftop water accumulation recognition model. After the drone obtains new rooftop images, they are input into the trained rooftop water accumulation recognition model, which outputs the marked water accumulation areas, thus enabling rapid identification of the corresponding rooftop water accumulation areas.

[0075] This concludes the embodiment.

[0076] Another embodiment of the present invention provides a rooftop water accumulation area identification system based on UAV vision. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above method steps S001 to S004.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying rooftop water accumulation areas based on UAV vision, characterized in that, The method includes the following steps: The drones were used to conduct inspections along a fixed route, collecting rooftop images at multiple times. Edge detection and connected component extraction are performed on the rooftop image, and several rooftop regions are obtained through semantic segmentation, which further segments several local rooftop regions. Based on the distribution and grayscale representation of each local rooftop region, several clusters are obtained. Based on the clusters, the local rooftop regions are combined to obtain several regions to be analyzed. Based on the contour regularity of the regions to be analyzed, several object regions and suspected water accumulation regions are identified. The system matches suspected water accumulation areas in rooftop images at different times. Based on the texture differences and gradient performance of the same suspected water accumulation area in different rooftop images, the system obtains the attention level of each suspected water accumulation area. Based on the distance between the suspected water accumulation area and the drain outlet in the rooftop area, and the distribution of objects near the drain outlet in the rooftop area, the system obtains the credibility of each suspected water accumulation area. Combined with the attention level, the system obtains the significant weight of each suspected water accumulation area. The specific method for obtaining the level of attention given to each suspected waterlogged area is as follows: For any suspected waterlogged area, obtain several rooftop images of the suspected waterlogged area, and denote them as the judged rooftop images of the suspected waterlogged area; obtain the gray-level co-occurrence matrix of the suspected waterlogged area in any two judged rooftop images, and obtain the energy value corresponding to the gray value of the suspected waterlogged area in the two judged rooftop images based on the gray-level co-occurrence matrix; For any suspected waterlogged area in the rooftop image, obtain the gradient value of each pixel and record the average gradient value of all pixels in the suspected waterlogged area in the rooftop image as the gradient performance value of the suspected waterlogged area in the rooftop image. The absolute value of the difference between the energy values ​​corresponding to the gray values ​​of the suspected water accumulation area in the two rooftop images is multiplied by the absolute value of the difference between the gradient performance values ​​of the suspected water accumulation area in the two rooftop images, and the texture difference factor of the suspected water accumulation area in the two rooftop images is used as the texture difference factor of the suspected water accumulation area in the two rooftop images. The mean and normalized result of all the texture difference factors between each pair of rooftop images corresponding to the suspected water accumulation area is taken as the degree of attention of the suspected water accumulation area. The specific methods for obtaining the credibility of each suspected waterlogged area are as follows: For any rooftop area, based on the construction drawings of the building corresponding to the rooftop area, the spatial location of the drainage outlet corresponding to the rooftop area is obtained, and the spatial location of the suspected water accumulation area and object area in the rooftop area is marked. The spatial location corresponding to the center of each suspected water accumulation area and the spatial location corresponding to the center of each object area in the rooftop area are obtained, and the shortest distance between them and the spatial location of the drainage outlet is used as the drainage distance of each suspected water accumulation area and each object area. For any suspected water accumulation area in the rooftop area, the drain outlet corresponding to the drainage distance of the suspected water accumulation area is recorded as the reference drain outlet of the suspected water accumulation area, and the drainage distance of the suspected water accumulation area is used as the distance threshold. The object area in the rooftop area whose drainage distance corresponding to the reference drain outlet is less than the distance threshold is recorded as the obstruction area of ​​the suspected water accumulation area. Obtain the mean of the reciprocals of the drainage distances of each blocked area, and multiply the mean of the reciprocals by the drainage distance of the suspected waterlogged area to determine the confidence level of the suspected waterlogged area. Based on the saliency weight of suspected water accumulation areas in rooftop images, saliency detection is performed on rooftop images to construct a rooftop water accumulation recognition model.

2. The method for identifying rooftop water accumulation areas based on UAV vision according to claim 1, characterized in that, The specific methods for performing edge detection and connected component extraction on the rooftop image, and obtaining several rooftop regions through semantic segmentation, are as follows: For any rooftop image at any given time, several edges of the rooftop image are obtained through Canny edge detection. Connected components are extracted based on closed edges to obtain several connected components of the rooftop image. Several connected components that are not contained in other connected components are taken as several maximum contour regions of the rooftop image. A large number of rooftop images are used as the training dataset, and several maximum contour regions are obtained from each rooftop image. The maximum contour regions belonging to the rooftop are manually labeled to construct a semantic segmentation network. The rooftop image and several maximum contour regions are input into the trained semantic segmentation network to obtain several rooftop regions of the rooftop image.

3. The method for identifying rooftop water accumulation areas based on UAV vision according to claim 1, characterized in that, The segmentation yields several local rooftop areas, and the specific methods include: For any rooftop area, construct a sliding window and slide it from the top left corner to the next column without overlapping, resulting in several areas framed by the sliding window, which serve as several partial rooftop areas of that rooftop area.

4. The method for identifying rooftop water accumulation areas based on UAV vision according to claim 1, characterized in that, The specific methods for obtaining several clusters are as follows: For any rooftop region in any rooftop image, obtain the center of each rooftop local region and the average gray value of the pixels in each rooftop local region; perform density clustering on all rooftop local regions in the rooftop region, and use the product of the absolute value of the difference between the Euclidean distance between the centers of the rooftop local regions and the average gray value of their respective pixels to obtain several clusters of the rooftop region.

5. The method for identifying rooftop water accumulation areas based on UAV vision according to claim 1, characterized in that, The specific method for obtaining the aforementioned regions to be analyzed is as follows: For several rooftop local areas within the same cluster, adjacent rooftop local areas are combined to obtain several combined areas of the same cluster, which are then used as several regions to be analyzed.

6. The method for identifying rooftop water accumulation areas based on UAV vision according to claim 1, characterized in that, The specific methods for obtaining several object regions and suspected water accumulation regions are as follows: For any region to be analyzed in any rooftop image, obtain all edge pixels and their tangent directions of the region to be analyzed; obtain the minimum bounding rectangle of the region to be analyzed, obtain the proportion of the area of ​​the region to be analyzed in the area of ​​its minimum bounding rectangle, and use the ratio of the proportion to the variance of the tangent directions of all edge pixels of the region to be analyzed as the contour regularity of the region to be analyzed. Obtain the contour regularity of each region to be analyzed in the rooftop image. Arrange all regions to be analyzed in ascending order of contour regularity. Obtain the difference between the contour regularity of sequentially adjacent regions to be analyzed. Use the contour regularity of the right region to be analyzed in the two regions corresponding to the maximum value of all differences as the rule threshold. The regions in the rooftop image whose contour regularity is less than the rule threshold are identified as suspected water accumulation areas in the rooftop image; the regions in the rooftop image whose contour regularity is greater than or equal to the rule threshold are identified as object regions in the rooftop image.

7. The method for identifying rooftop water accumulation areas based on UAV vision according to claim 1, characterized in that, The specific method for matching suspected water accumulation areas in rooftop images at different times includes: Obtain several suspected water accumulation areas and object areas from rooftop images at various times; for any two rooftop images at different times, match the respective rooftop areas to obtain the matched rooftop areas in the two rooftop images, and match the suspected water accumulation areas and object areas in the matched rooftop areas based on their positions.

8. A rooftop water accumulation area identification system based on UAV vision, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the rooftop water accumulation area identification method based on UAV vision as described in any one of claims 1-7.

Images