Intelligent identification method and device for urban waterlogging based on multispectral imaging of drone swarms

Abstract

This application proposes a method and device for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms, including the following steps: dividing the area to be monitored into multiple inspection areas; coordinating and allocating inspection tasks for the drone swarm based on environmental data, drone data, and mission data of the area to be monitored; obtaining multispectral images of the corresponding inspection areas taken by the drones; inputting the multispectral images into a pre-trained waterlogging data recognition model to output the waterlogging areas within the current inspection area, obtaining point cloud depth data of the waterlogging areas, and predicting the water depth of the waterlogging areas based on the point cloud depth data. This solution uses the collaborative work of drone swarms to inspect urban areas to obtain images of waterlogging areas, and processes the images of waterlogging areas through multispectral imaging to accurately detect the size of the waterlogging areas.

Description

Technical Field

[0001] The present application relates to the field of disaster monitoring, and in particular to a method and device for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms. Background Art

[0002] As global climate change intensifies, extreme weather events become more frequent, especially rainstorms and heavy rainfall, which are becoming increasingly severe. Cities, as an important part of modern society, often face serious risks of waterlogging and flooding due to their dense building structures, limited natural drainage systems, and land use issues. Waterlogging not only affects traffic and increases travel costs, but can also trigger chain reactions such as damaged buildings and paralyzed infrastructure, causing huge economic losses and social impacts. Therefore, timely and accurate monitoring and assessment of urban waterlogging areas has become a major issue. Traditional waterlogging monitoring methods rely on manual inspections, ground sensors, rain gauges, and other means, but they have significant limitations, such as low manual inspection efficiency, limited ground sensor coverage, susceptibility to interference in real-time and accuracy, and delayed monitoring data. These methods are unable to efficiently and accurately respond to the complex and ever-changing needs of urban waterlogging monitoring, and new intelligent and efficient technical means are urgently needed.

[0003] Remote sensing technology is an emerging detection technology. It is a comprehensive technology that uses various sensors based on the theory of electromagnetic waves to collect, process and image the reflected electromagnetic wave signals radiated by distant targets, thereby realizing the detection and identification of various ground scenes. Applying remote sensing technology to waterlogging detection can provide key data support for waterlogging monitoring and early warning with its unique advantages such as wide coverage, fast observation speed, adjustable temporal and spatial resolution, and few restrictions on ground conditions. It plays an important role in urban waterlogging monitoring, flood disaster assessment, water resources management and other fields. However, the imaging method of remote sensing technology is traditional optical imaging, such as RGB imaging, for waterlogging detection. Optical imaging is often affected by weather conditions such as clouds and haze, resulting in poor monitoring accuracy.

[0004] In summary, how to use remote sensing technology to quickly and accurately detect urban road waterlogging is a problem that needs to be solved urgently by existing technologies. Summary of the Invention

[0005] The embodiments of the present application provide a method and device for intelligently identifying urban waterlogging based on multispectral imaging of a drone swarm. The method patrols urban areas through the collaborative work of a drone swarm to obtain images of waterlogged areas, and processes the images of waterlogged areas through multispectral imaging to accurately detect the size of the waterlogged areas.

[0006] In a first aspect, an embodiment of the present application provides a method for intelligently identifying urban waterlogging based on multispectral imaging of a drone swarm, the method comprising:

[0007] Divide the area to be monitored into multiple inspection areas, and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas;

[0008] Coordinate and allocate inspection tasks to the drone fleet based on environmental data, drone data, and mission data of the area to be monitored, with at least one drone assigned to each inspection area;

[0009] Acquire a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images;

[0010] The multispectral image is input into a pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, obtain the point cloud depth data of the water accumulation area, and predict the water depth of the water accumulation area based on the point cloud depth data to obtain the water accumulation depth.

[0011] In a second aspect, an embodiment of the present application provides an intelligent urban waterlogging identification device based on multispectral imaging of a drone swarm, comprising:

[0012] The division module is used to divide the area to be monitored into multiple inspection areas and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas;

[0013] An allocation module coordinates and allocates inspection tasks to the drone fleet based on the environmental data of the monitored area, drone data, and task data, where at least one drone is allocated to each inspection area;

[0014] An acquisition module acquires a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images;

[0015] The water accumulation recognition module is used to input the multispectral image into the pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, obtain the point cloud depth data of the water accumulation area, and predict the water depth of the water accumulation area based on the point cloud depth data to obtain the water accumulation depth.

[0016] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for intelligently identifying urban waterlogging based on multispectral imaging of a drone swarm.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. When the program code is executed by a processor, an intelligent method for identifying urban waterlogging based on multispectral imaging of a drone swarm is implemented.

[0018] The main contributions and innovations of the present invention are as follows:

[0019] The embodiment of the present application monitors drone inspection data in real time. When the waterlogged area changes, the system automatically adjusts the division of the inspection area according to the new waterlogging data and updates the drone inspection tasks according to the new area information, thereby effectively responding to changes in waterlogging and new emergency tasks. When selecting the executing drone, this solution not only considers the basic flight capabilities of the drone, but also integrates factors such as real-time environmental data, weather changes, mission urgency, and drone collaboration, thereby improving the accuracy, real-timeness, and robustness of task execution. This solution combines image data from multiple different spectra, including visible light, near-infrared, and multispectral data. The combination of multispectral data enables this solution to accurately identify waterlogged areas under different lighting and weather conditions, especially in dark or low-contrast environments. This solution adaptively adjusts the window size based on the size and morphology of the water body in the multispectral feature map and performs window attention calculation. By integrating the features extracted at multiple scales through a weighted fusion mechanism, the key information of each scale is fully retained. Through cross-scale feature fusion, the model can better handle water areas of different scales, ensuring that both small-scale waterlogging and large-scale puddles can be accurately identified.

[0020] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 This is a flow chart of a method for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms according to an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of segmentation of an inspection area according to an embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of obtaining the size of a water accumulation area based on a water accumulation area mask according to an embodiment of the present application.

[0025] Figure 4 This is a structural block diagram of an intelligent urban waterlogging identification device based on multispectral imaging of drone swarms according to an embodiment of the present application;

[0026] Figure 5Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0028] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0029] Example 1

[0030] The embodiment of the present application provides an intelligent identification method for urban waterlogging based on multispectral imaging of drone swarms. The method uses the collaborative work of drone swarms to patrol urban areas to obtain waterlogged areas, and processes the images of waterlogged areas by combining multispectral imaging with point cloud depth data to accurately detect the size and depth of the waterlogged areas. Specifically, Figure 1 , the method comprising:

[0031] Divide the area to be monitored into multiple inspection areas, and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas;

[0032] Coordinate and allocate inspection tasks to the drone fleet based on environmental data, drone data, and mission data of the area to be monitored, with at least one drone assigned to each inspection area;

[0033] Acquire a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images;

[0034] The multispectral image is input into a pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, obtain the point cloud depth data of the water accumulation area, and predict the water depth of the water accumulation area based on the point cloud depth data to obtain the water accumulation depth.

[0035] In some embodiments, multiple base stations are set up in the area to be monitored, and each base station is connected to multiple drones. The area to be monitored is divided into multiple inspection areas according to the positions of the base stations in the area to be monitored, and the number of areas of the inspection area is dynamically adjusted according to the historical water accumulation data of each inspection area, wherein the historical water accumulation data includes the water accumulation area and the water depth of the water accumulation area.

[0036] In some specific implementations, the setting of base stations needs to take into account the city map, the take-off / landing requirements of drones, and the convenience of geographical location. Base stations should be deployed in key locations in the city, such as transportation centers, fire stations, urban management offices, etc., to ensure that emergency support can be provided at any time. It is also necessary to ensure that the base stations are evenly distributed in the monitored area to avoid excessive concentration of drones on a certain base station.

[0037] Specifically, the base station should be located in a moderate position in the inspection area to prevent the drone from flying too long a distance, and the reasonable distance between each base station should be calculated based on the battery life of the drone to ensure that the drone can better complete the inspection task.

[0038] Specifically, multiple base stations in the area to be monitored are set at initial fixed positions, and then the initial inspection area is divided based on the position of each base station using the Voronoi algorithm. The formula is expressed as:

[0039]

[0040] in, For the base station, For base stations Inspection area, p is any point in the area to be monitored, From point p to the base station distance, From point p to other base stations That is to say, this solution uses the Voronoi algorithm to determine the distance from each location in the monitored area to different base stations, and thus assigns each location to the nearest base station to complete the division of the inspection area.

[0041] In some specific embodiments, the severity of water accumulation at different locations in the monitored area is determined based on historical data, the base station location is dynamically updated based on the severity of water accumulation at different locations, and the inspection area is dynamically divided based on the updated base station location, wherein the severity of water accumulation is proportional to the density of base stations.

[0042] That is to say, in the area to be monitored, the more severe the waterlogging is, the denser the base stations are, that is, the more inspection areas are divided and the smaller the area of ​​each inspection area is, thereby achieving accurate monitoring of the location.

[0043] For example, the severity of waterlogging at different locations is obtained by taking weighted sum of the waterlogging areas and the water depths in the waterlogging areas at different locations within the monitoring area. As random monitoring is continuously carried out and areas with severe waterlogging are treated, when the severity of waterlogging in areas with severe waterlogging decreases, the base station density in the corresponding area will also decrease accordingly.

[0044] Specifically, the segmentation diagram of the inspection area is as follows: Figure 2 As shown, each inspection area in this solution corresponds to a base station. For example, if the waterlogging in an area is severe, the number of base stations in this area is increased, thereby dividing this area into multiple inspection areas to enhance the inspection accuracy of areas with severe waterlogging.

[0045] Specifically, the formula for dynamically updating the inspection area is as follows:

[0046]

[0047] in, For the updated base station, For base stations Inspection area, p is any point in the area to be monitored, From point p to the base station distance, From point p to other base stations That is to say, in areas with serious waterlogging, such as low-lying areas or waterlogging points, the algorithm in this solution will divide these areas into more inspection areas to improve detection accuracy.

[0048] In some specific embodiments, an inspection task is formulated for each inspection area, and an execution drone is selected in the base station corresponding to the inspection area to which the inspection task belongs. The inspection task is performed by the execution drone, wherein the drones in the base station that meet the inspection task execution requirements are used as pre-selected drones, and the environmental adaptability score of each pre-selected drone is calculated based on the environmental data of the inspection area, and the drone with the highest environmental adaptability score is selected as the execution drone.

[0049] Furthermore, the task data of the inspection task and the drone data of each drone in the base station are obtained, and the pre-selected drones are obtained based on the matching degree of the task data and the drone data, wherein the task data includes the flight time required for the task, the load required for the task, and the sensor type required for the task, and the drone data includes the maximum flight time of the drone, the maximum load of the drone, and the sensors carried by the drone.

[0050] Exemplarily, the task data requirements for the inspection task include the flight time required for the task, the payload required for the task, and the sensor type required for the task. That is to say, the maximum flight time, maximum payload, and onboard sensors of each drone in the corresponding base station are calculated, and drones whose maximum flight time is greater than the flight time required for the task, whose maximum payload is greater than the payload required for the task, and whose onboard sensors meet the sensor type required for the task are selected as pre-selected drones.

[0051] Specifically, the calculation formula for the maximum flight time of a drone is as follows:

[0052]

[0053] in, is the maximum flight time of the drone, is the remaining power of the drone battery, is the average power consumption of the UAV.

[0054] Specifically, use Indicates the maximum carrying capacity of the drone, Indicates the payload required for the mission. The following formula is used to determine whether the drone meets the mission execution requirements:

[0055]

[0056] Specifically, the calculation formula for the flight efficiency of the drone is as follows:

[0057]

[0058] in, It represents the flight efficiency of the UAV, Distance represents the mission distance of executing the waterlogging event, and Speed ​​represents the flight speed of the UAV.

[0059] In addition, this solution also calculates the degree of match between each drone and the task execution requirements of the inspection task according to the following formula:

[0060]

[0061] in, Indicates the matching degree between the pre-selected UAV and the task execution requirements of the flooding event, The battery capacity matching degree is used to indicate the matching degree between the remaining power of the drone battery and the flight time required for the mission. The flight time matching degree is used to indicate the matching degree between the maximum flight time of the UAV and the flight time required for the mission. The load capacity matching degree is used to indicate the matching degree between the load required by the mission and the maximum load of the UAV. The sensor matching degree is used to indicate the degree of matching between the sensors carried by the UAV and the sensor types required for the mission.

[0062] Specifically, the flight capability matching The drone is evaluated to see whether it has the flight capability required to perform a specific task. The higher the matching degree, the greater the possibility that the drone is suitable for the task. Therefore, in other embodiments, drones with a flight capability matching degree greater than a set threshold are selected as pre-selected drones.

[0063] In some specific embodiments, the environmental data of the inspection area includes wind speed, precipitation, and lighting conditions. The formula for calculating the environmental adaptability score of each pre-selected drone is as follows:

[0064]

[0065] in, Score the drone's environmental adaptability. Indicates the impact of wind speed on the flight stability of the drone. It represents the impact score of precipitation on the sensitivity of drone sensors and flight stability. Indicates the impact of low light or haze weather on image acquisition and sensor visual range. represents the adaptability score of the drone’s sensor performance, Rate the impact of battery life on task completion, is the corresponding weight coefficient, which is used to balance the influence of various factors.

[0066] Specifically, the environmental adaptability score reflects the comprehensive capabilities of each pre-selected drone in a specific environment and determines whether it is suitable for performing inspection tasks.

[0067] It is worth mentioning that the various scoring methods in this solution are set according to actual conditions. For example, in the threshold scoring method, if the wind speed is > xm / s, the score = y, etc. This solution does not provide a detailed explanation of the calculation method of each score.

[0068] Specifically, when selecting the drones to execute, this solution not only considers the basic flight capabilities of the drones, but also integrates factors such as real-time environmental data, weather changes, and mission urgency, thereby improving the accuracy, real-timeness, and robustness of mission execution, thereby improving the efficiency and reliability of the drone swarm when performing inspection tasks.

[0069] In some specific embodiments, when there are multiple waterlogging events, the mission urgency score of each waterlogging event is calculated based on the historical waterlogging depth, geographical location, traffic impact, and infrastructure impact of each inspection task, and drones are assigned to execute waterlogging events with high mission urgency scores first.

[0070] Specifically, the formula for task urgency scoring is as follows:

[0071]

[0072] in, Rate the urgency of the task for the flooding incident. is the historical water depth, usually in millimeters or centimeters, affecting the urgency of the task. is the geographical location, used to indicate the impact of proximity to important infrastructure, residential areas, transportation hubs, etc. Traffic impact is used to indicate the traffic conditions in the mission area and whether it will cause greater traffic congestion. Infrastructure risk is used to indicate the impact on urban infrastructure (such as drainage systems, power facilities, etc.). is the weight coefficient, which is used to balance various influencing factors.

[0073] Specifically, by assigning a task urgency score to each waterlogging event, the system is helped to dynamically adjust the priority of the task. The higher the task urgency score of the waterlogging event, the higher the priority of the waterlogging event, to ensure that serious waterlogging events are executed first.

[0074] In some specific embodiments, drone inspection tasks are preferentially assigned to areas with severe waterlogging that require priority treatment. By dynamically adjusting task priorities, the algorithm can achieve optimal resource scheduling and ensure that urgent tasks receive timely responses. The relationship between task priority and regional resource allocation can be optimized through the following methods:

[0075]

[0076] in, Indicates the priority of waterlogging event i in inspection area j, represents the weight of waterlogging event i, It represents the area of ​​the inspection area, and n is the number of all inspection areas.

[0077] In some specific embodiments, the urgency of the flooding event is dynamically adjusted based on the execution progress of the inspection task, the status of the executing drone, and environmental changes, thereby achieving dynamic task allocation. The formula is as follows:

[0078]

[0079] in, For the adjusted task urgency score of the flooding incident, The depth of accumulated water changes due to weather changes. In order to take into account the task redistribution caused by the change of UAV power consumption and flight time, This is to take into account the task adjustments caused by changes in water depth and traffic conditions.

[0080] Specifically, the dynamic task adjustment mechanism in this scheme ensures that tasks can be completed efficiently based on environmental changes and task execution progress by continuously adjusting the priority and allocation of tasks.

[0081] In some embodiments, the multispectral data fusion module normalizes the multispectral image of the waterlogged area and uses each spectral band as an independent input channel, and extracts features from each independent input channel through a convolution layer to obtain features of different spectral bands, and performs weighted fusion on the features of each spectral band to obtain a multispectral feature map.

[0082] Specifically, multispectral images mainly include visible light, near-infrared and multispectral data. Visible light (RGB) images are suitable for daytime waterlogging monitoring and can quickly identify waterlogging areas and ground features. Near-infrared (NIR) images can effectively identify water bodies, especially providing clear boundaries of waterlogged areas at night or in low-light environments. Multispectral (MS) images, including other bands (such as infrared, short-wave infrared, etc.), are used to provide additional ground object information and enhance recognition accuracy. Through optical data in different bands, more comprehensive information can be provided, thereby improving the accuracy and robustness of waterlogging monitoring.

[0083] Specifically, the formula for normalizing the multispectral image is expressed as follows:

[0084]

[0085] Where I is the multispectral image, , where H is the image height, W is the image width, and C is the number of bands. and are the mean and standard deviation of the image, is the normalized multispectral image.

[0086] Specifically, the image data collected by drones contains multiple spectral bands. Visible light images are used to capture detailed information about urban environments, while near-infrared bands can significantly enhance the recognition of water areas, especially in low-contrast, low-saturation water areas, such as flooded roads. In addition, multispectral images provide rich water information under different lighting and weather conditions. Therefore, this solution treats each spectral band as an independent input channel and extracts features from each input channel through a convolutional layer. The formula is as follows:

[0087]

[0088] in, is the extracted feature representation, Conv is the convolution layer, are independent input channels, where which are used to represent different spectral bands.

[0089] Furthermore, the formula for weighted fusion of the features of each spectral band is expressed as follows:

[0090]

[0091] in, is the multispectral feature map, is the weight of different spectral bands, and N is the number of spectral bands.

[0092] In some specific embodiments, urban water bodies have more complex forms, including water accumulation on roads, water accumulation under bridges, water accumulation in sewers, etc. These water features may have different visual appearances due to scale differences. However, traditional encoding layers usually use fixed-size windows for self-attention calculations, which cannot adapt to features of different scales. Therefore, this solution selects the window size based on the scale of the input feature map of each encoding layer.

[0093] That is to say, in a multi-scale coding module composed of multiple cascaded coding layers, the scale of the feature map processed by each coding layer is different. The window size is set based on the scale of the feature map processed by each coding layer, and the window attention calculation is performed with the set window size in each coding layer. After weighted fusion of all the window attention calculation results in the same coding layer, an intermediate feature map is obtained, and the intermediate feature map is input into the next coding layer. The output result of the last coding layer is a multi-scale water accumulation feature map.

[0094] Specifically, the formula for calculating the attention result of each window in the encoding layer is expressed as follows:

[0095]

[0096] in, and are query and key respectively, is the attention result of the i-th window.

[0097] Specifically, the formula for weighted fusion of the attention results of each window to obtain the multi-scale water feature map is as follows:

[0098]

[0099] in, is the intermediate feature map, is the weighting coefficient for each window, is the attention result for each window, It is the feature after window self-attention calculation.

[0100] Specifically, this solution integrates features extracted at multiple scales through a weighted fusion mechanism, fully preserving key information at each scale. This cross-scale feature fusion allows the model to better handle water bodies of varying sizes, ensuring accurate identification of both small areas of accumulated water and large puddles.

[0101] Specifically, in encoders at different levels, the window size is adjusted in advance based on the spatial structure of the image and the distribution of water bodies. For low-level detail feature extraction, a smaller window is used to extract fine-grained features, while for high-level global feature extraction, a larger window is used to process global information.

[0102] Specifically, the multi-scale window mechanism in this solution enables the model to capture the characteristics of water areas at different scales. Small windows focus on local features (such as the texture of water accumulation on a single street), while large windows focus on global features (such as large areas of water accumulation).

[0103] In some specific embodiments, the spatial features of the multi-scale water accumulation feature map are the boundaries and morphological features of the water accumulation, and the channel features are the color and brightness features of the water accumulation. The formula for performing joint attention calculation on the spatial features and channel features of the multi-scale water accumulation feature map is expressed as follows:

[0104]

[0105] in, is the spatial feature, is the channel feature.

[0106] Specifically, urban flooding often involves complex backgrounds and diverse water characteristics. Jointly modeling spatial and channel information is crucial for accurate water segmentation. Traditional self-attention mechanisms fail to fully consider the effective integration of spatial and channel features, leading to inaccurate segmentation of some details, particularly water edges. However, this proposed scheme, combining spatial and channel attention, effectively captures the complex characteristics of water areas.

[0107] In some specific embodiments, the enhanced water accumulation feature map is decoded by a multi-layer decoder. In each layer of the decoder, as the image resolution is gradually restored, the model will gradually refine the segmentation boundary of the water accumulation area. The decoder fuses the low-level detail information and the high-level abstract information through jump connections to ensure that the boundary of the water accumulation area in the final output image is clearer and avoid mis-segmentation in low-contrast areas (for example, edges, shadows). The present invention regards the water accumulation area segmentation task as an image binary classification task. A binary image is output, in which the water accumulation area is marked as 1 (or a high probability value) and the non-water accumulation area is marked as 0 (or a low probability value). The model calculates the probability of each pixel belonging to the water accumulation area, and uses a threshold operation to classify each pixel as water accumulation (1) or non-water accumulation (0) to obtain a water accumulation area mask. The schematic diagram of obtaining the size of the water accumulation area based on the water accumulation area mask is shown in FIG. Figure 3 shown.

[0108] Specifically, in the last decoder layer, a fully connected layer is used to perform binary classification on each pixel and output the probability value p of each pixel belonging to the water accumulation area, ranging from 0 to 1. The water accumulation area mask is obtained by thresholding the probability value of each pixel. The formula is expressed as:

[0109]

[0110] Among them, P is the water accumulation area mask, W is the weight matrix, and b is the bias term.

[0111] In some specific embodiments, point cloud depth data of the waterlogged area is acquired by executing a drone or other monitoring equipment, and a classification head is used to perform depth prediction on the point cloud depth data to obtain waterlogged depth information.

[0112] Specifically, the classification head is a classification model that has been trained in advance. During the training process, the point cloud depth data is input into the classification head to obtain the predicted depth information. Then, a regression loss function is constructed based on the real depth information and the predicted depth information. The parameters of the classification head are adjusted based on the value of the regression loss function. When the value of the regression loss function meets the set conditions, the parameters of the classification head are saved.

[0113] Specifically, the regression loss function is expressed as follows:

[0114]

[0115] Among them, L is the regression loss function, N is the number of samples, For real depth information, To predict depth information.

[0116] In some specific embodiments, the acquired waterlogged area and water depth are used as historical waterlogging data to adjust the inspection area in real time to complete the closed loop.

[0117] In some specific embodiments, a risk assessment value is constructed based on the waterlogging area, waterlogging depth, waterlogging growth rate and drainage capacity of the waterlogging area. When the risk assessment value is greater than a set threshold or a set threshold for the waterlogging depth, a waterlogging warning is triggered.

[0118] Specifically, the formula for calculating the risk assessment value is as follows:

[0119]

[0120] Where R is the risk assessment value, As a risk assessment function, a machine learning model can be used, where A is the waterlogged area, D is the waterlogged depth, V is the waterlogged growth rate, and P is the drainage capacity.

[0121] Example 2

[0122] Based on the same concept, refer to Figure 4 This application also proposes an intelligent urban waterlogging identification device based on multispectral imaging of drone swarms, including:

[0123] The division module is used to divide the area to be monitored into multiple inspection areas and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas;

[0124] An allocation module coordinates and allocates inspection tasks to the drone fleet based on the environmental data of the monitored area, drone data, and task data, where at least one drone is allocated to each inspection area;

[0125] An acquisition module acquires a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images;

[0126] The water accumulation recognition module is used to input the multispectral image into the pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, obtain the point cloud depth data of the water accumulation area, and predict the water depth of the water accumulation area based on the point cloud depth data to obtain the water accumulation depth.

[0127] Example 3

[0128] This embodiment also provides an electronic device, referring to Figure 5 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0129] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0130] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0131] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0132] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the urban waterlogging intelligent identification methods based on multispectral imaging of drone swarms in the above embodiments.

[0133] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0134] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0135] The input / output device 408 is used to input or output information. In this embodiment, the input information may be a multispectral image of a waterlogged area, and the output information may be the size of the waterlogged area.

[0136] Optionally, in this embodiment, the processor 402 may be configured to execute the following steps through a computer program:

[0137] Divide the area to be monitored into multiple inspection areas, and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas;

[0138] Coordinate and allocate inspection tasks to the drone fleet based on environmental data, drone data, and mission data of the area to be monitored, with at least one drone assigned to each inspection area;

[0139] Acquire a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images;

[0140] The multispectral image is input into a pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, obtain the point cloud depth data of the water accumulation area, and predict the water depth of the water accumulation area based on the point cloud depth data to obtain the water accumulation depth.

[0141] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0142] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0143] The embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components that are configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, it should be noted at this point that, for example, Figure 5 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0144] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above embodiments merely illustrate several embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An intelligent identification method for urban waterlogging based on multispectral imaging of drone swarms, characterized by: The following steps are involved: Divide the area to be monitored into multiple inspection areas, and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas; Coordinate and allocate inspection tasks to the drone fleet based on environmental data, drone data, and mission data of the area to be monitored, with at least one drone assigned to each inspection area; Acquire a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images; The multispectral image is input into the pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, wherein the water accumulation data recognition model includes a multispectral data fusion module, a multi-scale encoding module, a spatial attention module and a decoder, wherein the multispectral data fusion module normalizes the multispectral image of the water accumulation area and uses each spectral band as an independent input channel, and extracts features from each independent input channel through the convolution layer to obtain features of different spectral bands, and performs weighted fusion on the features of each spectral band to obtain a multispectral feature map, which is input into the multispectral feature map. In the scale coding module, the multi-scale coding module includes multiple cascaded coding layers. Windows of different sizes are used in each coding layer to perform window self-attention calculation to obtain a multi-scale water accumulation feature map. The multi-scale water accumulation feature map is input into the spatial attention module to perform joint attention calculation of spatial features and channel features to obtain an enhanced water accumulation feature map. The enhanced water accumulation feature map is input into the decoder for decoding and output to obtain a water accumulation area mask. The water accumulation area is determined based on the water accumulation area mask, and the point cloud depth data of the water accumulation area is obtained. The water depth of the water accumulation area is predicted based on the point cloud depth data to obtain the water accumulation depth.

2. The method for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms according to claim 1 is characterized in that: A plurality of base stations are set up in the area to be monitored, and each base station is connected to a plurality of drones. The area to be monitored is divided into a plurality of inspection areas according to the positions of the base stations in the area to be monitored, and the number of areas of the inspection area is dynamically adjusted according to the historical water accumulation data of each inspection area, wherein the historical water accumulation data includes the water accumulation area and the water depth of the water accumulation area.

3. The method for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms according to claim 1 is characterized in that: An inspection task is formulated for each inspection area, and an execution drone is selected in the base station corresponding to the inspection area to which the inspection task belongs. The execution drone performs the inspection task. Among them, the drones in the base station that meet the inspection task execution requirements are used as pre-selected drones. The environmental adaptability score of each pre-selected drone is calculated based on the environmental data of the inspection area, and the drone with the highest environmental adaptability score is selected as the execution drone.

4. The method for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms according to claim 3 is characterized in that: Obtain the task data of the inspection task and the drone data of each drone in the base station, and obtain the pre-selected drones based on the matching degree of the task data and the drone data. The task data includes the flight time required for the task, the required payload and the type of sensor required for the task. The drone data includes the maximum flight time of the drone, the maximum payload and the sensors carried by the drone.

5. The method for intelligently identifying urban waterlogging based on multispectral imaging of drone swarms according to claim 3 is characterized in that: When the same inspection area includes multiple inspection tasks, the task urgency score of the corresponding inspection task is evaluated based on the address of the inspection task location, historical water depth, traffic impact, and infrastructure impact, and inspection tasks with high task urgency scores are assigned to drones first.

6. An intelligent urban waterlogging identification device based on multispectral imaging of drone swarms, characterized by: include: The division module is used to divide the area to be monitored into multiple inspection areas and adjust the inspection areas in real time based on the historical water accumulation data of the inspection areas; An allocation module coordinates and allocates inspection tasks to the drone fleet based on the environmental data of the monitored area, drone data, and task data, where at least one drone is allocated to each inspection area; An acquisition module acquires a multispectral image of the corresponding inspection area taken by the drone, wherein the multispectral image is a combination of at least two different spectral images; The water accumulation recognition module is used to input the multispectral image into the pre-trained water accumulation data recognition model to output the water accumulation area in the current inspection area, wherein the water accumulation data recognition model includes a multispectral data fusion module, a multi-scale encoding module, a spatial attention module and a decoder, wherein the multispectral data fusion module normalizes the multispectral image of the water accumulation area and uses each spectral band as an independent input channel, and extracts features from each independent input channel through the convolution layer to obtain features of different spectral bands, and performs weighted fusion on the features of each spectral band to obtain a multispectral feature map, and the multispectral features are combined into a single image. The image is input into a multi-scale encoding module, which includes multiple cascaded encoding layers. Windows of different sizes are used in each encoding layer to perform window self-attention calculation to obtain a multi-scale water accumulation feature map. The multi-scale water accumulation feature map is input into a spatial attention module to perform joint attention calculation of spatial features and channel features to obtain an enhanced water accumulation feature map. The enhanced water accumulation feature map is input into a decoder for decoding and output to obtain a water accumulation area mask. The water accumulation area is determined based on the water accumulation area mask, and point cloud depth data of the water accumulation area is obtained. The water depth of the water accumulation area is predicted based on the point cloud depth data to obtain the water accumulation depth.

7. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for intelligently identifying urban waterlogging based on multispectral imaging of a drone swarm as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process. When the program code is executed by a processor, an intelligent urban waterlogging identification method based on multispectral imaging of a drone swarm as described in any one of claims 1 to 5 is implemented.

Images