A multi-area coverage inspection system using drone swarms

Abstract

This invention discloses a multi-area coverage inspection system using a drone swarm, specifically relating to the field of drone swarm inspection technology. The system includes: an inspection area division module, a drone swarm setting module, a road damage monitoring module, a traffic sign monitoring module, a monitoring response module, a working status judgment module, an inspection display and early warning module, and a database. This invention employs multi-area drone swarm inspection. In swarm mode, multiple drones can work collaboratively, achieving efficient inspection of large areas and multiple regions through coordinated flight and division of labor, greatly improving inspection speed and accuracy while reducing labor costs and safety risks. This invention divides highway inspection areas, specifically improving inspection speed and accuracy, ensuring inspection accuracy in high-risk areas, and increasing inspection efficiency in general areas.

Description

Technical Field

[0001] This invention relates to the field of drone swarm inspection technology, and more specifically to a multi-area coverage inspection system using drone swarms. Background Technology

[0002] The total length of highways is often very large, and the extensive inspection area makes manual inspection very time-consuming, costly, and difficult to guarantee comprehensive coverage. Traditional manual inspection often cannot achieve real-time and accurate monitoring of the entire road system, and cannot meet the growing demand. Therefore, it is necessary to study the multi-area coverage inspection technology of unmanned aerial vehicle (UAV) swarms for highways.

[0003] The existing technology, such as the invention patent application CN118960752A, discloses a method and system for full-process path planning of multi-UAV inspection on highways. The method includes: constructing a static obstacle grid map; determining the first path and the second path of the UAV within the static obstacle grid map according to an adaptive extended domain algorithm; calculating the length of the first path and the second path according to the first path and the second path respectively; determining whether the total length of the inspection process path meets the UAV's cruising distance requirements based on the first path length and the second path length; determining whether the road segment to be inspected can be covered for inspection based on the UAV's travel radius and the width of the road segment to be inspected; determining the third path, the fourth path, and the fifth path; and combining the third, fourth, and fifth paths of each UAV to obtain the complete path of each UAV.

[0004] Existing technology, such as the invention patent application CN118798591A, discloses an intelligent inspection system for highway equipment. This system includes: acquiring a parameter Z reflecting the aging rate of the highway through historical vehicle information and temperature and humidity data; integrating this parameter Z with the maintenance density of each road segment; and further allocating the number of inspections to each monitoring segment on the highway based on this. This invention also analyzes traffic flow information of each monitoring segment at different time periods and rationally selects the scheme with the minimum traffic flow or the least overall traffic flow impact as the final inspection scheme. This ensures that the drone inspection unit can reduce the obstruction of road surface by vehicles during inspection work, enabling the drone to collect as much complete road surface information as possible for each monitoring segment.

[0005] Based on the above solutions, it was found that existing technologies for highway inspection neglect the fact that, due to the vast length of highways, relying on a single drone for inspection can only cover a small portion of the area, failing to cover the entire highway. This results in low inspection efficiency, an inability to accurately and quickly inspect the actual road conditions and identify potential hazards, and consequently, compromises highway safety, increasing the risk of accidents. Furthermore, the technology neglects to assess the drone's status and lacks an efficient collaborative system to address abnormal drone operation, leading to operational risks and instability during drone inspections. Ultimately, this results in low efficiency for highway drone inspections, making it difficult to replace manual inspections for efficient and accurate highway patrols. Summary of the Invention

[0006] The purpose of this invention is to provide a multi-area coverage inspection system using a swarm of drones, which solves the problems existing in the background art.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a multi-area coverage inspection system using a swarm of unmanned aerial vehicles (UAVs), the system comprising:

[0008] The inspection area division module is used to obtain map information of the highway to be inspected and various flight parameters of the UAV, divide the highway to be inspected into inspection areas, and divide each inspection segment according to the division interval.

[0009] The drone swarm setting module obtains the road damage risk index and traffic sign risk index of each inspection area in each inspection section from historical inspections, classifies the inspection level of each inspection area in each inspection section, and sets the working mode of the drones based on the inspection level of each inspection area in each inspection section.

[0010] The road damage monitoring module is used to analyze road inspection videos of each inspection area in each inspection section, obtain various road parameters of each inspection area in each inspection section, and calculate the road damage risk index of each inspection area in each inspection section.

[0011] The traffic sign monitoring module is used to analyze the inspection videos of traffic signs in each inspection area of ​​each inspection section, obtain the parameters of traffic signs in each inspection area of ​​each inspection section, and calculate the risk index of traffic signs in each inspection area of ​​each inspection section.

[0012] The monitoring and response module is used to calculate the highway risk coefficient of each inspection area in each inspection section, assess the highway risk coefficient level of each inspection area in each inspection section, and implement response measures corresponding to the damage level of traffic signs in each inspection area in each inspection section.

[0013] The working status judgment module is used to monitor the working status parameters of the inspection drones in each inspection area of ​​each inspection section. If an abnormality is judged, the working status abnormality response measures are executed.

[0014] The inspection display and early warning module is used to build a 3D model of the highway to be inspected, and to display the inspection data and drone operating status information of each inspection area.

[0015] The advantages of this invention compared to the prior art are as follows:

[0016] I. This invention divides highways into inspection areas and, based on the risk levels of highways in previous inspections, classifies each inspection area into inspection levels. It sets specific parameters for inspection drones for inspection areas of different inspection levels, thereby improving the speed and accuracy of inspections in a targeted manner. This ensures the inspection accuracy of high-risk areas and improves the inspection efficiency of low-risk areas.

[0017] Second, this invention adopts a multi-area inspection by drone swarm. In swarm mode, multiple drones can work together. Through coordinated flight and division of labor, they can achieve efficient inspection of a large area and multiple regions, which greatly improves the speed and accuracy of inspection, while reducing labor costs and safety risks.

[0018] Third, this invention uses image recognition to identify the road surface conditions and traffic signs of highways, analyze the risk level of highways, and then take corresponding response measures to effectively solve the safety hazards existing on highways and ensure the safety of highway traffic.

[0019] Fourth, this invention establishes a working network architecture for a drone swarm system, consisting of a control center, a lead drone, and patrol drones. The lead drone monitors the working status of each patrol drone and responds promptly to patrol drones with abnormal working status, thereby increasing the stability of the drone swarm inspection and ensuring the controllability of the drone inspection process. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a system structure connection diagram of the present invention.

[0022] Figure 2 This is a schematic diagram of the network architecture of the present invention. Detailed Implementation

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0024] Reference Figure 1 As shown. This invention provides a multi-area coverage inspection system using a drone swarm. The system includes: an inspection area division module, used to acquire map information of the highway to be inspected and various flight parameters of the drones, to divide the highway to be inspected into inspection areas, and to divide each inspection segment according to the division interval.

[0025] The drone swarm setting module obtains the road damage risk index and traffic sign risk index of each inspection area in each inspection section from historical inspections, classifies the inspection level of each inspection area in each inspection section, and sets the working mode of the drones based on the inspection level of each inspection area in each inspection section.

[0026] The road damage monitoring module is used to analyze road inspection videos of each inspection area in each inspection section, obtain various road parameters of each inspection area in each inspection section, and calculate the road damage risk index of each inspection area in each inspection section.

[0027] The traffic sign monitoring module is used to analyze the inspection videos of traffic signs in each inspection area of ​​each inspection section, obtain the parameters of traffic signs in each inspection area of ​​each inspection section, and calculate the risk index of traffic signs in each inspection area of ​​each inspection section.

[0028] The monitoring and response module is used to calculate the highway risk coefficient of each inspection area in each inspection section, assess the highway risk coefficient level of each inspection area in each inspection section, and implement response measures corresponding to the damage level of traffic signs in each inspection area in each inspection section.

[0029] The working status judgment module is used to monitor the working status parameters of the inspection drones in each inspection area of ​​each inspection section. If an abnormality is judged, the working status abnormality response measures are executed.

[0030] The inspection display and early warning module is used to build a 3D model of the highway to be inspected, and to display the inspection data and drone operating status information of each inspection area.

[0031] The database stores the intervals for dividing inspection sections, map information of the highways to be inspected, flight parameters of drones, gridding ratio, road damage risk index and traffic sign risk index of each inspection area in each inspection section, judgment threshold of highway risk coefficient, flight speed and horizontal field of view of inspection drones in general and risk areas, influence factors of cracks per unit number, cracks per unit length, potholes per unit number, and potholes per unit area, risk level intervals of highways, and corresponding inspection frequencies of response measures for each risk level of highways.

[0032] It should be noted that, in a specific embodiment, a multi-area coverage inspection system using a drone swarm includes an inspection area division module connected to a drone swarm setting module, a drone swarm setting module connected to a road damage monitoring module, a road damage monitoring module connected to a traffic sign monitoring module, a traffic sign monitoring module connected to a monitoring response module, a monitoring response module connected to a working status judgment module, a working status judgment module connected to an inspection display and early warning module, and a database connected to the drone swarm setting module, the road damage monitoring module, the traffic sign monitoring module, and the inspection display and early warning module.

[0033] In a specific embodiment of the present invention, the specific analysis method for dividing the highway to be inspected into inspection areas is as follows: based on the map information of the highway to be inspected, the starting point coordinates, ending point coordinates, and length of the highway to be inspected are extracted. Based on the drone's various flight parameters, including its endurance speed and battery life Through the formula: The drone's remaining inspection range was calculated. .

[0034] Through the formula: The total number of inspection areas for the highway to be inspected is calculated. , This indicates the rounding up symbol, with the starting coordinates of the highway to be inspected serving as the starting point of the first inspection area. The inspection distance for each inspection area is defined as follows: the endpoint coordinates of the highway to be inspected are the endpoint coordinates of the last inspection area, which is then divided into sections. The inspection areas are divided into several sections, which are then used to obtain the inspection areas of the highway to be inspected.

[0035] It should be noted that, in specific embodiments, The symbol indicates rounding up. The flight speed and flight duration are obtained from the drone's factory specifications. Flight speed refers to the suitable flight speed obtained by the manufacturer based on the drone's flight duration test.

[0036] In a specific embodiment of the present invention, the method for dividing the inspection segments according to the division interval is as follows: extracting the division interval from the database. Starting with the coordinates of the starting point of the highway to be inspected, and by quantity Each inspection area is divided into an inspection section, denoted as . ,in , This indicates the number of each inspection section. This indicates the total number of inspection sections.

[0037] It should be noted that, in specific embodiments, the division interval is manually set by technicians based on a comprehensive evaluation of the UAV's operating parameters, and the division interval is extracted from the database. If the number of inspection areas included in each inspection section is set to 5, the starting point of the highway is used as the area division point, and each inspection section is divided into 5 inspection areas. If there are less than 5 inspection areas in the end, the less than 5 inspection areas are divided into a separate inspection section.

[0038] In a specific embodiment of the present invention, the specific analysis method for dividing the inspection level of each inspection area into each inspection section is as follows: based on the road damage risk index and traffic sign risk index of each inspection area in each inspection section in history, the average values ​​are processed to obtain the historical average road damage risk index of each inspection area in each inspection section. Historical average traffic sign risk index , , Indicates the number of each inspection area. This indicates the total number of areas inspected.

[0039] It should be noted that, in a specific embodiment, the road damage risk index and traffic sign risk index of each inspection area in each inspection section are extracted from the database for each historical inspection.

[0040] Through the formula: The historical highway risk coefficients for each inspection area of ​​each inspection section were calculated. ,like Then determine the first The first inspection section The inspection level of each inspection area is a general area, among which The threshold for judging the risk coefficient of highways extracted from the database.

[0041] like Then determine the first The first inspection section The inspection area is classified as a high-risk area.

[0042] The inspection levels of each inspection area in each inspection section are summarized.

[0043] It should be noted that, in specific embodiments, the threshold for judging the highway risk coefficient is set by technicians based on the impact of historical highway risk coefficients on subsequent highways.

[0044] This invention divides highways into inspection areas and, based on the risk levels of highways in previous inspections, classifies each inspection area into inspection levels. It sets specific parameters for inspection drones for inspection areas of different inspection levels, thereby improving the speed and accuracy of inspections in a targeted manner. This ensures the inspection accuracy of high-risk areas and increases the inspection efficiency of low-risk areas.

[0045] In a specific embodiment of the present invention, the method for setting the working mode of the UAV based on the inspection level of each inspection area of ​​each inspection section is as follows: each inspection section is equipped with a lead UAV, which is responsible for the coordinated command of the inspection UAVs in each inspection area of ​​each inspection section.

[0046] Based on the inspection level of each inspection area within each inspection segment, the navigation drone sets the flight speed of the inspection drones within the general area of ​​each inspection segment to a value extracted from the database. The horizontal field of view is set to be extracted from the database. The flight speed of the inspection drones within the risk area is set to be extracted from the database. The horizontal field of view is set to be extracted from the database. .

[0047] It should be noted that, in specific embodiments, the flight speed of the inspection drone in the general area is greater than that of the inspection drone in the risk area, and the horizontal field of view of the inspection drone in the general area is greater than that of the inspection drone in the risk area. These settings are all manually preset by technicians based on the actual environment.

[0048] The inspection drone's flight speed varies. A slower flight speed allows for the acquisition of higher quality images, which helps obtain more detailed inspection results in risk areas. A faster flight speed, on the other hand, can improve inspection efficiency while ensuring the acquisition of inspection information.

[0049] The inspection drone has a wide horizontal field of view, which allows for extensive information acquisition in general areas, improving inspection efficiency. A narrow field of view allows for detailed inspection tasks in high-risk areas, providing higher clarity and detail.

[0050] This invention employs a drone swarm for multi-area inspection. In swarm mode, multiple drones can work collaboratively, achieving efficient inspection of large areas and multiple regions through coordinated flight and division of labor. This greatly improves the speed and accuracy of inspection while reducing labor costs and safety risks.

[0051] In a specific embodiment of the present invention, the method for obtaining road surface parameters for each inspection area of ​​each inspection section and calculating the road surface damage risk index for each inspection area of ​​each inspection section is as follows: based on each frame of the road surface inspection video of each inspection area of ​​each inspection section, a deep learning model is used to automatically identify the number of cracks in each inspection area of ​​each inspection section. Number of pits The total length of cracks in each inspection area of ​​each inspection section The total area of ​​pits in each inspection area of ​​each inspection section .

[0052] It should be noted that, in a specific embodiment, based on a deep learning model, the model is trained on each frame of the road inspection video of each inspection area in each inspection section. Contour detection is used to extract the contours of each crack in each inspection area of ​​each inspection section. The actual length of each crack in each inspection area of ​​each inspection section is obtained by calculating the arc length of the contours. The total length of the cracks in each inspection area of ​​each inspection section is obtained by summing them. Contour detection is used to extract the contours of each pothole in each inspection area of ​​each inspection section. The number of pixels of each pothole is counted to obtain the total area of ​​the potholes in each inspection area of ​​each inspection section.

[0053] Through the formula: The road damage risk index of each inspection area in each inspection section was calculated. ,in This represents the influence factor for a unit number of cracks extracted from the database. This represents the influence factor per unit length of crack extracted from the database. This represents the impact factor for a unit number of potholes extracted from the database. This represents the influence factor of potholes per unit area extracted from the database.

[0054] In a specific embodiment of the present invention, the method for obtaining the parameters of traffic signs in each inspection area of ​​each inspection segment and calculating the risk index of traffic signs in each inspection area of ​​each inspection segment is as follows: based on each frame of the traffic sign inspection video of each inspection area of ​​each inspection segment, the images are divided according to the gridding ratio extracted from the database, and the images of each traffic sign in each inspection area of ​​each inspection segment are automatically identified using a deep learning model. The RGB value of each pixel at the cross intersection of the image grid of each traffic sign in each inspection area of ​​each inspection segment is extracted.

[0055] It should be noted that, in specific embodiments, the division based on the grid ratio extracted from the database specifically refers to, for example, dividing each frame of the traffic sign inspection video of each inspection area of ​​each inspection segment into a grid with an aspect ratio of 3 x 4, and obtaining the RGB value of each pixel at the cross intersection of each grid.

[0056] Extract the outlines of each traffic sign in each inspection area of ​​each inspection section. Based on the minimum bounding rectangle of the outlines of each traffic sign in each inspection area of ​​each inspection section, obtain the tilt angle of each traffic sign in each inspection area.

[0057] It should be noted that, in a specific embodiment, the computer image analysis software determines the tilt angle of each traffic sign in each inspection area by setting the minimum bounding rectangle for the image of each traffic sign in each inspection area of ​​each inspection section.

[0058] Image analysis software was used to compare the RGB values ​​of each pixel at the crossroads of the image grids of traffic signs in each inspection area of ​​each inspection section with the standard RGB values ​​of each pixel at the crossroads of the image grids of traffic signs in each inspection area of ​​each inspection section in the database. This yielded the color similarity of the traffic signs in each inspection area of ​​each inspection section. The average color similarity of the traffic signs in each inspection area of ​​each inspection section was then calculated by averaging the results. .

[0059] Similarly, the average tilt angle compliance of traffic signs in each inspection area of ​​each inspection section is obtained. .

[0060] Through the formula: The risk index of traffic signs in each inspection area of ​​each inspection section was calculated. .

[0061] In a specific embodiment of the present invention, the specific analysis method for calculating the highway risk coefficient of each inspection area in each inspection section, assessing the highway risk coefficient level of each inspection area in each inspection section, and implementing response measures corresponding to the damage level of traffic signs in each inspection area in each inspection section is as follows:

[0062] Based on the road damage risk index of each inspection area in each inspection section Risk index of traffic signs Through the formula: The highway risk coefficient for each inspection area of ​​each inspection section is calculated.

[0063] like If it falls within the first risk level zone of the highway, then it is determined to be the [missing information]. The first inspection section The highway risk level in the inspection area is high, and Level 1 highway risk response measures are implemented.

[0064] like If it falls within the second risk level range for highways, then the following determination will be made: The first inspection section The highway risk level in the inspection area is medium, and Level II highway risk response measures are implemented.

[0065] like If it falls within the third risk level range for highways, then it is determined to be level three. The first inspection section The highway risk level in the inspection area is low.

[0066] The aforementioned Level 1 risk response measures for highways involve sending inspection information from the lead drones of each inspection section to the control center. The control center then displays the information in 3D and issues warnings to highway maintenance personnel, who then carry out on-site repairs.

[0067] The aforementioned Level II risk response measures for highways will be implemented. The first inspection section The next inspection area will be designated as a risk area.

[0068] It should be noted that, in specific embodiments, the risk level intervals of the expressways are pre-set by technicians based on a comprehensive assessment of the actual impact of the risk coefficients of historical expressways on subsequent expressways, and stored in a database.

[0069] This invention uses image recognition to identify the road surface conditions and traffic signs of highways, analyze the risk level of highways, and then implement corresponding response measures to effectively solve the safety hazards existing on highways and ensure the safety of highway traffic.

[0070] In a specific embodiment of the present invention, the monitoring of the working status parameters of the inspection drones in each inspection area of ​​each inspection segment, if an abnormality is detected, will trigger an abnormal working status response measure. The specific analysis method is as follows: the lead drone of each inspection segment obtains the real-time battery level of the inspection drones in the inspection areas it is responsible for. The lead drone in each inspection section sends network test signals to the inspection drones in the inspection areas it is responsible for, in order to obtain the response time of the inspection drones in each inspection area of ​​each inspection section. .

[0071] It should be noted that, in a specific embodiment, the lead drone of each inspection segment sends a network test signal to the inspection drones of each inspection area under its responsibility at regular intervals. After receiving the network test signal, the inspection drones of each inspection area send a response signal to the lead drone of the inspection segment in that inspection area. The lead drone of each inspection segment counts the response time of the inspection drones in the inspection areas it is responsible for.

[0072] like or Then determine the first The first inspection section The inspection drone in the inspection area is malfunctioning. The first inspection section Each inspection area is designated as an abnormal inspection area, and abnormal work status response measures are implemented. This represents the battery threshold for normal operation of the inspection drone, extracted from the database. This indicates the network response time for the inspection drone to be operating normally, as retrieved from the database.

[0073] It should be noted that, in specific embodiments, the power threshold for the normal operation of the inspection drone is to ensure that the drone has enough power to return after completing the inspection task. This threshold is preset by technicians and stored in the database.

[0074] The network response time for normal operation of the inspection drone is preset by technicians based on the network response time between the inspection drone and the lead drone, which is the furthest from the lead drone, and a redundancy value is set. This preset time is then stored in the database.

[0075] The aforementioned abnormal operation response measures refer to the following: the lead drone of the inspection segment to which the abnormal inspection area belongs selects the inspection drones of the inspection areas on the left and right sides of the abnormal inspection area with a selection step size of 1 after the inspection is completed. If the remaining power is sufficient for the power requirement of a second inspection, the lead drone of the inspection segment sends a temporary inspection task to the inspection drones of the abnormal inspection area to perform the inspection of the abnormal inspection area.

[0076] If the power requirements for the second inspection are not met, the lead drone in this inspection segment will sequentially increase the screening step size, and use the screening step size to screen the remaining power of the inspection drones in the inspection areas on the left and right sides of the abnormal inspection area after the inspection is completed, and repeat the above operation.

[0077] It should be noted that, in a specific embodiment, the navigation drone first filters the remaining battery power of the inspection drones in the inspection area of ​​the highway in the direction of travel with a filtering step size of 1 after the inspection is completed. If the condition is met, the inspection drones in the inspection area of ​​the highway in the direction of travel with a filtering step size of 1 will perform a temporary inspection task. If the condition is not met, the remaining battery power of the inspection drones in the inspection area of ​​the highway in the opposite direction of travel will be filtered, and the operation will be repeated.

[0078] If the remaining current of the inspection drones in adjacent inspection areas does not meet the requirements after inspection, the remaining power of the inspection drones in the inspection areas along the highway travel direction is filtered again with a step size of 2. If the requirements are met, the inspection drones in the inspection areas along the highway travel direction with a step size of 2 will perform temporary inspection tasks. If the requirements are not met, the remaining power of the inspection drones in the inspection areas in the opposite direction of highway travel will be filtered, and the operation will be repeated.

[0079] If the inspection area in the abnormal inspection area does not have an inspection area under the responsibility of the inspection section in the direction of highway travel or in the opposite direction of highway travel, then the judgment for this direction is skipped.

[0080] If the inspection drones in each inspection area of ​​the inspection segment do not meet the power requirements for secondary inspections, the lead drone of the inspection segment sends an abnormal operation signal to the inspection control center. The inspection control center then wakes up the idle drones to perform temporary inspection tasks in the abnormal inspection areas of the inspection segment.

[0081] The screening step size increases sequentially. If all the inspection areas under the responsibility of the inspection segment to which the abnormal inspection area belongs have been screened and none of them meet the requirements, then the lead drone of that inspection segment sends an abnormal work signal to the inspection control center, and the inspection control center responds.

[0082] It should be noted that, in specific embodiments, the drone swarms set up by this system operate in multiple batches in rotation. While one batch of drones is conducting multi-area coverage inspections, the other batches of drones are in an idle state and can be woken up by the inspection control center to perform temporary inspection tasks.

[0083] This invention establishes a working network architecture for a drone swarm system, consisting of a control center, a lead drone, and patrol drones. The lead drone monitors the working status of each patrol drone and responds promptly to patrol drones with abnormal working status, thereby increasing the stability of the drone swarm inspection and ensuring the controllability of the drone inspection process.

[0084] In a specific embodiment of the present invention, the method for displaying the inspection data and drone working status information of each inspection area is as follows: based on the highway map of the inspection area, the division of each inspection area and each inspection drone, a three-dimensional display model is generated using three-dimensional modeling software to display the working status of the inspection drones in each inspection area of ​​each inspection section and the highway risk level.

[0085] Based on the highway risk level of each inspection area in each inspection section, the color warning corresponding to each risk level is displayed in the 3D display model.

[0086] Based on the working status parameters of the inspection drones in each inspection area of ​​each inspection section, color warnings corresponding to each abnormal working status parameter are displayed in the 3D display model.

[0087] It should be noted that, in specific embodiments, color-coded warnings corresponding to each risk level are displayed in the 3D display model. For example, a high-risk highway is matched with a red warning, a medium-risk highway with a yellow warning, and a low-risk highway with a green warning.

[0088] The 3D display model shows color-coded warnings for each abnormal operating status parameter. For example, if the real-time battery level of the inspection drone is abnormal, a blue warning is given; if the network response time of the inspection drone is abnormal, a purple warning is given.

[0089] Reference Figure 2As shown, the system architecture of this invention is divided into three layers: inspection control center, navigation drone, and inspection drone. The inspection drone in each inspection area acquires the highway inspection data of each inspection area and sends it to the navigation drone of the corresponding inspection section. The navigation drone of each inspection section acquires the working status data of the inspection drone in each inspection area it is responsible for, and determines whether the working status of the inspection drone in each inspection area is abnormal. The navigation drone of each inspection section sends the highway inspection data and the judgment result of whether the working status is abnormal in each inspection area it is responsible for to the inspection control center for display, early warning and abnormal response.

[0090] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A multi-area coverage inspection system using a swarm of unmanned aerial vehicles (UAVs), characterized in that, The system includes: The inspection area division module is used to obtain map information of the highway to be inspected and various flight parameters of the UAV, divide the highway to be inspected into inspection areas, and divide each inspection segment according to the division interval. The drone swarm setting module obtains the road damage risk index and traffic sign risk index of each inspection area in each inspection section from historical inspections, classifies the inspection level of each inspection area in each inspection section, and sets the working mode of the drone based on the inspection level of each inspection area in each inspection section. The road damage monitoring module is used to analyze the road inspection videos of each inspection area in each inspection section, obtain the road parameters of each inspection area in each inspection section, and calculate the road damage risk index of each inspection area in each inspection section. The traffic sign monitoring module is used to analyze the traffic sign inspection videos of each inspection area in each inspection section, obtain the parameters of each traffic sign in each inspection area in each inspection section, and calculate the traffic sign risk index of each inspection area in each inspection section. Based on the historical road damage risk index and traffic sign risk index of each inspection area in each inspection section, the average values ​​are calculated to obtain the historical average road damage risk index for each inspection area in each inspection section. Historical average traffic sign risk index , , Indicates the number of each inspection area. Indicates the total number of inspection areas; Through the formula: The historical highway risk coefficients for each inspection area of ​​each inspection section were calculated. ,like Then determine the first The first inspection section The inspection level of each inspection area is a general area, among which The threshold for judging the risk coefficient of highways extracted from the database; like Then determine the first The first inspection section The inspection area is classified as a high-risk area. The inspection levels of each inspection area in each inspection section are summarized. The monitoring and response module is used to calculate the highway risk coefficient of each inspection area in each inspection section, assess the highway risk coefficient level of each inspection area in each inspection section, and implement response measures corresponding to the damage level of traffic signs in each inspection area in each inspection section. The working status judgment module is used to monitor the working status parameters of the inspection drones in each inspection area of ​​each inspection section. If an abnormality is judged, the working status abnormality response measures are executed. The inspection display and early warning module is used to build a 3D model of the highway to be inspected, and to display the inspection data and drone operating status information of each inspection area.

2. A multi-area coverage inspection system using a drone swarm according to claim 1, characterized in that, The specific analysis method for dividing the highway to be inspected into inspection areas is as follows: Based on the map information of the highway to be inspected, extract the starting point coordinates, ending point coordinates, and length of the highway to be inspected. Based on the drone's various flight parameters, including its endurance speed and battery life Through the formula: The drone's remaining inspection range was calculated. ; Through the formula: The total number of inspection areas for the highway to be inspected is calculated. , This indicates the rounding up symbol, with the starting coordinates of the highway to be inspected serving as the starting point of the first inspection area. The inspection distance for each inspection area is defined as follows: the endpoint coordinates of the highway to be inspected are the endpoint coordinates of the last inspection area, which is then divided into sections. The inspection areas are divided into several sections, which are then used to obtain the inspection areas of the highway to be inspected.

3. A multi-area coverage inspection system using a drone swarm according to claim 2, characterized in that, The specific analysis method for dividing the inspection sections according to the interval is as follows: Extract the partition interval from the database Starting with the coordinates of the starting point of the highway to be inspected, and by quantity Each inspection area is divided into an inspection section, denoted as . ,in , This indicates the number of each inspection section. This indicates the total number of inspection sections.

4. A multi-area coverage inspection system using a drone swarm according to claim 1, characterized in that, The specific analysis method for setting the working mode of the UAV based on the inspection level of each inspection area in each inspection segment is as follows: Each inspection section is equipped with a lead drone, which is responsible for the coordinated command of inspection drones in each inspection area of ​​each inspection section; Based on the inspection level of each inspection area within each inspection segment, the navigation drone sets the flight speed of the inspection drones within the general area of ​​each inspection segment to a value extracted from the database. The horizontal field of view is set to be extracted from the database. The flight speed of the inspection drones within the risk area is set to be extracted from the database. The horizontal field of view is set to be extracted from the database. .

5. A multi-area coverage inspection system using a drone swarm according to claim 4, characterized in that, The specific analysis method for obtaining the road surface parameters of each inspection area in each inspection section and calculating the road surface damage risk index of each inspection area in each inspection section is as follows: Based on frames of road surface inspection videos from each inspection area of ​​each inspection section, a deep learning model is used to automatically identify the number of cracks in each inspection area of ​​each inspection section. Number of pits The total length of cracks in each inspection area of ​​each inspection section The total area of ​​pits in each inspection area of ​​each inspection section ; Through the formula: The road damage risk index of each inspection area in each inspection section was calculated. ,in This represents the influence factor for a unit number of cracks extracted from the database. This represents the influence factor per unit length of crack extracted from the database. This represents the impact factor for a unit number of potholes extracted from the database. This represents the influence factor of potholes per unit area extracted from the database.

6. A multi-area coverage inspection system using a drone swarm according to claim 5, characterized in that, The specific analysis method for obtaining the parameters of traffic signs in each inspection area of ​​each inspection section and calculating the risk index of traffic signs in each inspection area of ​​each inspection section is as follows: Based on the images of each frame of the traffic sign inspection video of each inspection area in each inspection section, the grid is divided according to the grid ratio extracted from the database. The images of each traffic sign in each inspection area of ​​each inspection section are automatically identified using a deep learning model. The RGB value of each pixel at the cross intersection of the image grid of each traffic sign in each inspection area of ​​each inspection section is extracted. Extract the outline of each traffic sign in each inspection area of ​​each inspection section. Based on the minimum bounding rectangle of the outline of each traffic sign in each inspection area of ​​each inspection section, obtain the tilt angle of each traffic sign in each inspection area. Image analysis software was used to compare the RGB values ​​of each pixel at the crossroads of the image grids of traffic signs in each inspection area of ​​each inspection section with the standard RGB values ​​of each pixel at the crossroads of the image grids of traffic signs in each inspection area of ​​each inspection section in the database. This yielded the color similarity of traffic signs in each inspection area of ​​each inspection section. The average color similarity of traffic signs in each inspection area of ​​each inspection section was then calculated by averaging the results. ; Similarly, the average tilt angle compliance of traffic signs in each inspection area of ​​each inspection section is obtained. ; Through the formula: The risk index of traffic signs in each inspection area of ​​each inspection section was calculated. .

7. A multi-area coverage inspection system using a drone swarm according to claim 6, characterized in that, The specific analysis method for assessing the risk coefficient level of each inspection area in each inspection section and implementing response measures corresponding to the damage level of traffic signs in each inspection area of ​​each inspection section is as follows: Based on the road damage risk index of each inspection area in each inspection section Risk index of traffic signs Through the formula: The highway risk coefficient for each inspection area of ​​each inspection section is calculated. like If it falls within the first risk level zone of the highway, then it is determined to be the [missing information]. The first inspection section The highway risk level in the inspection area is high, and the first-level highway risk response measures are implemented. like If it falls within the second risk level range for highways, then the following determination will be made: The first inspection section The highway risk level in the inspection area is medium, and Level II highway risk response measures are implemented. like If it falls within the third risk level range for highways, then it is determined to be level three. The first inspection section The highway risk level in the inspection area is low. The aforementioned Level 1 risk response measures for highways involve sending inspection information from the lead drones of each inspection section to the control center, where the control center provides a 3D display and issues warnings to highway maintenance personnel. The aforementioned Level II risk response measures for highways will be implemented. The first inspection section The next inspection area will be designated as a risk area.

8. A multi-area coverage inspection system using a drone swarm according to claim 7, characterized in that, The monitoring system tracks the operational status parameters of the inspection drones in each inspection area of ​​each inspection section. If an anomaly is detected, an anomaly response measure is implemented. The specific analysis method is as follows: The lead drone in each inspection section obtains the real-time battery level of the inspection drones in the inspection areas it is responsible for. The lead drone in each inspection section sends network test signals to the inspection drones in the inspection areas it is responsible for, in order to obtain the response time of the inspection drones in each inspection area of ​​each inspection section. ; like or Then determine the first The first inspection section The inspection drone in the inspection area is malfunctioning. The first inspection section Each inspection area is designated as an abnormal inspection area, and abnormal work status response measures are implemented. This represents the battery threshold for normal operation of the inspection drone, extracted from the database. This indicates the network response time for normal operation of the inspection drone, as extracted from the database. The aforementioned abnormal working status response measures refer to the fact that the lead drone of the inspection segment to which the abnormal inspection area belongs selects the remaining power of the inspection drones in the inspection areas with a selection step size of 1 on the left and right sides of the abnormal inspection area after the inspection is completed. If the power requirement for a second inspection is met, the lead drone of the inspection segment sends a temporary inspection task to the inspection drone in the abnormal inspection area to perform the inspection of the abnormal inspection area. If the power requirements for the second inspection are not met, the lead drone in this inspection segment will sequentially increase the screening step size, and use the screening step size to screen the remaining power of the inspection drones in the inspection areas on the left and right sides of the abnormal inspection area after the inspection is completed, and repeat the above operation. If the inspection drones in each inspection area of ​​the inspection segment do not meet the power requirements for secondary inspections, the lead drone of the inspection segment sends an abnormal operation signal to the inspection control center. The inspection control center then wakes up the idle drones to perform temporary inspection tasks in the abnormal inspection areas of the inspection segment.

9. A multi-area coverage inspection system using a drone swarm according to claim 1, characterized in that, The specific analysis method for displaying the inspection data and UAV operating status information of each inspection area in the early warning system is as follows: Based on the highway map, the division of each inspection area and each inspection drone, a 3D display model is generated using 3D modeling software to show the working status of the inspection drones in each inspection area of ​​each inspection section and the highway risk level. Based on the highway risk level of each inspection area in each inspection section, the color warning corresponding to each risk level is displayed in the three-dimensional display model; Based on the working status parameters of the inspection drones in each inspection area of ​​each inspection section, color warnings corresponding to each abnormal working status parameter are displayed in the 3D display model.

Images