Low-altitude unmanned aerial vehicle multi-machine linkage pavement disease detection system and method

Abstract

The invention relates to the technical field of pavement disease detection, and discloses a low-altitude unmanned aerial vehicle multi-machine linkage pavement disease detection system, which comprises a plurality of sets of low-altitude pavement detection unmanned aerial vehicles used for completing pavement shooting in sequence; the cloud storage server is used for receiving shooting data of the unmanned aerial vehicle; the data processing server is used for initiating task scheduling and performing defect position alignment on the data of each unmanned aerial vehicle; and the reasoning server is used for calculating the position and the size of the pavement disease. According to the invention, pavement disease inspection can be carried out without interference, and meanwhile, the problem that the pavement is easily blocked by traffic flow when the unmanned aerial vehicle shoots the pavement is solved. The completeness and precision of a detection result are effectively improved, and the convenience degree of detection is also improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pavement disease detection, and in particular to a low-altitude unmanned aerial vehicle (UAV) multi-machine linkage pavement disease detection system. Background Art

[0002] With the continuous development and progress of society and the rapid development of the transportation sector, the mileage of roads has increased rapidly and the traffic volume has continued to expand. Road damage is inevitable, so the task of road disease detection has become increasingly arduous.

[0003] At present, road defect detection technology mainly relies on vehicle-mounted camera detection, but its detection range is small, its applicable occasions are limited, its efficiency is low, and some manual intervention is required. For example, the driving speed of the detection vehicle cannot keep up with the speed of 120 kilometers per hour. Therefore, there are safety hazards when conducting detection on the highway, and the detection vehicle is easily blocked by other vehicles, resulting in a lot of data loss. Summary of the invention

[0004] The purpose of the present invention is to solve the problems existing in the prior art and to propose a low-altitude UAV multi-machine linkage road disease detection system.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A low-altitude UAV multi-machine linkage road surface disease detection system, comprising:

[0007] Multiple sets of low-altitude road inspection drones are used to complete road photography one after another;

[0008] Cloud storage server, used to receive the shooting data of the drone;

[0009] The data processing server is used to initiate task scheduling and align the defect locations of each drone data;

[0010] Inference server, used to calculate the location and size of pavement defects.

[0011] Preferably, the low-altitude road surface detection drone comprises:

[0012] 5G module for communicating with cloud storage servers;

[0013] High-definition camera for photographing the road surface;

[0014] Data acquisition module, used to control the high-definition camera to realize data acquisition;

[0015] A data processing module is used to pre-process the collected view data to ensure the accuracy and consistency of the data;

[0016] The pre-detection module automatically identifies whether the road is blocked by vehicles when the drone is shooting from the air;

[0017] Flight control module, used to control the flight of the drone.

[0018] A low-altitude UAV multi-machine linkage road surface disease detection method, using a low-altitude UAV multi-machine linkage road surface disease detection system, including the following steps:

[0019] Drones are dispersed on the same road, and their high-definition cameras are vertically positioned to capture road surface information.

[0020] S1: The first drone takes a photo of the road surface and detects whether the road surface is blocked by vehicles during the shooting process. After the first drone finds the blockage, it sends the position and image data to the server;

[0021] S2: The second and subsequent drones follow and shoot. At this time, the server sends the occlusion information to the second drone;

[0022] S3, when the second drone passes the blocked position, it determines whether there is still a vehicle blocking the road. If the road is no longer blocked, data collection is performed. If there is still a part of the area blocked, it waits for a certain period of time. After the waiting period, if the road information still cannot be photographed, the position is sent to the subsequent drones through the server, and the drones are transmitted and retaken in turn.

[0023] S4: Each drone transmits the captured images to the cloud storage server;

[0024] S6: After the storage server receives the image, the data processing server initiates task scheduling to align the defect positions of the data from each drone. After alignment, the data is inferred and calculated in turn to determine the location and size of the road surface defects.

[0025] Preferably, the process of aligning the defect positions includes:

[0026] Preprocessing: Preprocess the input image, including denoising, contrast enhancement, etc., to improve image quality and reduce the difficulty of subsequent processing;

[0027] Feature extraction: Use feature point detection algorithm to extract feature points in the image and calculate feature descriptors;

[0028] Feature matching: Use feature descriptors to match feature points and find corresponding points between two images;

[0029] Transformation matrix calculation: Calculate the transformation matrix based on the matched feature point pairs;

[0030] Image transformation: Apply a transformation matrix to transform the image to achieve alignment;

[0031] Post-processing: Post-process the aligned images to improve image quality;

[0032] Preferably, the specific steps of calculating the location and size of road surface defects include:

[0033] Determine the specific location of the defect in the image by matching the identified defect with a reference point in the image, where the reference point is a road sign or lane line;

[0034] Combined with the shooting position and angle of the image, the location of the defect in the actual road surface is further calculated;

[0035] For the road surface images that have been taken, image analysis of the defects is performed using the image processing software MATLAB;

[0036] By setting the scale or calibrating the image, the actual size of the disease in the image can be calculated.

[0037] Preferably, the specific process of detecting whether the road surface is blocked by the vehicle is as follows:

[0038] Preprocess the collected view data, including denoising, enhancement, calibration and synchronization, to ensure data accuracy and consistency;

[0039] The pre-detection module then uses a deep learning algorithm, a convolutional neural network (CNN), to analyze the pre-processed images to detect vehicles and other obstacles on the road, which involves analyzing the shape, size, position, and speed information of the vehicle.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention can conduct road disease inspection without interference, and solves the problem that the road surface is easily blocked by traffic when the drone is photographing the road surface. It effectively improves the integrity and accuracy of the detection results, and also improves the convenience of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the main component of the multi-machine linkage road surface detection system.

[0043] Figure 2 This is the workflow of the UAV pavement disease detection system.

[0044] Figure 3 This is the overall network structure of YOLOv7. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0046] like Figure 1-Figure 3 As shown, a low-altitude UAV multi-machine linkage road surface disease detection system includes:

[0047] Multiple sets of low-altitude road inspection drones are used to complete road photography one after another;

[0048] Cloud storage server, used to receive the shooting data of the drone;

[0049] The data processing server is used to initiate task scheduling and align the defect locations of each drone data;

[0050] Inference server, used to calculate the location and size of pavement defects.

[0051] In this embodiment, the low-altitude road surface detection drone includes:

[0052] 5G module for communicating with cloud storage servers;

[0053] High-definition camera for photographing the road surface;

[0054] Data acquisition module, used to control the high-definition camera to realize data acquisition;

[0055] A data processing module is used to pre-process the collected view data to ensure the accuracy and consistency of the data;

[0056] The pre-detection module automatically identifies whether the road is blocked by vehicles when the drone is shooting from the air;

[0057] Flight control module, used to control the flight of the drone.

[0058] A low-altitude UAV multi-machine linkage road surface disease detection method, using a low-altitude UAV multi-machine linkage road surface disease detection system, including the following steps:

[0059] S1: The first drone takes a photo of the road surface and detects whether the road surface is blocked by vehicles during the shooting process. After the first drone finds the blockage, it sends the position and image data to the server;

[0060] S2: The second and subsequent drones follow and shoot. At this time, the server sends the occlusion information to the second drone;

[0061] S3, when the second drone passes the blocked position, it determines whether there is still a vehicle blocking the road. If the road is no longer blocked, data collection is performed. If there is still a part of the area blocked, it waits for a certain period of time. After the waiting period, if the road information still cannot be photographed, the position is sent to the subsequent drones through the server, and the drones are transmitted and retaken in turn.

[0062] S4: Each drone transmits the captured images to the cloud storage server;

[0063] S6: After the storage server receives the image, the data processing server initiates task scheduling to align the defect positions of the data from each drone. After alignment, the data is inferred and calculated in turn to determine the location and size of the road surface defects.

[0064] Compared with the prior art, the present invention adopts multiple sets of drones to collect road surface data. If the road surface is blocked by traffic when the front drone is shooting, the rear drone will take supplementary photos, which can realize non-interference road disease inspection and solve the problem that the road surface is easily blocked by traffic when the drone is shooting. It also solves the current road disease detection method mainly based on vehicle-mounted camera detection, which has a small detection range, limited applicable occasions, low efficiency and partial need for manual intervention. For example, the driving speed of the detection vehicle cannot keep up with the speed of 120 kilometers per hour, so there are safety hazards when traveling on the highway for detection, and the detection vehicle is easily blocked by other vehicles, resulting in a lot of data loss, which effectively improves the integrity and accuracy of the detection results, and also improves the convenience of detection.

[0065] In this embodiment, the process of aligning the defect positions includes:

[0066] Preprocessing: Preprocess the input image, including denoising, contrast enhancement, etc., to improve image quality and reduce the difficulty of subsequent processing;

[0067] Feature extraction: Use feature point detection algorithm to extract feature points in the image and calculate feature descriptors;

[0068] Feature matching: Use feature descriptors to match feature points and find corresponding points between two images;

[0069] Transformation matrix calculation: Calculate the transformation matrix based on the matched feature point pairs;

[0070] Image transformation: Apply a transformation matrix to transform the image to achieve alignment;

[0071] Post-processing: Post-process the aligned images to improve image quality;

[0072] In this embodiment, the specific steps of calculating the location and size of the road surface disease include:

[0073] Determine the specific location of the defect in the image by matching the identified defect with a reference point in the image, where the reference point is a road sign or lane line;

[0074] Combined with the shooting position and angle of the image, the location of the defect in the actual road surface is further calculated;

[0075] For the road surface images that have been taken, image analysis of the defects is performed using the image processing software MATLAB;

[0076] By setting the scale or calibrating the image, the actual size of the disease in the image can be calculated.

[0077] Preferably, the specific process of detecting whether the road surface is blocked by the vehicle is as follows:

[0078] Preprocess the collected view data, including denoising, enhancement, calibration and synchronization, to ensure data accuracy and consistency;

[0079] The pre-detection module then uses a deep learning algorithm, a convolutional neural network (CNN), to analyze the pre-processed images to detect vehicles and other obstacles on the road, which involves analyzing the shape, size, position, and speed information of the vehicle.

[0080] Working principle:

[0081] Multi-machine linkage road surface detection includes the following steps:

[0082] (1) Training cloud-based disease detection models;

[0083] (2) Training the front-end occlusion detection model;

[0084] (3) The front-end occlusion detection model is placed in the front-end pre-detection module of the front-end controller carried by the UAV;

[0085] (4) The cloud disease detection system is located in the server reasoning module;

[0086] Among them, the cloud-based disease detection model is based on the YOLOv7 network structure. The YOLOv7 network structure is based on the essence of the YOLO series of algorithms and is carefully optimized for target detection tasks. Its network architecture is divided into four parts: input end, backbone network, neck and head.

[0087] The backbone network constitutes the core framework of YOLOv7, which is composed of CBS, ELAN, and MP modules stacked together to ensure the efficiency and accuracy of feature extraction. The neck network is responsible for receiving the features of the backbone network and efficiently fusing multi-scale features through FPN and PAN structures, significantly enhancing the network's ability to recognize targets of different scales. The head network is the final output stage of YOLOv7. It uses a specially designed REP module for target prediction and guides the parameter optimization process through a loss function. In this way, YOLOv7 achieves a smooth transition from low-level visual features to high-level semantic understanding, providing strong support for real-time target detection tasks.

[0088] YOLOv7 also uses re-parameterized convolution, auxiliary head detection, and dynamic label allocation in its training method, which enhances the training effect and speed of the network model. The trained YOLOv7 model is deployed to the pavement disease detection system of the server, and then the collected pavement images are transmitted to the model for detection. The model outputs the detection results, including the location, type, and confidence of the disease.

[0089] (5) The system includes multiple drones, each of which has the ability to collect road images, detect occlusions, and upload data.

[0090] (6) After the drone collects data, it uploads the data to the server for road disease detection.

[0091] Embodiment 1:

[0092] The user uses two sets of front-end detection drones to inspect road defects. The first drone takes off first and flies forward at a speed of 50 kilometers per hour at an altitude of about 10 meters above the road.

[0093] The high-definition camera carried by the drone takes photos vertically while moving forward. During the shooting process, some sections of the road are occupied by vehicles, so it is impossible to take photos of that section of the road. Therefore, the photos and GPS information of that section are sent back to the server. The server transmits them to the second drone that is following behind and taking photos.

[0094] After the second drone moves to the corresponding location, it detects whether there is a vehicle blocking the road. If there is no vehicle blocking the road, it takes a photo and uploads it.

[0095] If there are still vehicles blocking the way at this time, you can set a waiting time and capture unobstructed road information within the waiting time before moving forward. If there is still obstruction after the set waiting time, report the location and take additional photos the next time you perform the task.

[0096] The road surface information uploaded by the two drones is automatically detected for defects in the cloud and a report is generated.

[0097] Embodiment 2:

[0098] The user uses three sets of front-end detection drones to inspect road surface defects. The first drone takes off first and flies forward at a speed of 50 kilometers per hour at an altitude of about 10 meters above the road. The second drone follows and flies 500 meters away from the front drone.

[0099] If there are vehicles blocking the road during the first drone's filming, the information will be transmitted to the second drone. If the road is still blocked when the second drone passes through the section, the information will be transmitted to the third drone. After the third drone finishes filming, it will send the information to the cloud server.

[0100] After the cloud server receives the data from the three drones, it first aligns the information through GPS. Then it performs automatic inference on the AI ​​model to generate a complete inspection report for the road section.

[0101] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A low-altitude UAV multi-machine linkage road surface disease detection system, characterized by: include: Multiple sets of low-altitude road inspection drones are used to complete road photography one after another; Cloud storage server, used to receive the shooting data of the drone; The data processing server is used to initiate task scheduling and align the defect locations of each drone data; Inference server, used to calculate the location and size of pavement defects.

2. The low-altitude UAV multi-machine linkage road disease detection system according to claim 1 is characterized by: The low-altitude road surface detection drone comprises: 5G module for communicating with cloud storage servers; High-definition camera for photographing the road surface; Data acquisition module, used to control the high-definition camera to realize data acquisition; A data processing module is used to pre-process the collected view data to ensure the accuracy and consistency of the data; The pre-detection module automatically identifies whether the road is blocked by vehicles when the drone is shooting from the air; Flight control module, used to control the flight of the drone.

3. A low-altitude UAV multi-machine linkage road surface disease detection method, using a low-altitude UAV multi-machine linkage road surface disease detection system according to claim 1 or 2, characterized in that: The following steps are involved: S1: The first drone takes a photo of the road surface and detects whether the road surface is blocked by vehicles during the shooting process. After the first drone finds the blockage, it sends the position and image data to the server; S2: The second and subsequent drones follow and shoot. At this time, the server sends the occlusion information to the second drone; S3, when the second drone passes the blocked position, it determines whether there is still a vehicle blocking the road. If the road is no longer blocked, data collection is performed. If there is still a part of the area blocked, it waits for a certain period of time. After the waiting period, if the road information still cannot be photographed, the position is sent to the subsequent drones through the server, and the drones are transmitted and retaken in turn. S4: Each drone transmits the captured images to the cloud storage server; S6: After the storage server receives the image, the data processing server initiates task scheduling to align the defect positions of the data from each drone. After alignment, the data is inferred and calculated in turn to determine the location and size of the road surface defects.

4. The method for detecting road defects by using multiple low-altitude unmanned aerial vehicles in linkage mode according to claim 3 is characterized in that: The process of defect position alignment includes: Preprocessing: Preprocess the input image, including denoising, contrast enhancement, etc., to improve image quality and reduce the difficulty of subsequent processing; Feature extraction: Use feature point detection algorithm to extract feature points in the image and calculate feature descriptors; Feature matching: Use feature descriptors to match feature points and find corresponding points between two images; Transformation matrix calculation: Calculate the transformation matrix based on the matched feature point pairs; Image transformation: Apply a transformation matrix to transform the image to achieve alignment; Post-processing: Post-process the aligned images to improve image quality.

5. The low-altitude UAV multi-machine linkage road disease detection method according to claim 3 is characterized by: The specific steps of calculating the location and size of the road surface disease include: Determine the specific location of the defect in the image by matching the identified defect with a reference point in the image, where the reference point is a road sign or lane line; Combined with the shooting position and angle of the image, the location of the defect in the actual road surface is further calculated; For the road surface images that have been taken, image analysis of the defects is performed using the image processing software MATLAB; By setting the scale or calibrating the image, the actual size of the disease in the image can be calculated.

6. The method for detecting road defects by using multiple low-altitude unmanned aerial vehicles in linkage mode according to claim 5 is characterized in that: The specific process of detecting whether the road surface is blocked by a vehicle is as follows: Preprocess the collected view data, including denoising, enhancement, calibration and synchronization, to ensure data accuracy and consistency; The pre-detection module then uses a deep learning algorithm, a convolutional neural network (CNN), to analyze the pre-processed images to detect vehicles and other obstacles on the road, which involves analyzing the shape, size, position, and speed information of the vehicle.

Images