A bridge disease detection system and detection method based on drone

Abstract

The present application discloses a bridge defect detection system based on an unmanned aerial vehicle and a detection method thereof, including an unmanned aerial vehicle system and a ground station. The unmanned aerial vehicle system includes a rotor power unit, a navigation control unit, a communication unit and an intelligent detection unit. The rotor power unit provides flight power, and the navigation control unit is responsible for environmental perception, positioning and three-dimensional point cloud model generation, guiding the unmanned aerial vehicle system to fly stably along a predetermined route. The communication unit ensures the instant transmission of flight status information, environmental information and video images between the unmanned aerial vehicle system and the ground station. The intelligent detection unit uses a camera and a gimbal to capture images of the bridge, and analyzes and identifies the type, size and location of the defect in real time. The ground station displays the flight status, environmental information and route execution of the unmanned aerial vehicle system in real time, supports route planning, and displays the defect identification results. It solves the problem of how to conduct inspections when it is difficult for the unmanned aerial vehicle to obtain effective GPS positioning information.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a bridge disease detection system and a detection method thereof based on unmanned aerial vehicles. Background Art

[0002] Bridge inspection refers to a comprehensive and systematic inspection and evaluation of bridge structures and their ancillary facilities to ensure the safe operation of bridges and extend their service life. Traditional bridge inspections mainly rely on bridge inspection vehicles as a platform, with inspectors conducting bridge inspections through visual inspections and auxiliary equipment.

[0003] The rapid development of drone technology has provided a safe and efficient solution for bridge inspection. Drones are equipped with high-definition cameras and other detection equipment to inspect bridge structures.

[0004] However, in complex environments such as under bridges, it is difficult for drones to obtain effective GPS positioning information. Due to the lack of GPS positioning information, flight control mainly relies on manual operation by the pilot. This not only places high demands on the professional technical capabilities of the pilot, but also increases the risk of operational errors and crashes.

[0005] Therefore, how to solve the inspection problem when it is difficult for drones to obtain effective GPS positioning information has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In the embodiments of the present application, by providing a drone-based bridge disease detection system and a detection method thereof, the problem of how to conduct inspections when it is difficult for the drone to obtain effective GPS positioning information is solved.

[0007] In a first aspect, an embodiment of the present application provides a bridge disease detection system based on a drone, including a drone system and a ground station; the drone system includes a rotor power unit, a navigation control unit, a communication unit and an intelligent detection unit; the rotor power unit is configured to provide flight power for the drone system, so that the drone system can stably and continuously perform the bridge disease detection task; the navigation control unit is configured to be responsible for environmental perception and positioning control during the flight of the drone system, generate a three-dimensional point cloud model of the area where the drone system is located, and enable the drone system to stably fly on a predetermined route; the communication unit is configured to allow data transmission between the drone system and the ground station, including flight status information, environmental information and captured video images of the drone system, to ensure the instant transmission and reception of information; the intelligent detection unit is configured to use a camera and a gimbal mounted on the drone system to capture an image of the bridge during the flight of the drone system, and to perform real-time analysis on the captured image, identify the type, size and location of the disease existing in the bridge, and generate a recognition result of the bridge disease; the ground station is configured to display the flight status information, environmental information and route execution of the drone system in real time, allowing users to plan and design routes, and display the recognition results of bridge diseases returned by the intelligent detection unit.

[0008] On the second aspect, an embodiment of the present application provides a detection method for a bridge disease detection system based on a drone, including: starting a rotor power unit, performing the first manual flight of the drone system, using a navigation control unit to collect environmental information, generating a three-dimensional point cloud model, and transmitting the three-dimensional point cloud model to a ground station through a communication unit; performing route planning and design on the ground station, inputting the coordinate positions of the waypoints, and in each subsequent flight, the drone system will automatically fly according to the predetermined route; the navigation control unit continuously performs environmental perception and positioning control; during the flight of the drone system, the intelligent detection unit uses the camera and gimbal mounted on the drone system to capture images of the bridge, and performs real-time analysis on the captured images to identify the type, size and location of the bridge disease, and generate a bridge disease identification result; the ground station displays the flight status information, environmental information and route execution status of the drone system in real time, allowing users to plan and design routes, and display the bridge disease identification results returned by the intelligent detection unit.

[0009] In a possible implementation, the use of a navigation control unit to collect environmental information and generate a three-dimensional point cloud model includes: the navigation control unit collects environmental information of the area where the drone system is located, and uses the environmental information to generate a three-dimensional point cloud model; during the flight of the drone system, the position of the drone system in the three-dimensional point cloud model is determined so that the drone system can fly stably on a predetermined route.

[0010] In a possible implementation, the navigation control unit collects environmental information of the area where the UAV system is located and generates a three-dimensional point cloud model using the environmental information, including: the navigation control unit performs environmental perception and positioning control by fusing laser radar and IMU data; two laser radars are installed on the UAV system; one laser radar is horizontally installed on the body of the UAV system, responsible for collecting point cloud information in the horizontal direction, and the other laser radar is vertically installed on the body of the UAV system, responsible for collecting point cloud information in the vertical direction; and a three-dimensional point cloud model of the area where the UAV system is located is generated using the point cloud information in the horizontal direction and the point cloud information in the vertical direction.

[0011] In a possible implementation, during the flight of the drone system, the position of the drone system in the three-dimensional point cloud model is determined so that the drone system can fly stably on a predetermined route, including: using the generated three-dimensional point cloud model to determine the coordinate positions of the waypoints, inputting the coordinate positions of the waypoints one by one into the ground station, and the ground station controls the distance between the drone system and the bridge to remain constant based on the coordinate positions of the waypoints; sending the data collected by the laser radar and the IMU to the airborne computer in real time; wherein the laser radar provides three-dimensional point cloud information of the environment, and the IMU data provides flight status information of the drone system; the airborne computer fuses the received laser radar and IMU data to generate the position information of the drone system in the three-dimensional point cloud model; and sending the position information of the drone system in the three-dimensional point cloud model to the flight control system, and the flight control system adjusts the flight attitude and speed of the drone system accordingly so that the drone system can fly stably on the predetermined route.

[0012] In a possible implementation, the method further includes: after the coordinate position of the waypoint is input, all the input waypoint information is displayed in the three-dimensional point cloud model; the waypoint information is checked in the three-dimensional point cloud model to ensure that there are no obstacles around each waypoint position; a reasonable hovering time is set for each waypoint, and the number of photos and the interval between photos at the waypoint are determined.

[0013] In a possible implementation, the intelligent detection unit includes an image processing subunit and a storage system; when the drone system flies to the location of a waypoint, the camera's photo-taking function is triggered; the image processing subunit receives the image of the bridge from the camera and processes the image. During the processing, the image processing subunit identifies and marks the type of bridge damage, measures the size and location of the bridge, and generates a bridge damage identification result; during the flight of the drone system, the collected bridge image and the generated bridge damage identification result are stored in real time in the storage system.

[0014] In a possible implementation, the ground station analyzes and evaluates the damage condition of the bridge by reading data from the storage system.

[0015] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects:

[0016] The embodiment of the present application provides a bridge disease detection system and detection method based on a drone. Through the stable flight capability of the rotor power unit and the environmental perception and positioning technology of the navigation control unit, the drone system can be ensured to stably fly and perform detection tasks in a complex and changeable bridge environment. The intelligent detection unit can capture the image of the bridge in real time and accurately identify the type, size and location of the disease, which significantly improves the efficiency and accuracy of bridge disease detection. The three-dimensional point cloud model of the area where the drone system is located generated by the navigation control unit not only provides a reliable basis for the stable flight of the drone system, but also provides intuitive visualization support for subsequent route planning and design. Users can easily plan the optimal route based on the three-dimensional model, further improving the efficiency and quality of the detection work. The efficient data transmission capability of the communication unit ensures the real-time transmission and reception of information between the drone system and the ground station. The ground station can display the flight status information, environmental information and route execution status of the drone system in real time, so that users can grasp the dynamics of the drone system at any time and ensure the safe implementation of the detection task. The user operation interface provided by the ground station is concise and clear, which not only allows users to plan and design routes, but also can intuitively display the bridge disease identification results returned by the intelligent detection unit. It solves the problem of how to conduct inspections when it is difficult for the drone to obtain effective GPS positioning information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a bridge disease detection system based on a drone provided in an embodiment of the present application;

[0019] Figure 2 A flow chart of a detection method of a bridge disease detection system based on a drone provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a drone-based bridge disease detection server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0023] Figure 1 A schematic diagram of a bridge disease detection system based on a drone provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, it includes an unmanned aerial vehicle system and a ground station. The unmanned aerial vehicle system includes a rotor power unit, a navigation control unit, a communication unit and an intelligent detection unit. The rotor power unit is configured to provide the flight power of the unmanned aerial vehicle system, so that the unmanned aerial vehicle system can stably and continuously perform the detection task of bridge defects. The navigation control unit is configured to be responsible for environmental perception and positioning control during the flight of the unmanned aerial vehicle system, generate a three-dimensional point cloud model of the area where the unmanned aerial vehicle system is located, and enable the unmanned aerial vehicle system to stably fly on the predetermined route. The communication unit is configured to transmit data between the unmanned aerial vehicle system and the ground station, including the flight status information, environmental information and captured video images of the unmanned aerial vehicle system, to ensure the instant transmission and reception of information. The intelligent detection unit is configured to use the camera and gimbal mounted on the unmanned aerial vehicle system to capture the image of the bridge during the flight of the unmanned aerial vehicle system, and to perform real-time analysis on the captured image, identify the type, size and location of the disease existing in the bridge, and generate the recognition result of the bridge disease. The ground station is configured to display the flight status information, environmental information and route execution of the unmanned aerial vehicle system in real time, allowing users to plan and design routes, and display the recognition results of bridge diseases returned by the intelligent detection unit.

[0024] It should be noted that the flight control system in the present application is arranged in the navigation control unit.

[0025] The present application embodiment provides a detection method of a bridge disease detection system based on a drone, such as Figure 2 As shown, the method includes steps S101 to S105. Among them, Figure 2 This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of a detection method of a bridge disease detection system based on a drone. In the case where the final result can be achieved, Figure 2 The steps shown may be performed in parallel or reversed.

[0026] S101: Start the rotor power unit, conduct the first manual flight of the UAV system, use the navigation control unit to collect environmental information, generate a three-dimensional point cloud model, and transmit the three-dimensional point cloud model to the ground station through the communication unit.

[0027] The navigation control unit is used to collect environmental information and generate a three-dimensional point cloud model, including: the navigation control unit collects environmental information of the area where the UAV system is located, and generates a three-dimensional point cloud model using the environmental information. During the flight of the UAV system, the position of the UAV system in the three-dimensional point cloud model is determined, so that the UAV system can stably fly on a predetermined route.

[0028] The navigation control unit collects environmental information of the area where the UAV system is located and uses the environmental information to generate a three-dimensional point cloud model, including: the navigation control unit performs environmental perception and positioning control by fusing laser radar and IMU data. Two laser radars are installed on the UAV system. Among them, one laser radar is installed horizontally on the body of the UAV system and is responsible for collecting point cloud information in the horizontal direction, and the other laser radar is installed vertically on the body of the UAV system and is responsible for collecting point cloud information in the vertical direction. The three-dimensional point cloud model of the area where the UAV system is located is generated using the point cloud information in the horizontal direction and the point cloud information in the vertical direction.

[0029] It should be noted that the IMU in this application is an inertial measurement unit.

[0030] Specifically, in the initial stage of bridge inspection, the operator will find a relatively flat area under the bridge that is suitable for placing the UAV system, place the UAV system securely and mark the location. Then, a safe place away from the UAV system is selected to place the portable ground station. Once the UAV system and the ground station are powered on and a communication connection is successfully established, the operator can intuitively see the 3D point cloud model of the bridge and its surrounding environment on the display screen of the ground station.

[0031] Furthermore, in order to enhance the accuracy of the 3D point cloud model and the density of the point cloud, and ensure the accurate positioning of the UAV system at the bottom of the bridge, the operator will start the rotor power unit and conduct the first manual flight of the UAV system. During this flight, the UAV system will fly under the bridge along the length of the bridge. At the same time, the operator will closely monitor the generation of the 3D point cloud model on the ground station, and judge the effect of the flight by observing the degree of match between the model and the actual environment at the bottom of the bridge. When the match between the two reaches a high standard, the operator will stop the flight in time and land the UAV system.

[0032] In the process of building a 3D point cloud model, the navigation control unit plays a core role. It is responsible for collecting information about the bridge and its surroundings, and generating a 3D point cloud model through calculation. This collection and calculation process is parallel, so the generated 3D point cloud model can be transmitted to the ground station through the communication unit in real time. In manual flight mode, the operator only needs to maintain a safe distance between the UAV system and the ground and the bottom of the bridge, and fly once along the length of the bridge to complete the construction of the 3D point cloud model. This method requires relatively low professional and technical level of flight operators, and the flight time is relatively short, which effectively avoids fatigue problems caused by long-term flight control and further ensures flight safety.

[0033] During the flight of the UAV system, the position of the UAV system in the three-dimensional point cloud model is determined so that the UAV system can fly stably on the predetermined route, including: using the generated three-dimensional point cloud model to determine the coordinate position of the waypoint, inputting the coordinate position of the waypoint into the ground station one by one, and the ground station controls the distance between the UAV system and the bridge to remain constant according to the coordinate position of the waypoint. The data collected by the laser radar and the IMU are sent to the onboard computer in real time. Among them, the laser radar provides the three-dimensional point cloud information of the environment, and the IMU data provides the flight status information of the UAV system. The onboard computer fuses the received laser radar and IMU data to generate the position information of the UAV system in the three-dimensional point cloud model. The position information of the UAV system in the three-dimensional point cloud model is sent to the flight control system, and the flight control system adjusts the flight attitude and speed of the UAV system accordingly, so that the UAV system can fly stably on the predetermined route.

[0034] Specifically, after the three-dimensional point cloud model is built, the coordinate positions of each waypoint can be determined according to the actual structure and requirements of the bridge, and the coordinate positions of these waypoints can be entered into the ground station one by one. This process provides clear guidance for the flight path of the UAV system. In the practice of bridge inspection, in order to ensure that the intelligent identification of the potential defect locations of the bridge can achieve high accuracy, the quality and size consistency of the collected bridge bottom images must be strictly guaranteed. Therefore, when the UAV system performs the inspection task, it is necessary to accurately control the distance between it and the bottom of the bridge to keep it constant. The realization of this requirement depends on the advanced positioning and control technology of the UAV system. When planning and designing the predetermined route, the coordinate positions of all waypoints are determined relative to the take-off point of the UAV system. Taking the flat bridge as an example, in order to ensure that the UAV system can always stay at the same height during the flight, the Z-axis coordinates of all waypoints can be set to the same value. This setting not only simplifies the complexity of flight control, but also further improves the accuracy and efficiency of inspection.

[0035] Furthermore, the method for determining the coordinate position of the waypoint also includes: defining an objective function, obtaining a waypoint coordinate position set by minimizing the objective function, and inputting the waypoint coordinate position set into the ground station. The objective function is: .in, is the objective function, is the coordinate position set of the waypoints, is the number of waypoints, Waypoint The coverage index is defined as the waypoints The number of point clouds within the preset range. Waypoint The safety index is defined as the waypoint distance to bridge structures or other obstructions, Waypoint The efficiency index is defined as the time it takes for the UAV system to fly from the previous waypoint to the waypoint Flight time, , and is the weight coefficient, which is used to measure the relative importance of different indicators. The objective function can be minimized using the gradient descent method. The value of the waypoint coordinate position set is continuously adjusted through the iterative process. In each iteration, the objective function value is calculated and the waypoint coordinates are updated. The optimization process is monitored to ensure that it converges to the optimal solution within a reasonable time. When the objective function value reaches the preset convergence condition, the iteration process is stopped. The convergence condition can be set as the relative change of the objective function value is less than a certain threshold, or the number of iterations reaches a preset upper limit.

[0036] Specifically, after the coordinate position of the waypoint is input, all the input waypoint information is displayed in the 3D point cloud model. The waypoint information is checked in the 3D point cloud model to ensure that there are no obstacles around each waypoint. A reasonable hovering time is set for each waypoint, and the number of photos and the interval between photos at the waypoint are determined.

[0037] Specifically, in the subsequent stages of the route planning and design process, once the coordinates of the waypoints are entered, this information will be immediately reflected in the 3D point cloud model. This allows operators to intuitively view and verify the accuracy and safety of all waypoints. Specifically, when the coordinate data of the waypoints are entered into the ground station one by one and synchronized to the 3D point cloud model, all waypoint locations will be clearly displayed in the 3D point cloud model. These waypoints are presented in the form of virtual markers, providing an intuitive visual reference for operators. The operator will check the information of these waypoints in the 3D point cloud model. Ensure that there are no potential obstacles around each waypoint location to ensure the safety of the UAV system during flight. By carefully reviewing the terrain, buildings and other possible obstacles in the 3D point cloud model, the operator can adjust the waypoint location in time to avoid potential collision risks. After ensuring that the waypoint location is safe and correct, the operator will set a reasonable hovering time for each waypoint. By setting the hovering time reasonably, it can be ensured that the UAV system has enough time to obtain high-quality image information at each waypoint. In addition, the operator also needs to determine the number of photos and the interval between photos at each waypoint. The number of photos to be taken depends on the resolution and detail of the required images and the requirements for subsequent analysis. The interval between photos needs to take into account the flight speed, stability and continuity of image acquisition of the UAV system. By planning the number of photos and the interval between photos, it can be ensured that the UAV system can efficiently obtain image information of the bottom of the bridge during flight.

[0038] Furthermore, setting a reasonable hovering time for each waypoint can be calculated using the following formula: .in, The hover time set for each waypoint, is the number of photos the drone system can take per second, is the total number of photos to be taken at each waypoint, is the time required for image processing to complete, Additional time is reserved to cope with unforeseen circumstances, such as changes in wind speed, fluctuations in drone system performance, etc.

[0039] It should be noted that if the performance of the UAV system degrades during continuous shooting due to thermal effects or other factors, additional time can be added to ensure image quality. If the inspection task has strict time requirements, the additional time can be reduced to improve efficiency.

[0040] S102: Plan and design the route on the ground station and input the coordinates of the waypoints. In each subsequent flight, the UAV system will automatically fly according to the predetermined route.

[0041] Specifically, in the entire inspection task process, the first inspection requires manual flight modeling, generating a 3D point cloud model, and inputting the coordinates of the waypoints to establish a predetermined route. In each subsequent flight, the drone system will automatically perform the entire inspection process based on the predetermined route set during the first modeling, without the need for manual intervention again.

[0042] S103: The navigation control unit continues to perform environment perception and positioning control.

[0043] S104: During the flight of the UAV system, the intelligent detection unit uses the camera and gimbal mounted on the UAV system to capture images of the bridge, and performs real-time analysis on the captured images to identify the type, size and location of the bridge defects, and generate identification results of the bridge defects.

[0044] The intelligent detection unit includes an image processing subunit and a storage system. When the drone system flies to the location of the waypoint, the camera's photo function is triggered. The image processing subunit receives the image of the bridge from the camera and processes the image. During the processing, the image processing subunit identifies and marks the type of bridge damage, measures the size and location of the bridge, and generates a recognition result of the bridge damage. During the flight of the drone system, the collected bridge image and the generated bridge damage recognition result are stored in the storage system in real time.

[0045] Specifically, at the ground station, the operator only needs to click on the execution task command, and the UAV system can automatically start the take-off procedure and fly stably in strict accordance with the preset route. This efficient operation is due to the intelligent detection unit on the UAV system. When the UAV system arrives at the location of each waypoint, its onboard camera will be automatically triggered to quickly capture the image of the bottom of the bridge. These image information is then immediately transmitted to the image processing module for analysis and processing. The image processing module performs fine processing on the received original image of the bottom of the bridge, which can not only accurately identify and mark the types of bridge defects, such as cracks and corrosion, but also accurately measure the size and location of these defects. On this basis, the recognition results of bridge defects will be generated. It is worth mentioning that the entire image processing process is completed in real time on the airborne end, that is, during the flight of the UAV system, without relying on manual recognition at the back end. This efficient processing method greatly improves the inspection efficiency and makes bridge detection work faster and more accurate. At the same time, the collected bridge images and the generated bridge disease recognition results will be properly stored in the storage system for subsequent reference and analysis.

[0046] It should be noted that a deep learning algorithm specifically for images is embedded in the image processing subunit. This algorithm performs deep learning training on a large number of high-definition bridge disease defect images, continuously optimizes and iterates, and finally realizes the recognition of bridge diseases. The algorithm can be a convolutional neural network model (referred to as: CNN). CNN is a deep learning algorithm that is particularly suitable for processing image data. In the task of bridge disease recognition, the CNN model can extract useful feature information such as texture, shape, color, etc. from the input bridge image, and then use this feature information to classify and locate the bridge disease. Suppose there is a set of high-definition image data sets containing a variety of bridge diseases, such as cracks, corrosion, and shedding. These images can be input into the CNN model for training. During the training process, the model will learn how to extract feature information from the image and classify the disease based on this feature information. Through continuous iteration and optimization, the model will gradually improve the accuracy and efficiency of recognition. When the training is completed, the trained CNN model can be embedded in the image processing subunit. When the UAV system captures an image of a bridge, the image processing subunit uses the model to analyze the image in real time, identify the type, size and location of the defects on the bridge, and generate corresponding identification results.

[0047] S105: The ground station displays the flight status information, environmental information and route execution of the UAV system in real time, allowing users to plan and design routes, and display the identification results of bridge defects returned by the intelligent detection unit. The ground station analyzes and evaluates the damage of the bridge by reading the data in the storage system.

[0048] Specifically, at the ground station, operators can easily read the data in the storage system, which covers the information collected by the drone system during the bridge inspection process. During the bridge inspection process, the 3D point cloud model constructed by the drone system and the waypoint information entered by the operator can also be easily exported and saved at the ground station. This function provides a solid foundation for subsequent bridge condition analysis, inspection report preparation, and possible maintenance plan design. Through this comprehensive management platform of the ground station, the efficiency and quality of bridge inspection work have been significantly improved.

[0049] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0050] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0051] The methods, devices or modules described in this application can be implemented in the form of computer-readable program codes. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) that can be executed by the (micro)processor, logic gates, switches, application-specific integrated circuits (English: Application Specific Integrated Circuit, referred to as: ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program codes, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0052] like Figure 3 As shown, an embodiment of the present application also provides a drone-based bridge defect detection server, including a memory 301 and a processor 302; the memory 301 is used to store computer-executable instructions; the processor 302 is used to execute computer-executable instructions to implement a detection method of a drone-based bridge defect detection system described above in an embodiment of the present application.

[0053] An embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When a computer executes the executable instructions, it can implement the detection method of the drone-based bridge disease detection system described in the embodiment of the present application.

[0054] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method described in the embodiment of the present application.

[0055] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations.

[0056] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A bridge disease detection system based on drones, characterized in that: Includes UAV systems and ground stations; The UAV system includes a rotor power unit, a navigation control unit, a communication unit and an intelligent detection unit; The rotor power unit is configured to provide flight power for the UAV system, so that the UAV system can stably and continuously perform the bridge disease detection task; The navigation control unit is configured to be responsible for environmental perception and positioning control during the flight of the UAV system, generate a three-dimensional point cloud model of the area where the UAV system is located, and enable the UAV system to fly stably on a predetermined route; The communication unit is configured to perform data transmission between the UAV system and the ground station, including flight status information, environmental information and captured video images of the UAV system, ensuring instant delivery and reception of information; The intelligent detection unit is configured to use a camera and a gimbal mounted on the UAV system to capture images of the bridge during the flight of the UAV system, and to perform real-time analysis on the captured images to identify the type, size and location of the disease existing on the bridge, and generate an identification result of the bridge disease; The ground station is configured to display the flight status information, environmental information and route execution status of the UAV system in real time, allowing the user to plan and design routes, and display the identification results of bridge defects returned by the intelligent detection unit; Start the rotor power unit, conduct the first manual flight of the UAV system, use the navigation control unit to collect environmental information, generate a three-dimensional point cloud model, and transmit the three-dimensional point cloud model to the ground station through the communication unit; Plan and design the route on the ground station, input the coordinates of the waypoints, and the UAV system will automatically fly according to the predetermined route in each subsequent flight; The method for determining the coordinate position of a waypoint includes: defining an objective function, obtaining a waypoint coordinate position set by minimizing the objective function, and inputting the waypoint coordinate position set into a ground station; the objective function is: ;in, is the objective function, is the coordinate position set of the waypoints, is the number of waypoints, Waypoint The coverage index is defined as the waypoints The number of point clouds within the preset range. Waypoint The safety index is defined as the waypoint distance to bridge structures or other obstructions, Waypoint The efficiency index is defined as the time it takes for the UAV system to fly from the previous waypoint to the waypoint Flight time, , and is the weight coefficient, which is used to measure the relative importance of different indicators; The intelligent detection unit includes an image processing subunit and a storage system; when the UAV system flies to the location of the waypoint, the camera's photo function is triggered; the image processing subunit receives the image of the bridge from the camera and processes the image. During the processing, the image processing subunit identifies and marks the type of bridge damage, measures the size and location of the bridge, and generates a recognition result of the bridge damage; during the flight of the UAV system, the collected bridge image and the generated bridge damage recognition result are stored in the storage system in real time; The image processing module processes the received original image of the bottom of the bridge, which can not only accurately identify and mark the types of bridge defects, including cracks and corrosion, but also accurately measure the size and location of these defects. The entire image processing process is completed in real time on the airborne end during the flight of the UAV system. The image processing subunit is embedded with a deep learning algorithm specifically for images, which includes a convolutional neural network model. It also includes: setting a reasonable hovering time for each waypoint, and determining the number of photos and the interval between photos taken at the waypoint; The reasonable hovering time for each waypoint is calculated by the following formula: ;in, The hovering time set for each waypoint, is the number of photos the drone system can take per second, is the total number of photos to be taken at each waypoint, is the time required for image processing to complete, Extra time is reserved to deal with unforeseen situations, including changes in wind speed, fluctuations in drone system performance, etc.; if the drone system performance degrades during continuous shooting due to thermal effects or other factors, extra time can be added to ensure image quality; if the inspection task has strict time requirements, the extra time can be reduced to improve efficiency.

2. The detection method of the bridge disease detection system based on drone according to claim 1 is characterized in that: include: Start the rotor power unit, conduct the first manual flight of the UAV system, use the navigation control unit to collect environmental information, generate a three-dimensional point cloud model, and transmit the three-dimensional point cloud model to the ground station through the communication unit; Plan and design the route on the ground station, input the coordinates of the waypoints, and the UAV system will automatically fly according to the predetermined route in each subsequent flight; The navigation control unit continuously performs environmental perception and positioning control; During the flight of the UAV system, the intelligent detection unit uses the camera and gimbal mounted on the UAV system to capture images of the bridge, and performs real-time analysis on the captured images to identify the type, size and location of the bridge defects, and generate identification results of the bridge defects; The ground station displays the UAV system’s flight status information, environmental information, and route execution in real time, allowing users to plan and design routes, and displaying the identification results of bridge defects returned by the intelligent detection unit.

3. The detection method of the bridge disease detection system based on drone according to claim 2 is characterized in that: The method of collecting environmental information using the navigation control unit to generate a three-dimensional point cloud model includes: The navigation control unit collects environmental information of the area where the UAV system is located and uses the environmental information to generate a three-dimensional point cloud model; During the flight of the UAV system, the position of the UAV system in the three-dimensional point cloud model is determined so that the UAV system can fly stably on the predetermined route.

4. The detection method of the bridge disease detection system based on drone according to claim 3 is characterized in that: The navigation control unit collects environmental information of the area where the UAV system is located and generates a three-dimensional point cloud model using the environmental information, including: The navigation control unit performs environmental perception and positioning control by fusing lidar and IMU data; Two laser radars are installed on the UAV system; one laser radar is installed horizontally on the body of the UAV system and is responsible for collecting point cloud information in the horizontal direction, and the other laser radar is installed vertically on the body of the UAV system and is responsible for collecting point cloud information in the vertical direction; The point cloud information in the horizontal direction and the point cloud information in the vertical direction are used to generate a three-dimensional point cloud model of the area where the UAV system is located.

5. The detection method of the bridge disease detection system based on drone according to claim 4 is characterized in that: During the flight of the UAV system, determining the position of the UAV system in the three-dimensional point cloud model so that the UAV system can stably fly on a predetermined route includes: The method for determining the coordinate positions of the waypoints also includes: determining the coordinate positions of the waypoints using the generated three-dimensional point cloud model, and inputting the coordinate positions of the waypoints into the ground station one by one; The ground station controls the distance between the UAV system and the bridge to remain constant based on the coordinate position of the waypoint; The data collected by the laser radar and IMU are sent to the onboard computer in real time; the laser radar provides three-dimensional point cloud information of the environment, and the IMU data provides the flight status information of the UAV system; The onboard computer fuses the received LiDAR and IMU data to generate the position information of the UAV system in the 3D point cloud model; The position information of the UAV system in the three-dimensional point cloud model is sent to the flight control system, and the flight control system adjusts the flight attitude and speed of the UAV system accordingly so that the UAV system can fly stably on the predetermined route.

6. The detection method of the bridge disease detection system based on drone according to claim 5 is characterized in that: Also includes: After the coordinate position of the waypoint is input, all the input waypoint information is displayed in the 3D point cloud model; Check the waypoint information in the 3D point cloud model to ensure that there are no obstacles around each waypoint location.

7. The detection method of the bridge disease detection system based on drone according to claim 6 is characterized in that: The ground station analyzes and evaluates the damage condition of the bridge by reading the data in the storage system.

Images