Navigation-free autonomous flight bridge bottom inspection control method and unmanned aerial vehicle-mounted control equipment

Abstract

The invention discloses a navigation-free autonomous flight bridge bottom inspection control method and unmanned aerial vehicle-mounted control equipment, and the method comprises the steps: responding to the obtained three-dimensional point cloud data of a to-be-inspected bridge, planning and selecting a bridge bottom inspection path through the three-dimensional point cloud data, and enabling the starting point of the bridge bottom inspection path to start from the outside of the bridge; and generating a bridge bottom inspection route based on the bridge bottom inspection path and executing inspection, wherein the inspection comprises bridge bottom navigation-free flight control, autonomous obstacle avoidance, roof-imitating flight and inspection waypoint image acquisition. After the unmanned aerial vehicle flies out of the bridge, when the electric quantity of the unmanned aerial vehicle is too low or the unmanned aerial vehicle is manually triggered to return, the return flight route of the unmanned aerial vehicle is automatically called to execute a return flight task during return flight. The method disclosed by the invention can be efficiently operated on light embedded equipment, so that the unmanned aerial vehicle can realize real-time perception and real-time flight control on the bridge in the flight process, a more efficient and safer bridge bottom inspection flight control solution is provided, and the development of the unmanned aerial vehicle in the bridge inspection field is promoted.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) application technology, specifically relating to a navigation-free autonomous flight bridge underpass inspection control method and UAV-borne control equipment. Background Technology

[0002] With the continuous advancement of infrastructure construction in my country, the number and scale of various bridges are constantly increasing. As important transportation hubs, the safety of bridges directly affects the smooth flow of transportation and the safety of people's lives and property. As a crucial piece of infrastructure requiring high attention, ensuring the safe and reliable operation of bridges necessitates comprehensive, timely, and efficient inspections. Statistics show that my country currently has millions of bridges of various types, with a considerable number being added each year. Among these bridges, the bridge deck, due to its special location and complex structure, is often a high-risk area for safety hazards. The bridge deck structure is exposed to the natural environment for extended periods, and is susceptible to cracks, spalling, and corrosion due to various factors such as wind and rain erosion, vehicle collisions, and ship impacts. If these problems are not detected and addressed in a timely manner, they may lead to serious safety accidents, such as bridge collapse.

[0003] Traditional bridge under-bridge inspections rely primarily on manual labor, requiring inspectors to use boats, suspended platforms, and other equipment to approach the bridgebed. However, this method is not only inefficient but also poses significant safety risks. Furthermore, the complex environment under bridges makes it difficult to guarantee the accuracy of manual inspections. As infrastructure construction continues, the number and size of bridges will continue to grow, placing immense pressure on bridge inspection and maintenance. In addition, some bridges are located in remote or inaccessible areas, making manual inspections even more difficult and costly. Therefore, the rapid development of drone technology presents new opportunities for bridge under-bridge inspections.

[0004] Due to their advantages of flexibility, low cost, and high efficiency, drones can quickly reach the bottom of bridges for inspection. Currently, although some drones are used for bridge inspection, most rely on satellite navigation systems for flight control. However, due to the complex environment and significant signal interference under bridges, satellite navigation systems often cannot function properly, posing a significant challenge to the autonomous flight control of drones. Therefore, there is an urgent need for a bridge inspection control method that enables drone-borne control equipment to achieve autonomous flight in environments without satellite navigation signals. Summary of the Invention

[0005] The summary section of this application is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] To address the problems and shortcomings of existing technologies, this invention aims to provide a navigation-free autonomous flight bridge underpass inspection control method and UAV-borne control equipment. By using the UAV-borne control equipment as the core, the UAV itself as the carrier, and UAV-borne lidar and gimbal cameras as the main sensors, it enables autonomous flight control and precise inspection of the UAV even in bridge underpass environments without satellite navigation signals, providing strong technical support for bridge safety management. This solves the problems mentioned in the background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] As a first aspect of this application, the present invention discloses a bridge underpass inspection control method for unguided autonomous flight, wherein the bridge underpass inspection control method is integrated into an unmanned aerial vehicle (UAV) onboard control device, and includes the following steps.

[0009] Step 1, in response to acquiring the 3D point cloud data of the bridge to be inspected;

[0010] Step 2: Plan and select the inspection path under the bridge using the three-dimensional point cloud data. The starting point of the inspection path under the bridge must be outside the bridge.

[0011] Step 3: Generate an inspection route under the bridge based on the inspection path under the bridge and perform the inspection. The inspection includes unguided flight control under the bridge, autonomous obstacle avoidance, top-following flight and inspection waypoint image acquisition.

[0012] Step 4: First, fly the drone outside the bridge, then return it to its starting point if the drone's battery is too low or if it is manually activated.

[0013] Step 5: Upon returning to base, the drone's return route will be automatically invoked to execute the return mission;

[0014] During inspections, the IMU inertial sensor built into the lidar in the UAV-borne control equipment is used to fuse the laser inertial SLAM autonomous flight control in order to cope with the drift phenomenon after the satellite navigation connection is lost.

[0015] Preferably, in step 1, a DJI M350 drone is used with a DJI Zenmuse L2 gimbal to record point clouds, and the results of the point cloud recording are then used with DJI Terra to reconstruct the 3D point cloud to obtain a 3D point cloud model. Finally, 3D point cloud data is obtained based on the 3D point cloud model.

[0016] Preferably, in step 2, when planning the bridge under-inspection path using 3D point cloud data, there is no need to consider the interval between photo points. The 3D point cloud data is opened and reconstructed using geographic information system software, and the required bridge under-inspection path is selected from the 3D point cloud data according to the set inspection distance.

[0017] Preferably, in step 3, generating the bridge under-inspection route based on the inspection path involves first calculating the length of the inspection path based on the bridge under-inspection path, then obtaining the field of view and overlap rate of the gimbal camera installed under the fuselage of the unmanned aerial vehicle to calculate the interval between the photo waypoints, and using the interval between the photo waypoints, generating multiple inspection waypoints one by one from the starting point to the end point of the bridge under-inspection path along the direction vector of the inspection path to form the bridge under-inspection route.

[0018] Preferably, the navigation-free flight control in step 3 further includes the following steps:

[0019] Step 3.1.1: Assign an altitude value to the inspection waypoint, and calculate the target displacement between the inspection waypoint and the current position of the UAV.

[0020] Step 3.1.2: Calculate the remaining distance from the current UAV to the target waypoint based on the target displacement;

[0021] Step 3.1.3: Use the deceleration distance and remaining distance of the UAV's current speed to determine whether it is in the deceleration or acceleration zone;

[0022] Step 3.1.4: For the deceleration range and acceleration range, respectively, adjust the current speed with acceleration to maintain uniform flight speed;

[0023] Step 3.1.5: Decompose the corrected current velocity into velocity components in three axial directions and send them to the virtual joystick to execute the flight of the unmanned aerial vehicle.

[0024] Preferably, the autonomous obstacle avoidance in step 3 further includes the following steps:

[0025] Step 3.2.1: Acquire radar point cloud data around the unmanned aerial vehicle using the lidar;

[0026] Step 3.2.2: Filter the radar point cloud data to remove all existing noise.

[0027] Step 3.2.3: Calculate the angle between each point in the filtered radar point cloud data and the direction vector, and select points whose angles are within a specific range to form an obstacle dataset;

[0028] Step 3.2.4: For each point in the obstacle dataset, calculate its Euclidean distance to the center point of the lidar, and take the minimum value of these distances as the closest distance to the obstacle in a specific direction;

[0029] Step 3.2.5: Calculate the nearest distance to obstacles in all directions in sequence and provide timely feedback for flight control.

[0030] Preferably, in step 3, the top-flying is as follows: first, the distance between the UAV and the first waypoint under the bridge is recorded as the reference distance; then, the distance between the UAV and the bridge bottom at a certain moment during the inspection is recorded as the distance from the top. When the reference distance is greater than the distance from the top, the UAV needs to be compensated to move away from the bridge bottom; otherwise, the UAV needs to be compensated to move closer to the bridge bottom. Finally, the compensated height is added to the vertical component of the three velocity components in step 3.1.5.

[0031] Preferably, in step 3, the image acquisition of the inspection waypoints is carried out by the UAV calling the UAV gimbal camera control interface through the PSDK interface after arriving at each inspection waypoint, and then moving to the next inspection waypoint in sequence until all inspection waypoints have been photographed.

[0032] Preferably, the laser inertial SLAM autonomous flight control is as follows: first, the relative motion of the IMU between two scans of the lidar is calculated; then, prior information for attitude estimation of the unmanned aerial vehicle is obtained through a Kalman filter; the linear velocity of the unmanned aerial vehicle is transformed into the IMU coordinate system; the residual is calculated by the difference between the linear velocity displacement of the unmanned aerial vehicle and the relative distance estimated by the prior IMU; then, the residual and the Kalman gain are substituted to obtain the posterior state; and finally, the motion trajectory of the unmanned aerial vehicle is obtained by superimposing the relative motion information.

[0033] As a second aspect of this application, the present invention also discloses a navigation-free autonomous flight bridge under-inspection UAV-borne control device. The UAV-borne control device is installed on the top of the UAV and includes a lidar, an edge computing module, and a coaxial cable. The lidar integrates an inertial measurement unit (IMU) and is used to sense the distance to the bridge underside and the surrounding environment in real time. The edge computing module provides computing power support for the navigation-free autonomous flight bridge under-inspection control method. The coaxial cable is used to connect the UAV-borne control device to power it and can also connect the edge computing module to the UAV to enable communication between the two. The gimbal camera is installed under the fuselage of the UAV and is controlled by calling the UAV gimbal camera control interface through the PSDK interface.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] This invention provides a navigation-free autonomous flight control method and UAV-borne control equipment for bridge underpass inspection. Responding to the acquisition of 3D point cloud data of the bridge to be inspected, an underpass inspection path is planned and selected based on the 3D point cloud data. The starting point of the underpass inspection path must be outside the bridge. An underpass inspection route is generated based on the inspection path and the inspection is executed. The inspection includes navigation-free flight control under the bridge, autonomous obstacle avoidance, top-following flight, and waypoint image acquisition. After the UAV flies to the outside of the bridge, it returns to base when the battery is low or manually triggered. During the return, the return route is automatically invoked to execute the return mission. This invention, through an efficient, safe, and stable waypoint flight algorithm, better supports the smooth execution of underpass inspection tasks.

[0036] In practical applications, traditional UAV bridge inspection methods require complex satellite navigation positioning under the bridge or rely on manual flight. The method of this invention, however, can achieve safe and stable flight even in environments without satellite navigation. Traditional bridge inspection relies on manual flight, which has a large blind spot from the human perspective, making it impossible to accurately perceive the distance between the aircraft and the bridge. With the increasing demand for bridge inspection, manual inspection not only faces a severe shortage of manpower but also contradicts the current trend of rising labor costs. Therefore, this invention, by installing an edge computing module on the UAV and enabling the UAV's virtual joystick, allows the UAV to operate independently of manual control, thereby significantly improving inspection efficiency and saving labor costs. The lidar on the UAV's onboard control equipment can perceive the distance between the aircraft and the bridge in real time with millimeter-level accuracy, greatly improving the safety of bridge inspections.

[0037] The method of this invention can operate efficiently on lightweight embedded devices, enabling UAVs to achieve real-time perception and flight control of bridges during flight. This provides a more efficient and safer flight control solution for bridge inspection in practical applications, significantly promoting the further development of UAV technology in the field of bridge inspection. Attached Figure Description

[0038] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application.

[0039] In the attached diagram:

[0040] Figure 1 This is a flowchart illustrating the steps of the bridge underpass inspection control method without navigation in an embodiment of the present invention.

[0041] Figure 2 This is a control diagram for the bridge bottom inspection control method without navigation autonomous flight in this embodiment of the invention.

[0042] Figure 3 This is a control diagram for flight control within the bridge underpass inspection control method without navigation in this embodiment of the invention;

[0043] Figure 4 This is a control diagram for attitude perception within the bridge bottom inspection control method for unguided autonomous flight in this embodiment of the invention.

[0044] Figure 5 This is a schematic diagram of the inspection path within the bridge bottom inspection control method without navigation in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the bridge underpass inspection control method without navigation in an embodiment of the present invention, showing the return flight within the bridge.

[0046] Figure 7 This is a connection diagram of the UAV-borne control device in an embodiment of the present invention;

[0047] Figure 8 This is a perspective view of the UAV-borne control device in an embodiment of the present invention;

[0048] Figure 9 This is a side view of the UAV-borne control device in an embodiment of the present invention;

[0049] Figure 10 This is an internal connection diagram of the unmanned aerial vehicle (UAV) onboard control device in an embodiment of the present invention. Detailed Implementation

[0050] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0051] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0052] Example 1

[0053] In this embodiment of the invention, a navigation-free autonomous flight bridge underpass inspection control method and an unmanned aerial vehicle (UAV) onboard control device are disclosed. The navigation-free autonomous flight bridge underpass inspection control method is integrated into an UAV onboard control device mounted on top of the UAV. (Refer to...) Figures 7 to 10As shown, the UAV-borne control equipment includes a lidar, an edge computing module, and a coaxial cable, with the lidar integrating an inertial measurement unit (IMU). The lidar, as a core component, is used to sense the distance to the bridge underside and the surrounding environment in real time. The edge computing module provides computing power for the navigation-free autonomous flight bridge underside inspection control method of this invention. The coaxial cable connects the control equipment, providing power, and also enables communication between the edge computing module and the UAV, facilitating subsequent control of the basic UAV.

[0054] The navigation-free autonomous flight bridge inspection control method of this invention mainly comprises two parts: flight control and attitude perception. When the UAV executes flight control, the position of each waypoint relative to the current UAV is decomposed into absolute geodetic coordinate offsets, which include the UAV's absolute altitude offset and the aircraft's horizontal offset based on the "northeast-sky" coordinate system. Subsequently, a steady-state algorithm is applied to these offsets separately using acceleration to stabilize the aircraft's instantaneous velocity, maintain stable movement, and ensure that the aircraft does not sway violently. Finally, the three-axis velocity components in the "northeast-sky" direction are output. After the steady-state algorithm, the displacement of the aircraft from the target waypoint is converted into velocity, and different transmission quantities are set according to the DJI virtual joystick control mode, which are sent to the virtual joystick to execute corresponding actions. Situational perception runs in parallel throughout the entire method, used to perceive the state of the surrounding environment and the distance to the bridge top in real time. When an obstacle is approaching the current UAV, the inspection is automatically paused and avoidance is performed. At the same time, the waypoint correction algorithm is activated to correct the waypoint based on the obstacle position obtained from the situational perception and continue to execute the subsequent inspection waypoint flight tasks. It can automatically control the distance between the UAV and the bridge bottom according to the pre-set inspection distance when the UAV gets closer or farther away, so that the distance error is less than 0.5 meters.

[0055] Specifically, refer to Figures 1 to 4 As shown, the present invention mainly includes the following steps:

[0056] Step 1, in response to acquiring the 3D point cloud data of the bridge to be inspected;

[0057] Step 2: Plan and select the inspection path under the bridge using 3D point cloud data. The starting point of the inspection path under the bridge must be outside the bridge.

[0058] Step 3: Generate an inspection route under the bridge based on the inspection path and execute the inspection. The inspection includes unguided flight control under the bridge, autonomous obstacle avoidance, top-following flight, and inspection waypoint image acquisition.

[0059] Step 4: After flying the drone outside the bridge, return it to base when the drone's battery is too low or when it is manually activated.

[0060] Step 5: When returning to home, the return route of the unmanned aerial vehicle is automatically called to perform the return mission.

[0061] First, the process involves acquiring the 3D point cloud data of the bridge to be inspected. During bridge inspection, it's necessary to obtain the 3D point cloud data of the bridge beforehand. This point cloud data must contain all bridge points within the inspection area, and each bridge point must be represented by latitude, longitude, and altitude coordinates. Furthermore, the acquisition of the bridge's 3D point cloud data can be achieved using a DJI M350 drone paired with a DJI Zenmuse L2 gimbal for point cloud recording. The recorded point cloud data is then used to reconstruct the 3D point cloud using DJI Terra. Based on the reconstructed 3D point cloud model, further 3D point cloud data is obtained. Since the point cloud data is acquired using the aforementioned existing equipment, detailed descriptions are omitted here.

[0062] Then, the inspection path under the bridge is planned and selected based on the collected 3D point cloud data. Specifically, the inspection path under the bridge is planned using the 3D point cloud data of the bridge to be inspected, and the starting point of the inspection path must be outside the bridge. When planning the inspection path under the bridge, it is not necessary to consider the interval between photo points; simply open the reconstructed point cloud data using geographic information system software such as GlobalMapper, and select the path to be inspected from the point cloud data, such as... Figure 5 As shown. Set the inspection distance (i.e., the distance for taking photos under the bridge). Manually select and export the inspection path, then input it along with the inspection distance into the subsequent bridge under-bridge inspection algorithm. The inspection distance is the distance between the UAV and the top of the bridge when it takes pictures under the bridge, i.e., the distance from the shooting surface. The default inspection distance is 5 meters.

[0063] Step 3 involves generating an inspection route under the bridge based on the inspection path and performing the inspection. The inspection includes unguided flight control under the bridge, autonomous obstacle avoidance, top-following flight, and image acquisition of inspection waypoints.

[0064] After receiving the inspection path under the bridge, the system generates inspection waypoints based on the inspection distance to form an inspection route. Specifically, the entire inspection path under the bridge is composed of several smaller inspection paths. Let the starting point and ending point of a certain inspection path be... and Therefore, the inspection path direction vector can be calculated using the coordinates of the inspection path's endpoint and starting point, and is represented as follows: Then calculate the length of the inspection path. Length of the inspection path The Euclidean distance between the starting point and the ending point is... Next, the field of view of the gimbal camera is obtained. and overlap rate The gimbal camera is typically mounted under the fuselage of the unmanned aerial vehicle (UAV) and is controlled via the PSDK interface, which calls the UAV's gimbal camera control interface. The PSDK interface is a development toolkit provided by DJI to support developers in creating payload devices that can be mounted on DJI drones. (Gimbal camera field of view) This is an inherent parameter of the camera, referring to the field of view that the lens can capture. It is usually expressed as a horizontal, vertical, or diagonal angle, with the shorter side being the vertical field of view. Overlap rate. This represents the percentage of overlap between two adjacent waypoint images captured by the gimbal camera, within the width of the waypoint image, and ranges from 0 to 1. Overlap Rate The default value is 0.1, meaning there is a 10% overlap between photos of two adjacent waypoints. This is based on the gimbal camera's field of view. and overlap rate Calculate the interval between each photo waypoint , is represented as:

[0065] ;

[0066] in, Expressed as overlap rate, This is expressed as the field of view width. The field of view width... The field of view of the gimbal camera can be used The distance is calculated from the inspection distance and is expressed as:

[0067] ;

[0068] in, Indicated as inspection distance, This refers to the field of view of the gimbal camera. The inspection distance here... This is the distance from the top of the bridge when the drone takes pictures from under the bridge.

[0069] From the starting point of the inspection route Starting, along the inspection path direction vector Step-by-step generation of photo waypoints until the destination The first inspection route point Each point is represented as ,in The index of the waypoint for taking photos is counted starting from 0. Represented as:

[0070] ;

[0071] in, It is represented as a unit direction vector, and its calculation formula is: That is, the direction from the starting point of the inspection to the end point of the inspection. This indicates the number of times the inspection is inserted at equal intervals from the start point to the end point of the inspection. Each point is a patrol waypoint, resulting in multiple patrol waypoints. It can form a patrol route under the bridge.

[0072] For unguided flight control under bridges, the following steps are also included:

[0073] Step 3.1.1: Assign an altitude value to the inspection waypoint and calculate its target displacement relative to the current position of the UAV, using the inspection waypoint as the target waypoint.

[0074] Step 3.1.2: Calculate the remaining distance from the current UAV to the target waypoint based on the target displacement;

[0075] Step 3.1.3: Use the deceleration distance and remaining distance of the UAV's current speed to determine whether it is in the deceleration or acceleration zone;

[0076] Step 3.1.4: For both the deceleration and acceleration intervals, adjust the current speed using acceleration to maintain constant speed flight.

[0077] Step 3.1.5: Decompose the corrected current velocity into velocity components in three axial directions and send them to the virtual joystick to execute the flight of the unmanned aerial vehicle.

[0078] Specifically, after obtaining the inspection waypoints, the flight control phase begins, and flight control is performed one by one according to the generated inspection waypoints. This invention represents the coordinate system of the inspection waypoints as "North-East-Sky," which is a local spatial coordinate system with a specific reference point as the origin and due north, due east, and vertically upward as the three positive directions. This system is used to accurately describe the relative position of the target "how far and how high" it is from the reference point. Since the coordinates generated during the inspection route generation phase do not consider the height dimension, and because the waypoint height is closely related to the distance to the top of the bridge and changes constantly during the inspection process, we can initially assign an initial value to the height. Default initial value The value is the bridge surface point cloud height minus the inspection distance. The bridge deck point cloud height is the elevation value of each 3D point cloud data point on the upper surface of the bridge deck. The inspection waypoints obtained from the above steps... Mark as the target waypoint The current location of the unmanned aerial vehicle is The displacement of the unmanned aerial vehicle that needs to move to the target waypoint is then represented as... That is, the offsets in the three axes can be obtained through the Euclidean coordinate system.

[0079] The drone's flight control utilizes DJI's PSDK virtual joystick. The PSDK virtual joystick is a drone control interface developed by DJI for embedded edge computing modules, employing a speed control mode. DJI provides a dedicated API interface; simply sending the speeds of the "North, East, Sky" axes via the API achieves the corresponding virtual joystick movement. However, directly sending the target waypoint to the PSDK virtual joystick results in significant drone swaying. Considering flight safety, the movement is decomposed into horizontal and vertical motions, and steady-state control in both directions is incorporated. By providing the drone's acceleration, the drone's speed is smoothly increased to achieve stable flight. The maximum horizontal flight speed of the drone is set to... Its velocity at the previous moment (i.e., its initial velocity). initial acceleration The maximum speed at the current moment is expressed as .in, This represents the time interval for sending control commands to the DJI virtual joystick; the default value is 20 milliseconds.

[0080] To ensure the unmanned aerial vehicle (UAV) flies in a straight line horizontally, a smooth velocity output is needed based on an acceleration mechanism until it reaches the target waypoint. Specifically, we calculate the remaining distance from the UAV to the target waypoint based on the displacement made therefrom, denoted as [missing information]. ,Right now Current speed of the unmanned aerial vehicle Represented as This means the current velocity is the resultant velocity along the three axes. The deceleration distance of the unmanned aerial vehicle at its current velocity is expressed as... So when the remaining distance Less than or equal to deceleration distance When the unmanned aerial vehicle enters the deceleration zone, and when the remaining distance... Greater than deceleration distance The aircraft enters the uniform acceleration range. The acceleration speed of the unmanned aerial vehicle during this range is... The speed in the deceleration range is When the acceleration zone reaches its maximum speed. At that time, before the aircraft enters the deceleration zone, it will be at... The aircraft is flying at a constant speed. The current speed is decomposed into three axes. Then, the velocity components of each axis can be obtained directly and sent to the DJI virtual joystick for execution, which are represented as follows: , and By using DJI's virtual joystick speed control mode, Sending a message to the virtual joystick enables the virtual joystick to control the unmanned aerial vehicle.

[0081] Because there is no RTK signal under the bridge, and the UAV experiences altitude drift after the RTK signal is lost under the bridge, we introduce laser inertial SLAM autonomous flight control technology and top-flying simulation technology to solve these problems. Laser inertial SLAM autonomous flight control technology provides precise positioning of the UAV under the bridge, while top-flying simulation provides a stable shooting distance. Both technologies are integrated into the UAV-borne control device of this invention. We integrate an inertial measurement unit (IMU) inside the lidar, and perform positioning by fusing SLAM using the lidar's built-in inertial sensor. The UAV's flight control speed is obtained from the DJI drone's PSDK interface. The PSDK interface can obtain the UAV's instantaneous velocity in the "northeast-sky" coordinate system. Fusing these instantaneous velocities into the laser inertial SLAM compensates for the IMU's acceleration drift, ultimately outputting stable position coordinates.

[0082] Specifically, this invention integrates the horizontal linear velocity of the UAV's flight control system, building upon existing multi-sensor fusion SLAM localization and reconstruction methods. Existing multi-sensor fusion SLAM localization calculates the relative motion of the IMU between two LiDAR scans, obtaining prior information for UAV attitude estimation via a Kalman filter. The Kalman filter then updates the IMU prior information to obtain the final posterior state quantity, i.e., the relative motion information between the two LiDAR scan frames. The UAV's trajectory is obtained by superimposing this relative motion information. This invention, however, introduces the observation of the UAV's linear velocity after completing the IMU prior estimation, defining the Kalman gain as g. Among them, g Represented as the Kalman gain matrix, and Let them represent the observation noise coefficient matrix and its transpose, respectively. Represented as the observation noise matrix, It is represented as the prior variance matrix.

[0083] Since the IMU's prior coordinate system is the "front-left-upper" coordinate system, the UAV's linear velocity in the "north-east-sky" coordinate system needs to be transformed to the IMU's "front-left-upper" coordinate system. The UAV's attitude angle is defined as follows: "North-East Sky" coordinates Rotate to "front left top" coordinates This can be done using the following formula:

[0084] ;

[0085] No. The linear velocity displacement of the unmanned aerial vehicle at any given moment is the displacement of the unmanned aerial vehicle. flight It can be done Time and Time difference Linear velocity of unmanned aerial vehicles flight The calculation shows that:

[0086] flight flight ;

[0087] No. Time residuals The relative distance difference between the front left and upper left is estimated using the aircraft displacement in the "front left and upper left" coordinate system and IMU prior estimation. The calculation formula is as follows:

[0088] ;

[0089] The relative distance difference is the displacement between the linear velocity displacements of the unmanned aerial vehicle at two adjacent moments, i.e., the first... Time relative to The displacement at time -1, and the relative distance difference estimated by the IMU prior, are similarly calculated. Wherein, imu Indicates the forward distance of the IMU. imu Indicates the leftward distance of the IMU. imu Indicates the IMU up-distance. flight Indicates the forward distance of the aircraft's displacement. flight This indicates the leftward displacement of the aircraft. flight This represents the upward distance of the aircraft's displacement. The general form of the posterior state is:

[0090] ;

[0091] Substituting the residuals and Kalman gain into the equation yields the posterior state of the final fused linear velocity displacement of the spacecraft. The posterior covariance matrix is... G is a three-dimensional identity matrix. Represents the Kalman gain matrix. It is the observation noise coefficient matrix. The prior covariance matrix, This is the posterior state. This is the prior state. This is the residual.

[0092] For autonomous obstacle avoidance, the following steps are also included:

[0093] Step 3.2.1: Acquire radar point cloud data around the unmanned aerial vehicle using the lidar inside the UAV's onboard control equipment;

[0094] Step 3.2.2: Filter the radar point cloud data to remove all existing noise.

[0095] Step 3.2.3: Calculate the angle between each point in the filtered radar point cloud data and the direction vector, and select points whose angles are within a specific range to form an obstacle dataset;

[0096] Step 3.2.4: For each point in the obstacle dataset, calculate its Euclidean distance to the center point of the lidar, and take the minimum of these distances as the closest distance to the obstacle in a specific direction;

[0097] Step 3.2.5: Calculate the nearest distance to obstacles in all directions in sequence and provide timely feedback for flight control.

[0098] Specifically, the UAV's onboard control unit, located in the nose of the drone, incorporates a Livox Mid360 LiDAR. Operating at 10Hz, this LiDAR provides relatively real-time data updates, ensuring the drone can promptly perceive changes in its surroundings during flight. By emitting a laser beam and measuring the return time of the reflected light, the LiDAR accurately acquires point cloud data of the scene surrounding the drone. Since a point cloud is a collection of numerous three-dimensional points, each containing its spatial location information (X, Y, Z coordinates) and intensity information (intensity of reflected light), this point cloud data can depict the detailed three-dimensional structure of the drone's surrounding environment, including the shape and location of terrain, buildings, trees, and obstacles such as other aircraft.

[0099] Further processing of point cloud data can extract distance information of obstacles in specific directions, such as in front of and above the lidar. Because point cloud data contains noise, statistical filtering of the lidar point cloud data is necessary before using it. Specifically, assuming the lidar point cloud data is represented as... , where each point It is a three-dimensional coordinate For each point Select points within its neighborhood. Then calculate the mean and standard deviation, which are expressed as follows:

[0100] ;

[0101] ;

[0102] in, Represented as the number of points in the neighborhood. Belongs to the field A point within the range. For each point... Calculate its deviation from the neighborhood mean, expressed as .if This means that the point deviates significantly from the whole, so the point is... These are marked as noise points. This represents a custom threshold, used to control the unit defining the degree of offset; the default value is 2. Then, from the point cloud data... Remove all points marked as noise to obtain filtered point cloud data. Represented as By simply extracting the point closest to the radar in front of it, using the Euclidean distance formula to calculate the distance between this point and the lidar, and then taking the minimum of these distances as the closest distance to obstacles in that direction, the distances to the lidar in each direction can be obtained sequentially.

[0103] Specifically, to extract points in front of the lidar, a directional region needs to be defined. Assume the area in front of the lidar is a cone-shaped region centered on the lidar or a fan-shaped region in a specific direction. More specifically, assume the direction vector in front of the lidar is... Define an angle range This is used to represent the angle of the fan-shaped region in front of the lidar. For the filtered point cloud data... Each point in Calculate its relationship with the direction vector The included angle , represented as Each point Defined by the three-dimensional coordinate system Then, we choose the included angle. Within a specific angular range The points inside are These selected points are combined to form a new obstacle dataset. For obstacle datasets For each point, calculate its Euclidean distance to the center point of the lidar. Taking the center point of the lidar as the origin, the Euclidean distance of each point is expressed as:

[0104] ;

[0105] From all distances The minimum value is found, and this minimum value is the closest distance to the obstacle. The distances to obstacles in other directions are calculated in the same way as the distance in front, so they will not be repeated here. Since the lidar and the UAV are fixedly connected, meaning there is no relative motion between them, the obstacle distance measured by the lidar can be considered the distance between the UAV and the obstacle. This equivalence makes lidar a very important sensor in UAV obstacle avoidance systems, directly providing accurate obstacle avoidance information to the UAV's control system. When an obstacle is detected, the distance information is promptly fed back to the UAV through the PSDK interface for flight control, enabling the UAV to brake in time to avoid the risk of collision.

[0106] For top-hinged flight, specifically, after detecting the distances to the lidar in each direction in the above steps, the top-hinged flight phase only requires using the distance between the top of the UAV and the bottom of the bridge. The purpose of top-hinged flight is to keep this distance stable within a consistent range. When this distance increases or decreases, the distance value is fed back to the flight control unit, causing the UAV to move up or down to compensate. Specifically, the distance from the first waypoint of the UAV entering the bottom of the bridge to the bottom of the bridge is taken as the baseline distance. During the inspection, at a certain moment, the distance between the aircraft and the bottom of the bridge is the distance from the top. Set the distance variation threshold range as , The default value is 1 meter. When... When the distance is less than the reference distance, it means the drone's altitude is below the reference distance, and the drone needs to be moved away from under the bridge. When the distance is greater than the reference distance, it indicates that the unmanned aerial vehicle (UAV) is at an altitude greater than the reference distance, and in this case, the UAV needs to be moved closer to the bottom of the bridge. Both of these situations require altitude compensation for the UAV. By adding a compensation altitude to the vertical component of the target waypoint from step 3.1.5 at a distance of meters, the aircraft can maintain a reference altitude for inspection. Finally, waypoint images are acquired. After reaching each waypoint, the UAV calls the UAV gimbal camera control interface through the PSDK interface to control the camera to trigger a photo. After taking a photo, it moves to the next waypoint until the photo-taking task for all waypoints is completed.

[0107] Steps 4 and 5 involve flying the drone outside the bridge. If the drone's battery is low or it is manually triggered to return to home, the drone will automatically use its return-to-home route. The purpose of flying outside the bridge is to return to home. Because DJI drones will ascend before returning to home if they are more than 50 meters from the takeoff point, ascending when the drone is under the bridge poses a safety hazard; therefore, it is necessary to fly the drone outside the bridge. The timing for triggering the return-to-home action is when the drone's battery is low or it is manually triggered. At this point, the drone will automatically fly outside the bridge and then automatically return to home. The return-to-home process will automatically use the drone's return-to-home route. The specific return-to-home execution method uses existing technology and will not be elaborated further here. Figure 6 This is a schematic diagram illustrating the internal return-to-base method for bridge-bottom inspection control without navigation.

[0108] Example 2

[0109] Based on the content of Embodiment 1 above, performance testing was further conducted in this embodiment. The UAV-borne control device of this invention integrates an edge computing module and a LiDAR, connected via DJI coaxial cable. The DJI coaxial cable is a dedicated connection cable provided by DJI Innovations for the connection between its flight platform and external load devices, primarily used for transmitting data, power, and control signals. The coaxial cable is used to connect the control device for power supply and also to facilitate communication between the edge computing module and the UAV, enabling subsequent basic control of the UAV. The LiDAR used is a Livox-Mid360, and the edge computing module is an RK3588. The RK3588 is a high-performance edge AI computing chip; its powerful CPU, GPU, and NPU combination makes it an ideal platform for deploying and running various algorithms on the device side (edge). The performance parameters of the RK3588 edge computing module are shown in Table 1 below.

[0110]

[0111] Table 1

[0112] Using a DJI M350-RTK equipped with a Zenmuse H30 camera, bridge bottom inspection tests were conducted on several bridges in Shaanxi, Shandong, and Chongqing. The navigation-free autonomous flight control method using this patented technology successfully completed the bridge bottom inspection operations safely and efficiently.

[0113] The RK3588 edge computing module platform was used for three months of continuous testing in three regions. Performance metrics were evaluated by extracting algorithm runtime and aircraft attitude data from 100 sets of data, and average values ​​were calculated. The average runtime for target detection while the UAV is hovering was set at 100 milliseconds, and the average runtime for wind turbine status estimation while the aircraft is running was 0.3 milliseconds.

[0114] The performance evaluation metrics of this invention mainly include algorithm runtime and aircraft attitude stability. The stability of the UAV's flight attitude is assessed by generating a number of swaying motions greater than 10 degrees using the method of this invention. The experiment also examines the frequency of human intervention required when encountering dangerous situations during mission execution. The experimental results show that, regarding UAV attitude stability, without this method, the average swaying angle in a single mission was 15.2 degrees. However, with this method, the swaying angle was only 2.7 degrees, representing a 5-fold improvement in UAV flight stability. In terms of safety, through 100 real-world tests, the number of human interventions required during bridge inspections using this flight control algorithm was 0.

[0115] All technologies not described in detail in this invention are existing technologies. It should be understood that the indicated orientations or positional relationships in the description of this invention are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0116] The above description is merely an explanation of some preferred embodiments of this disclosure and the technical principles employed. Those skilled in the art should understand that, in addition to the scope of the invention as described in the embodiments of this disclosure, the present invention may have other implementations. It is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure. For those skilled in the art, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A method for bridge underpass inspection control without navigation, characterized in that: The bridge underpass inspection and control method is integrated into the UAV-borne control equipment and includes the following steps: Step 1, in response to acquiring the 3D point cloud data of the bridge to be inspected; Step 2: Plan and select the inspection path under the bridge using the three-dimensional point cloud data. The starting point of the inspection path under the bridge must be outside the bridge. Step 3: Generate an inspection route under the bridge based on the inspection path under the bridge and perform the inspection. The inspection includes unguided flight control under the bridge, autonomous obstacle avoidance, top-following flight and inspection waypoint image acquisition. Step 4: First, fly the drone outside the bridge, then return it to its starting point if the drone's battery is too low or if it is manually activated. Step 5: Upon returning to base, the drone's return route will be automatically invoked to execute the return mission; During inspections, the IMU inertial sensor built into the lidar in the UAV-borne control equipment is used to fuse the laser inertial SLAM autonomous flight control in order to cope with the drift phenomenon after the satellite navigation connection is lost.

2. The bridge underpass inspection control method without navigation as described in claim 1, characterized in that: In step 1, a DJI M350 drone is used with a DJI Zenmuse L2 gimbal to record point clouds. The recorded point clouds are then reconstructed into a 3D point cloud model using DJI Terra. Finally, 3D point cloud data is obtained based on the 3D point cloud model.

3. The bridge underpass inspection control method without navigation as described in claim 2, characterized in that: In step 2, when planning the bridge under-slope inspection path using 3D point cloud data, there is no need to consider the interval between photo points. The 3D point cloud data is opened and reconstructed using geographic information system software, and the required bridge under-slope inspection path is selected from the 3D point cloud data according to the set inspection distance.

4. The bridge underpass inspection control method without navigation for autonomous flight according to claim 3, characterized in that: In step 3, the bridge inspection route is generated based on the inspection path. First, the length of the inspection path is calculated based on the bridge inspection path. Then, the field of view and overlap rate of the gimbal camera installed under the fuselage of the UAV are obtained to calculate the interval between the photo waypoints. Using the interval between the photo waypoints, multiple inspection waypoints are generated one by one from the starting point to the end point of the bridge inspection path along the direction vector of the inspection path to form the bridge inspection route.

5. The bridge underpass inspection control method without navigation as described in claim 4, characterized in that: The navigation-free flight control in step 3 also includes the following steps: Step 3.1.1: Assign an altitude value to the inspection waypoint, and calculate the target displacement between the inspection waypoint and the current position of the UAV. Step 3.1.2: Calculate the remaining distance from the current UAV to the target waypoint based on the target displacement; Step 3.1.3: Use the deceleration distance and remaining distance of the UAV's current speed to determine whether it is in the deceleration or acceleration zone; Step 3.1.4: For the deceleration range and acceleration range, respectively, adjust the current speed with acceleration to maintain uniform flight speed; Step 3.1.5: Decompose the corrected current velocity into velocity components in three axial directions and send them to the virtual joystick to execute the flight of the unmanned aerial vehicle.

6. The bridge underpass inspection control method without navigation as described in claim 5, characterized in that: The autonomous obstacle avoidance in step 3 also includes the following steps: Step 3.2.1: Acquire radar point cloud data around the unmanned aerial vehicle using the lidar; Step 3.2.2: Filter the radar point cloud data to remove all existing noise. Step 3.2.3: Calculate the angle between each point in the filtered radar point cloud data and the direction vector, and select points whose angles are within a specific range to form an obstacle dataset; Step 3.2.4: For each point in the obstacle dataset, calculate its Euclidean distance to the center point of the lidar, and take the minimum value of these distances as the closest distance to the obstacle in a specific direction; Step 3.2.5: Calculate the nearest distance to obstacles in all directions in sequence and provide timely feedback for flight control.

7. The bridge underpass inspection control method without navigation as described in claim 6, characterized in that: In step 3, the simulated top flight involves first recording the distance from the first waypoint of the UAV to the bottom of the bridge as the reference distance, then recording the distance from the UAV to the bottom of the bridge at a certain moment during the inspection process as the distance from the top. When the reference distance is greater than the distance from the top, the UAV needs to be compensated to move away from the bottom of the bridge; otherwise, it needs to be compensated to move closer to the bottom of the bridge. Finally, the compensated height is added to the vertical component of the three velocity components in step 3.1.

5.

8. The bridge underpass inspection control method for unguided autonomous flight according to claim 7, characterized in that: In step 3, the image acquisition of the inspection waypoints is as follows: after the UAV arrives at each inspection waypoint, it calls the UAV gimbal camera control interface through the PSDK interface to trigger the taking of a picture. After taking the picture, it moves to the next inspection waypoint in sequence until all the inspection waypoints have been photographed.

9. The bridge underpass inspection control method for unguided autonomous flight according to claim 8, characterized in that: The laser inertial SLAM autonomous flight control method first calculates the relative motion of the IMU between two laser radar scans, obtains prior information for UAV attitude estimation through a Kalman filter, transforms the UAV linear velocity to the IMU coordinate system, calculates the residual by the difference between the UAV linear velocity displacement and the relative distance estimated by the IMU prior, and then substitutes the residual and Kalman gain to obtain the posterior state. Finally, the UAV's trajectory is obtained by superimposing the relative motion information.

10. A navigation-free, autonomous flight unmanned aerial vehicle (UAV) for bridge underpass inspection, characterized in that: The UAV-borne control equipment is installed on the top of the UAV and includes a lidar, an edge computing module, and a coaxial cable. The lidar integrates an inertial measurement unit (IMU) and is used to sense the distance to the bridge underside and the surrounding environment in real time. The edge computing module provides computing power support for the bridge underside inspection control method without navigation. The coaxial cable is used to connect the UAV-borne control equipment to power it and can also connect the edge computing module to the UAV to enable communication between the two. The gimbal camera is installed under the fuselage of the UAV and is controlled by calling the UAV gimbal camera control interface through the PSDK interface.

Images