Closed channel unmanned aerial vehicle autonomous inspection system and method based on multiple sensors

Abstract

The invention provides a multi-sensor-based closed channel unmanned aerial vehicle autonomous inspection system and method.The system comprises a laser radar, a task computer and a flight control computer, and the laser radar is used for collecting three-dimensional point cloud data in a pipeline area in the flight process of an unmanned aerial vehicle and sending the three-dimensional point cloud data to the task computer; the three-dimensional point cloud data is sent to the task computer; the task computer is used for performing wall sensing on the pipeline area passed by the unmanned aerial vehicle in the flight process according to the three-dimensional point cloud data to obtain wall feature information; according to the wall feature information, constructing a flight control instruction corresponding to the unmanned aerial vehicle; and the flight control computer is used for adjusting the flight state of the unmanned aerial vehicle according to the flight control instruction sent by the task computer. The application scene and space of the unmanned aerial vehicle inspection system are expanded.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to an autonomous inspection system and method for UAVs in closed passages based on multiple sensors. Background Technology

[0002] In recent years, drones have been widely used in infrastructure inspection, urban pipeline inspection, and industrial equipment inspection due to their high maneuverability and flexible viewing angles. They are particularly advantageous in complex, dangerous, or confined environments, where they can replace manual labor for remote inspections. Conventional drones in outdoor or open environments rely on high-precision positioning systems (such as RTK-GNSS) for autonomous navigation and mission execution. In open-air scenarios with good satellite signals, RTK technology can provide centimeter-level positioning accuracy, effectively supporting refined route planning and flight control.

[0003] However, in enclosed passages (such as subway tunnels, underground utility tunnels, and industrial pipelines), RTK signals cannot effectively cover the area due to severe structural obstruction. Currently, to solve the positioning problem, the industry generally adopts a holistic solution centered on SLAM (Simultaneous Localization and Mapping) technology. Its main technical approach involves equipping the drone with a high-performance computing unit and a multi-sensor fusion system (such as depth cameras, LiDAR, and IMU), using visual SLAM, LiDAR SLAM, or multi-source information fusion SLAM algorithms to achieve autonomous positioning and mapping of the drone within the pipeline. During inspection, the drone collects surrounding environmental data in real time, synchronously calculates its own spatial pose within the pipeline using SLAM algorithms, dynamically generates an environmental map, and completes the inspection task by combining preset tasks or its own self-generated flight path.

[0004] While SLAM technology provides a feasible path for UAVs to achieve autonomous localization in environments without satellite signals, it faces numerous challenges in special scenarios such as enclosed pipes. First, the confined space, linear extension, and highly repetitive structure of pipes lack rich and unique visual or laser features, making it difficult for feature-matching-based visual SLAM or laser SLAM algorithms to achieve stable tracking, leading to localization drift and increased cumulative error. Second, pipes often present complex conditions such as low light, strong reflections, dust, or water mist, affecting the imaging and measurement accuracy of cameras or lidar, thus reducing the robustness of the SLAM system. Third, the long length of pipes makes it difficult for a single SLAM system to maintain accuracy over long periods without external references, and the cost of building high-precision electronic maps for SLAM-based UAV navigation systems is high. These factors contribute to the technical bottlenecks faced by UAVs in enclosed pipe scenarios, including unstable localization, large cumulative errors, easy loss of localization, and high operating costs. Therefore, there is an urgent need for a multi-sensor-based autonomous inspection system and method for UAVs operating in enclosed passages to address these issues. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an autonomous inspection system and method for unmanned aerial vehicles (UAVs) in closed passages based on multiple sensors.

[0006] This invention provides an autonomous inspection system for unmanned aerial vehicles (UAVs) in closed passages based on multiple sensors, including a lidar, a mission computer, and a flight control computer, wherein: The lidar is used to collect three-dimensional point cloud data in the pipeline area during the flight of the UAV, and send the three-dimensional point cloud data to the mission computer. The task computer is used to perform wall perception on the pipeline area passed by the UAV during flight based on the three-dimensional point cloud data, obtain wall feature information, and construct flight control commands corresponding to the UAV based on the wall feature information. The flight control computer is used to adjust the flight status of the UAV according to the flight control commands sent by the mission computer.

[0007] According to the present invention, a multi-sensor-based autonomous inspection system for unmanned aerial vehicles (UAVs) in a closed passage is provided. The system further includes an image acquisition device for acquiring images of markers in front of the UAV during flight and sending the acquired marker image data to the mission computer. The markers are set at preset positions within the passage area to indicate that the UAV executes corresponding flight control commands at the preset positions. The mission computer is also used to determine the flight action control command to be executed by the UAV at the preset position based on the image data of the marker.

[0008] According to the present invention, a multi-sensor-based autonomous inspection system for unmanned aerial vehicles (UAVs) in closed passages includes a task computer comprising a data acquisition module, an environmental perception and flight path generation module, an obstacle detection module, a marker recognition module, a task management module, and a control module, wherein: The data acquisition module is used to acquire the 3D point cloud data and the marker image data, and to perform time synchronization processing and data caching processing on the 3D point cloud data and the marker image data to obtain the original point cloud data and the original image data. The environmental perception and flight path generation module is used to determine the wall location information and the wall distribution type corresponding to the wall location information in the pipeline area based on the wall feature information obtained by wall perception from the original point cloud data, and generate the initial flight path information corresponding to the UAV based on the wall location information and the wall distribution type. The obstacle detection module is used to detect obstacles in front of the UAV based on the original point cloud data and obtain obstacle detection results. The identifier recognition module is used to identify the flight action control command of the UAV at the current preset position based on the identifier image data; The task management module is used to construct the target flight path information corresponding to the UAV based on the initial flight path information, the obstacle detection results, and the flight action control commands. The control module is used to generate the flight control commands based on the target route information.

[0009] According to the present invention, an autonomous inspection system for unmanned aerial vehicles (UAVs) in a closed passage based on multiple sensors is provided, wherein the environmental perception and flight path generation module is specifically used for: Based on the longitudinal axis of symmetry of the UAV, the original point cloud data is divided into point cloud data on the left side of the UAV and point cloud data on the right side of the UAV. Linear fitting operations were performed on the point cloud data on the left and right sides of the UAV respectively to obtain the linear structure information of the pipe wall. The wall position information is determined based on the straight line of the wall corresponding to the linear structure information of the pipe wall; If, based on the linear structure information of the pipe wall, it is determined that there are straight wall lines on both the left and right sides of the pipe area, the initial flight path information is generated based on the midpoint between the straight wall lines on the left and right sides of the pipe area. If, based on the linear structure information of the pipe wall, it is determined that there is a straight wall on one side of the pipe area, the initial flight path information is generated based on the preset safety distance of the UAV and the offset distance between the straight wall on one side of the pipe area.

[0010] According to the present invention, an autonomous inspection system for unmanned aerial vehicles (UAVs) in a closed passage based on multiple sensors is provided, wherein the environmental perception and flight path generation module is further used for: The original point cloud data is projected onto the XY plane of the UAV body coordinate system to obtain the projected planar point cloud data; The projected planar point cloud data is downsampled to obtain downsampled planar point cloud data. Based on the longitudinal axis of symmetry of the UAV, the downsampled planar point cloud data is divided into point cloud data on the left side of the UAV and point cloud data on the right side of the UAV.

[0011] According to the present invention, an autonomous inspection system for unmanned aerial vehicles (UAVs) in a closed passage based on multiple sensors is provided, wherein the task management module is specifically used for: If, based on the obstacle detection results, it is determined that there are no obstacles ahead of the drone, the initial flight path information is used as the target flight path information. If, based on the obstacle detection results, it is determined that there is an obstacle in front of the UAV, the initial flight path information is adjusted based on a preset obstacle scheduling decision to obtain the target flight path information; If the flight maneuver control command is obtained during the flight of the UAV, the flight maneuver control command is added to the corresponding position in the initial flight path information of the UAV to obtain the target flight path information; The preset obstacle scheduling decision includes: When the distance between the drone and the obstacle meets the preset safe distance, the drone waiting timer process is executed; If the obstacle is removed during the drone waiting timer process, the unfinished initial flight path information will be used as the target flight path information. If the obstacle is not removed during the drone waiting timer process, the drone return information will be used as the target flight path information.

[0012] This invention also provides a multi-sensor-based autonomous inspection method for unmanned aerial vehicles (UAVs) in closed passages, comprising: During the flight of the drone, three-dimensional point cloud data of the pipeline area is collected; Based on the three-dimensional point cloud data, wall perception is performed on the pipeline area passed by the UAV during flight to obtain wall feature information; Based on the wall feature information, the flight control commands corresponding to the UAV are constructed; The flight status of the UAV is adjusted according to the flight control commands.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the autonomous inspection method for unmanned aerial vehicles based on multiple sensors in a closed passage as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-sensor-based autonomous inspection method for unmanned aerial vehicles in closed passages as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-sensor-based autonomous inspection method for unmanned aerial vehicles in closed passages as described above.

[0016] The present invention provides a multi-sensor-based autonomous inspection system and method for closed passage UAVs. During the flight of the UAV, three-dimensional point cloud data of the pipeline area is collected by lidar and transmitted to the mission computer. The mission computer then uses wall perception to obtain feature information and construct flight control commands. Finally, the flight control computer adjusts the flight state of the UAV according to the commands. Therefore, it does not rely on high-precision positioning methods such as RTK or SLAM. It can still achieve autonomous navigation and stable flight of UAVs in environments with no GNSS signals, repetitive structures, or scarce visual features, thus expanding the application scenarios and space of UAV inspection systems. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A schematic diagram of the structure of the multi-sensor-based autonomous inspection system for closed-channel drones provided by the present invention; Figure 2 This invention provides a schematic diagram of the flight path output of the two side walls of the drone. Figure 3 This is a schematic diagram of the flight path output for a single-sided wall of a drone provided by the present invention; Figure 4 A flowchart illustrating the multi-sensor-based autonomous inspection method for closed-channel drones provided by this invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] Figure 1 This is a schematic diagram of the structure of the multi-sensor-based autonomous inspection system for unmanned aerial vehicles in a closed passage provided by the present invention, as shown below. Figure 1 As shown, this invention provides an autonomous inspection system for unmanned aerial vehicles (UAVs) in a closed passage based on multiple sensors, including a lidar 101, a mission computer 102, and a flight control computer 103, wherein: The lidar 101 is used to collect three-dimensional point cloud data in the pipeline area during the flight of the UAV and send the three-dimensional point cloud data to the mission computer 102. The task computer 102 is used to perform wall perception on the pipe area passed by the UAV during flight based on the three-dimensional point cloud data, obtain wall feature information, and construct flight control commands corresponding to the UAV based on the wall feature information. The flight control computer 103 is used to adjust the flight status of the UAV according to the flight control commands sent by the mission computer 102.

[0021] In this invention, through the collaborative work of multiple sensors, the UAV can autonomously and safely plan its route and complete inspection tasks without relying on RTK and SLAM positioning. The closed-channel UAV autonomous inspection system mainly consists of three core parts: a lidar 101, a mission computer 102, and a flight control computer 103. These parts cooperate with each other to complete functions such as environmental perception, decision-making and planning, and flight control.

[0022] Specifically, the LiDAR 101 serves as the primary environmental perception sensor, collecting real-time 3D point cloud data of the pipeline area in front of and around the UAV during its flight. This 3D point cloud data accurately reflects the spatial features of the pipeline walls and obstacles, providing fundamental data support for subsequent wall perception and flight path planning.

[0023] The lidar 101 operates continuously during the drone's flight, transmitting the collected 3D point cloud data to the mission computer 102 for further processing and analysis. For example, during pipeline inspection, the lidar 101 continuously scans the surrounding environment, acquiring the 3D coordinate information of various locations inside the pipeline, forming 3D point cloud data.

[0024] After receiving the 3D point cloud data sent by the lidar 101, the mission computer 102 first preprocesses the point cloud data. Selecting point clouds within a predetermined range above and below the UAV ensures that the analyzed data accurately reflects the spatial state of the UAV's current flight path, avoiding interference from irrelevant data. Simultaneously, the point cloud data is projected onto a horizontal plane to highlight the cross-sectional structure of the pipe, facilitating subsequent wall detection. For example, when processing the point cloud data, point clouds that are too far or too close to the UAV, or outside the current flight path range, are filtered out, retaining only those within the effective range. These point clouds are then projected onto a horizontal plane, making the cross-sectional structure of the pipe clearer and facilitating the identification of wall locations.

[0025] Furthermore, the mission computer 102 divides the preprocessed point cloud data into left and right point clouds according to the left-right direction of the UAV (based on the body coordinate system). Then, it performs line fitting processing on the point clouds in these two regions respectively, with the aim of identifying and fitting the feature planes of the walls on the left and right sides. Through the line fitting algorithm, the line that best represents the wall features can be found from the point cloud data, thereby determining the position and orientation of the wall.

[0026] In this invention, the detection of the left and right walls mainly includes the following two situations: Simultaneously, the system detects the walls on both sides: by analyzing and calculating the spatial positions of the walls, it automatically finds the midpoint between the two walls and uses this midpoint as a reference point for the drone's current flight path. This effectively ensures that the drone flies safely along the centerline of the pipeline, avoiding collisions with the walls due to deviation from the centerline. For example, in a straight pipeline, after detecting the walls on both sides, the midpoint position is calculated, and the drone flies with this point as a reference, always staying near the center of the pipeline.

[0027] If only one wall is detected: Based on the spatial location information of the wall, the system translates the wall feature in a straight line to the centerline of the pipeline according to a preset safety distance, generating a new reference point as the current flight path point. This processing method allows the drone to pass safely and smoothly even when there is only a wall on one side. For example, at the turnout of a railway tunnel or the branching point of a pipeline, there may only be a wall on one side. In this case, the system translates the flight path according to a preset safety distance, generating a suitable flight path reference point for the drone and ensuring flight safety.

[0028] Furthermore, the mission computer 102 constructs flight control commands for the UAV based on the wall feature information obtained from wall sensing. These commands include flight path information, speed information, and attitude adjustment information that the UAV needs to follow during flight, guiding the UAV on how to safely and efficiently complete the inspection task. For example, after determining the UAV's flight path based on the wall feature information, the mission computer generates corresponding control commands to make the UAV fly along that path.

[0029] The flight control computer 103 receives flight control commands from the mission computer 102 and adjusts the UAV's flight status accordingly. The flight control computer 103 is the direct executor of UAV flight control, capable of responding to mission computer commands in real time and controlling the UAV's motors, servos, and other actuators to achieve various flight maneuvers such as takeoff, landing, hovering, forward movement, backward movement, and turning.

[0030] In this invention, the flight control computer 103 continuously monitors the flight control commands sent by the mission computer 102. Once a command is received, the flight status of the UAV is immediately adjusted. For example, when the mission computer 102 sends a command requiring the UAV to turn left, the flight control computer 103 will control the corresponding servo motor of the UAV to make the UAV turn left, ensuring that the UAV flies along the predetermined route.

[0031] The autonomous inspection system for closed passages provided by this invention uses a multi-sensor-based drone to collect three-dimensional point cloud data of the pipeline area via lidar during drone flight and transmits it to the mission computer. The mission computer then uses wall perception to obtain feature information and constructs flight control commands. Finally, the flight control computer adjusts the drone's flight status according to these commands. This eliminates the need to rely on high-precision positioning methods such as RTK or SLAM, enabling autonomous navigation and stable flight of the drone even in environments with no GNSS signals, repetitive structures, or scarce visual features. This expands the application scenarios and scope of drone inspection systems.

[0032] Based on the above embodiments, the system further includes an image acquisition device 104, used to acquire images of markers in front of the UAV and send the acquired marker image data to the mission computer 102; wherein, the markers are set at a preset position in the pipeline area to indicate that the UAV executes corresponding flight action control commands at the preset position. The mission computer 102 is also used to determine the flight action control command to be executed by the UAV at the preset position based on the image data of the marker.

[0033] In this invention, reference may be made to Figure 1 As shown, in addition to core components such as the lidar 101, mission computer 102, and flight control computer 103, the closed-channel UAV autonomous inspection system also includes an image acquisition device 104 (such as a camera), which further enhances the system's environmental perception and mission execution capabilities. The image acquisition device 104 works in conjunction with the mission computer 102, acquiring image data of specific markers to provide crucial information for the UAV's precise flight control within the pipeline, ensuring that the UAV can complete the inspection task according to preset requirements.

[0034] The image acquisition device 104 is used to acquire images of markers in front of the drone during its flight. As the drone flies along the pipeline, it captures images of markers in the environment in front of it in real time. These markers are specific objects that are pre-set in the pipeline area and have specific features such as shape, color or pattern, for example, QR codes, color marks and reflective strips.

[0035] After acquiring the image data of the markers, the image acquisition device 104 immediately sends this image data to the task computer 102 to ensure that the task computer 102 can obtain the latest marker information in a timely manner, providing a basis for subsequent decision-making and control. For example, during the high-speed flight of the UAV, the image acquisition device 104 continuously acquires images of the markers ahead and quickly transmits the data to the task computer 102, enabling the task computer 102 to make accurate judgments based on real-time information.

[0036] In this invention, markers are placed at predetermined locations within the pipeline area, typically associated with critical areas or specific tasks within the pipeline. For example, markers are placed at pipeline bends, branches, and near equipment requiring inspection, enabling the drone to accurately identify and perform the corresponding operations.

[0037] Each marker represents a specific flight control command that the drone needs to perform at that preset location. Different markers have different meanings. By identifying the type or characteristics of the marker, the drone can determine the flight actions it should take at that location, such as deceleration, turning, hovering, and acceleration. For example, a reflective strip of a specific color may indicate that the drone needs to decelerate and hover for a period of time to conduct a detailed inspection of the surrounding environment; while a QR code marker may contain more complex instructions, guiding the drone to perform a specific inspection route or operational task.

[0038] After receiving the marker image data sent by the image acquisition device 104, the mission computer 102 analyzes and processes the image using image recognition algorithms and preset rules. By identifying the type, characteristics, and position of the marker in the image, the mission computer 104 can accurately determine the flight control commands to be executed by the UAV at the preset position. For example, the mission computer 104 decodes the acquired QR code image to obtain the command information contained therein, and then generates corresponding flight control commands based on this information, such as adjusting the flight altitude and changing the flight direction.

[0039] Specifically, the mission computer 102 first receives the marker image data sent by the image acquisition device 104, and preprocesses this data, such as denoising and enhancement, to improve image quality and facilitate subsequent recognition and analysis. Then, it uses an image recognition algorithm to identify markers in the preprocessed image, determining the type and feature information of the markers in the image by comparing them with a preset marker feature library. For example, it identifies reflective strips in the image and determines their color, shape, and position. Next, based on the identified marker information, and in conjunction with preset rules and mission requirements, the mission computer 102 generates flight control commands for the UAV to execute at the preset location. For example, if the identified marker indicates a need to turn at that location, the mission computer 102 will generate corresponding turning control commands, including parameters such as turning angle and speed. Finally, the generated flight control commands are sent to the flight control computer 103, which executes these commands, adjusts the UAV's flight state, and enables it to complete the flight mission according to preset requirements.

[0040] This invention, through the collaborative work of image acquisition device 104 and mission computer 102, collects and identifies preset markers in the pipeline, providing precise flight control commands for the UAV, and realizing autonomous and accurate inspection of the UAV in a closed pipeline environment.

[0041] Based on the above embodiments, the task computer includes a data acquisition module, an environmental perception and route generation module, an obstacle detection module, a marker recognition module, a task management module, and a control module, wherein: The data acquisition module is used to acquire the 3D point cloud data and the marker image data, and to perform time synchronization processing and data caching processing on the 3D point cloud data and the marker image data to obtain the original point cloud data and the original image data. The environmental perception and flight path generation module is used to determine the wall location information and the wall distribution type corresponding to the wall location information in the pipeline area based on the wall feature information obtained by wall perception from the original point cloud data, and generate the initial flight path information corresponding to the UAV based on the wall location information and the wall distribution type. The obstacle detection module is used to detect obstacles in front of the UAV based on the original point cloud data and obtain obstacle detection results. The identifier recognition module is used to identify the flight action control command of the UAV at the current preset position based on the identifier image data; The task management module is used to construct the target flight path information corresponding to the UAV based on the initial flight path information, the obstacle detection results, and the flight action control commands. The control module is used to generate the flight control commands based on the target route information.

[0042] In this invention, the mission computer, acting as the "intelligent hub" of the closed-channel UAV autonomous inspection system, integrates multiple functional modules. These modules work collaboratively to achieve autonomous inspection tasks for the UAV in complex pipeline environments. These modules include a data acquisition module, an environmental perception and flight path generation module, an obstacle detection module, a marker recognition module, a task management module, and a control module. Each module has its unique function and role, working together to ensure the UAV completes its inspection work safely and efficiently.

[0043] Specifically, the data acquisition module is the source of data acquisition for the entire system, used to collect data from all onboard sensors of the UAV (such as LiDAR and cameras). The LiDAR collects 3D point cloud data, which can accurately reflect the spatial position and shape of objects (such as walls and obstacles) within the pipe; the camera collects image data of markers, used to identify various pre-set markers within the pipe. Simultaneously, the data acquisition module performs time synchronization processing on the collected 3D point cloud data and marker image data to ensure that the data collected by different sensors are time-matched, avoiding data errors caused by time differences. Furthermore, the collected data is cached to form raw point cloud data and raw image data, providing efficient and reliable raw input for subsequent functional modules.

[0044] In this invention, during the drone's flight within the pipeline area, the lidar continuously scans the surrounding environment, generating 3D point cloud data; the camera captures images of the foreground in real time, acquiring image data of landmarks. The data acquisition module simultaneously receives both types of data, processes them synchronously according to a unified time standard, and stores the processed data in a cache, awaiting access from other modules. For example, at a certain moment, the lidar acquires a set of point cloud data, and the camera captures an image containing a QR code. The data acquisition module marks both sets of data with the same timestamp and then stores them as raw point cloud data and raw image data, respectively.

[0045] Furthermore, the environmental perception and flight path generation module performs perception processing based on the raw point cloud data provided by the data acquisition module to achieve perception of the left and right side walls of the UAV.

[0046] The environmental perception and flight path generation module analyzes and processes point cloud data to extract feature information of the walls, thereby determining the wall location information and the corresponding wall distribution type (such as double-sided walls, single-sided walls, etc.) within the pipeline area. Based on this wall location information and distribution type, it dynamically generates initial flight path information in the UAV's body coordinate system, ensuring that the UAV can fly along a safe and reasonable path.

[0047] In this invention, the environmental perception and flight path generation module first preprocesses the raw point cloud data to remove noise and invalid data, improving data quality. Then, a specific algorithm (such as a line fitting algorithm) is used to process the point cloud data, identifying the characteristic planes of the left and right walls and determining their location information. Next, the wall distribution type is determined based on the wall location information; for example, if walls are detected on both sides, the wall distribution type is double-sided walls; if only one wall is detected, it is single-sided walls. Finally, based on the wall location and distribution type, combined with the UAV's flight parameters (such as fuselage size, safety distance, etc.), the environmental perception and flight path generation module generates initial flight path information, such as the flight path's height, width, and turning radius.

[0048] Meanwhile, the obstacle detection module uses raw point cloud data provided by the data acquisition module to detect and assess potential obstacles (such as foreign objects, construction equipment, etc.) in front of the drone in real time. By analyzing the shape, size, and position of objects in the point cloud data, it determines whether obstacles exist and sends the obstacle detection results to the task management module so that the drone's flight strategy can be adjusted in a timely manner to avoid collisions with obstacles.

[0049] In this invention, the obstacle detection module continuously receives raw point cloud data and processes and analyzes the data in real time. It uses a target detection algorithm (such as a deep learning-based point cloud target detection algorithm) to scan the point cloud data and identify obstacles. If an obstacle is detected, it records information such as the obstacle's position and size, and sends the detection results (such as whether an obstacle exists and its type) to the task management module. For example, during drone flight, if the obstacle detection module detects a large object ahead, it immediately sends this information to the task management module, which then adjusts the drone's flight path or takes obstacle avoidance measures based on this information.

[0050] Furthermore, the identification module detects and identifies various pre-set identification objects (such as QR codes, reflective strips, color markers, etc.) within the pipeline based on the identification object image data (or point cloud data) provided by the data acquisition module. By identifying the identification objects, the module accurately locates the key tasks of the drone within the pipeline, such as determining the specific operations the drone needs to perform at that location (such as hovering, taking pictures, collecting data, etc.), and identifies the flight action control commands of the drone at the current pre-set location, sending the identification results to the task management module.

[0051] In this invention, the identifier recognition module preprocesses the identifier image data, such as image enhancement and noise reduction, to improve image quality. Then, an image recognition algorithm (such as a QR code recognition algorithm or a color recognition algorithm) is used to identify the identifier in the image. If a QR code is identified, it is decoded to obtain the instruction information it contains; if a color mark or reflective strip is identified, the instruction it represents is determined according to preset rules. Finally, the identifier recognition module sends the recognition result (such as a flight control command) to the task management module. For example, if a drone flies to a location marked with a QR code, the identifier recognition module recognizes and decodes the QR code, obtaining the instruction to "hover for 5 seconds," and then sends this instruction to the task management module.

[0052] Furthermore, the task management module undertakes full-process management functions such as workflow scheduling, status monitoring, and anomaly handling for UAV inspection tasks. Based on the initial flight path information provided by the environmental perception and flight path generation module, the obstacle detection results provided by the obstacle detection module, and the flight action control commands provided by the marker recognition module, the task management module comprehensively analyzes the current flight status and environmental conditions of the UAV, flexibly adjusts the task processing flow, and constructs the target flight path information corresponding to the UAV to ensure that the UAV can safely and smoothly complete the inspection task.

[0053] Specifically, after receiving initial flight path information, obstacle detection results, and flight maneuver control commands, the task management module comprehensively analyzes this information. If the obstacle detection results indicate an obstacle ahead, the task management module adjusts the initial flight path information based on the obstacle's position and size to avoid it and generate new target flight path information. If the identifier recognition module sends flight maneuver control commands, the task management module incorporates these commands into the target flight path information to ensure the UAV performs the corresponding actions at specific locations. For example, if the initial flight path information plans for the UAV to fly in a straight line, but the obstacle detection module detects an obstacle ahead, the task management module adjusts the flight path so that the UAV can bypass the obstacle and continue flying. Simultaneously, if the identifier recognition module sends a command to "hover and take a picture at a certain location," the task management module sets the corresponding hovering point and picture-taking command in the target flight path information.

[0054] Finally, based on the target flight path information output by the task management module, the control module generates flight control commands in real time and sends them to the flight control computer to control the UAV's flight. These flight control commands include parameters such as the UAV's flight speed, direction, altitude, and attitude, ensuring that the UAV can fly accurately according to the target flight path information and achieve autonomous inspection tasks.

[0055] In this invention, after receiving the target flight path information from the task management module, the control module converts it into specific flight control commands. For example, based on the flight speed requirement in the target flight path information, it generates corresponding motor speed control commands; based on the flight direction requirement, it generates servo control commands to adjust the UAV's flight attitude. Then, these flight control commands are sent to the flight control computer, which controls the various actuators of the UAV (such as motors and servos) according to the commands to achieve flight control of the UAV. For example, if the target flight path information requires the UAV to fly forward at a speed of 2 m / s, the control module generates corresponding motor speed commands to enable the UAV to reach the specified flight speed.

[0056] Based on the above embodiments, the environment perception and route generation module is specifically used for: Based on the longitudinal axis of symmetry of the UAV, the original point cloud data is divided into point cloud data on the left side of the UAV and point cloud data on the right side of the UAV. Linear fitting operations were performed on the point cloud data on the left and right sides of the UAV respectively to obtain the linear structure information of the pipe wall. The wall position information is determined based on the straight line of the wall corresponding to the linear structure information of the pipe wall; If, based on the linear structure information of the pipe wall, it is determined that there are straight wall lines on both the left and right sides of the pipe area, the initial flight path information is generated based on the midpoint between the straight wall lines on the left and right sides of the pipe area. If, based on the linear structure information of the pipe wall, it is determined that there is a straight wall on one side of the pipe area, the initial flight path information is generated based on the preset safety distance of the UAV and the offset distance between the straight wall on one side of the pipe area.

[0057] In this invention, the environmental perception and flight path generation module is a key component of the UAV autonomous inspection system in a closed pipeline environment. Its core task is to dynamically generate flight paths that adapt to the actual environment by using point cloud data collected in real time by airborne lidar and combining it with intelligent algorithms, without relying on RTK and SLAM positioning. This enables the UAV to plan its flight path autonomously and safely within the closed pipeline, ensuring that the UAV can complete the inspection task along a suitable path.

[0058] Specifically, the environmental perception and flight path generation module divides the raw point cloud data along the longitudinal axis of symmetry of the UAV body, obtaining point cloud data for the left and right sides of the UAV. This is to facilitate subsequent analysis and processing of the point cloud data from both sides, thereby accurately identifying the wall conditions on both sides of the pipeline. Because environmental information from both sides is crucial for flight path planning when the UAV flies inside the pipeline, this division method allows for more targeted acquisition and processing of relevant data.

[0059] During the drone's flight, the lidar continuously collects point cloud data of the surrounding environment. This raw point cloud data contains information from all directions within the pipe. After acquiring this raw point cloud data, the environmental perception and flight path generation module divides the point cloud data into left and right parts, using the drone's longitudinal axis of symmetry as the dividing line. For example, assuming the drone is flying horizontally within the pipe, with its longitudinal axis of symmetry perpendicular to the flight direction, then all point cloud data located to the left of the axis of symmetry is classified as left-side point cloud data, and those to the right are classified as right-side point cloud data.

[0060] Then, the environmental perception and flight path generation module performs line fitting operations on the segmented point cloud data from the left and right sides of the UAV, respectively. The purpose is to extract the linear structural information of the pipe wall from the point cloud data. Since the pipe wall usually has a relatively regular straight line shape, line fitting can more accurately identify the characteristics of the wall, providing a basis for subsequent determination of the wall position and generation of flight paths.

[0061] For the left-side point cloud data, the environmental perception and flight path generation module uses a suitable straight-line fitting algorithm (such as the least squares straight-line fitting algorithm) to process the data. This algorithm finds a straight line that best represents the distribution trend of the left-side point cloud data; this line represents the linear structure information of the left-side wall. Similarly, the same straight-line fitting operation is performed on the right-side point cloud data to obtain the linear structure information of the right-side wall. For example, when processing the left-side point cloud data, the algorithm analyzes the coordinate position of each point, and through calculation and optimization, determines a straight line that minimizes the sum of the distances from all left-side point cloud data points to this line; this straight line represents the linear structure of the left-side wall.

[0062] Furthermore, the environmental perception and flight path generation module determines the specific location information of the pipe wall based on the linear structural information of the pipe wall obtained from line fitting. The wall location information is a crucial basis for generating flight paths; only by accurately knowing the wall's location can the drone's flight path be rationally planned, ensuring the drone's safe flight within the pipe. In this invention, key parameters, such as the slope and intercept of the line, can be extracted from the linear equation obtained from line fitting. Combined with the drone's own position and attitude information (which can be obtained through the drone's own sensors), the specific location of the wall can be calculated. For example, given the linear equations of the left and right walls, combined with the drone's position coordinates in the body coordinate system, geometric calculation methods can be used to determine the specific position information of the left and right walls relative to the drone, such as the distance between the walls and the drone, and the relative position of the walls in the left and right directions of the drone.

[0063] Figure 2 The schematic diagram of the flight path output of the two side walls of the UAV provided by this invention can be referred to. Figure 2 As shown, when it is determined that there are straight walls on both sides of the pipeline area based on the linear structure information of the pipeline walls, initial flight path information can be generated by calculating the midpoint between the two straight walls. This strategy ensures that the UAV flies along the centerline of the pipeline, maintaining a relatively uniform safe distance from the walls on both sides during flight, thus improving flight safety and stability.

[0064] Figure 3 This is a schematic diagram of the flight path output for a single-sided wall of a drone provided by the present invention, which can be referred to. Figure 3 As shown, when a straight wall is determined to exist on one side of the pipeline area based on the linear structure information of the pipeline wall, initial flight path information is generated according to the preset safety distance of the UAV and the offset distance between the straight wall on that side. This strategy is designed to address situations where there is a wall on only one side of the pipeline (such as a railway tunnel turnout or a pipeline branch), ensuring that the UAV can pass safely and smoothly even under these circumstances.

[0065] For example, suppose there is only a straight wall on the left side, and the preset safe distance for the drone is d. First, determine the position of the straight wall. Then, based on the preset safe distance d, offset the straight wall by a distance d towards the centerline of the pipeline to obtain a new straight line. The trajectory formed by the points on this new straight line is the initial flight path information, and the drone will fly along this path. Simultaneously, the environmental perception and flight path generation module will continuously output the generated flight path as trajectory points in the drone's coordinate system, providing the drone's flight control module with real-time navigation targets and dynamically updating the flight path according to changes in the pipeline environment, achieving autonomous and continuous flight inspection.

[0066] Based on the above embodiments, the environment perception and route generation module is further used for: The original point cloud data is projected onto the XY plane of the UAV body coordinate system to obtain the projected planar point cloud data; The projected planar point cloud data is downsampled to obtain downsampled planar point cloud data. Based on the longitudinal axis of symmetry of the UAV, the downsampled planar point cloud data is divided into point cloud data on the left side of the UAV and point cloud data on the right side of the UAV.

[0067] In this invention, for a closed pipe environment, the UAV focuses on the structural condition of the pipe at the current altitude cross section during flight, such as the position and shape of the walls on both sides of the pipe and the distribution of possible obstacles in the pipe on the cross section.

[0068] The environmental perception and flight path generation module projects the original 3D point cloud data onto the XY plane of the UAV's body coordinate system, which can focus the point cloud information that was originally scattered in 3D space onto a 2D plane, and more intuitively display the spatial structure of the pipeline cross-section, thereby simplifying the subsequent analysis and processing of pipeline environmental characteristics.

[0069] Meanwhile, this projection processing helps to remove some redundant information that has little impact on flight path planning in the vertical direction (Z-axis direction), allowing the environmental perception and flight path generation module to focus more on cross-sectional information that is closely related to the UAV's horizontal flight and flight path planning, thereby improving processing efficiency and accuracy.

[0070] Specifically, assuming the drone is flying horizontally inside the pipe, its X-axis points in the direction of its flight, its Y-axis is perpendicular to the direction of flight and lies in the horizontal plane, and its Z-axis is perpendicular to the horizontal plane and pointing upwards. The raw point cloud data contains the coordinate information of each point inside the pipe in three-dimensional space. After receiving this raw point cloud data, the environmental perception and flight path generation module ignores the Z-coordinate value for each point cloud data, retaining only the X-axis and Y-axis coordinates, and projects the point cloud data onto the XY plane to obtain the projected planar point cloud data. This data only contains the position information of the points on the cross-section of the pipe.

[0071] Even after projection, the raw point cloud data, though now on a two-dimensional plane, can still be very large. A large amount of point cloud data complicates subsequent calculations and processing, consumes significant computing resources and time, and reduces the system's real-time performance. Downsampling aims to reduce the amount of point cloud data and improve processing efficiency. Simultaneously, during downsampling, it's crucial to preserve the main spatial distribution characteristics of the wall to ensure that the downsampling point cloud data accurately reflects the structural information of the pipe's cross-section, without affecting subsequent flight path generation and UAV flight safety.

[0072] In this invention, a suitable downsampling algorithm, such as random sampling or voxel grid filtering sampling, is employed. This method represents the originally dense point cloud data with fewer representative points, thereby achieving downsampling, significantly reducing the amount of point cloud data, while preserving the approximate shape and location information of key features such as walls.

[0073] Furthermore, by dividing the downsampled planar point cloud data according to the longitudinal axis of symmetry of the UAV, the position, shape, and other information of the left and right side walls can be studied independently, providing a basis for generating appropriate flight paths based on the wall conditions. For example, when there are walls on both sides of the pipeline, it is necessary to determine the positions of the left and right side walls separately, and then calculate the midpoint as a flight path reference; when there is only a wall on one side, it is also necessary to generate a flight path based on the position information of that side wall.

[0074] In this invention, assuming the longitudinal axis of symmetry of the UAV coincides with the Y-axis (in the body coordinate system), for each point (x, y) in the downsampled planar point cloud data, the sign of its x-coordinate determines whether the point is located on the left or right side of the UAV. If x < 0, the point belongs to the left side of the UAV's point cloud data; if x > 0, the point belongs to the right side; if x = 0, it can be specially processed according to specific needs, such as being classified as left or right, or ignored. In this way, the downsampled planar point cloud data is accurately divided into left and right parts, preparing for subsequent wall detection and flight path generation.

[0075] This invention effectively avoids the risks of yaw and collision caused by unstable 3D positioning and incomplete walls by using point cloud XY plane projection and linear feature recognition, ensuring flight safety and flight path accuracy, and ensuring that the UAV always flies along the center line of the pipeline or a safe passage.

[0076] Based on the above embodiments, the task management module is specifically used for: If, based on the obstacle detection results, it is determined that there are no obstacles ahead of the drone, the initial flight path information is used as the target flight path information. If, based on the obstacle detection results, it is determined that there is an obstacle in front of the UAV, the initial flight path information is adjusted based on a preset obstacle scheduling decision to obtain the target flight path information; If the flight maneuver control command is obtained during the flight of the UAV, the flight maneuver control command is added to the corresponding position in the initial flight path information of the UAV to obtain the target flight path information; The preset obstacle scheduling decision includes: When the distance between the drone and the obstacle meets the preset safe distance, the drone waiting timer process is executed; If the obstacle is removed during the drone waiting timer process, the unfinished initial flight path information will be used as the target flight path information. If the obstacle is not removed during the drone waiting timer process, the drone return information will be used as the target flight path information.

[0077] In this invention, during the drone's flight, the obstacle detection module monitors the environment ahead in real time. When the detection results show no obstacles ahead, it indicates that the drone can fly smoothly as planned without adjusting its flight path. At this point, the mission management module directly uses the initial flight path information provided by the environmental perception and flight path generation module as the target flight path information, allowing the drone to continue flying along this predetermined path to ensure the continuity and efficiency of the flight mission.

[0078] For example, when a drone is conducting an inspection mission inside a straight pipe, the environmental perception and route generation module plans an initial route from the pipe entrance to the exit. The obstacle detection module continuously scans the path ahead during flight. If no obstacles are detected, the mission management module determines that the initial route is safe and feasible, directly setting it as the target route, and the drone continues flying along this route.

[0079] When the obstacle detection module reports an obstacle ahead, the drone cannot fly directly along the initial flight path, otherwise a collision may occur, causing equipment damage or even a safety accident. Therefore, the task management module needs to adjust the initial flight path according to the preset obstacle scheduling decisions and generate a new target flight path to ensure the drone's flight safety.

[0080] Specifically, when the distance between the drone and the obstacle meets the preset safe distance, the drone waiting timer process is executed: assuming the preset safe distance is 5 meters, when the obstacle detection module detects that the distance between the drone and the obstacle is 5 meters, the task management module will instruct the drone to hover at the safe distance in front of the obstacle and start the waiting timer process. For example, the waiting time is set to 30 seconds.

[0081] If the obstacle is removed within the 30-second waiting period, it indicates that the road ahead is clear. At this point, the task management module will assume that the drone can continue flying along the initial route. Therefore, it will use the incomplete initial route information as the target route information and instruct the drone to resume the inspection route and continue moving forward.

[0082] If the obstacle remains within the 30-second waiting period and shows no signs of removal, the task management module will determine that the flight path is blocked and consider that continuing to wait is meaningless. To ensure the safety of the drone, the module will use the drone's return-home information as the target flight path information, control the drone to turn back, and re-perceive and output the return-home trajectory to allow the drone to safely return to the starting point or other designated location.

[0083] In this invention, besides obstacles, the UAV may also need to perform specific flight maneuvers during flight due to other mission requirements, such as taking photos, hovering, uploading data, turning back, and switching missions. These flight maneuver control commands may come from the marker recognition module or other external commands. After obtaining these commands, the mission management module needs to make corresponding adjustments based on the initial flight path information, add flight maneuver control commands, and generate new target flight path information to meet diverse mission requirements.

[0084] For example, during drone flight, the marker recognition module detects a specific marker. According to the pre-set mission strategy, the drone needs to take a picture at the marker's location. After receiving this flight control command (take a picture), the mission management module finds the corresponding location (i.e., the marker's location) in the initial flight path information, adds the picture command at that location, and generates target flight path information. Based on this target flight path information, the flight control module controls the drone to automatically perform the picture-taking action when it reaches the marker's location.

[0085] This invention enhances the intelligent response capabilities of drones in complex environments such as enclosed pipelines, enabling real-time detection and response to dynamic obstacles within the pipeline. This allows the drone to automatically hover and wait, resume flight after obstacles disappear, or intelligently return to base if obstacles persist, significantly improving the safety and autonomy of the operation. Simultaneously, it achieves efficient autonomous multi-tasking, supporting marker-based point-to-point actions such as automatic return, photography, data reporting, and seamless switching between multiple inspection tasks, meeting diverse pipeline inspection and operation needs and improving overall operational efficiency and intelligence. Furthermore, this invention simplifies the system architecture, reducing implementation costs. Relying on point cloud environmental perception and trajectory planning in the body coordinate system, it significantly reduces dependence on complex positioning systems, feature-rich environments, and high-performance hardware, making the system easier to deploy, maintain, and expand.

[0086] Figure 4 This is a flowchart illustrating the multi-sensor-based autonomous inspection method for closed-channel drones provided by the present invention, as shown below. Figure 4 As shown, this invention provides a multi-sensor-based autonomous inspection method for unmanned aerial vehicles (UAVs) in closed passages, comprising: Step 401: During the flight of the UAV, collect three-dimensional point cloud data in the pipeline area.

[0087] In special environments such as enclosed pipes where GNSS signals are unavailable and visual features may be scarce or structures may be highly repetitive, this invention collects three-dimensional point cloud data to provide rich environmental information for subsequent wall perception, UAV flight path generation, and flight control.

[0088] Specifically, in this invention, the three-dimensional point cloud data collected by devices such as lidar can accurately describe the shape, position, and spatial distribution of objects within a pipe, providing reliable data support for the autonomous flight of drones in complex environments. The drone is equipped with a lidar sensor; during flight, the lidar emits laser beams at a certain frequency into the surrounding environment and receives the reflected laser signals.

[0089] Furthermore, based on the time difference between laser emission and reception and the angle information of the laser beam, the distance and orientation between each laser point and the UAV are calculated, thereby obtaining the three-dimensional coordinate information of a large number of points within the pipeline area, which constitute three-dimensional point cloud data.

[0090] Step 402: Based on the three-dimensional point cloud data, perform wall perception on the pipe area passed by the UAV during flight to obtain wall feature information.

[0091] In this invention, the collected 3D point cloud data is projected onto the UAV's XY plane to highlight the spatial distribution characteristics within the pipeline cross-section, simplifying subsequent processing complexity. Then, the projected point cloud data is downsampled and fitted with straight lines along the walls, allowing for rapid identification and fitting of the left and right walls of the pipeline, obtaining wall feature information such as position, shape, and orientation. This wall feature information is crucial for generating safe flight paths, helping the UAV accurately understand the pipeline environment and avoid collisions with the walls.

[0092] Step 403: Based on the wall feature information, construct the flight control commands corresponding to the UAV.

[0093] In this invention, the system can intelligently select the centerline of both walls or a reference point after a safe offset from one wall as the flight path based on the actual detected pipe wall conditions (both sides / one side). This allows for the construction of flight control commands based on wall feature information, enabling intelligent flight path adjustments and ensuring safe flight of the UAV in various spatial changes or structural anomalies. In this invention, the flight control commands guide the UAV to fly along a safe route, avoiding collisions with walls while simultaneously meeting the requirements of the inspection task.

[0094] Specifically, if walls are detected on both sides of the pipeline, the center line of the straight line between the two walls is calculated as the flight path of the drone. Then, control commands are constructed for the drone to fly along the center line, such as controlling the drone's flight direction to always be consistent with the center line direction and maintaining a certain flight altitude.

[0095] If only one side of the wall is detected, a safe offset is made to that side of the wall to obtain a reference point as the flight path. For example, taking the left side wall as an example, according to the preset safe distance d, the left side wall is offset in a straight line by a distance d into the pipeline to obtain a new reference line as the flight path.

[0096] Step 404: Adjust the flight status of the UAV according to the flight control command.

[0097] In this invention, after the flight control commands are constructed, they are translated into actual actions of the UAV to adjust its flight status and ensure safe flight along a predetermined route, such as flight direction, altitude, and speed. Based on these commands, corresponding motor control signals are calculated and issued to adjust the speed of the UAV's motors. For example, to make the UAV turn right, the speed of the two right-side motors is appropriately increased while the speed of the two left-side motors is decreased, thus enabling the UAV to perform a right turn. By continuously adjusting the motor speeds, the UAV's flight attitude and trajectory can be precisely controlled, allowing it to fly safely within the pipeline according to the flight control commands and complete the inspection task within the enclosed pipeline.

[0098] The present invention provides a multi-sensor-based autonomous inspection method for closed passages using unmanned aerial vehicles (UAVs). During the flight of the UAV, three-dimensional point cloud data of the pipeline area is collected by lidar and transmitted to the mission computer. The mission computer then uses wall perception to obtain feature information and constructs flight control commands. Finally, the flight control computer adjusts the UAV's flight status according to the commands. Therefore, it does not rely on high-precision positioning methods such as RTK or SLAM. It can still achieve autonomous navigation and stable flight of UAVs in environments with no GNSS signals, repetitive structures, or scarce visual features, thus expanding the application scenarios and space of UAV inspection systems.

[0099] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, communication interface 502, and memory 503 communicate with each other via the communication bus 504. The processor 501 can call logical instructions in the memory 503 to execute a multi-sensor-based autonomous inspection method for a closed-channel UAV. This method includes: collecting three-dimensional point cloud data within a pipeline area during the UAV's flight; performing wall perception on the pipeline area traversed by the UAV during flight based on the three-dimensional point cloud data to obtain wall feature information; constructing flight control commands corresponding to the UAV based on the wall feature information; and adjusting the UAV's flight state according to the flight control commands.

[0100] Furthermore, the logical instructions in the aforementioned memory 503 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by the computer, the computer can execute the multi-sensor-based autonomous inspection method for closed-channel UAVs provided by the above methods. The method includes: collecting three-dimensional point cloud data in a pipeline area during the flight of the UAV; performing wall perception on the pipeline area passed by the UAV during flight based on the three-dimensional point cloud data to obtain wall feature information; constructing flight control instructions corresponding to the UAV based on the wall feature information; and adjusting the flight state of the UAV according to the flight control instructions.

[0102] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the multi-sensor-based autonomous inspection method for closed-channel drones provided in the above embodiments. The method includes: collecting three-dimensional point cloud data in a pipeline area during the flight of the drone; performing wall perception on the pipeline area traversed by the drone during flight based on the three-dimensional point cloud data to obtain wall feature information; constructing flight control commands corresponding to the drone based on the wall feature information; and adjusting the flight state of the drone according to the flight control commands.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-sensor-based autonomous inspection system for unmanned aerial vehicles (UAVs) in a closed passage, characterized in that, This includes lidar, mission computer, and flight control computer, among which: The lidar is used to collect three-dimensional point cloud data in the pipeline area during the flight of the UAV, and send the three-dimensional point cloud data to the mission computer. The task computer is used to perform wall perception on the pipeline area passed by the UAV during flight based on the three-dimensional point cloud data, obtain wall feature information, and construct flight control commands corresponding to the UAV based on the wall feature information. The flight control computer is used to adjust the flight status of the UAV according to the flight control commands sent by the mission computer.

2. The multi-sensor-based autonomous inspection system for unmanned aerial vehicles in closed passages according to claim 1, characterized in that, The system also includes an image acquisition device for acquiring images of markers in front of the UAV and sending the acquired marker image data to the mission computer; wherein the markers are set at a preset position in the pipeline area to indicate that the UAV executes corresponding flight action control commands at the preset position; The mission computer is also used to determine the flight action control command to be executed by the UAV at the preset position based on the image data of the marker.

3. The multi-sensor-based autonomous inspection system for unmanned aerial vehicles in closed passages according to claim 2, characterized in that, The mission computer includes a data acquisition module, an environmental perception and route generation module, an obstacle detection module, a marker recognition module, a mission management module, and a control module, wherein: The data acquisition module is used to acquire the 3D point cloud data and the marker image data, and to perform time synchronization processing and data caching processing on the 3D point cloud data and the marker image data to obtain the original point cloud data and the original image data. The environmental perception and flight path generation module is used to determine the wall location information and the wall distribution type corresponding to the wall location information in the pipeline area based on the wall feature information obtained by wall perception from the original point cloud data, and generate the initial flight path information corresponding to the UAV based on the wall location information and the wall distribution type. The obstacle detection module is used to detect obstacles in front of the UAV based on the original point cloud data and obtain obstacle detection results. The identifier recognition module is used to identify the flight action control command of the UAV at the current preset position based on the identifier image data; The task management module is used to construct the target flight path information corresponding to the UAV based on the initial flight path information, the obstacle detection results, and the flight action control commands. The control module is used to generate the flight control commands based on the target route information.

4. The multi-sensor-based autonomous inspection system for unmanned aerial vehicles in closed passages according to claim 3, characterized in that, The environmental perception and route generation module is specifically used for: Based on the longitudinal axis of symmetry of the UAV, the original point cloud data is divided into point cloud data on the left side of the UAV and point cloud data on the right side of the UAV. Linear fitting operations were performed on the point cloud data on the left and right sides of the UAV respectively to obtain the linear structure information of the pipe wall. The wall position information is determined based on the straight line of the wall corresponding to the linear structure information of the pipe wall; If, based on the linear structure information of the pipe wall, it is determined that there are straight lines of walls on both the left and right sides of the pipe area, the initial flight path information is generated based on the midpoint between the straight lines of walls on the left and right sides of the pipe area. If, based on the linear structure information of the pipe wall, it is determined that there is a straight wall on one side of the pipe area, the initial flight path information is generated based on the preset safe distance of the UAV and the offset distance between the straight wall on one side of the pipe area.

5. The multi-sensor-based autonomous inspection system for unmanned aerial vehicles in closed passages according to claim 4, characterized in that, The environmental perception and route generation module is also used for: The original point cloud data is projected onto the XY plane of the UAV body coordinate system to obtain the projected planar point cloud data; The projected planar point cloud data is downsampled to obtain downsampled planar point cloud data. Based on the longitudinal axis of symmetry of the UAV, the downsampled planar point cloud data is divided into point cloud data on the left side of the UAV and point cloud data on the right side of the UAV.

6. The multi-sensor-based autonomous inspection system for unmanned aerial vehicles in closed passages according to claim 3, characterized in that, The task management module is specifically used for: If, based on the obstacle detection results, it is determined that there are no obstacles ahead of the drone, the initial flight path information is used as the target flight path information. If, based on the obstacle detection results, it is determined that there is an obstacle in front of the UAV, the initial flight path information is adjusted based on a preset obstacle scheduling decision to obtain the target flight path information; If the flight maneuver control command is obtained during the flight of the UAV, the flight maneuver control command is added to the corresponding position in the initial flight path information of the UAV to obtain the target flight path information; The preset obstacle scheduling decision includes: When the distance between the drone and the obstacle meets the preset safe distance, the drone waiting timer process is executed; If the obstacle is removed during the drone waiting timer process, the unfinished initial flight path information will be used as the target flight path information. If the obstacle is not removed during the drone waiting timer process, the drone return information will be used as the target flight path information.

7. A method for autonomous inspection of closed passages using unmanned aerial vehicles based on multiple sensors, characterized in that, include: During the flight of the drone, three-dimensional point cloud data of the pipeline area is collected; Based on the three-dimensional point cloud data, wall perception is performed on the pipeline area passed by the UAV during flight to obtain wall feature information; Based on the wall feature information, the flight control commands corresponding to the UAV are constructed; The flight status of the UAV is adjusted according to the flight control commands.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the autonomous inspection method for unmanned aerial vehicles in closed passages based on multiple sensors as described in claim 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the autonomous inspection method for unmanned aerial vehicles in closed passages based on multiple sensors as described in claim 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the autonomous inspection method for unmanned aerial vehicles in closed passages based on multiple sensors as described in claim 7.

Images