Unmanned aerial vehicle-based air-ground cooperative off-road scene three-dimensional terrain detection method and system

Abstract

This invention discloses a method and system for 3D terrain detection in off-road scenarios based on UAV (Unmanned Aerial Vehicle) collaborative air-ground approach. The method includes: a vehicle-mounted terminal planning the target driving area and issuing terrain detection commands to the UAV; the UAV collecting obstacle information and multi-source terrain data within the area, fusing and processing these data to generate and transmit a 3D sparse terrain model; and after detection, deep fusion data generating and transmitting a high-precision 3D terrain model. The vehicle-mounted terminal integrates the two models to form hierarchical terrain data, performs visual segmentation on the high-precision model, extracts terrain feature parameters, determines risk levels, and identifies and visualizes terrain risks and passable areas. This method utilizes the UAV's aerial perspective to eliminate blind spots in vehicle-mounted perception, accurately captures 3D information of off-road terrain, improves the comprehensiveness and accuracy of detection, provides a reliable basis for driving decisions, and effectively avoids safety risks during off-road driving.

Description

Technical Field

[0001] This invention belongs to the field of terrain detection technology, specifically relating to a method and system for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Terrain detection in off-road scenarios is a crucial link in ensuring vehicle traffic safety, but it still faces two major technological challenges in practical applications: First, traditional vehicle-mounted sensors are limited by installation location and detection angle, resulting in significant field of view limitations and blind spots in complex off-road environments, making it difficult to achieve full-range terrain perception; Second, off-road terrain has strong three-dimensional features, with undulating terrain and varied landforms, making it difficult for existing detection methods to accurately capture multi-dimensional terrain information, which can easily lead to misjudgments of road conditions and thus cause driving safety risks. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for three-dimensional terrain detection in off-road scenarios based on air-ground cooperation using unmanned aerial vehicles (UAVs), in order to solve the technical defects of traditional vehicle-mounted sensors in off-road terrain detection, such as limited field of view and blind spots, and the difficulty of existing detection methods in accurately capturing three-dimensional multi-dimensional information of off-road terrain, which can easily lead to misjudgment of road conditions.

[0004] To achieve the above objectives, this application provides the following technical solution: The first aspect of this application provides a method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs), including: The vehicle-mounted system acquires planning information of the target area where the vehicle intends to drive in an off-road scenario, and generates terrain detection commands based on the planning information and sends them to the drone. The UAV receives and executes the terrain detection command, collects distance and orientation information of obstacles in the target area to be driven and multi-source terrain data, and generates a three-dimensional sparse terrain model after fusing the multi-source terrain data. The drone will transmit the collected 3D sparse terrain model, obstacle distance and orientation information back to the vehicle terminal in real time, and the vehicle terminal will display it visually. After the UAV completes the execution of the terrain detection command, it performs deep fusion processing on the multi-source terrain data collected throughout the process to generate a high-precision three-dimensional terrain model, and then transmits the high-precision three-dimensional terrain model back to the vehicle terminal. The vehicle-mounted system integrates high-precision 3D terrain models with 3D sparse terrain models to form a hierarchical output of terrain data for the target driving area. The vehicle-mounted system performs visual segmentation processing on the high-precision 3D terrain model in the hierarchical output results, extracts terrain feature parameters, completes terrain risk level determination, identifies terrain risk areas and passable areas within the target driving area, and displays them visually.

[0005] Furthermore, the multi-source terrain data is acquired simultaneously by the visual camera module and the lidar module carried by the UAV; The multi-source terrain data includes visual image data and three-dimensional point cloud data; The visual image data is used to analyze the terrain texture and color features of the intended driving target area; The three-dimensional point cloud data is used to calculate the terrain undulation, road slope, and geometric feature parameters of obstacles in the target area to be driven.

[0006] Furthermore, the terrain feature parameters include the road surface slope, obstacle height, and road width of the target area to be driven. After the vehicle terminal extracts the terrain feature parameters from the high-precision three-dimensional terrain model, it combines the built-in vehicle passability parameters to determine the terrain risk level. The vehicle passability parameters include vehicle length, width, height, minimum ground clearance, departure angle, and approach angle.

[0007] Furthermore, when the recognition results are visualized, interactive operations are performed on the in-vehicle infotainment system. These interactive operations include: It allows for view adjustment operations such as rotation, scaling, and translation of the 3D terrain model, as well as real-time quantitative measurement operations of the area of ​​the target driving area, the straight-line distance between two points, and the road surface slope.

[0008] Furthermore, the information transmission between the drone and the vehicle is achieved through a multi-link redundant communication module. The communication module uses a radio communication RC link as the main transmission link and a combination of 4G and 5G cellular communication and access point AP links as auxiliary redundant links. The main transmission link is used to transmit real-time flight control commands and flight status information of the UAV, while the auxiliary redundant link is used for terrain data backhaul.

[0009] Furthermore, the process of collecting distance and orientation information of obstacles within the target driving area, as well as multi-source terrain data, includes: A preset flight control strategy is executed based on the off-road scene characteristics of the intended driving target area, where the off-road scene characteristics correspond to sandy, mountainous, and Gobi off-road scenarios. When the UAV executes the preset flight control strategy, it dynamically adjusts its flight altitude according to the terrain undulations of the target area to achieve adaptive adjustment of flight altitude, and completes environmental perception and precise obstacle avoidance throughout the flight through the onboard lidar module.

[0010] A second aspect of this application provides a UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios, comprising: The drone terminal, the vehicle-mounted terminal, and the communication module are provided, wherein the drone terminal and the vehicle-mounted terminal establish a two-way communication connection through the communication module. The drone terminal is used to receive terrain detection commands from the vehicle terminal, collect obstacle information and multi-source terrain data in the target area to be driven, and generate a three-dimensional sparse terrain model and a high-precision three-dimensional terrain model after data fusion processing, and then transmit the model and obstacle information back to the vehicle terminal. The vehicle-mounted terminal is used to plan the target area for driving and generate terrain detection commands to be sent to the drone terminal. It receives the model and data returned by the drone terminal and completes the visualization display. It integrates the terrain model to form a hierarchical output result, and at the same time identifies terrain risk areas and passable areas. The communication module adopts a multi-link redundancy design to ensure the real-time performance and stability of the transmission of detection commands, terrain data, and 3D terrain models between the UAV and the vehicle-mounted terminal.

[0011] Furthermore, the UAV terminal is equipped with a visual camera module, a lidar module, an edge computing module, and a data fusion module, all of which are electrically connected to the UAV terminal. The visual camera module and the lidar module are used to synchronously collect multi-source terrain data of the target area to be driven. The edge computing module is used to fuse multi-source terrain data to generate a 3D sparse terrain model and a high-precision 3D terrain model. The data fusion module is used to spatially align and weightedly fuse image data and point cloud data based on a multi-source data fusion algorithm, and output a high-precision three-dimensional terrain model in a unified coordinate system.

[0012] Furthermore, the vehicle-mounted terminal is equipped with a task planning module, a data receiving module, a model integration module, a visual segmentation module, and a visualization interaction module; The task planning module is used to plan the target area for driving and generate terrain detection instructions; The data receiving module is used to receive the model and data transmitted back from the UAV. The model integration module is used to integrate terrain models to form hierarchical output results; The visual segmentation module is used to identify terrain-risk areas and passable areas; The visualization and interaction module is used to visualize the data and recognition results and support driver interaction.

[0013] Furthermore, the communication module is configured with a main transmission link unit and an auxiliary redundant link unit; The main transmission link unit is a radio communication RC link, used to transmit UAV flight control commands and status information; The auxiliary redundant link unit includes 4G and 5G cellular communication links and access point (AP) links, which are used to work with the main transmission link unit to realize the real-time transmission of detection commands, terrain data and three-dimensional terrain models.

[0014] Compared with the prior art, the present invention has the following beneficial effects: By leveraging the aerial perspective of drones, this technology overcomes the installation and angle limitations of traditional vehicle-mounted sensors, eliminating field-of-view limitations and blind spots, and enabling full-range terrain perception of the target area. Relying on drones to complete multi-source terrain data collection and fusion modeling, the vehicle-mounted system performs visual segmentation and risk assessment, accurately capturing three-dimensional multi-dimensional information of off-road terrain, avoiding road condition misjudgments, and transmitting three-dimensional sparse terrain models in real time. After detection, it can also generate high-precision models, integrating them into a hierarchical data result and visually displaying risks and passable areas. It accurately extracts terrain features and determines risk levels, significantly improving the comprehensiveness and accuracy of off-road terrain detection, providing a reliable basis for driving decisions, and effectively avoiding driving safety risks. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart of a UAV-based air-ground cooperative three-dimensional terrain detection method for off-road scenarios is provided by the present invention; Figure 2 A schematic diagram of a UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios is provided for this invention. Figure 3 This invention provides an operational schematic diagram of a UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios. In the image: 1. Vehicle; 2. Drone terminal; 3. Target area to be traveled. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0018] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0020] The present invention will now be described in further detail with reference to the accompanying drawings: like Figure 1 As shown, in a first aspect of the present invention, a method for three-dimensional terrain detection in an air-ground cooperative off-road scenario based on unmanned aerial vehicles (UAVs) is provided, comprising: S101. The vehicle terminal obtains the planning information of the target area where the vehicle intends to drive in the off-road scenario, and generates a terrain detection command based on the planning information and sends it to the drone. For example, the driver can select the target area and plan the scanning task through the vehicle's APP. The scanning path of the drone can be customized according to the needs of off-road driving. The vehicle system automatically generates the optimal flight path of the drone based on the planned target area range and scanning path, and generates terrain detection instructions containing the target area range and scanning path. The instructions are sent to the drone through a multi-link redundant communication module via a radio communication RC main transmission link to ensure low latency and high reliability of instruction transmission.

[0021] After receiving the mission, the drone takes off autonomously and flies steadily to the target area along the preset route without human intervention. During the flight, it relies on the onboard LiDAR to achieve fully autonomous obstacle avoidance. It can also quickly plan obstacle avoidance and detour paths based on the real-time detection data of the LiDAR to avoid collisions with obstacles and ensure the progress of the scanning mission. At the same time, the LiDAR continuously detects various obstacles in the target area and transmits accurate information such as the real-time relative distance between the obstacle and the drone and the obstacle's azimuth coordinates back to the vehicle's infotainment system. The vehicle's infotainment system displays this information in the form of icons, values, and directional markers on the on-device visual interface, presenting the driver with obstacle warnings for the road conditions ahead and allowing the driver to understand the distribution of obstacles in the target area in advance.

[0022] S102. The UAV receives and executes the terrain detection command, collects the distance and orientation information of obstacles in the target area to be driven and multi-source terrain data, and generates a three-dimensional sparse terrain model after fusing the multi-source terrain data. For example, after receiving the terrain detection command, the UAV first matches and executes the preset flight control strategy according to the off-road scene characteristics such as sand, forest, and Gobi corresponding to the target area. At the same time, it dynamically adjusts the flight altitude according to the terrain undulation characteristics in the area to achieve adaptive adjustment of the flight altitude. It also completes environmental perception and precise obstacle avoidance throughout the flight through the onboard lidar module.

[0023] After the drone arrives at the target area, the lidar enters high-speed acquisition mode, working synchronously with the visual camera module to complete the synchronous and comprehensive acquisition of multi-source terrain data and obstacle information. The lidar generates hundreds of thousands of point cloud data per second, collecting 3D point cloud data for calculating terrain undulation, road slope, and obstacle geometric feature parameters. At the same time, it detects and collects the distance and orientation information of obstacles in the target area in real time. The visual camera module collects visual image data for analyzing terrain texture and color features. The two work together to ensure the integrity and consistency of terrain data.

[0024] The drone uses a built-in edge computing module and a high-performance processing chip to perform local edge processing on the collected visual image data and 3D point cloud data. This enables rapid data parsing, preprocessing, and fusion modeling. By combining information such as terrain undulations, textures, and colors, the drone can quickly construct and generate a 3D sparse terrain model of the target area that clearly presents the basic outline and main features of the terrain.

[0025] S103. The UAV will transmit the collected three-dimensional sparse terrain model, obstacle distance and orientation information back to the vehicle terminal in real time, and the vehicle terminal will display it visually. For example, the drone transmits the generated 3D sparse terrain model and the collected obstacle distance and orientation information back to the vehicle terminal in real time through a multi-link redundant communication module. The terrain data transmission is completed by relying on 4G / 5G cellular communication and access point AP auxiliary redundant links, ensuring the stability and efficiency of the transmission.

[0026] After receiving the information, the vehicle-mounted system completes the fusion processing, accurately marking the distance and orientation information of obstacles on the corresponding positions of the 3D sparse terrain model, realizing an integrated visualization display. The driver can rotate, zoom, and pan the model on the vehicle-mounted system to intuitively and comprehensively view the basic terrain outline, obstacle distribution, and relative positions of the target area, and understand the basic terrain conditions of the area in advance.

[0027] After completing the terrain scanning task of the target area, the drone can flexibly select the flight mode according to the instructions issued by the vehicle. It supports autonomous return mode and can also start synchronous flight mode with the vehicle at a set altitude. It also supports one-click autonomous return and precise landing operation throughout the process, improving the convenience of operation and flight safety.

[0028] During autonomous return or synchronous flight following a vehicle, the UAV uses its built-in data fusion and edge computing modules to perform deep fusion processing on the multi-source terrain data collected in the early stages. Based on the multi-source data fusion algorithm, it spatially aligns and weights the 3D point cloud data and high-resolution texture image data, eliminating spatial biases and achieving complementary data advantages. This results in a high-precision 3D terrain model in a unified coordinate system, accurately reproducing all terrain details of the target area. Simultaneously, the UAV can intelligently adapt its modeling mode according to ambient lighting conditions. During sufficient daylight, it combines point cloud and texture image data to construct a full-color 3D terrain model. At night, when lighting is insufficient, it automatically switches to a laser point cloud 3D terrain modeling mode, relying on point cloud data to ensure modeling accuracy and guaranteeing high-precision terrain modeling under different lighting conditions.

[0029] S104. After the UAV completes the execution of the terrain detection command, it performs deep fusion processing on the multi-source terrain data collected throughout the process to generate a high-precision three-dimensional terrain model, and then transmits the high-precision three-dimensional terrain model back to the vehicle terminal. For example, after the UAV completes the scanning task corresponding to the terrain detection command, it uses the built-in data fusion module and edge computing module to perform deep fusion processing on the visual image data and 3D point cloud data collected throughout the process. Based on the multi-source data fusion algorithm, the two types of data are spatially aligned and weighted to eliminate spatial deviations between data, achieve complementary advantages of data, and output a high-precision 3D terrain model in a unified coordinate system. This accurately restores all terrain details such as terrain undulations, precise size and position of obstacles in the target area to be driven. At the same time, the UAV can intelligently adapt the modeling mode according to the ambient lighting conditions. When there is sufficient daylight, it builds a full-color 3D terrain model, and when there is insufficient light at night, it automatically switches to the laser point cloud 3D terrain modeling mode to ensure the modeling accuracy under different lighting conditions.

[0030] Subsequently, the UAV transmits the generated high-precision 3D terrain model back to the vehicle-mounted terminal via a multi-link redundant communication relay module. This communication relay module adopts a multi-channel redundancy design with a radio link as the main link and 4G and 5G cellular communication and mobile hotspot links as backup links. The vehicle-mounted terminal uses a 2.4GHz unlicensed frequency band RC radio link as the main link, dedicated to transmitting real-time flight control commands and flight status information of the UAV, ensuring ultra-low latency and accurate transmission. At the same time, the 4G+5G cellular communication link and the AP mobile hotspot link created by the vehicle-mounted terminal are used as auxiliary redundant links, dedicated to the transmission of large-capacity terrain data and models. While maintaining a stable connection on the RC main link throughout the flight, the UAV automatically connects to the AP mobile hotspot created by the vehicle-mounted terminal, constructing a multi-channel parallel transmission architecture. Relying on this architecture, the high-precision 3D terrain model is transmitted back, which not only greatly improves the data transmission rate, but also effectively avoids the data transmission interruption problem caused by the failure of a single link, ensuring the stability and efficiency of terrain model data transmission in all aspects. After the vehicle's infotainment system receives the high-precision 3D terrain model completely through the communication relay module, it integrates it with the previously transmitted 3D sparse model at the spatial and data levels. It performs precise fusion and calibration of the terrain feature information of the two types of models, organically combining the basic outline information of the 3D sparse model with the fine feature information of the high-precision 3D terrain model to form a hierarchical output of terrain data of the target driving area from basic outline to fine features. This enables the hierarchical display and flexible access of terrain information. Drivers can choose to view terrain models of different levels of detail according to their needs, balancing the convenience and accuracy of terrain viewing.

[0031] S105. The vehicle-mounted system integrates the high-precision 3D terrain model with the 3D sparse terrain model to form a hierarchical output result of the terrain data of the target area to be driven. For example, after the vehicle-mounted system receives the high-precision 3D terrain model transmitted back by the drone through the communication module, it performs spatial and data-level collaborative integration with the previously received 3D sparse terrain model. The terrain feature information of the two types of models is accurately fused and calibrated. The basic terrain outline information presented by the 3D sparse terrain model is organically combined with the fine terrain feature information contained in the high-precision 3D terrain model to form a hierarchical output result of terrain data from basic outline to fine features of the target area to be driven. This enables the hierarchical display and flexible access of terrain information. The driver can choose to view terrain models of different levels of detail according to the needs of off-road driving.

[0032] After the vehicle's infotainment system retrieves the hierarchical terrain data, it uses the built-in image processing module to perform visual segmentation on the high-precision 3D terrain model. Based on the preset terrain feature recognition algorithm, it accurately identifies and quantitatively calibrates key terrain feature parameters such as road slope, obstacle height, and road width. Combined with the off-road vehicle passability threshold, it automatically distinguishes between terrain risk areas such as steep slopes and deep pits and flat, passable areas. It also performs differentiated color marking and clear boundary delineation for the two types of areas, achieving an intuitive distinction between risky and passable areas.

[0033] After visual segmentation is completed, the vehicle-mounted system allows drivers to interactively measure terrain parameters such as the area of ​​the target region, the straight-line distance between two points, and the road slope. At the same time, the display and interaction module displays a high-precision 3D terrain model with high definition, accurately presenting the location, range, and risk type of risk areas, as well as the direction and width of passable areas. Drivers can rotate, zoom, and pan the model to view the terrain from any angle, and can also interactively measure terrain parameters by selecting points and boxes to quickly obtain key data.

[0034] S106. The vehicle-mounted unit performs visual segmentation processing on the high-precision three-dimensional terrain model in the hierarchical output result, extracts terrain feature parameters, completes terrain risk level determination, identifies terrain risk areas and passable areas within the target driving area, and displays them visually.

[0035] For example, after the vehicle-mounted system retrieves the hierarchical output results, it performs visual segmentation processing on the high-precision 3D terrain model based on the built-in image processing module. Through the preset terrain feature recognition algorithm, it accurately extracts core terrain feature parameters such as road slope, obstacle height, and road width. Combined with the vehicle's built-in vehicle length, width, height, minimum ground clearance, departure angle, and passability parameters such as departure angle and approach angle, it performs a comprehensive analysis of the terrain and completes the risk level determination. It automatically identifies terrain risk areas such as steep slopes, deep pits, and large obstacles, as well as flat and open passable areas.

[0036] The vehicle-mounted system uses differentiated color markings and clear boundary delineation for the two types of areas, and completes high-definition visualization in the display and interaction module. During the display, the driver can perform interactive operations, such as rotating, scaling, and panning the marked 3D terrain model. The driver can also perform real-time quantitative measurements of the area of ​​the target driving area, the straight-line distance between two points, and the road slope by selecting points and boxes. This allows the driver to intuitively grasp the location and range of risk areas and the direction of passable areas, providing accurate terrain references for planning off-road driving routes.

[0037] This method achieves accurate full-area terrain detection of the target driving area through air-to-ground collaboration between vehicles and drones, overcoming the limitations of traditional vehicle-mounted sensing devices in terms of field of view and blind spots. Relying on multi-source data acquisition and fusion modeling technology of LiDAR and visual cameras, it accurately captures the three-dimensional terrain features of off-road scenarios, solving the problem of low detection accuracy in traditional technologies. A multi-link redundant communication architecture ensures the stable and reliable transmission of terrain data and models. The hierarchical output and visual segmentation processing of terrain data provide drivers with comprehensive, intuitive, and interactive terrain information, helping to scientifically plan driving routes, effectively reducing the probability of accidents such as getting stuck and overturning, and significantly improving the safety, intelligence level, and traffic efficiency of off-road driving.

[0038] In a second aspect, this invention provides a three-dimensional terrain detection system for air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs), such as... Figure 2 As shown, it consists of a drone terminal 2, a vehicle-mounted terminal, and a multi-link redundant communication module. The three are electrically connected and achieve bidirectional transmission of data and commands through wireless communication. The vehicle-mounted terminal serves as the core of system control and interaction, the drone terminal 2 serves as the core of aerial perception, and the multi-link redundant communication module serves as the communication relay unit between the two. The three work together to break through the field of view limitations of traditional vehicle-mounted perception devices, accurately capture the three-dimensional terrain features of off-road scenarios, and achieve full-area, high-precision terrain detection of the target area 3. This provides comprehensive, accurate, and real-time terrain information support for the off-road vehicle 1, effectively avoids driving risks, and improves the safety, intelligence level, and traffic efficiency of off-road driving.

[0039] The UAV terminal 2 serves as the core of the system's aerial perception, equipped with a main control unit that includes a visual camera module, a lidar module, an edge computing module, a data fusion module, and an integrated flight control module. Each module is electrically connected to the main control unit, undertaking tasks such as obstacle detection, multi-source terrain data acquisition, 3D model construction, and data transmission within the target area 3. It supports flexible flight modes including autonomous return and synchronous flight with vehicle 1, enabling one-click autonomous return and precise landing throughout the entire process. The visual camera module, upon reaching the target area 3, activates synchronously with the lidar module, continuously acquiring high-resolution texture image data of the target area 3. This provides texture and color support for the 3D model construction, ensuring the terrain model accurately reflects the actual terrain features.

[0040] The lidar module has dual functions of high-speed point cloud acquisition and full-process autonomous obstacle avoidance. It can generate hundreds of thousands of point cloud data per second, accurately construct the terrain geometry of the target area, and restore the three-dimensional geometric features such as terrain undulation, slope, and obstacle size. The acquired three-dimensional point cloud data and image data complement each other to form multi-source terrain data. At the same time, it detects obstacles in real time throughout the flight, plans obstacle avoidance and detour paths for the UAV, and transmits information such as obstacle distance and azimuth coordinates back to the vehicle terminal in real time.

[0041] The edge computing module rapidly analyzes, preprocesses, and lightweight models the collected multi-source terrain data, quickly generating a 3D sparse model that clearly presents the basic outline and main features of the terrain in the target area 3. This model has a small data volume, fast transmission speed, and can be transmitted back to the vehicle terminal in real time to achieve real-time terrain information perception. The data fusion module performs deep fusion processing on all multi-source terrain data after the UAV 2 completes the scanning task. First, it eliminates spatial deviations through a spatial alignment algorithm, and then uses a weighted fusion algorithm to achieve complementary data advantages, outputting a high-precision 3D terrain model in a unified coordinate system that accurately restores all terrain details of the target area 3. Simultaneously, it can intelligently adapt the modeling mode according to lighting conditions, constructing a full-color 3D terrain model during the day and switching to a laser point cloud modeling mode at night to ensure modeling accuracy under different lighting conditions.

[0042] After the UAV terminal 2 completes the scan, it can flexibly switch flight modes according to the instructions of the vehicle terminal. In the autonomous return mode, the UAV terminal 2 returns to the vicinity of the vehicle 1 along the optimal path and lands accurately. In the follow flight mode, the UAV terminal 2 flies synchronously with the vehicle 1 at the set altitude and continuously detects the terrain of the target area 3 ahead to meet the needs of off-road driving.

[0043] The vehicle-mounted terminal is installed on the off-road vehicle 1 and serves as the core of the system's control, processing, and interaction. It is equipped with a task planning module, a data receiving module, a model integration module, a visual segmentation module, a visualization interaction module, and a main control unit. Each module is electrically connected to the main control unit, allowing the driver to issue all operation commands through the interactive interface, thereby achieving automation and intelligence throughout the entire terrain detection process.

[0044] The task planning module allows drivers to customize the target area 3 and the scanning path 2 of the drone in the vehicle's APP. The module automatically generates the optimal flight path for the drone based on the planning content and issues terrain scanning tasks through a multi-link redundant communication module. It also supports one-click take-off commands to achieve rapid task planning and issuance.

[0045] The data receiving module can simultaneously receive obstacle information, multi-source terrain data, three-dimensional sparse model and high-precision three-dimensional terrain model of the target area to be driven transmitted from different communication links, ensuring the integrity and real-time performance of data reception. The data is synchronously transmitted to the image processing module for processing, and the highly real-time obstacle information is transmitted to the display and interaction module for real-time display.

[0046] The model integration module has two main functions: First, it integrates and precisely calibrates the 3D sparse model with the high-precision 3D terrain model at the spatial and data levels, forming a hierarchical output of terrain data for the target driving area 3, from basic outline to fine features, enabling layered display and flexible access to terrain information. Second, based on a preset terrain feature recognition algorithm, it accurately identifies and quantitatively calibrates the core parameters of the high-precision 3D terrain model, such as road slope, obstacle height, and road width. Combined with the vehicle's passability parameters (body length / width / height, minimum ground clearance, departure angle, and passing angle), it determines the terrain risk level, automatically distinguishes between risky areas such as steep slopes and deep pits and flat, open, passable areas, and performs differentiated feature calibration for the two types of areas.

[0047] The display and interaction module provides drivers with visualization and interactive operation functions. First, it provides high-definition visualization of obstacle information, hierarchical terrain data, and labeled high-precision 3D terrain models of the target area. Obstacle information is displayed using icons and numerical values, and it supports free switching between sparse and high-precision 3D models. Differentiated color markings and clear boundary delineation are used for risky and passable areas to intuitively present core terrain information. Second, it allows drivers to rotate, scale, and translate the 3D model of the target area, enabling comprehensive perception of terrain information. Third, it supports real-time quantitative measurement of the area of ​​the target area, the straight-line distance between two points, and the road slope through point selection and box selection, quickly obtaining key terrain parameters.

[0048] The multi-link redundant communication module adopts a multi-channel redundant design, and is equipped with RC radio links, 4G+5G cellular communication links, and AP mobile hotspot links. It adopts a collaborative working mode of dedicated control of the main link and transmission of auxiliary links to ensure the real-time performance, stability and reliability of command and data transmission in complex off-road environments, and avoid command interruption or data loss caused by single link failure.

[0049] The RC radio link (main link) operates in the 2.4GHz unlicensed frequency band and adopts frequency hopping spread spectrum technology. It has strong anti-co-channel interference capability and is dedicated to transmitting real-time flight control commands (take-off, landing, heading adjustment, etc.) and flight status information (flight attitude, battery level, etc.) from the UAV terminal 2. It achieves millisecond-level ultra-low latency transmission, enabling precise and real-time control of the UAV terminal 2 from the vehicle terminal.

[0050] The 4G+5G cellular communication link (auxiliary redundant link) is built on the public mobile network and features high bandwidth and high transmission rate. It is specifically designed to undertake the high-capacity transmission of multi-source terrain data and three-dimensional terrain models of the target area, thereby relieving the pressure on the main link and improving the overall data transmission efficiency.

[0051] The AP mobile hotspot link (auxiliary redundant link) is a local mobile hotspot created by the vehicle-mounted terminal. The UAV terminal 2 automatically connects after takeoff. It is a near-field communication link. When the distance between the vehicle 1 and the UAV terminal 2 is close, the transmission rate is fast and the signal is stable. It complements the cellular communication link and further improves the stability of terrain data transmission in the target area 3.

[0052] The UAV maintains a stable RC radio link throughout the flight and automatically connects to the AP mobile hotspot, forming a dual-auxiliary redundant data transmission channel with the 4G+5G cellular communication link. This enables parallel transmission of terrain data and models of the target area. Even if one auxiliary link is interrupted, the other can still complete the data transmission, providing communication assurance for the stable operation of the system.

[0053] The driver plans the target area 3 and customizes the scanning path through the vehicle-mounted display and interaction module. The task planning module generates the optimal flight route and sends the terrain scanning task to the UAV terminal 2 through the RC radio link. The driver can issue a one-click take-off command.

[0054] After receiving the command, the UAV terminal 2 takes off autonomously and flies to the target area 3 along the preset route. During the flight, the lidar module autonomously avoids obstacles throughout the process. At the same time, it transmits the distance and orientation information of obstacles in the target area 3 back to the vehicle terminal through the auxiliary link, and the vehicle terminal displays the information in real time.

[0055] After the drone 2 flies to the target area 3, the visual camera and lidar module collect multi-source terrain data synchronously and without blind spots. The edge computing module processes the data in real time and generates a three-dimensional sparse model, which is transmitted back to the vehicle terminal through an auxiliary link. The vehicle terminal integrates the model with the obstacle information of the target area 3 and displays it, allowing the driver to intuitively view the basic outline of the terrain.

[0056] After the UAV terminal 2 completes the scanning of the target area 3, it switches the flight mode according to the instructions of the vehicle terminal. During the flight, the data fusion module performs deep fusion processing on all data to generate a high-precision three-dimensional terrain model. The modeling mode is adapted according to the lighting conditions, and the model is transmitted back to the vehicle terminal through dual auxiliary redundant links.

[0057] After receiving the high-precision model, the vehicle-mounted data receiving module transmits it to the image processing module. The module integrates it with the 3D sparse model to form 3-level terrain data of the target driving area. At the same time, it performs visual segmentation processing on the high-precision model to identify and mark risk / passable areas.

[0058] The vehicle-mounted display and interaction module will display all terrain information of the target driving area 3 in high definition, and differentiate the risk / passable areas. The driver can adjust the view of the model, measure the terrain parameters, and scientifically plan the off-road driving route of vehicle 1 in combination with the terrain information to effectively avoid terrain risks.

[0059] The driver can issue commands as needed, and the drone terminal 2 can return to home autonomously with one click and land accurately, or activate the follow flight mode to continuously detect the terrain of the target area 3 ahead, providing continuous terrain information support for the vehicle 1 to drive off-road.

[0060] This system effectively addresses the core technical shortcomings of traditional off-road terrain detection technologies, such as limited field of view, neglect of 3D terrain features, low detection accuracy, and poor communication stability, through an air-ground collaborative perception architecture, multi-source data fusion processing, multi-link redundant communication design, and intelligent human-computer interaction. Compared to traditional technologies, the UAV-based aerial perception architecture overcomes the ground-based field of view limitations of traditional vehicle-mounted perception devices, achieving full-area, blind-spot-free terrain detection of the target area, completely eliminating blind spots. Multi-source sensor data acquisition combined with refined data fusion processing accurately captures multi-dimensional changes in the target area, restoring all detailed features of the 3D terrain, solving the problems of simplified terrain and low detection accuracy in traditional technologies. The multi-link redundant communication design, combined with frequency hopping spread spectrum... Featuring high-frequency technology and a multi-channel parallel transmission architecture, this system boasts strong anti-interference and fault tolerance capabilities, making it suitable for various complex off-road communication environments. From mission planning to UAV-side detection and modeling, and then to vehicle-side processing and display, the entire process is automated, requiring only simple commands from the driver, significantly improving the efficiency and convenience of terrain detection. The system provides drivers with intuitive, comprehensive, and interactive terrain information through three-tiered terrain data output for the target area, differentiated color labeling combined with model view adjustment, and interactive terrain parameter measurement, enhancing the scientific and rational nature of path planning. Only the sensor acquisition parameters and terrain feature recognition thresholds need to be adjusted to adapt to various off-road scenarios such as forests, snowfields, and deserts, without requiring any changes to the overall system architecture and core modules, demonstrating excellent adaptability and scalability.

[0061] In summary, this system significantly improves the comprehensiveness, accuracy, and reliability of terrain detection in off-road scenarios, effectively reduces the probability of accidents such as vehicles getting stuck or overturning, and significantly enhances the safety, intelligence, and traffic efficiency of off-road driving. It has high practical application and promotion value.

[0062] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs), characterized in that, include: The vehicle-mounted system acquires planning information of the target area where the vehicle intends to drive in an off-road scenario, and generates terrain detection commands based on the planning information and sends them to the drone. The UAV receives and executes the terrain detection command, collects distance and orientation information of obstacles in the target area to be driven and multi-source terrain data, and generates a three-dimensional sparse terrain model after fusing the multi-source terrain data. The drone will transmit the collected 3D sparse terrain model, obstacle distance and orientation information back to the vehicle terminal in real time, and the vehicle terminal will display it visually. After the UAV completes the execution of the terrain detection command, it performs deep fusion processing on the multi-source terrain data collected throughout the process to generate a high-precision three-dimensional terrain model, and then transmits the high-precision three-dimensional terrain model back to the vehicle terminal. The vehicle-mounted system integrates high-precision 3D terrain models with 3D sparse terrain models to form a hierarchical output of terrain data for the target driving area. The vehicle-mounted system performs visual segmentation processing on the high-precision 3D terrain model in the hierarchical output results, extracts terrain feature parameters, completes terrain risk level determination, identifies terrain risk areas and passable areas within the target driving area, and displays them visually.

2. The method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The multi-source terrain data is acquired synchronously by the visual camera module and the lidar module carried by the UAV; The multi-source terrain data includes visual image data and three-dimensional point cloud data; The visual image data is used to analyze the terrain texture and color features of the intended driving target area; The three-dimensional point cloud data is used to calculate the terrain undulation, road slope, and geometric feature parameters of obstacles in the target area to be driven.

3. The method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The terrain feature parameters include the road slope, obstacle height and road width of the target area to be driven. After the vehicle terminal extracts the terrain feature parameters from the high-precision three-dimensional terrain model, it combines the built-in vehicle passability parameters to determine the terrain risk level. The vehicle passability parameters include vehicle length, width, height, minimum ground clearance, departure angle, and approach angle.

4. The method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, When the recognition results are visualized, interactive operations are performed on the vehicle-mounted system. These interactive operations include: It allows for view adjustment operations such as rotation, scaling, and translation of the 3D terrain model, as well as real-time quantitative measurement operations of the area of ​​the target driving area, the straight-line distance between two points, and the road surface slope.

5. The method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, Information transmission between the drone and the vehicle is achieved through a multi-link redundant communication module. The communication module uses a radio communication RC link as the main transmission link and a combination of 4G and 5G cellular communication and access point (AP) links as auxiliary redundant links. The main transmission link is used to transmit real-time flight control commands and flight status information of the UAV, while the auxiliary redundant link is used for terrain data backhaul.

6. The method for three-dimensional terrain detection in air-ground cooperative off-road scenarios based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of collecting distance and orientation information of obstacles and multi-source terrain data within the target driving area includes: A preset flight control strategy is executed based on the off-road scene characteristics of the intended driving target area, where the off-road scene characteristics correspond to sandy, mountainous, and Gobi off-road scenarios. When the UAV executes the preset flight control strategy, it dynamically adjusts its flight altitude according to the terrain undulations of the target area to achieve adaptive adjustment of flight altitude, and completes environmental perception and precise obstacle avoidance throughout the flight through the onboard lidar module.

7. A UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios, characterized in that, include: The drone terminal, the vehicle-mounted terminal, and the communication module are provided, wherein the drone terminal and the vehicle-mounted terminal establish a two-way communication connection through the communication module. The drone terminal is used to receive terrain detection commands from the vehicle terminal, collect obstacle information and multi-source terrain data in the target area to be driven, and generate a three-dimensional sparse terrain model and a high-precision three-dimensional terrain model after data fusion processing, and then transmit the model and obstacle information back to the vehicle terminal. The vehicle-mounted terminal is used to plan the target area for driving and generate terrain detection commands to be sent to the drone terminal. It receives the model and data returned by the drone terminal and completes the visualization display. It integrates the terrain model to form a hierarchical output result, and at the same time identifies terrain risk areas and passable areas. The communication module adopts a multi-link redundancy design to ensure the real-time performance and stability of the transmission of detection commands, terrain data, and 3D terrain models between the UAV and the vehicle-mounted terminal.

8. The UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios according to claim 7, characterized in that, The UAV terminal is equipped with a visual camera module, a lidar module, an edge computing module, and a data fusion module, all of which are electrically connected to the UAV terminal. The visual camera module and the lidar module are used to synchronously collect multi-source terrain data of the target area to be driven. The edge computing module is used to fuse multi-source terrain data to generate a 3D sparse terrain model and a high-precision 3D terrain model. The data fusion module is used to spatially align and weightedly fuse image data and point cloud data based on a multi-source data fusion algorithm, and output a high-precision three-dimensional terrain model in a unified coordinate system.

9. The UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios according to claim 7, characterized in that, The vehicle-mounted terminal is equipped with a task planning module, a data receiving module, a model integration module, a visual segmentation module, and a visualization interaction module. The task planning module is used to plan the target area for driving and generate terrain detection instructions; The data receiving module is used to receive the model and data transmitted back from the UAV. The model integration module is used to integrate terrain models to form hierarchical output results; The visual segmentation module is used to identify terrain-risk areas and passable areas; The visualization and interaction module is used to visualize the data and recognition results and support driver interaction.

10. The UAV-based air-ground cooperative three-dimensional terrain detection system for off-road scenarios according to claim 7, characterized in that, The communication module is configured with a main transmission link unit and an auxiliary redundant link unit; The main transmission link unit is a radio communication RC link, used to transmit UAV flight control commands and status information; The auxiliary redundant link unit includes 4G and 5G cellular communication links and access point (AP) links, which are used to work with the main transmission link unit to realize the real-time transmission of detection commands, terrain data and three-dimensional terrain models.

Images