Air-ground cooperative autonomous navigation method and system in unknown environment

Abstract

The invention discloses an air-ground cooperative autonomous navigation method and system in an unknown environment, relates to the technical field of intelligent traffic, and provides real-time aerial view semantic occupation grids for a sensorless ground unmanned vehicle through an unmanned aerial vehicle which is only provided with a lightweight RGB camera, so that expensive hardware such as a laser radar and an RTK-GPS (Real-Time Kinematic-Global Positioning System) is thoroughly omitted, and the system is simple and convenient to operate. The system cost and complexity are reduced to the minimum; semantic segmentation, projection and occupation marking are completed on the generated grid map at the unmanned aerial vehicle end, passable areas and dangerous obstacles are directly depicted, and the grid map can be aligned with a vehicle-mounted odometer without high-precision positioning. And then, the grids serve as conditions to be input into a diffusion model and a guide model to explicitly avoid obstacles, smooth tracks and strictly meet starting and ending point constraints in the sampling and denoising process, real-time planning of one graph and one safe road is achieved, and the autonomous navigation safety, reliability and universality of an air-ground heterogeneous system under the conditions of GPS rejection and perception deficiency are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method and system for air-ground cooperative autonomous navigation in unknown environments. Background Technology

[0002] Autonomous and safe navigation of ground-based unmanned vehicles (UAVs) in unknown and complex environments has always been one of the core challenges in intelligent unmanned systems research. Traditional navigation methods typically rely on onboard multimodal sensors, such as LiDAR, depth cameras, or millimeter-wave radar, to construct environmental characterizations and perform path planning. However, under conditions of limited resources or impaired perception capabilities, relying solely on the UAV itself is insufficient to achieve stable and efficient navigation. Furthermore, the high cost and power consumption of high-performance sensors such as LiDAR also limit their widespread application in large-scale complex scenarios.

[0003] With the continuous improvement of drones' low-altitude perception and rapid deployment capabilities, air-ground collaborative technology is considered an effective way to overcome the perception and navigation bottlenecks of ground-based unmanned vehicles. Drones can provide a wide-range, global bird's-eye view, compensating for the limited field of vision of ground-based unmanned vehicles, and have shown significant advantages in scenarios such as exploration and rescue, intelligent transportation, and planetary exploration. However, most existing air-ground collaborative methods often rely on expensive sensing equipment and complex communication modules, limiting their deployment on low-cost platforms. In addition, even with the perception support provided by drones, planning safe paths for ground-based unmanned vehicles in unknown environments remains difficult, especially in obstacle-rich scenarios.

[0004] Currently, most autonomous navigation methods for ground-based unmanned vehicles (UAVs) adopt two traditional paradigms: sampling-based and optimization-based. While these methods can achieve certain results in low-dimensional or structured environments, they expose new challenges in air-ground cooperative scenarios. On the one hand, air-ground cooperation often deals with large-scale unknown environments, making sampling-based methods more prone to redundant exploration in complex environments, leading to inefficiency. On the other hand, optimization-based methods heavily rely on accurate environmental modeling and reasonable cost functions, but under cross-domain perception conditions, maps often contain errors or omissions, making the planning process more susceptible to getting trapped in local optima. Therefore, how to achieve cross-domain perception under low-cost conditions and combine it with efficient and robust planning methods has become a critical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides a method and system for air-ground cooperative autonomous navigation in unknown environments, aiming to solve the problem of how existing technologies can achieve cross-domain perception under low-cost conditions and combine it with efficient and robust planning methods.

[0006] On the one hand, a method for air-ground cooperative autonomous navigation in unknown environments is provided, the method comprising: Images of the target area are acquired from an aerial platform and semantic perception is performed to form a bird's-eye view semantic occupancy representation that represents obstacles and passable areas; Obtain the position and heading of the ground vehicle in the ground coordinate system, and align the bird's-eye view semantic occupancy representation with the local coordinate system of the ground vehicle; Trajectory planning is modeled as a conditional generation problem driven by both task constraints and scene semantic constraints. A generative trajectory planning model is used to output feasible paths that satisfy the constraints. The path representation is transformed to the ground carrier reference coordinate system, and the ground carrier performs trajectory tracking control, combined with odometry for relative pose estimation.

[0007] Optionally, in the above scheme, the semantic perception adopts a lightweight image model to classify the acquired images at the pixel level and aggregate them into a bird's-eye view semantic occupancy representation. The representation records the spatial distribution of obstacles and accessibility in the form of region labels or grids.

[0008] In the above scheme, optionally, the bird's-eye view semantic occupancy representation is obtained by a pixel-to-ground point mapping established by the camera imaging model and ground plane constraints. The mapping is based on camera calibration parameters and platform attitude information, and uniformly maps image coordinates to ground coordinates.

[0009] In the above scheme, optionally, the position and heading of the ground carrier in the ground coordinate system are realized by visual mark recognition or similar visual positioning methods. Based on the detection and geometric calculation of marks or features in the aerial platform image, the two-dimensional position and heading angle of the ground carrier in the ground coordinate system are obtained.

[0010] In the above scheme, optionally, the coordinate alignment transforms the obstacles, passable areas and target points in the bird's-eye view semantic occupancy representation to a local coordinate system with the ground carrier as the reference. The local coordinate system takes the current position of the carrier as the origin and defines the axis with the vehicle's forward direction.

[0011] Optionally, in the above scheme, the trajectory planning adopts a generative trajectory planning model, which simultaneously introduces task constraints and semantic constraints into the condition input. The task constraints include start and end point requirements, and the semantic constraints specify the avoidance of obstacle areas and the priority selection of passable areas.

[0012] In the above scheme, optionally, the generative trajectory planning model generates trajectory samples through an iterative process of noise injection and step-by-step restoration, treating the trajectory as a state sequence that evolves over time, and converging to a feasible solution that satisfies the constraints during the restoration process based on the conditional input.

[0013] Optionally, in the above scheme, the trajectory generation process introduces a cost guidance mechanism: in each iteration, the current candidate trajectory is directionally corrected according to the cost function. The cost function consists of multiple weighted cost components, including at least reachability, trajectory continuity and smoothness, avoidance of semantic risk areas, and satisfaction of ground vehicle kinematic constraints.

[0014] In the above scheme, optionally, the feasible path is transformed from a global ground coordinate representation to a local path representation with the ground vehicle as a reference, and the path point sequence is consistent with the attitude reference of the ground vehicle after the coordinate transformation; the ground vehicle execution steps include: performing trajectory tracking control on discrete path points, calculating control quantities based on the currently estimated relative pose and driving the movement, wherein the relative pose is continuously estimated by the ground vehicle's odometry and used to update the trajectory execution state.

[0015] On the other hand, an air-ground cooperative autonomous navigation system for unknown environments is provided, the system comprising: Acquisition module: used to acquire images of the target area from the aerial platform and perform semantic perception to form a bird's-eye view semantic occupancy representation of obstacles and passable areas; Coordinate alignment module: used to obtain the position and heading of the ground carrier in the ground coordinate system, and to align the bird's-eye view semantic occupancy representation with the local coordinate system of the ground carrier; Trajectory planning module: This module models trajectory planning as a conditional generation problem driven by both task constraints and scene semantic constraints, and uses a generative trajectory planning model to output feasible paths that satisfy the constraints. The trajectory tracking control module is used to transform the path representation to the ground carrier reference coordinate system, so that the ground carrier can perform trajectory tracking control and perform relative pose estimation in conjunction with the odometer.

[0016] Compared with the prior art, this application has at least the following beneficial effects: Based on further analysis and research into the problems of existing technologies, this application recognizes the challenge of achieving cross-domain perception and combining it with efficient and robust planning methods under low-cost conditions. It utilizes a drone equipped only with a lightweight RGB camera as the "eye in the sky," providing a real-time bird's-eye view semantic occupancy grid for sensorless ground-based unmanned vehicles, completely eliminating the need for expensive hardware such as LiDAR and RTK-GPS, minimizing system cost and complexity. The generated grid map undergoes semantic segmentation, projection, and occupancy labeling on the drone, directly depicting passable areas and dangerous obstacles, and aligning with the vehicle's odometer without high-precision positioning. Subsequently, this grid serves as a conditional input diffusion model, guiding the model to explicitly avoid obstacles, smooth trajectories, and strictly satisfy start-endpoint constraints during sampling and denoising, achieving real-time planning of "one map, one safe path." The entire solution has no special requirements on the drone / unmanned vehicle model, size, or dynamic parameters. Simulation training can be transferred to various unknown scenarios such as forests, ruins, and parks after a single training session. It supports multi-vehicle collaboration and dynamic target expansion, significantly improving the safety, reliability, and versatility of autonomous navigation in air-ground heterogeneous systems under GPS denial and perception-deficient conditions. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an air-ground cooperative autonomous navigation method in an unknown environment, as provided in one embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] In one embodiment, such as Figure 1 As shown, an air-ground cooperative autonomous navigation method for unknown environments is provided, including the following steps: Images of the target area are acquired from an aerial platform and semantic perception is performed to form a bird's-eye view semantic occupancy representation that represents obstacles and passable areas; Obtain the position and heading of the ground vehicle in the ground coordinate system, and align the bird's-eye view semantic occupancy representation with the local coordinate system of the ground vehicle; Trajectory planning is modeled as a conditional generation problem driven by both task constraints and scene semantic constraints. A generative trajectory planning model is used to output feasible paths that satisfy the constraints. The path representation is transformed to the ground carrier reference coordinate system, and the ground carrier performs trajectory tracking control, combined with odometry for relative pose estimation.

[0020] In this embodiment, the RGB camera mounted on the UAV acquires top-down images of the task area at a fixed frequency. A lightweight semantic segmentation network is used to process the acquired images frame by frame to extract obstacles and passable areas. The segmentation network can be a lightweight model with real-time performance and efficiency, such as Grounded-SAM2 or YOLO-Seg, and is not limited to a single implementation. The output of the semantic segmentation is a semantic prediction map of the same size as the original image, where each pixel is assigned a corresponding semantic label, such as road, ground, obstacle, or vehicle. To facilitate subsequent map construction, this embodiment simplifies the semantic results into a binary obstacle mask. in This represents the image pixel coordinates. When the pixel belongs to an obstacle region, let... The remaining passable areas are set as =0. This mask directly defines the distribution of obstacles in the environment. To transform semantic information from the image to a ground reference plane, this embodiment assumes the ground is a single plane (z=0 in the world coordinate system). Considering that the UAV may experience pitch and yaw angle changes during flight, a unified projection relationship from pixels to ground points needs to be established. Let the camera intrinsic parameters be... The extrinsic parameters of the camera to the world coordinate system are: in For rotation matrix, Let be the translation matrix. Let the pixel value after distortion removal be... Then its corresponding point on the ground. It is given by the following formula: in They are respectively The first two lines.

[0021] Through the above projection, all obstacles in the mask... Projecting the pixels onto the ground coordinate system yields the obstacle point set: This point set characterizes the spatial distribution of obstacles on the ground as observed by the drone.

[0022] To form a structured map representation, this embodiment further discretizes the obstacle point set into a two-dimensional occupancy grid map. Let the map extent be... The size should be generated as follows: Two-dimensional raster image ,in Any obstacle point Mapping to raster index The method is as follows: and set This indicates that the cell is occupied. Grid cells not covered by obstacle points are set to [position missing] by default. , indicating a passable area.

[0023] AprilTag markers are placed on the top of the ground unmanned vehicle. The position and heading angle of the ground unmanned vehicle in the ground plane coordinate system are calculated using the detection results of the drone's RGB camera. The obstacles and target points in the bird's-eye view (BEV) semantic occupancy grid are mapped to the local coordinate system of the ground unmanned vehicle to achieve cross-domain perception and coordinate alignment. To obtain the initial pose of the UGV on the map, an AprilTag marker is placed on top of it, and the detection results from the UAV image are used for calculation. Once the AprilTag is recognized, combined with the camera's intra-camera height information, the two-dimensional position of the UGV in the ground plane coordinate system can be directly obtained. and the yaw angle extracted from the rotation matrix The origin of the ground coordinate system is directly below the UAV, and the coordinate axes are referenced to the UAV coordinate system.

[0024] After obtaining the initial pose, we can also obtain the positions of all coordinates within the BEV's occupied grid in the ground plane coordinate system within the local coordinate system centered on the UGV. Let any point in the global BEV be... The location of UGV is Then, the representation of this point in the UGV local coordinate system is: in For heading angle Defined two-dimensional rotation matrix: This transformation allows obstacles and planned paths to be re-expressed in the UGV's local reference frame, enabling the path to be defined from the UGV's own starting point in the direction of the vehicle's front. Navigation is performed in the coordinate system of the axes. The UGV initializes the odometer and begins relative positioning and navigation each time it receives BEV information. This embodiment models trajectory planning as a conditional generation problem, uses a diffusion model to gradually denoise and generate a path from a noisy trajectory, and introduces a cost function to guide the sampling process. The cost function includes start and end point constraints, trajectory smoothness constraints, and semantic risk constraints. A. Dataset preparation and model training: All training samples are derived from a simulation environment. Specifically, obstacles and target points are pre-placed in a constructed two-dimensional simulation environment, and then a large number of feasible trajectories are generated using traditional path planning algorithms (such as A*, RRT*, and trajectory optimization methods). These trajectories are represented in sequence form as follows: The motion of the ground robot is modeled as a discrete path, consisting of a series of positions in two-dimensional space. composition.

[0025] B. Model Training: The diffusion model achieves a mapping from random noise to high-quality trajectories by constructing a forward diffusion process and a reverse denoising process. Specifically, the forward diffusion process generates high-quality trajectories from real trajectories. Starting from this point, Gaussian noise is gradually added to form a Markov chain, the conditional distribution of which is: Where t=1,…,T are the diffusion time steps. This is the noise scale at the t-th time step. Further, it can be derived that for any time step... The trajectory state satisfies: in Indicates the cumulative retention rate. This represents accumulated noise. The goal of model training is to learn the inverse process of the forward diffusion process, whose conditional distribution is also Gaussian: in The denoised mean of the model output is represented by a conditional neural network with time-step encoding. To simplify training, a fixed diagonal covariance matrix is ​​used. The model is then supervised by a noise prediction target, and the loss function is defined as: in This represents the noise component predicted by the model.

[0026] After completing the forward diffusion process Modeling and inverse process After training, the diffusion model can be used to progressively generate high-quality trajectories from Gaussian noise. The generation process starts from a random initial state. Initially, backsampling is performed iteratively to obtain the executable trajectory. : This process is essentially equivalent to prior distribution of the trajectory. Sampling can capture prior characteristics such as continuity and dynamic feasibility in the real trajectory distribution. C. Path Planning: In this system, the ground robot lacks any perception capabilities and relies entirely on a BEV occupancy map containing semantic information transmitted from the aerial drone for navigation. We adopt a motion planning method based on a diffusion model, viewing the trajectory planning problem as a process of sampling from a conditional posterior distribution. Specifically, given the constraints or objective conditions of the planning task, our goal is to find the path from this conditional posterior distribution. A path that meets the task requirements was effectively sampled, in which... This represents task constraint information, including start and end points, and obstacle information in the occupancy map. Therefore, the trajectory planning problem is transformed into a maximum a posteriori probability inference problem, i.e., finding: In each step of backsampling, target conditions are introduced, such as semantic occupancy graphs. Start and end points Defined cost function This transforms the sampling process into an approximate posterior sampling problem: Considering that the target distribution is difficult to sample directly, a gradient-guided strategy can be adopted, introducing the gradient of the cost function in each denoising step to correct the denoising trajectory. The update formula is as follows: in This indicates the step size for gradient guidance; The gradient of the trajectory cost can contain multiple weighted cost components, each optimized for a different objective; To enhance generation diversity, a certain degree of randomness is retained. This update is equivalent to adding the original denoised trajectory in each sampling step. Fine-tuning in a more optimal direction, thereby achieving optimal trajectory generation under conditional guidance.

[0027] In this embodiment, the generated trajectory is converted to the coordinate system of the ground unmanned vehicle, and the ground unmanned vehicle executes the trajectory points in sequence. The relative positioning is performed in combination with the local odometer, so as to achieve safe autonomous navigation in unknown and complex environments.

[0028] A. Planning coordinate transformation: Path point sequence output by the diffusion model The default coordinate system is defined in the global ground plane coordinate system. To facilitate direct tracking and execution by the autonomous vehicle, the trajectory points need to be transformed into a local coordinate system with the autonomous vehicle as the origin. The transformation scheme is described in part of step two. Through this transformation, the trajectory points are unified into the vehicle's own reference system, that is, the vehicle's current position is the origin, and the direction of the vehicle's head is the x-axis direction.

[0029] B. Path Execution: The converted trajectory point sequence is sent to the motion control module of the unmanned vehicle (UAV). Path points are stored in discrete form and tracked sequentially. The UAV generates corresponding steering, acceleration, or deceleration control commands by calculating the error between its current position and the target point. In implementation, trajectory execution can be achieved using a pure tracking algorithm, PID control, or other path tracking controllers; this invention does not limit the specific method. The positioning information does not rely on expensive high-precision positioning equipment. The UAV estimates relative displacement and heading angle changes in real time using its onboard odometer (such as a wheel speedometer or inertial measurement unit) to obtain its current pose. This relative positioning is initialized at the same frequency as the semantic map information transmitted by the UAV, ensuring that planning and navigation are within a unified reference map scale. Because the UAV updates the BEV semantic grid and trajectory planning results at a certain frequency, the UAV's positioning error only accumulates between adjacent updates and does not accumulate over multiple frames, thus ensuring navigation stability.

[0030] C. Iterative update planning and execution: When executing the trajectory, the autonomous vehicle always performs motion control based on the latest BEV map and diffusion model planning results.

[0031] Through the above mechanism, even in conditions where the global environment is unknown and high-precision positioning equipment is lacking, ground unmanned vehicles can still achieve low-cost, safe, and reliable autonomous navigation.

[0032] In this embodiment, the drone is equipped with an RGB camera to acquire environmental images, and a lightweight semantic segmentation network is used to extract obstacles and passable areas, including: The drone's RGB camera acquires top-down images of the mission area at a fixed frequency. A lightweight semantic segmentation network processes the acquired images frame by frame to extract obstacles and passable areas. The segmentation network is either a lightweight Grounded-SAM2 or YOLO-Seg model. The semantic segmentation output is a semantic prediction map of the same size as the original image, where each pixel is assigned a corresponding semantic label, which can be road, ground, obstacle, or vehicle. The semantic result is simplified into a binary obstacle mask. in, This represents the image pixel coordinates. When the pixel belongs to an obstacle region, let... The remaining passable areas are set as =0, the mask is used to define the distribution of obstacles in the environment.

[0033] In this embodiment, projecting pixel coordinates onto the ground plane by combining camera intrinsic parameters and flight altitude includes: Establish a unified projection relationship from pixels to ground points, with the camera intrinsic parameters being... The extrinsic parameters of the camera to the world coordinate system are: in, For rotation matrix, The translation matrix is ​​used to distort the pixels. The corresponding point on the ground for: in, They are respectively The first two lines; By projection, all obstacles in the mask... Projecting the pixels onto the ground coordinate system yields the obstacle point set: .

[0034] In this embodiment, generating a bird's-eye view semantic occupancy raster map includes: The obstacle point set describes the spatial distribution of obstacles on the ground as observed by the UAV; The obstacle point set is discretized into a two-dimensional occupancy grid map, with a map extent of [missing information]. The generated size is Two-dimensional raster image ,in Any obstacle point Mapping to raster index The method is as follows: and set This indicates that the cell is occupied; grid cells not covered by obstacle points are set to this by default. , indicating a passable area.

[0035] In this embodiment, the step of deploying AprilTag markers on the top of the unmanned ground vehicle, using the detection results of the RGB camera on the drone to calculate the position and heading angle of the unmanned ground vehicle in the ground plane coordinate system, and mapping obstacles and target points in the semantic occupancy grid map of the bird's-eye view to the local coordinate system of the unmanned ground vehicle to achieve cross-domain perception and coordinate alignment specifically includes: An AprilTag marker is placed on the top of the unmanned ground vehicle, and the detection results from the drone image are used for calculation. Once the AprilTag is identified, combined with the altitude information from the camera, the two-dimensional position of the unmanned ground vehicle in the ground plane coordinate system is obtained. and the yaw angle extracted from the rotation matrix The origin of the ground coordinate system is directly below the UAV, and the coordinate axis directions are referenced from the UAV coordinate system. After obtaining the initial pose, the positions of all coordinates in the grid occupied by the bird's-eye view in the ground plane coordinate system are obtained in the local coordinate system centered on the ground autonomous vehicle. Let any point in the global bird's-eye view be... The location of the unmanned ground vehicle is Then, the representation of any point in the local coordinate system of the unmanned ground vehicle is: in For heading angle Defined two-dimensional rotation matrix: In this embodiment, the dataset preparation for the diffusion model includes: All training samples are derived from a simulation environment. Obstacles and target points are pre-placed in the constructed two-dimensional simulation environment. A preset number of feasible trajectories are generated using traditional path planning algorithms. These feasible trajectories are represented in sequence form as follows: The motion of the ground robot is modeled as a discrete path consisting of multiple points in two-dimensional space. composition.

[0036] In this embodiment, the diffusion model achieves a mapping from random noise to high-quality trajectories by constructing a forward diffusion process and a reverse denoising process. Specifically, the forward diffusion process generates high-quality trajectories from real trajectories. Starting from this point, Gaussian noise is added to form a Markov chain, whose conditional distribution is: Where t=1,…,T are the diffusion time steps. It is the noise scale at the t-th time step; Arbitrary time step The trajectory state satisfies: in, Indicates the cumulative retention rate. This represents cumulative noise, and the model training condition distribution is also in Gaussian form: in, The denoised mean of the model output is represented by a conditional neural network with time-step encoding. To simplify training, a fixed diagonal covariance matrix is ​​used. The model is then supervised by a noise prediction target, and the loss function is defined as: in, This represents the noise component predicted by the model. After completing the forward diffusion process Modeling and inverse process After training, the diffusion model is used to progressively generate high-quality trajectories from Gaussian noise, with the generation process starting from a random initial state. Initially, backsampling is performed iteratively to obtain the executable trajectory. : .

[0037] In this embodiment, the motion planning method based on the diffusion model treats the trajectory planning problem as a process of sampling from a conditional posterior distribution, specifically including: given the constraints or target conditions of the planning task, sampling from the target conditional posterior distribution. A path that meets the task requirements is sampled from the data. The task constraint information includes the start and end points, and obstacle information in the occupancy map. The trajectory planning problem is a maximum a posteriori probability inference problem, employing a gradient-guided strategy. In each denoising step, the gradient of the cost function is introduced to correct the denoised trajectory, as shown in the following formula: in This indicates the step size for gradient guidance; This indicates that the gradient of the trajectory cost contains multiple weighted cost components, each of which optimizes for a different objective; Retaining a certain degree of randomness enhances the diversity of generation.

[0038] In this embodiment, transforming the generated trajectory to the ground-based unmanned vehicle coordinate system using the cost function includes: The path point sequence output by the diffusion model The default definition is in the global ground plane coordinate system. The trajectory points are transformed to the local coordinate system with the autonomous vehicle as the origin. The trajectory points are unified to the vehicle's own reference system, with the vehicle's current position as the coordinate origin and the direction of the vehicle's head as the x-axis direction.

[0039] In this embodiment, the ground-based unmanned vehicle sequentially executes trajectory points and performs relative positioning using local odometry to achieve safe autonomous navigation in unknown and complex environments, including: The converted trajectory point sequence is sent to the motion control module of the unmanned vehicle. The path points are stored in discrete form and tracked step by step in sequence. The unmanned vehicle generates corresponding steering, acceleration or deceleration control commands by calculating the error between the current position and the target point. The trajectory execution is achieved by using a pure tracking algorithm and PID control. The positioning information is obtained by the unmanned vehicle through its own onboard odometer to estimate the relative displacement and heading angle changes in real time to obtain the current pose. When executing the trajectory, the autonomous vehicle always performs motion control based on the latest BEV map and diffusion model planning results.

[0040] In this embodiment, the semantic perception adopts a lightweight image model to classify the acquired images at the pixel level and aggregate them into a bird's-eye view semantic occupancy representation. The representation records the spatial distribution of obstacles and accessibility in the form of region labels or grids.

[0041] In this embodiment, the bird's-eye view semantic occupancy representation is obtained by a pixel-to-ground point mapping established by the camera imaging model and ground plane constraints. The mapping is based on camera calibration parameters and platform attitude information, and uniformly maps image coordinates to ground coordinates.

[0042] In this embodiment, the position and heading of the ground carrier in the ground coordinate system are achieved through visual marker recognition or similar visual positioning methods. Based on the detection and geometric calculation of markers or features in the aerial platform image, the two-dimensional position and heading angle of the ground carrier in the ground coordinate system are obtained.

[0043] In this embodiment, the coordinate alignment transforms the obstacles, passable areas, and target points in the bird's-eye view semantic occupancy representation to a local coordinate system with the ground carrier as the reference. The local coordinate system has the current position of the carrier as the origin and the axial direction defined by the vehicle's forward direction.

[0044] In this embodiment, the trajectory planning adopts a generative trajectory planning model, which simultaneously introduces task constraints and semantic constraints into the condition input. The task constraints include start and end point requirements, and the semantic constraints specify the avoidance of obstacle areas and the priority selection of passable areas.

[0045] In this embodiment, the generative trajectory planning model generates trajectory samples through an iterative process of noise injection and step-by-step restoration, treating the trajectory as a state sequence that evolves over time, and converging to a feasible solution that satisfies the constraints during the restoration process based on the conditional input.

[0046] In this embodiment, the trajectory generation process introduces a cost-guided mechanism: in each iteration, the current candidate trajectory is directionally corrected based on a cost function. The cost function consists of multiple weighted cost components, including at least reachability, trajectory continuity and smoothness, avoidance of semantic risk areas, and satisfaction of ground vehicle kinematic constraints.

[0047] In this embodiment, the feasible path is transformed from a global ground coordinate representation to a local path representation with the ground carrier as a reference, and the path point sequence remains consistent with the attitude reference of the ground carrier after the coordinate transformation. The ground vehicle execution steps include: performing trajectory tracking control on discrete path points, calculating control quantities based on the currently estimated relative pose and driving the movement, wherein the relative pose is continuously estimated by the ground vehicle's odometer and used to update the trajectory execution status.

[0048] In one embodiment, an air-ground cooperative autonomous navigation system for unknown environments is provided, the system comprising: Acquisition module: used to acquire images of the target area from the aerial platform and perform semantic perception to form a bird's-eye view semantic occupancy representation of obstacles and passable areas; Coordinate alignment module: used to obtain the position and heading of the ground carrier in the ground coordinate system, and to align the bird's-eye view semantic occupancy representation with the local coordinate system of the ground carrier; Trajectory planning module: This module models trajectory planning as a conditional generation problem driven by both task constraints and scene semantic constraints, and uses a generative trajectory planning model to output feasible paths that satisfy the constraints. The trajectory tracking control module is used to transform the path representation to the ground carrier reference coordinate system, so that the ground carrier can perform trajectory tracking control and perform relative pose estimation in conjunction with the odometer.

[0049] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for air-ground cooperative autonomous navigation in an unknown environment, characterized in that, The method includes: Images of the target area are acquired from an aerial platform and semantic perception is performed to form a bird's-eye view semantic occupancy representation that represents obstacles and passable areas; Obtain the position and heading of the ground vehicle in the ground coordinate system, and align the bird's-eye view semantic occupancy representation with the local coordinate system of the ground vehicle; Trajectory planning is modeled as a conditional generation problem driven by both task constraints and scene semantic constraints. A generative trajectory planning model is used to output feasible paths that satisfy the constraints. The path representation is transformed to the ground carrier reference coordinate system, and the ground carrier performs trajectory tracking control, combined with odometry for relative pose estimation.

2. The method according to claim 1, characterized in that, The semantic perception uses a lightweight image model to classify the acquired images at the pixel level and aggregate them into a bird's-eye view semantic occupancy representation. The representation records the spatial distribution of obstacles and accessibility in the form of region labels or grids.

3. The method according to claim 1, characterized in that, The bird's-eye view semantic occupancy representation is obtained by a pixel-to-ground point mapping established by the camera imaging model and ground plane constraints. The mapping is based on camera calibration parameters and platform attitude information, and uniformly maps image coordinates to ground coordinates.

4. The method according to claim 1, characterized in that, The position and heading of the ground vehicle in the ground coordinate system are achieved through visual marker recognition or similar visual positioning methods. Based on the detection and geometric calculation of markers or features in the aerial platform image, the two-dimensional position and heading angle of the ground vehicle in the ground coordinate system are obtained.

5. The method according to claim 1, characterized in that, The coordinate alignment transforms the obstacles, passable areas, and target points in the bird's-eye view semantic occupancy representation to a local coordinate system with the ground carrier as the reference. The local coordinate system has the current position of the carrier as the origin and the axial direction defined by the vehicle's forward direction.

6. The method according to claim 1, characterized in that, The trajectory planning adopts a generative trajectory planning model, which simultaneously introduces task constraints and semantic constraints into the condition input. The task constraints include start and end point requirements, and the semantic constraints specify the avoidance of obstacle areas and the priority selection of passable areas.

7. The method according to claim 1, characterized in that, The generative trajectory planning model generates trajectory samples through an iterative process of noise injection and step-by-step restoration, treating the trajectory as a state sequence that evolves over time. During the restoration process, it converges to a feasible solution that satisfies the constraints based on the conditional input.

8. The method according to claim 1, characterized in that, The trajectory generation process introduces a cost-guided mechanism: in each iteration, the current candidate trajectory is directionally corrected based on a cost function. The cost function consists of multiple weighted cost components, including at least reachability, trajectory continuity and smoothness, avoidance of semantic risk areas, and satisfaction of ground vehicle kinematic constraints.

9. The method according to claim 1, characterized in that, The feasible path is transformed from a global ground coordinate representation to a local path representation with the ground carrier as a reference, and the path point sequence remains consistent with the attitude reference of the ground carrier after the coordinate transformation. The ground vehicle execution steps include: performing trajectory tracking control on discrete path points, calculating control quantities based on the currently estimated relative pose and driving the movement, wherein the relative pose is continuously estimated by the ground vehicle's odometer and used to update the trajectory execution status.

10. An air-ground cooperative autonomous navigation system for unknown environments, characterized in that, The system includes: Acquisition module: used to acquire images of the target area from the aerial platform and perform semantic perception to form a bird's-eye view semantic occupancy representation of obstacles and passable areas; Coordinate alignment module: used to obtain the position and heading of the ground carrier in the ground coordinate system, and to align the bird's-eye view semantic occupancy representation with the local coordinate system of the ground carrier; Trajectory planning module: This module models trajectory planning as a conditional generation problem driven by both task constraints and scene semantic constraints, and uses a generative trajectory planning model to output feasible paths that satisfy the constraints. The trajectory tracking control module is used to transform the path representation to the ground carrier reference coordinate system, so that the ground carrier can perform trajectory tracking control and perform relative pose estimation in conjunction with the odometer.

Images