An air-ground cooperative path planning system and method for Mars exploration

Abstract

This invention specifically relates to an air-ground collaborative path planning system and method for Mars exploration, comprising: acquiring multi-scale images of the Martian surface through air-ground collaboration; constructing a high-precision panoramic map based on a proposed adaptive sampling affine scale-invariant feature transformation algorithm; and combining a deep learning rock segmentation algorithm and a fast travel method to achieve global obstacle avoidance and energy-optimized path planning in complex unstructured Martian terrain environments. This enhances the autonomous navigation capability in complex Martian terrain and solves the technical problems of limited environmental perception range, insufficient obstacle recognition accuracy, and poor path planning efficiency and smoothness in existing Martian ground exploration path planning technologies. It is applicable to cross-view environmental mapping, rock obstacle recognition, and global path planning for Mars UAVs and Mars ground exploration rovers in complex unstructured terrain.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation and intelligent unmanned system control technology for deep space exploration, specifically to an air-ground collaborative path planning system and method for Mars exploration. Background Technology

[0002] In recent years, Mars, due to its Earth-like environmental features and unique geological evolution history, has become a core target for human deep space exploration and planetary science research. The Martian surface is characterized by dense rock formations, undulating terrain, significant dust and shadow interference, and large variations in sunlight. Exploration missions primarily using wheeled ground rovers (such as "Zhurong" and "Perseverance") are limited by sensor field of view and ground obstructions, making it difficult to obtain early information on distant obstacle distributions during long-distance autonomous navigation, thus affecting path safety and travel efficiency.

[0003] Existing path planning schemes based solely on ground-based perspectives typically rely on local grid maps or short-range point clouds. When faced with the high obstacle density environment of Mars, these schemes are prone to problems such as frequent replanning, increased detour costs, or getting trapped in local optima.

[0004] In 2021, the successful maiden flight of the "Ingenuity" Mars drone marked a new stage in Mars exploration, characterized by collaborative "air-ground" operations. By constructing a heterogeneous collaborative system between drones and ground rovers, the wide-area coverage capability of UAVs and the detailed observation advantages of UGVs can be deeply integrated, providing new possibilities for efficient and safe global exploration. However, existing technologies still face the following key challenges: At the environmental perception level: there is a lack of robust fusion methods for images with large parallax and large-scale differences from heterogeneous aerial and ground perspectives, making it difficult to construct globally consistent high-precision terrain maps. For rock obstacles with similar textures and varied shapes on the Martian surface, traditional threshold-based or rule-based segmentation algorithms struggle to distinguish rock entities from their shadow boundaries, resulting in inaccurate obstacle modeling.

[0005] At the path planning level: Existing path planning algorithms are computationally inefficient in high-resolution, high-obstacle-density global maps, and the generated paths often have many polylines and poor smoothness; At the system collaboration level: existing research focuses on a single perception or planning stage, failing to achieve a closed-loop system that integrates multi-source data fusion, terrain semantic understanding, and global smooth path generation, resulting in limited overall system autonomy and adaptability.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] This invention provides an air-ground collaborative path planning system and method for Mars exploration, a computer-readable storage medium, and a computer program product, which can effectively overcome the defects existing in the prior art.

[0008] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0009] According to a first aspect of the present invention, an air-ground cooperative path planning system for Mars exploration is provided, the system comprising: The multi-source sensing module is used to acquire aerial image sequences collected by UAVs, ground image sequences collected by ground exploration vehicles, and pose information. The panoramic map construction module is used to perform cross-view registration of aerial image sequences and ground image sequences using adaptive sampling affine scale-invariant feature transformation to construct a panoramic map. The obstacle recognition module is used to identify obstacles in the panoramic map using an obstacle recognition model and generate a cost map. The global path planning module is used to search for paths on the obstacle map based on the pose information of the ground exploration vehicle and to determine the driving path of the ground exploration vehicle using a fast travel algorithm. The task execution module is used to generate control commands by combining the driving path and the kinematic constraints of the probe vehicle to drive the probe vehicle.

[0010] In some exemplary embodiments, the panoramic map construction module includes: The feature matching unit is used to filter out pairs of aerial and ground image frames with overlapping observation areas from the aerial image sequence and the ground image sequence based on the time synchronization results; and to perform affine scale-invariant feature transformation processing based on adaptive sampling on the aerial and ground image frame pairs to obtain candidate matching point pairs. The geometric verification unit is used to verify the geometric consistency of candidate matching point pairs and determine the registration transformation model when the geometric consistency verification is passed. The panoramic stitching unit is used to map pairs of air and ground images to a unified coordinate system and stitch them together to construct a panoramic map based on a registration transformation model.

[0011] In some exemplary embodiments, the feature matching unit includes: Affine parameter space determination sub-unit is used to extract initial feature points from the overhead image and ground image in the air-ground image frame pair, and to calculate the feature point density index. When the feature point density index represents less image texture information or the matching difficulty is high, increase the upper limit of the tilt parameter, increase the number of tilt layers, and decrease the rotation angle step size. When the feature point density index represents rich image texture information or the matching difficulty is low, reduce the upper limit of the tilt parameter, reduce the number of tilt layers, and increase the rotation angle step size. The virtual view generation and matching subunit is used to generate a virtual view using a sample set, and to perform scale-invariant feature extraction and matching on the virtual view to obtain candidate matching point pairs.

[0012] In some exemplary embodiments, the geometric verification unit includes: The consistency verification subunit is used to perform geometric consistency verification on candidate matching point pairs using the nearest neighbor ratio method, eliminate erroneous candidate matching point pairs, and obtain the target matching point pairs. The matching quality index between images is calculated based on the target matching point pairs; the matching quality index includes: number of inliers and inlier ratio; The registration model determination sub-unit is used to determine the corresponding registration transformation model when the number of inliers or the inlier rate meets the preset consistency qualification judgment conditions.

[0013] In some exemplary embodiments, the obstacle recognition module includes: The obstacle mask generation unit is used to input the panoramic map into the obstacle recognition model to obtain the obstacle mask; wherein, the obstacle recognition model is a ResU-Net network, the encoder in the ResU-Net network adopts the ResNet residual structure to enhance the feature extraction capability, and the decoder in the ResU-Net network adopts the U-Net decoding structure to gradually restore the spatial resolution. The obstacle area expansion unit is used to extract the obstacle boundary based on the obstacle mask and expand the obstacle boundary according to a preset safety distance to obtain a safe avoidance area. The obstacle map generation unit is used to map the safe avoidance area onto a planning grid in a unified coordinate system to generate a cost map.

[0014] In some exemplary embodiments, the global path planning module includes: The start-end point determination unit is used to determine the starting point of the ground exploration vehicle in the obstacle map based on the pose information of the ground exploration vehicle, and to obtain the corresponding target point; The velocity field construction unit is used to construct the velocity field corresponding to the two-dimensional grid map based on the starting point, the target point, and the distribution of obstacle areas in the obstacle map. The arrival time field solution unit is used to establish the Eikonal equation satisfied by the wavefront propagation arrival time based on the velocity field, and to discretize and solve the Eikonal equation to obtain the arrival time field formed by propagation from the starting point outward. The path backtracking unit is used to backtrack from the target point to the starting point step by step along the direction of decreasing arrival time based on the arrival time field, so as to obtain the driving path of the ground exploration vehicle.

[0015] In some exemplary embodiments, the system further includes a path shaping and replanning module, which is used to: Monitor environmental changes and obstacle map updates of the ground exploration vehicle to determine whether the current travel path meets the path replanning trigger conditions. When the judgment result shows that the current driving path meets the path replanning trigger condition, path replanning is triggered, and the panoramic map and obstacle map are updated. When the judgment result indicates that the current travel path does not meet the path replanning trigger condition, the ground exploration vehicle is controlled to continue to perform the task according to the current travel path.

[0016] According to a second aspect of the present invention, an air-ground cooperative path planning method for Mars exploration is provided, applied to the aforementioned system, characterized in that the method comprises: Acquire aerial image sequences captured by UAVs, ground image sequences captured by ground exploration vehicles, and pose information; Cross-view registration is performed on sequences of aerial and ground images to construct a panoramic map; Obstacle recognition models are used to identify obstacles in panoramic maps and generate cost maps. Based on the pose information of the ground exploration vehicle, a fast travel algorithm is used to search for paths on the obstacle map and determine the driving path of the ground exploration vehicle. Control commands are generated by combining the driving path and the kinematic constraints of the probe vehicle to drive the probe vehicle.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, the device where the storage medium is located controls the execution of the above-described air-ground cooperative path planning method for Mars exploration.

[0018] According to a fourth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the above-described air-ground cooperative path planning method for Mars exploration.

[0019] According to a fifth aspect of the present invention, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to implement the aforementioned air-ground collaborative path planning method for Mars exploration by executing the executable instructions.

[0020] The embodiments of this invention provide an air-ground collaborative path planning system for Mars exploration. Addressing the characteristics of large viewing angle differences, significant scale variations, and non-uniform illumination in Mars-Earth heterogeneous images, it introduces an adaptive sampling affine scale-invariant feature transform. This dynamically adjusts the sampling density of the affine parameter space based on image difficulty, improving the success rate and registration accuracy of panoramic map construction. To address the issues of high similarity in Martian surface textures, strong shadow interference, and unclear obstacle boundaries, it uses target-shadow association constraints and safety margin processing to stably convert visual segmentation results into obstacle grids that can be directly used for planning. This reduces the risk of passage due to shadow misjudgment and improves the robustness and physical safety of obstacle recognition. Furthermore, to address the problems of heavy computational burden, path polymorphism, and difficulty in comprehensively optimizing energy consumption costs in traditional grid planning under high-resolution global maps, it introduces a passage cost that decreases with distance from obstacles into the cost map. This allows the path to have better continuity and smoothness while meeting obstacle avoidance safety requirements, while simultaneously improving global planning efficiency and reducing the overall cost of long-distance missions. This solves the technical problems of limited environmental perception range, insufficient obstacle recognition accuracy, and poor path planning efficiency and smoothness in existing Mars ground exploration path planning technologies.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 This schematic diagram illustrates the structure of an air-ground collaborative path planning system for Mars exploration, an exemplary embodiment of the present invention. Figure 2 This schematic diagram illustrates the process of air-to-ground image fusion and panoramic image construction based on AS-ASIFT, an exemplary embodiment of the present invention. Figure 3 This schematically illustrates a visualization of keypoint matching between air-to-ground frame pairs, representing an exemplary embodiment of the present invention. Figure 4 This schematic diagram illustrates the obstacle recognition model of an exemplary embodiment of the present invention in a test set for recognizing rocks on the surface of Mars. Figure 5 This illustration schematically shows a feasible path planned by the global path planning module in an exemplary embodiment of the present invention; Figure 6 This schematically illustrates a flowchart of an air-ground collaborative path planning method for Mars exploration, an exemplary embodiment of the present invention. Figure 7 The illustration shows a schematic diagram of the overall process of an air-ground cooperative path planning method for Mars exploration, an exemplary embodiment of the present invention. Figure 8 The diagram illustrates the composition of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0025] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] To address the shortcomings and deficiencies of existing technologies, this exemplary embodiment provides an air-ground collaborative path planning system for Mars exploration. (Reference) Figure 1 As shown, it can specifically include: The multi-source sensing module 10 is used to acquire the sequence of overhead images collected by the UAV, the sequence of ground images collected by the ground exploration vehicle, and pose information; The panoramic map construction module 12 is used to perform cross-view registration of the overhead image sequence and the ground image sequence using adaptive sampling affine scale-invariant feature transformation to construct a panoramic map. The obstacle recognition module 14 is used to identify obstacles in the panoramic map using an obstacle recognition model and generate a cost map. The global path planning module 16 is used to perform path search on the obstacle map based on the pose information of the ground exploration vehicle and to determine the driving path of the ground exploration vehicle by using a fast travel algorithm. The task execution module 18 is used to generate control commands by combining the driving path and the kinematic constraints of the probe vehicle to drive the probe vehicle.

[0027] The following will describe in more detail the various modules of an air-ground collaborative path planning system for Mars exploration in this exemplary embodiment, with reference to the accompanying drawings and embodiments.

[0028] For example, refer to Figure 1 As shown, Figure 1 This is a schematic diagram of an air-ground collaborative path planning system for Mars exploration. The multi-source sensing module 10 includes: a Mars unmanned aerial vehicle (UAV) sensing unit and a ground rover sensing unit.

[0029] The Mars UAV perception unit 101 includes a visual camera, a lidar, and a high-precision positioning and timing component, used to collect environmental data and pose information on the Mars UAV. The visual camera is used to collect RGB image data of the area around or below the Mars UAV, acquiring visual information such as surface texture, rock distribution, and shadow boundaries; the lidar is used to emit pulsed laser beams to measure the distance, shape, and surface features of objects around and below the spacecraft, generating 3D point cloud data of the Mars UAV's downward-facing area, used to acquire geometric information such as terrain undulations and rock protrusions, and to support dynamic / static obstacle detection; the high-precision positioning and timing component is used to provide pose estimation and time reference for the Mars UAV to support air-to-ground data alignment, cross-view registration, and panoramic map construction.

[0030] The ground rover's perception unit 102 includes a visual camera, a lidar, a high-precision positioning and timing component, and an inertial navigation system (INS), used to collect environmental data and pose information on the Mars rover. The visual camera acquires RGB image data of the area surrounding the Mars rover, obtaining close-range details of rocks, sand textures, and surface boundary information. The lidar scans the Martian surface terrain and generates a 3D point cloud or 2D ground grid map near the rover, thereby identifying passable areas and obstacle areas. The high-precision positioning and timing component provides pose estimation and a time reference for the Mars rover, supporting spatiotemporal alignment with observation data from a Mars drone. The inertial navigation system (INS) uses inertial sensors (accelerometers and gyroscopes) to measure the rover's acceleration and angular velocity, and calculates the rover's position, velocity, and attitude through integration calculations. This provides auxiliary positioning and attitude constraints in situations where positioning information is unstable or errors accumulate, thereby improving the continuity and robustness of air-ground collaborative mapping and path planning.

[0031] Furthermore, the air-to-ground collaborative path planning system for Mars exploration also includes an air-to-ground communication module 11 for enabling air-to-ground data exchange and synchronization. The air-to-ground communication module 11 is connected to the air-to-ground collaborative decision-making and planning subsystem, which includes a panoramic map construction module 12, an obstacle recognition module 14, a global path planning module 16, and a path shaping and replanning module 19. The air-to-ground communication module 11 is signal-connected to the multi-source sensing module 10 and the panoramic map construction module 12, transmitting the collected air-to-ground data stream to the panoramic map construction module 12 and transmitting control commands to the multi-source sensing module 10. The air-to-ground communication module 11 is signal-connected to the obstacle recognition module 14, the global path planning module 16, the mission execution module 18, and the path shaping and replanning module 19, generating obstacle distribution maps and planning paths, and driving the rover to perform path tracking based on the planned paths.

[0032] The panoramic map construction module 12 receives aerial images from the Mars drone and ground images from the rover. It first performs preprocessing to correct distortion and normalize brightness. Then, it samples affine parameters from each image to generate a virtual view, and performs SIFT feature extraction on the virtual view. After matching the air-to-ground features, it removes mismatches and estimates the homography matrix / affine transformation, finally generating a panoramic map in a unified coordinate system. The obstacle recognition module 14 inputs the panoramic map into ResU. The Net network outputs a pixel-level rock probability map, which is then thresholded to obtain a binary mask. Further connected component analysis of the mask yields statistics such as rock quantity, area, and equivalent diameter, and the mask is mapped to obstacle occupancy cells or cost cells in the planning grid. The global path planning module 16 constructs a velocity field / cost field based on the aforementioned obstacle grid or cost map, and calculates the arrival time field using the Fast Journey Method (FMM). A path is obtained by backtracking from the target point along the negative gradient direction; this path is smoother and has fewer turns in a continuous spatial sense. The task execution module 18 converts the backtracked path into a waypoint sequence, and performs sparsification and local smoothing based on the minimum turning radius and speed limit of the probe vehicle. The path shaping and replanning triggering module 19 triggers incremental stitching and replanning when a new UAV image is received or the probe vehicle detects a sudden increase in obstacle density.

[0033] For example, the panoramic map construction module 12 includes: The feature matching unit 121 is used to filter out pairs of aerial and ground image frames with overlapping observation areas from the aerial image sequence and the ground image sequence based on the time synchronization result; and to perform affine scale-invariant feature transformation processing based on adaptive sampling on the pairs of aerial and ground image frames to obtain candidate matching point pairs. Geometric verification unit 122 is used to perform geometric consistency verification on candidate matching point pairs and determine the registration transformation model when the geometric consistency verification is passed. The panoramic stitching unit 123 is used to map pairs of air and ground images to a unified coordinate system and stitch them together based on a registration transformation model to construct a panoramic map.

[0034] Specifically, distortion correction, brightness normalization, and time synchronization are performed on the aerial and ground image sequences, and aerial-ground image frame pairs with overlapping observation areas are selected as the input for registration. The Adaptive Sampling ASIFT (AS-ASIFT) algorithm is used to extract and match features from the aerial-ground image frame pairs to obtain candidate matching point pairs. False matches are eliminated using the nearest neighbor ratio method, and the registration transformation model is determined when the Random Sample Consensus (RANSAC) geometric consistency verification is passed. Based on the transformation model that finally passes the consistency qualification criteria, the UAV images and the rover images are coordinate registered and projected onto a unified coordinate system. Fusion processing is performed in the overlapping areas to output a unified aerial-ground panoramic map, which can be further mapped to a unified raster map.

[0035] For example, the feature matching unit 121 includes: Affine parameter space determination subunit 1211 is used to extract initial feature points from the overhead image and ground image in the air-ground image frame pair, and calculate the feature point density index. When the feature point density index represents less image texture information or the matching difficulty is high, increase the upper limit of the tilt parameter, increase the number of tilt layers, and decrease the rotation angle step size. When the feature point density index represents rich image texture information or the matching difficulty is low, reduce the upper limit of the tilt parameter, reduce the number of tilt layers, and increase the rotation angle step size. The virtual view generation and matching subunit 1212 is used to generate a virtual view using a sample set, and to perform scale-invariant feature extraction and matching on the virtual view to obtain candidate matching point pairs.

[0036] Specifically, key points for scale-invariant feature transformation (SIFT) are extracted from the overhead and ground images in the air-ground image frame pair, and the feature point density is calculated. ,in Here, W represents the number of key points, and H represents the image width and height, respectively.

[0037] The affine parameter sampling set is adaptively determined based on the feature point density ρ to generate multiple sets of corresponding virtual views; wherein, the sampling set includes at least: the upper limit of the tilt parameter. Number of inclined layers and rotation angle step size When ρ is below a preset threshold, increase ,Increase and reduce Encrypted sampling is used; sparse sampling is used when ρ is higher than a preset threshold to reduce... ,reduce and increase To reduce the amount of computation.

[0038] Affine transformation can be expressed as:

[0039] in, The coordinates of the original image are... For virtual view coordinates, For affine matrices, It is a translation vector.

[0040] Through the Parameter set Multiple virtual views are generated through sampling. A sample set is used in the first round of sampling. A virtual view is generated, which includes at least the feature point locations, scales, main directions, and their descriptors for subsequent cross-view matching. If the subsequent consistency qualification check fails, a second round of encrypted sampling is triggered to generate a sample set. For example, increase ,Increase or reduce ), and set an upper limit on the number of virtual views. Or view the maximum quantity To balance registration robustness and computational cost under low-texture and non-uniform lighting conditions on Mars.

[0041] Furthermore, a full feature matching is performed on all virtual view feature sets of the UAV and all virtual view feature sets of the probe vehicle.

[0042] For example, the geometric verification unit 122 includes: The consistency verification subunit 1221 is used to perform geometric consistency verification on candidate matching point pairs using the nearest neighbor ratio method, eliminate erroneous candidate matching point pairs, and obtain the target matching point pairs. The matching quality index between images is calculated based on the target matching point pairs; the matching quality index includes: number of inliers and inlier ratio; The registration model determination subunit 1222 is used to determine the corresponding registration transformation model when the number of inliers or the inlier rate meets the preset consistency qualification judgment conditions.

[0043] Specifically, the nearest neighbor ratio method is used to filter out ambiguous matching pairs among the candidate matching point pairs, resulting in a set of candidate matching pairs. The nearest neighbor ratio method is shown in the following formula:

[0044] in, For the descriptor to be matched, , These are the nearest neighbor and second nearest neighbor descriptors, respectively. The threshold is used to remove unreliable matches.

[0045] Specifically, the transformation model parameters required for air-to-ground image registration are estimated based on the target matching points to obtain the number of inliers. Interior point ratio Matching quality indicators; based on preset thresholds, the consistency of the matching quality indicators is judged as qualified: when or Output transformation model when threshold requirement is met; when or If the threshold requirement is not met, a second round of encrypted sampling is triggered, and the virtual view generation, candidate matching point pair elimination, and consistency qualification determination are repeated until the result is deemed qualified or the computational budget limit is reached. The homography model satisfies the following formula:

[0046] in, This represents the pixel coordinates of feature points in the image to be registered (such as UAV images in air-to-ground vision). This represents the pixel coordinates of the corresponding feature point in the reference image (such as a baseline image obtained by a probe). This is the scale factor, used for homogeneous coordinate normalization. Let be a homography matrix, which is a The matrix describes the projection mapping relationship between the two planes.

[0047] Furthermore, the reprojection error is used as the interior point criterion:

[0048] in, For the first Reprojection error of the target matching point; These are the actual observed pixel coordinates of the feature points in the reference image, typically represented as a two-dimensional vector. ; Let be the homogeneous coordinates of the feature points in the image to be registered, represented as a three-dimensional vector. . This is a projection mapping function, whose function is to transform the homogeneous coordinate vector. Mapping back to a two-dimensional pixel coordinate system, the calculation method is as follows: .

[0049] set up The threshold value is a preset tolerance range; when the calculated error... When the value is below this threshold, the matching pair is considered an interior point that meets geometric consistency. When a point is identified as an interior point, its number is counted. With interior point ratio Ultimately, the option with the largest number of interior points or the smallest error is selected. As the optimal registration model.

[0050] For details, please refer to Figure 2 As shown, Figure 2 This is a schematic diagram of the process of aerial-ground image fusion and panoramic image construction based on AS-ASIFT.

[0051] (1) Obtain the preprocessed aerial images of Mars taken by the UAV (UAV image Ia) and the ground image sequence collected by the Mars ground rover (rover image Ig), and construct air-to-ground image frame pairs based on the time synchronization results; the time synchronization threshold is preferably selected as follows. The image pairs with an overlap of at least 0.20% of the observation area were selected as the input for registration; the candidate frame pairs were quality-screened, with Laplacian variance being the preferred sharpness metric. The frame pairs are preserved, while the average gray level of the 8-bit image is not... Frame pairs within the range are identified as having exposure issues and are removed.

[0052] (2) Extract initial SIFT key points from UAV image Ia and rover image Ig respectively, and calculate feature point density. As an indicator of matching difficulty, among , Here are the keypoints, and W and H are the image width and height in pixels; based on Adaptive selection of affine parameter sampling strategy and determination of sampling set The sampling set must contain at least the upper limit of the tilt parameter. Number of inclined layers and rotation angle step size ;when When the value is high, sparse sampling is used to reduce computational cost; when Dense sampling is used to improve robustness when the parallax is low or large; the preferred range of sampling parameters is: , , (Easy to match); , , (Difficult to match).

[0053] (3) Based on the sampling set S, perform affine parameter sampling on Ia and Ig respectively to generate multiple sets of virtual views, which are used to simulate the tilt, rotation and scale changes of the air-to-ground image caused by the viewpoint difference; the affine transformation can be expressed as Preferred sampling method for tilt parameters To control computational load, it is preferable to set an upper limit on the number of virtual views. The value is preferably 120 to 300, and more preferably 240.

[0054] (4) Perform SIFT feature extraction on each virtual view of Ia and each virtual view of Ig to obtain the feature set of all virtual views of the UAV and the feature set of all virtual views of the probe vehicle. The feature set includes at least the feature point position, scale, main direction and its descriptor, and records the virtual view identifier to which the feature belongs for subsequent tracing.

[0055] (5) Perform full feature matching on all virtual view feature sets of the UAV and all virtual view feature sets of the probe vehicle, and use the nearest neighbor ratio method to filter out ambiguous matching pairs to obtain a candidate matching pair set; the nearest neighbor ratio method satisfies ,in The optimal value is 0.75. Maximum number of matches. The preferred value is 150 to 400, and more preferably 250.

[0056] (6) Perform geometric consistency verification on the candidate matching pair set input to RANSAC, eliminate mismatches, and estimate the transformation model parameters required for air-to-ground image registration; when using the homography model, satisfy the following: and reprojection error As an interior point criterion, the reprojection error threshold is preferably selected as follows: The preferred RANSAC confidence level is 0.99, and the preferred maximum number of iterations is 2000; when and It was judged as qualified at that time; among them The preferred value is 20-60; The preferred value is 0.75 to 0.95; when a value is determined to be unqualified and the budget limit is not reached, encrypted sampling is triggered: by reducing... or increase Or increase Generate a denser set of samples Then return to steps (3) to (6) for the second round of matching; when the budget limit is reached and it is still not qualified, the frame pair is determined to be registered and the empty ground image frame pair is reselected.

[0057] (7) After the consistency qualification is passed, Ia and Ig are registered according to the optimal registration model and projected to a unified coordinate system. The image stitching is completed by performing fusion processing in the overlapping area, and a unified panoramic map of air and ground is output. Feathering fusion is preferred to reduce brightness difference and suppress stitching seams and ghosting. The width of the feathering fusion transition zone is preferably 10 to 80 pixels, more preferably 30 pixels. Linear brightness compensation is performed on the overlapping area to improve stitching consistency.

[0058] (8) Output panoramic map.

[0059] For example, the obstacle recognition module 14 includes: The obstacle mask generation unit 141 is used to input the panoramic map into the obstacle recognition model to obtain the obstacle mask; wherein, the obstacle recognition model is a ResU-Net network, the encoder in the ResU-Net network adopts the ResNet residual structure to enhance the feature extraction capability, and the decoder in the ResU-Net network adopts the U-Net decoding structure to gradually restore the spatial resolution. The obstacle area expansion unit 142 is used to extract the obstacle boundary according to the obstacle mask and expand the obstacle boundary according to a preset safe distance to obtain a safe avoidance area. The obstacle map generation unit 143 is used to map the safe avoidance area onto a planning grid under a unified coordinate system to generate a cost map.

[0060] Specifically, to improve rock recognition accuracy, a ResU-Net network is used as the obstacle recognition model. The ResU-Net network uses a ResNet residual structure as a U-Net encoder to extract subtle textures and multi-scale features of Martian rocks; the U-Net decoder restores spatial resolution, and high-frequency detail information from the encoder is passed to the decoder via skip connections, thereby improving rock boundary segmentation accuracy. The residual units alleviate the vanishing gradient problem through cross-layer connections, supporting stable training of deep networks and satisfying the following form:

[0061] in, For input features, For convolution mapping, These are network parameters.

[0062] In the training process of the obstacle recognition model, the training dataset is derived from real Martian images (e.g., Martian surface images taken by the "Zhurong" Mars rover). Image samples with varying rock distributions are selected from publicly available image data to construct a segmentation dataset. The dataset is randomly divided into training, validation, and test sets for network training, hyperparameter selection, and recognition performance evaluation. LabelMe and other annotation tools are used to perform pixel-level annotations on the images, primarily targeting rock regions larger than a preset threshold. Simultaneously, since shadowed rock regions created by illumination are closely adjacent to the rocks and easily affect recognition, these shadowed regions can be labeled as obstacle regions or as independent categories.

[0063] Specifically, the panoramic map is input into the trained ResU-Net network, which outputs a pixel-level rock category probability map. The probability map is then thresholded or subjected to maximum probability discrimination to generate a rock obstacle mask. The binary mask satisfies the following conditions:

[0064] in, For pixel position, For the probability of rock, The threshold value is used.

[0065] Project / map rock obstacle masks onto the panoramic view. Figure 1 A unified coordinate system is established, and an obstacle grid is generated according to the planned resolution: the grid cells corresponding to the mask are set as impassable, while the remaining grid cells are set as passable. Furthermore, to meet the safety margin and size constraints of the detection vehicle, the mask is morphologically expanded to generate safe obstacle zones, and the obstacle grid is updated accordingly. Additionally, a passage cost that decreases with distance from the obstacle is set for the area near the obstacle boundary, ensuring that the planned path maintains a safe distance while satisfying feasibility, thus generating a cost map.

[0066] For example, the global path planning module 16 includes: The start-end point determination unit 161 is used to determine the starting point of the ground exploration vehicle in the obstacle map based on the pose information of the ground exploration vehicle, and to obtain the corresponding target point; The velocity field construction unit 162 is used to construct the velocity field corresponding to the two-dimensional grid map based on the starting point, the target point, and the distribution of obstacle areas in the obstacle map; The arrival time field solving unit 163 is used to establish the Eikonal equation satisfied by the wavefront propagation arrival time based on the velocity field, and to perform discrete solution of the Eikonal equation to obtain the arrival time field formed by propagation from the starting point outward. The path backtracking unit 164 is used to backtrack from the target point to the starting point step by step along the direction of decreasing arrival time based on the arrival time field, so as to obtain the driving path of the ground exploration vehicle.

[0067] Specifically, a wavefront propagation velocity field is defined on a two-dimensional grid cost map. Set the grid cells corresponding to the obstacle area as impassable areas, and set their speed to... Set the grid cell corresponding to the passable area as the passable area, and set its speed to... (For example, take) Or a positive velocity obtained from cost mapping).

[0068] In a velocity field, the wavefront propagates from its origin to any point. Shortest arrival time Satisfies the Eikonal equation:

[0069] in, Indicates the location point. For wavefront propagation speed, To spread from the starting point to the point The shortest arrival time.

[0070] Rasterize the two-dimensional space, and set These are the grid points corresponding to the actual map coordinates. , Let these represent the rows and columns of the raster, respectively, denoted as:

[0071]

[0072] in, , The grid is respectively in and Spatial spacing in the direction, and Coordinates Backward and forward differences on.

[0073] Substituting the difference scheme into the Eikonal equation yields the discrete form:

[0074] Further orders:

[0075] Then in only When the variable is unknown, an equivalent two-dimensional discrete update form can be obtained:

[0076] Starting grid Set the initial region The remaining passable grids Initialize to infinity; starting from the nearest points of the solved points, follow the FMM wavefront extension strategy and use the discrete Eikonal equations to solve for the unknown points. Perform iterative updates until the arrival time of the endpoint grid is determined, thus obtaining the global arrival time field. .

[0077] Starting from the grid corresponding to the target point, the system backtracks by progressively selecting adjacent grids with smaller arrival times along the negative gradient direction of the arrival time field until it reaches the starting grid, thus obtaining the global path from the starting point to the target point.

[0078] For example, the task execution module 18 is used to receive the driving path output by the global path planning module, and, in conjunction with the current position, attitude information, and kinematic constraints of the ground exploration vehicle, to track and control the driving path, generating steering control commands and speed control commands to drive the ground exploration vehicle along the driving path. For example, based on the kinematic constraints of the exploration vehicle (minimum turning radius, maximum gradient, speed limit, etc.), the path is sparsified and smoothed, converting the global path into a continuous trajectory or a discrete waypoint sequence.

[0079] For example, the system further includes a path shaping and replanning module 19, which is used to: Monitor environmental changes and obstacle map updates of the ground exploration vehicle to determine whether the current travel path meets the path replanning trigger conditions. When the judgment result shows that the current driving path meets the path replanning trigger condition, path replanning is triggered, and the panoramic map and obstacle map are updated. When the judgment result indicates that the current travel path does not meet the path replanning trigger condition, the ground exploration vehicle is controlled to continue to perform the task according to the current travel path.

[0080] The triggering conditions include at least: entering an unknown area, a sudden increase in obstacle density, the restoration of communication and the acquisition of new aerial imagery, or the vehicular yaw exceeding a threshold.

[0081] Further, refer to Figure 3 As shown, Figure 3 For keypoint matching visualization of the same representative air-ground frame pair, only the RANSAC inlier connections are drawn. Fixed-sampling ASIFT, with an attempt=1, yields a low number of inliers, leading to matching failures. However, AS-ASIFT, after failing the consistency pass check, triggers attempt=2 encrypted sampling, increasing the number of inliers and maintaining a high inlier rate, thereby improving geometric consistency and registration success probability, resulting in successful matching. This result demonstrates that, while maintaining a controllable computational budget, the proposed method achieves more robust feature matching performance through an on-demand encrypted sampling mechanism.

[0082] In one specific embodiment, reference Figure 4 As shown, Figure 4 This diagram illustrates the ResU-Net network's performance in recognizing rocks on the Martian surface in the test set. The recognition results are shown in (a) and (b) for sparsely distributed rocks, (c) and (d) for densely distributed rocks, and (e) and (f) for images with rover interference. As can be seen, the model can accurately identify rocks of various scales and shapes, avoid interference from other objects in the image, and provide the global path planning module with spatial distribution characteristics of rock obstacles on the Martian surface.

[0083] In one specific embodiment, reference Figure 5 As shown, Figure 5 This refers to feasible paths planned by the global path planning module. In the air-ground collaborative planning system, assuming the Martian surface rocks are identified, the binary representation of the Martian surface grid map is used as input to the global path planning module. The area represents the obstacle area (e.g.) Figure 5 (Dark parts) The area is a feasible area (e.g.) Figure 5 (Light-colored section). See the section on feasible paths obtained through the global path planning module. Figure 5 .

[0084] This example implementation provides an air-ground cooperative path planning method for Mars exploration, applied to the aforementioned air-ground cooperative path planning system for Mars exploration. (Reference) Figure 6 As shown, it can specifically include: Step S61: Obtain the sequence of overhead images captured by the UAV, the sequence of ground images captured by the ground exploration vehicle, and pose information; Step S62: Perform cross-view registration on the aerial image sequence and the ground image sequence to construct a panoramic map; Step S63: Use the obstacle recognition model to identify obstacles in the panoramic map and generate a cost map; Step S64: Based on the pose information of the ground exploration vehicle, a fast travel algorithm is used to search for a path on the obstacle map to determine the driving path of the ground exploration vehicle. Step S65: Combine the driving path and the kinematic constraints of the probe vehicle to generate control commands to drive the probe vehicle.

[0085] For example, refer to Figure 7 As shown, Figure 7 This is a flowchart of the overall path planning method. The method includes: image acquisition and preprocessing, ASIFT registration and stitching, and ResU... Net obstacle segmentation, FMM global planning and path backtracking, waypoints and path tracing, and replanning loop.

[0086] 1) System initialization: Load the relevant programs and model files for air-ground collaborative path planning, establish the data communication link between the UAV and the exploration vehicle, and complete the initialization of camera calibration parameters, coordinate system transformation relationships and map cache area.

[0087] 2) Determine the starting point and target point: Obtain the current pose of the probe and calculate the starting point. Obtain the location of the task target and convert it into the target point. At the same time, the task area range and planning boundaries are set.

[0088] 3) Perform Mars UAV collaboration determination: Determine whether the UAV is in the air, whether communication is available, whether the UAV's field of view covers the planned area, and whether air-to-ground data can be synchronized; if the collaboration conditions are met, proceed to step 4); otherwise, proceed to step 13) Perform autonomous navigation of the rover.

[0089] 4) Acquiring air-to-ground images: The Mars UAV performs aerial photography along the mission area to acquire an aerial image set and records the corresponding timestamps and pose information; the Mars ground rover acquires ground image sequences during its journey and records the corresponding timestamps and pose information.

[0090] 5) Preprocessing and synchronizing aerial and ground images: distortion correction, brightness normalization and denoising are performed on the aerial image set and ground image sequence, respectively; frame alignment and synchronization are performed based on timestamps, and aerial and ground image pairs with overlapping observation areas are selected as fusion inputs.

[0091] 6) Adaptive generation of virtual views and feature extraction based on AS-ASIFT: First, SIFT key points are extracted from UAV image Ia and rover image Ig, and the feature point density is calculated as a matching difficulty index; based on the feature point density, an affine parameter sampling strategy is adaptively selected to determine the sampling set S, the sampling parameters are dynamically adjusted according to the feature point density, and an upper limit is set on the number of virtual views; multiple sets of virtual views are generated based on the sampling set S, and SIFT feature extraction is performed on each virtual view to form the virtual view feature sets of the UAV end and the rover end.

[0092] 7) Feature Matching, Geometric Verification, and Consistency Determination: Perform feature matching on the feature sets of the virtual views at both ends under budget constraints, and filter out ambiguous matches using the nearest neighbor / second nearest neighbor ratio method; perform geometric consistency verification on the candidate matches against the input RANSAC, eliminate erroneous matches, estimate the air-to-ground image registration transformation model, and output the number of inliers and the inlier rate. If the preset consistency qualification condition is not met, trigger encrypted sampling. Repeat steps 6) to this step under the constraint of not exceeding the upper limit of the view; when the consistency qualification condition is met, output the registration transformation model.

[0093] 8) Panoramic map generation and update: Based on registration transformation, the air-ground images are projected onto a unified coordinate system, overlapping areas are fused and a panoramic map is output; when a new air-ground frame pair arrives, the panoramic map is incrementally stitched and updated.

[0094] 9) ResU-Net Semantic Segmentation: Input the panoramic map (or its sliding window submap) into the trained ResU-Net semantic segmentation network, output a pixel-level probability map of rock obstacles, and generate a rock obstacle mask.

[0095] 10) Obstacle map construction: Map the rock obstacle mask to the planning grid coordinate system to generate the obstacle grid; combine the shape of the detection vehicle body and the safety margin to expand the obstacle area, and form a cost map for planning on the basis of the obstacle grid, thus obtaining the obstacle map.

[0096] 11) Global path planning based on FMM: Construct a velocity field based on the obstacle map, and use the Fast Journey algorithm (FMM) to iteratively calculate the arrival time field; backtrack from the target point along the negative gradient direction of the arrival time field to obtain the global path, and perform sparsification and smoothing on the path to generate the global optimal path.

[0097] 12) Replanning Judgment and Closed-Loop Update: Monitor changes in the environment ahead of the probe vehicle and the map update results to determine whether the current path conflicts with new obstacles, whether there are impassable areas, or whether the tracking deviation has increased; if replanning is triggered, return to steps 4) to 11) to update the panoramic map and obstacle map and replan; if not triggered, proceed to step 14).

[0098] 13) Autonomous navigation of the probe vehicle: When the conditions for UAV collaboration are not available, the probe vehicle constructs a local environmental representation based on its own perception and performs local obstacle avoidance and navigation; when the collaborative conditions are restored, it returns to step 3) to switch to air-ground collaborative planning.

[0099] 14) Path Discretization and Tracking Control: Discretize the global optimal path into a sequence of waypoints, combine the kinematic constraints of the probe vehicle to generate executable control commands, drive the probe vehicle to track the path along the sequence of waypoints, and continuously update the vehicle status information.

[0100] 15) Target arrival determination: Determine whether the probe has reached the target point or entered the target area; if it has reached the target, the process ends; if it has not reached the target, return to step 12) and continue to execute the closed loop of "replanning determination - path tracking".

[0101] The beneficial effects of this invention are as follows: (1) In view of the large difference in perspective, significant scale changes and non-uniform illumination of Mars air-ground heterogeneous images, this invention makes targeted improvements to the cross-view registration process, introduces adaptive sampling affine scale invariant feature transform (AS-ASIFT), dynamically adjusts the sampling density of the affine parameter space according to the image difficulty, and triggers encrypted sampling when the consistency judgment is not met and is subject to budget constraints, thereby increasing the number of effective matches and geometric consistency under controlled computational cost, and improving the success rate and registration accuracy of panoramic map construction.

[0102] (2) In view of the problems of high similarity of Martian surface texture, strong shadow interference and unclear obstacle boundaries, this invention adapts the semantic recognition and result expression method in an engineering manner. Through target and shadow association constraints and safety margin processing, the visual segmentation result is stably converted into an obstacle grid that can be directly used for planning, reducing the passage risk caused by shadow misjudgment and improving the robustness and physical safety of obstacle recognition.

[0103] (3) In view of the problems of heavy computational burden, path polyline and energy consumption cost in traditional grid planning under high resolution global map, this invention introduces the passage cost that decreases with the distance to the obstacle in the cost map, and generates the path based on the wavefront propagation time field, so that the path has better continuity and smoothness under the premise of satisfying obstacle avoidance safety, while improving the efficiency of global planning and reducing the comprehensive cost of long-distance tasks.

[0104] (4) This invention constructs an air-ground collaborative closed-loop system for working conditions such as weak texture, strong shadows, large parallax between air and ground and limited communication on Mars: the AS-ASIFT algorithm optimized for large parallax and non-uniform illumination, the ResU-Net semantic segmentation network enhanced for rock targets and shadow interference, and the FMM planning algorithm combined with kinematics and safety margin constraints are linked in a scenario-based manner to realize the closed loop of link from environmental perception, obstacle recognition to path planning, thereby improving the stability, continuity and mission success rate of path planning in the complex dynamic environment of Mars.

[0105] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0106] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0107] Figure 8 A schematic diagram of an electronic device suitable for implementing embodiments of the present invention is shown.

[0108] It should be noted that, Figure 8 The electronic device 1000 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0109] like Figure 8 As shown, the electronic device 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage section 1008 into Random Access Memory (RAM) 1003. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004. Furthermore, the electronic device 1000 also includes an FPGA device and a System-on-a-Chip (SoC) device.

[0110] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0111] In particular, according to embodiments of the present invention, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0112] Specifically, the aforementioned electronic devices can be airborne intelligent electronic devices.

[0113] It should be noted that the storage medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein computer-readable program code is carried. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0116] It should be noted that, as another aspect, this application also provides a storage medium, which may be included in an electronic device or may exist independently without being assembled into the electronic device. The aforementioned storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to perform the methods described in the following embodiments. For example, the electronic device may perform... Figure 6 The steps of the method shown.

[0117] In one embodiment, this application provides a computer program product including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0118] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0119] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0120] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A space-ground collaborative path planning system for Mars exploration, characterized in that, The system includes: The multi-source sensing module is used to acquire aerial image sequences collected by UAVs, ground image sequences collected by ground exploration vehicles, and pose information. The panoramic map construction module is used to perform cross-view registration of aerial image sequences and ground image sequences using adaptive sampling affine scale-invariant feature transformation to construct a panoramic map. The obstacle recognition module is used to identify obstacles in the panoramic map using an obstacle recognition model and generate a cost map. The global path planning module is used to search for paths on the obstacle map based on the pose information of the ground exploration vehicle and to determine the driving path of the ground exploration vehicle using a fast travel algorithm. The task execution module is used to generate control commands by combining the driving path and the kinematic constraints of the probe vehicle to drive the probe vehicle.

2. The system according to claim 1, characterized in that, The panoramic map construction module includes: The feature matching unit is used to filter out pairs of aerial and ground image frames with overlapping observation areas from the aerial image sequence and the ground image sequence based on the time synchronization results; and to perform affine scale-invariant feature transformation processing based on adaptive sampling on the aerial and ground image frame pairs to obtain candidate matching point pairs. The geometric verification unit is used to verify the geometric consistency of candidate matching point pairs and determine the registration transformation model when the geometric consistency verification is passed. The panoramic stitching unit is used to map pairs of air and ground images to a unified coordinate system and stitch them together to construct a panoramic map based on a registration transformation model.

3. The system according to claim 2, characterized in that, The feature matching unit includes: Affine parameter space determination sub-unit is used to extract initial feature points from the overhead image and ground image in the air-ground image frame pair, and to calculate the feature point density index. When the feature point density index represents less image texture information or the matching difficulty is high, increase the upper limit of the tilt parameter, increase the number of tilt layers, and decrease the rotation angle step size. When the feature point density index represents rich image texture information or the matching difficulty is low, reduce the upper limit of the tilt parameter, reduce the number of tilt layers, and increase the rotation angle step size. The virtual view generation and matching subunit is used to generate a virtual view using a sample set, and to perform scale-invariant feature extraction and matching on the virtual view to obtain candidate matching point pairs.

4. The system according to claim 2, characterized in that, The geometric verification unit includes: The consistency verification subunit is used to perform geometric consistency verification on candidate matching point pairs using the nearest neighbor ratio method, eliminate erroneous candidate matching point pairs, and obtain the target matching point pairs. The matching quality index between images is calculated based on the target matching point pairs; the matching quality index includes: number of inliers and inlier ratio; The registration model determination sub-unit is used to determine the corresponding registration transformation model when the number of inliers or the inlier rate meets the preset consistency qualification judgment conditions.

5. The system according to claim 1, characterized in that, The obstacle recognition module includes: The obstacle mask generation unit is used to input the panoramic map into the obstacle recognition model to obtain the obstacle mask; wherein, the obstacle recognition model is a ResU-Net network, the encoder in the ResU-Net network adopts the ResNet residual structure to enhance the feature extraction capability, and the decoder in the ResU-Net network adopts the U-Net decoding structure to gradually restore the spatial resolution. The obstacle area expansion unit is used to extract the obstacle boundary based on the obstacle mask and expand the obstacle boundary according to a preset safety distance to obtain a safe avoidance area. The obstacle map generation unit is used to map the safe avoidance area onto a planning grid in a unified coordinate system to generate a cost map.

6. The system according to claim 1, characterized in that, The global path planning module includes: The start-end point determination unit is used to determine the starting point of the ground exploration vehicle in the obstacle map based on the pose information of the ground exploration vehicle, and to obtain the corresponding target point; The velocity field construction unit is used to construct the velocity field corresponding to the two-dimensional grid map based on the starting point, the target point, and the distribution of obstacle areas in the obstacle map. The arrival time field solution unit is used to establish the Eikonal equation satisfied by the wavefront propagation arrival time based on the velocity field, and to discretize and solve the Eikonal equation to obtain the arrival time field formed by propagation from the starting point outward. The path backtracking unit is used to backtrack from the target point to the starting point step by step along the direction of decreasing arrival time based on the arrival time field, so as to obtain the driving path of the ground exploration vehicle.

7. The system according to claim 1, characterized in that, The system also includes a path shaping and replanning module, which is used for: Monitor environmental changes and obstacle map updates of the ground exploration vehicle to determine whether the current travel path meets the path replanning trigger conditions. When the judgment result shows that the current driving path meets the path replanning trigger condition, path replanning is triggered, and the panoramic map and obstacle map are updated. When the judgment result indicates that the current travel path does not meet the path replanning trigger condition, the ground exploration vehicle is controlled to continue to perform the task according to the current travel path.

8. A method for air-ground cooperative path planning for Mars exploration, applied to the system described in any one of claims 1 to 7, characterized in that, The method includes: Acquire aerial image sequences captured by UAVs, ground image sequences captured by ground exploration vehicles, and pose information; Cross-view registration is performed on sequences of aerial and ground images to construct a panoramic map; Obstacle recognition models are used to identify obstacles in panoramic maps and generate cost maps. Based on the pose information of the ground exploration vehicle, a fast travel algorithm is used to search for paths on the obstacle map and determine the driving path of the ground exploration vehicle. Control commands are generated by combining the driving path and the kinematic constraints of the probe vehicle to drive the probe vehicle.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method of claim 8.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to claim 8.

Images