Air-ground collaborative exploration system in unknown environment

Abstract

The invention provides an air-ground collaborative exploration system in an unknown environment. The air-ground collaborative exploration system comprises an unmanned aerial vehicle and an unmanned vehicle, the unmanned aerial vehicle is provided with a downward-looking camera; the unmanned vehicle is equipped with a laser radar and a foresight camera, and is provided with a mapping positioning module, a motion resolving module, a visual target detection module, a visual target execution module, an autonomous exploration module and an unknown environment path planning module. The system has three states: an autonomous exploration state, a target planning state and a target execution state. According to whether the unmanned aerial vehicle successfully obtains the target information or not, the system state is switched in real time, and when the target is visible, short-distance path planning is performed, so that the ground exploration efficiency is greatly improved while the completion degree of the search task is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent unmanned systems, and particularly to an air-ground collaborative exploration system in an unknown environment. Background Art

[0002] With the rapid development of artificial intelligence and unmanned system technologies, unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) have been widely used in various application scenarios. UAVs are usually used for tasks such as aerial monitoring, environmental perception, and target recognition, while UGVs are good at functions such as ground patrol, material transportation, and autonomous navigation. However, in practical applications, the limitations of UAVs and UGVs are gradually emerging. Although UAVs can provide a high-altitude perspective and quickly cover a large area, they are often unable to handle complex terrains or perform fine operations. On the other hand, although UGVs have strong ground mobility and can traverse various terrains, their perception range and environmental awareness are limited by the ground perspective, making it difficult to obtain global information in a timely manner.

[0003] Most existing unmanned systems rely on a single platform and are often unable to balance the acquisition and processing of multi-dimensional information in the air and on the ground when facing complex and unknown environments. For example, in disaster relief or military reconnaissance missions, the environment is often complex and dynamically changing. Relying solely on UAVs or UGVs to operate independently, it is difficult to quickly complete target positioning, path planning, and task execution. In addition, traditional air-ground collaborative systems mainly rely on manual command and control, lacking intelligent autonomous collaboration capabilities and being difficult to meet the task requirements of high-risk and strong time constraints.

[0004] To solve the above problems, researchers have proposed some preliminary air-ground collaborative exploration solutions, attempting to provide more comprehensive environmental perception and task execution capabilities by integrating the advantages of UAVs and UGVs. However, these solutions still face many challenges in practical applications. First, communication and coordination between the air and ground platforms often require precise time synchronization and high-bandwidth communication links, otherwise it is easy to cause information delay or loss. Second, the autonomy and intelligence of air-ground collaboration are still not high enough. Many systems rely on predefined tasks and rules and are difficult to cope with dynamically changing environments. Finally, there are still bottlenecks in key technologies such as environmental perception, target recognition, and path planning in existing systems, especially when dealing with high-complexity and multi-dimensional environments, showing significant limitations.

[0005] Therefore, there is an urgent need for a more intelligent and collaborative air-ground exploration system that can give full play to the high-altitude monitoring capabilities of UAVs and the ground operation capabilities of UGVs to solve the problems of autonomous exploration and target positioning in complex environments. Such a system should not only have high autonomy and intelligence but also be able to cooperate efficiently in a multi-platform and multi-dimensional environment, thereby improving the accuracy and efficiency of task execution. Summary of the Invention

[0006] In view of this, the present invention provides an air-ground collaborative exploration system in an unknown environment, aiming to solve the challenges of autonomous exploration and target positioning in a complex unknown environment. By combining the advantages of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), the system innovatively realizes efficient and accurate environment exploration and target recognition functions.

[0007] To solve the above technical problems, the present invention is implemented as follows.

[0008] An air-ground collaborative exploration system in an unknown environment, comprising: a UAV and a UGV; there is a communication link between the UAV and the UGV;

[0009] The UAV is equipped with a downward-looking camera; the UGV is equipped with a lidar, a forward-looking camera, and has a mapping and positioning module, a motion solution module, a visible target detection module, a visible target execution module, an autonomous exploration module, and an unknown environment path planning module; the lidar collects point cloud data of the environment; the mapping and positioning module constructs a three-dimensional map based on the point cloud data and performs real-time positioning;

[0010] The system has three states: autonomous exploration state, target planning state, and target execution state;

[0011] In the autonomous exploration state: the autonomous exploration module works, explores the logical path and determines the target point according to the three-dimensional mapping; the unknown environment path planning module performs local path planning according to the target point determined by the autonomous exploration module to generate an execution path; the motion solution module calculates the control quantity according to the execution path and drives the UGV to move; at the same time, the UAV hovers at a specified height above the UGV to perform high-altitude monitoring and target recognition, and when the target information is recognized, it is sent to the UGV;

[0012] Target planning state: when the UGV receives the target information sent by the UAV, it switches to the target planning state. In an environment where the map is not fully constructed, the unknown environment path planning module performs global path planning and local path planning according to the target point from the UAV to generate an execution path; the motion solution module calculates the control quantity according to the execution path and drives the UGV to move;

[0013] Target execution state: during the movement of the UGV, the visible target detection module continuously detects the targets within the visible range through the forward-looking camera; when the target point is recognized, it switches to the target execution state, and the visible target execution module uses a position control algorithm to calculate the control quantity and drives the UGV to move and converge to the desired position.

[0014] Preferably, the priorities of the autonomous exploration state, the target planning state, and the target execution state increase gradually; a high-priority state can interrupt the execution of a low-priority state; when the high-priority task is completed, the system automatically resumes the previous state until the entire exploration task is completed.

[0015] Preferably, after the system is started, it first enters the autonomous exploration state. If the UAV obtains target information during the autonomous exploration state, the exploration is interrupted and the target planning state is entered; if the visual target detection module recognizes the target, the exploration is interrupted and the target execution state is directly entered;

[0016] In the target planning state or the autonomous exploration state, if the visual target detection module recognizes the target, the current state is interrupted and the target execution state is entered;

[0017] In the target execution state, if the convergence of the target position is completed but it is not the target point planned in this round, the target planning state is returned;

[0018] If the target point planned in this round is completed but there are still target points not completed, the autonomous exploration state is returned;

[0019] If the UAV fails to obtain target information, the unmanned vehicle continues to conduct autonomous exploration until all target tasks are completed;

[0020] When all task targets are completed, the system enters the end state.

[0021] Preferably, the UAV is a quadrotor UAV; the unmanned vehicle is a Mecanum wheel unmanned vehicle; the lidar equipped on the unmanned vehicle is a 360-degree lidar; the UAV and the unmanned vehicle communicate data through the ROS platform.

[0022] Preferably, the autonomous exploration module includes an environmental information update unit, a viewpoint generation and screening unit, a traveling salesman problem calculation unit, and a target point determination unit;

[0023] The entire exploration space is divided by grids to obtain global exploration units. The global exploration units are further divided by grids to obtain local exploration units. Multiple point clouds are included in the local exploration units; the local exploration units are determined to be in three states: unknown, occupied, and unoccupied according to the statistics of the internal point cloud states; the global exploration units are determined to be in three states: under exploration, unexplored, and explored according to the statistics of the internal local exploration unit states; the exploration window is an area composed of multiple global exploration units centered on the unmanned vehicle; the passable area is an area composed of local exploration units determined to be unoccupied; the obstacle area is an area composed of local exploration units determined to be occupied; the unknown area is an area composed of local exploration units determined to be unknown;

[0024] The environmental information update unit is used to update the environmental information according to the 3D map when the driverless vehicle crosses the central global exploration unit within the exploration window, including local exploration unit update, global exploration unit update, exploration window sliding update, passable area and obstacle area update; among them, the local exploration unit update is to update the state according to the point cloud within the unit; the global exploration unit update is to update the states of multiple global exploration units that leave after the exploration window slides, and add the global exploration units updated to the exploring state to the exploration queue;

[0025] The viewpoint generation and screening unit is used to fill the candidate viewpoints at set distance intervals in the passable area, and screen out a set of preferred viewpoints according to the observable obstacle conditions corresponding to the candidate viewpoints and the cost for the driverless vehicle to reach the candidate viewpoints;

[0026] The traveling salesman problem calculation unit is used to calculate the asymmetric traveling salesman problem of the set of preferred viewpoints to obtain the exploration order of this set of preferred viewpoints, that is, to obtain the local exploration logical path; calculate the traveling salesman problem of all the exploring global exploration units in the exploration queue to obtain the exploration order of these exploring global exploration units, that is, to obtain the global exploration logical path; merge the local exploration logical path and the global exploration logical path to obtain the overall logical path and send it to the target point determination unit;

[0027] The target point determination unit is used to determine the next target point of the driverless vehicle according to the overall logical path, and send the determined target point to the unknown environment path planning module for local path planning.

[0028] Preferably, in the viewpoint generation and screening unit, the method of screening the preferred viewpoints is as follows:

[0029] Update the parameters of all candidate viewpoints, and the parameters include the observable obstacle area Observable_n within the exploration window, the observable boundary size Frontier_n between the passable area and the unknown area within the exploration window, and the cost Inertial_n for the driverless vehicle to reach the candidate viewpoints;

[0030] The method for determining the cost Inertial_n for the driverless vehicle to reach the candidate viewpoints is:

[0031] Inertial_n = k1·orientation_from_robot + k2·distance_from_robot

[0032] + k3·orientation_from_path

[0033] Among them, orientation_from_robot is the angle between the candidate viewpoint orientation and the heading angle of the driverless vehicle; distance_from_robot is the absolute distance from the candidate viewpoint to the driverless vehicle; orientation_from_path is the angle between the candidate viewpoint orientation and the path orientation of the driverless vehicle, where the path orientation is represented by the average value of the heading angles of the driverless vehicle in the past period of time; k1, k2, and k3 are weight values;

[0034] Perform the first round of screening: If the Observable_n parameter of the candidate viewpoint is less than the area threshold, or the Frontier_n parameter is less than the boundary size threshold, the current candidate viewpoint is eliminated;

[0035] Perform the second round of screening: For the remaining candidate viewpoints, calculate the weighted sum of the three parameters, sort them, and select a set number of candidate viewpoints as the preferred viewpoints in descending order of scores.

[0036] Preferably, the target point determination unit determines the next target point of the driverless vehicle according to the overall logical path as:

[0037] Determine the head viewpoint and the tail viewpoint of the overall logical path, calculate the weighted sum of orientation_from_robot and distance_from_robot for the head viewpoint and the tail viewpoint respectively, and take the larger one as the next target point of the driverless vehicle; where orientation_from_robot is the angle between the candidate viewpoint orientation and the heading angle of the driverless vehicle; distance_from_robot is the absolute distance from the candidate viewpoint to the driverless vehicle.

[0038] Preferably, the unknown environment path planning module includes a geometric mapping unit, a global planning unit, and a local planning unit;

[0039] In the target planning state, the geometric mapping unit transmits target information based on the unmanned aerial vehicle, executes the Far-Planner algorithm, constructs a two-dimensional map based on the three-dimensional point cloud, fits the obstacles into geometric shapes, and constructs a geometric map; the global planning unit plans the global path of the driverless vehicle to the target point using the A* algorithm according to the two-dimensional map; while executing the global path, the local planning unit performs real-time local path planning using the TEB local path planning algorithm according to the global path.

[0040] In the autonomous exploration state, the local planning unit is also responsible for performing local path planning according to the target point determined by the active exploration module.

[0041] Preferably, the 3D mapping module adopts the Point-LIO laser odometry calculation method; the visual target execution module adopts the PID controller as the position control algorithm.

[0042] Preferably, the system further includes a visualization module for providing a visualization interface to display real-time mapping information, UAV information, unmanned vehicle information, target point information, and path information.

[0043] Based on the above system, the specific steps during its operation are as follows:

[0044] Step S1 Environment perception and initial exploration: After the system starts, the unmanned vehicle uses lidar for 3D mapping and real-time positioning and enters the autonomous exploration state. Meanwhile, the UAV takes off to a high point and uses a downward-looking camera to perform high-altitude monitoring and target recognition tasks.

[0045] Step S2 Target recognition and information transmission: When the UAV recognizes a specific target, such as a danger safety sign or a QR code, at a high altitude, it transmits the coordinates, dimensions, and related information of the target to the ground unmanned vehicle through the ROS platform.

[0046] Step S3 Path planning and execution: After receiving the target information from the UAV, the unmanned vehicle interrupts the autonomous exploration state, switches to the target point navigation state, performs path planning in an environment where the map is not fully built, and quickly navigates to the target point.

[0047] Step S4 Local path optimization and dynamic obstacle avoidance: While executing the global path, the unmanned vehicle uses a front-view depth camera and lidar to perceive obstacles in real time and perform local path optimization. The system generates a safe and smooth driving path through the TEB algorithm to ensure that the unmanned vehicle can avoid dynamic obstacles and accurately reach the target position.

[0048] Step S5 Unmanned vehicle target execution state: When the front-view camera equipped on the unmanned vehicle recognizes the target point, that is, the confirmation of the target is completed, the unmanned vehicle enters the target execution state. Through the target recognition algorithm, the relative position between the target point and the unmanned vehicle is obtained, and the linear velocity and angular velocity of the unmanned vehicle are controlled by PID to quickly converge to the desired position.

[0049] Step S6 State priority and task switching: The state of the unmanned vehicle consists of three modes: autonomous exploration, target point navigation, and target point execution. The system performs dynamic state switching according to the task priority, and the high-priority state can interrupt the execution of the low-priority state. When the high-priority task is completed, the system will automatically resume to the previous state until the entire exploration task is completed.

[0050] Step S7 End State: When the driverless vehicle has completed the work tasks at all target points, the system enters the end state. The system quickly plans a path in the established three-dimensional map, and the driverless vehicle and the drone respectively stay in the end area to end the task.

[0051] Beneficial Effects:

[0052] (1) Multi-platform collaborative work: The collaborative work of the drone and the driverless vehicle can achieve air-ground integrated environmental perception and exploration, improving the task execution efficiency and accuracy of the system. Moreover, when the driverless vehicle is performing the target search task, it has three states: autonomous exploration, target planning, and target execution. According to whether the drone successfully obtains the target information, the system state is switched in real time, which not only ensures the completion degree of the search task but also greatly improves the efficiency of ground exploration.

[0053] (2) Autonomous perception ability in unknown environments: The system has the ability to autonomously explore, map, and search for targets in unknown environments, and can flexibly adjust the path and avoid obstacles in a dynamically changing environment, providing multiple task possibilities and robustness for unknown scenarios.

[0054] (3) Multi-level autonomous exploration mechanism: When the system plans the logical path in the autonomous exploration module, it adopts an exploration strategy that combines global planning and local planning. It can not only determine the exploration path of the driverless vehicle from a global perspective but also make flexible adjustments in the local environment. Through the generation and optimized selection of candidate viewpoints, it ensures that the exploration path of the driverless vehicle is more reasonable and efficient, thus significantly improving the speed and accuracy of task completion. At the same time, considering that the beginning and end of the path are often similar and it is difficult to make a choice, resulting in the driverless vehicle shaking its head, the present invention uses the angle between the viewpoint azimuth and the heading angle of the driverless vehicle and the distance between the viewpoint and the driverless vehicle to characterize the characteristics of the beginning and end of the logical path, so as to reasonably select the target point.

[0055] (4) Intelligent state management: Through intelligent state priority management, the system can efficiently switch task modes in complex task scenarios, ensuring the completion degree and high efficiency of the tasks.

[0056] (5) Wide application potential: The system can be widely applied in fields such as disaster rescue, environmental monitoring, and military reconnaissance, providing strong technical support for task execution in various complex environments. Description of the Drawings

[0057] Figure 1 It is the overall structure diagram of the air-ground collaborative exploration system provided by the present invention;

[0058] Figure 2 It is the module diagram of the air-ground collaborative exploration system provided by the present invention;

[0059] Figure 3Schematic diagram of the autonomous exploration function of the unmanned vehicle system provided by the present invention;

[0060] Figure 4 Flowchart of the state transition of the unmanned vehicle system provided by the present invention;

[0061] Figure 5 Schematic diagram of the design and physical object of the unmanned vehicle hardware system provided by the present invention;

[0062] Figure 6 Schematic diagram of the unknown environment path planning module;

[0063] Figure 7 Schematic diagram of the autonomous exploration module;

[0064] Figure 8 Schematic diagram of the unmanned aerial vehicle hardware system provided by the present invention. Detailed implementation manners

[0065] The present invention will be described in detail below with reference to the accompanying drawings and by way of examples.

[0066] The present invention provides an air-ground collaborative exploration system in an unknown environment. As Figure 1 shown, the system includes an unmanned aerial vehicle and an unmanned vehicle, and there is a communication link between the unmanned aerial vehicle and the unmanned vehicle. In this embodiment, the unmanned aerial vehicle and the unmanned vehicle can communicate data through the ROS platform. In the system, a display can also be connected to display the visualization interface of the air-ground collaborative exploration process. Information such as real-time mapping information, unmanned aerial vehicle information, unmanned vehicle information, target point information, and path information can be displayed on the visualization interface. One or more displays can be set for the visualization interface.

[0067] The unmanned aerial vehicle can be a quadrotor unmanned aerial vehicle, equipped with a downward-looking camera, a target detection algorithm, and a ROS transmission node. The unmanned aerial vehicle is mainly responsible for high-altitude monitoring. It uses the downward-looking camera and the target detection algorithm to obtain and identify important information in the environment, such as specific hazard safety signs and two-dimensional codes. When the system executes a task, the unmanned aerial vehicle hovers at a specified high altitude. It can perform real-time image acquisition and processing through the equipped downward-looking camera at an altitude of about 2 meters, obtain target information under a wide field of view, and transmit its position information to the unmanned vehicle on the ground through the ROS protocol to assist the unmanned vehicle in performing the target search task on the ground. In this embodiment, the target detection algorithm uses the Yolo algorithm.

[0068] The driverless vehicle can be a Mecanum wheel driverless vehicle. The driverless vehicle is equipped with a 360-degree lidar and a forward-looking camera. The driverless vehicle has the functions of environmental perception and target detection, and is capable of performing target search and autonomous exploration tasks in an unknown environment. In the absence of target position information provided by a drone, the driverless vehicle can independently complete ground exploration tasks, realize 3D mapping and target search. In the case of obtaining target position information provided by the drone, the driverless vehicle can quickly perform target navigation, quickly plan a path to reach the target position in an unknown environment, perform 3D mapping along the way, and finally identify and confirm the target through the forward-looking camera, enabling the driverless vehicle to perform high-precision position convergence on the target after approaching the target point.

[0069] To achieve the above functions, as Figure 1 and Figure 2 shown, the driverless vehicle further includes a basic function module, an unknown environment path planning module, an active exploration module, and a visible target execution module.

[0070] Among them, the basic function module specifically includes a visible target detection module, a 3D mapping module, and a motion solution module. The lidar collects the point cloud data of the environment. The mapping and positioning module constructs a 3D map based on the point cloud data and performs real-time positioning. The visible target detection module is used to perform target recognition based on the images collected by the forward-looking camera. The motion solution module is used to calculate the control quantity according to the planned execution path to drive the driverless vehicle to move. In this embodiment, the visible target detection module adopts the Yolo algorithm, the 3D mapping module adopts the Point-LIO laser odometry calculation method, and the motion solution module adopts the movebase algorithm.

[0071] The system of the present invention designs three states, and the unknown environment path planning module, the active exploration module, and the visible target execution module correspond to the three states. These three states include the autonomous exploration state, the target planning state, and the target execution state.

[0072] In the autonomous exploration state: the autonomous exploration module works, explores the logical path according to the 3D map and determines the target point; the unknown environment path planning module performs local path planning according to the target point determined by the autonomous exploration module to generate an execution path; the motion solution module calculates the control quantity according to the execution path to drive the driverless vehicle to move; at the same time, the drone hovers at a specified height above the driverless vehicle to perform high-altitude monitoring and target recognition, and when the target information is recognized, it is sent to the driverless vehicle;

[0073] Target planning state: When the driverless vehicle receives the target information sent by the drone, it switches to the target planning state. In an environment where the map is not fully constructed, the unknown environment path planning module performs global path planning and local path planning according to the target point from the drone to generate an execution path; the motion solution module calculates the control quantity according to the execution path to drive the driverless vehicle to move;

[0074] Target execution state: During the movement of the unmanned vehicle, the visible target detection module continuously detects targets within the visible range through the front-view camera; when a target point is recognized, it switches to the target execution state, and the visible target execution module uses a position control algorithm to calculate the control quantity and drive the unmanned vehicle to converge to the desired position.

[0075] Figure 3 This is the flowchart of the air-ground collaborative exploration system provided by the present invention. After the system is started, the unmanned vehicle enters the autonomous exploration state. In the exploration state, the unmanned vehicle simultaneously executes the exploration planning algorithm and the target detection algorithm in the unknown environment, and detects the target while exploring the unknown environment; the unmanned aerial vehicle starts the reconnaissance task, that is, flies to a high altitude and uses the downward-looking camera to detect the target; when the unmanned aerial vehicle detects the position and size information of the target, it will immediately communicate with the unmanned vehicle and transmit the specific data; after receiving the target data, the unmanned vehicle interrupts the exploration state and enters the target planning state, that is, plans the global path and local path of the target point in the unknown environment; when the front-view camera of the unmanned vehicle recognizes the target, the unmanned vehicle enters the target execution state, that is, uses the PID position tracking control method to converge to the desired position to execute the task; in the target execution state, if the convergence of the target position is completed but it is not the target point planned in this round, it returns to the target planning state; if the unmanned aerial vehicle fails to obtain the target information, the unmanned vehicle continues to perform autonomous exploration until all target tasks are completed; when all task targets are completed, the system enters the end state, and the unmanned vehicle and the unmanned aerial vehicle move to the end point respectively. If there is no end point, they return to the starting point.

[0076] Figure 4 This is a schematic diagram of the state switching of the three states of the unmanned vehicle system provided by the present invention. The states of the unmanned vehicle include the exploration state, the target planning state, and the target execution state, and the priorities of the three increase gradually. The high-priority state can interrupt the execution of the low-priority state; when the high-priority task is completed, the system automatically resumes to the previous state until the entire exploration task is completed. In the exploration state, the unmanned vehicle executes 3D mapping, target detection (visible camera), and autonomous exploration tasks; in the target planning state, the unmanned vehicle executes 3D mapping, target detection, and path planning tasks; in the target execution state, the unmanned vehicle executes the position convergence task. The initial state of the unmanned vehicle is the exploration state. In the exploration state, if target information is obtained, the exploration is interrupted and the target planning state is entered. If a target is recognized, the exploration is interrupted and the target execution state is directly entered. In the target planning state, if a target is recognized, the planning is interrupted and the target execution state is entered. In the target execution state, if the convergence of the target position is completed but it is not the target point planned in this round, it returns to the target planning state; if the target point planned in this round is completed but there are still target points not completed, it returns to the exploration state; if all targets are completed, it enters the end state.

[0077] Figure 5An example of the unmanned vehicle provided by the present invention includes a Mecanum wheel chassis, an electronic control module, a control module (host computer NUC), a lidar, a depth camera, etc.

[0078] Figure 6 It is a schematic diagram of the unknown environment path planning module. The unknown environment path planning module includes a geometric mapping unit, a global planning unit, and a local planning unit;

[0079] In the target planning state, the geometric mapping unit transmits target information based on the unmanned aerial vehicle, executes the Far-Planner algorithm, constructs a two-dimensional map based on the three-dimensional point cloud, fits the obstacles into geometric shapes, and constructs a geometric map; the global planning unit plans the global path of the unmanned vehicle to the target point using the A* algorithm according to the two-dimensional map; while executing the global path, the local planning unit simultaneously performs local path planning in real time using the TEB (Time Elastic Band) local path planning algorithm according to the global path.

[0080] In the autonomous exploration state, the local planning unit is also responsible for performing local path planning according to the target point determined by the active exploration module.

[0081] Figure 7 It is a schematic diagram of the autonomous exploration module. As shown in the figure, the autonomous exploration module includes an environmental information update unit, a viewpoint generation and screening unit, a traveling salesman problem calculation unit, and a target point determination unit;

[0082] The entire exploration space obtains global exploration units through grid division, the global exploration units are further divided into local exploration units, and multiple point clouds are included in the local exploration units; the local exploration units are identified as three states: unknown, occupied, and unoccupied according to the statistical status of the internal point clouds; the global exploration units are identified as three states: under exploration, unexplored, and explored according to the statistical status of the internal local exploration units; the exploration window is an area composed of multiple global exploration units centered on the unmanned vehicle; the passable area is an area composed of local exploration units determined to be unoccupied; the obstacle area is an area composed of local exploration units determined to be occupied; the unknown area is an area composed of local exploration units determined to be unknown;

[0083] The environmental information update unit is used to update the environmental information according to the three-dimensional map when the unmanned vehicle crosses the central global exploration unit within the exploration window, including local exploration unit update, global exploration unit update, exploration window sliding update, passable area and obstacle area update; among them, the local exploration unit update is to update the status according to the point clouds within the unit; the global exploration unit update is to update the status of multiple global exploration units that leave after the exploration window slides, and add the global exploration units updated to the exploration state to the exploration queue.

[0084] A viewpoint generation and screening unit is configured to evenly cover candidate viewpoints at a set distance interval in a passable area, and screen out a set of preferred viewpoints according to the observable obstacle conditions corresponding to the candidate viewpoints and the cost for the driverless vehicle to reach the candidate viewpoints.

[0085] Among them, the method for screening preferred viewpoints is as follows:

[0086] Update the parameters of all candidate viewpoints. The parameters include the observable obstacle area Observable_n within the exploration window, the observable boundary size Frontier_n between the passable area and the unknown area within the exploration window, and the cost Inertial_n for the driverless vehicle to reach the candidate viewpoint.

[0087] The method for determining the cost Inertial_n for the driverless vehicle to reach the candidate viewpoint is as follows:

[0088] Inertial_n = k1·orientation_from_robot + k2·distance_from_robot

[0089] + k3·orientation_from_path

[0090] Among them, orientation_from_robot is the angle between the candidate viewpoint azimuth and the heading angle of the driverless vehicle; distance_from_robot is the absolute distance from the candidate viewpoint to the driverless vehicle; orientation_from_path is the angle between the candidate viewpoint azimuth and the path azimuth of the driverless vehicle, where the path azimuth is represented by the average value of the heading angles of the driverless vehicle in the past period of time; k1, k2, and k3 are weights.

[0091] Conduct the first round of screening: If the Observable_n parameter of the candidate viewpoint is less than the area threshold, or the Frontier_n parameter is less than the boundary size threshold, the current candidate viewpoint is eliminated.

[0092] Conduct the second round of screening: For the remaining candidate viewpoints, calculate the weighted sum of the three parameters, sort them, and select a set number of candidate viewpoints with the highest scores as the preferred viewpoints according to the scores from high to low.

[0093] A traveling salesman problem calculation unit is configured to calculate the asymmetric traveling salesman problem of the set of preferred viewpoints to obtain the exploration order of the set of preferred viewpoints, that is, obtain the local exploration logic path; calculate the traveling salesman problem of all global exploration units in the exploration queue during exploration to obtain the exploration order of these global exploration units during exploration, that is, obtain the global exploration logic path; merge the local exploration logic path and the global exploration logic path to obtain the overall logic path, and send it to the target point determination unit.

[0094] The target point determination unit is used to determine the next target point of the driverless vehicle according to the overall logical path, and send the determined target point to the unknown environment path planning module for local path planning. In the prior art, since the beginning and the end of the path are relatively similar, it is difficult to make a choice, resulting in the phenomenon that the driverless vehicle shakes its head. The system has improved the method for determining the target point after obtaining the logical path, and uses the angle between the viewpoint azimuth and the heading angle of the driverless vehicle, and the distance between the viewpoint and the driverless vehicle to characterize the features of the beginning and the end of the logical path, so as to reasonably select the target point. Specifically: determine the head viewpoint and the tail viewpoint of the overall logical path, calculate the weighted sum of orientation_from_robot and distance_from_robot for the head viewpoint and the tail viewpoint respectively, and take the larger one as the next target point of the driverless vehicle; where orientation_from_robot is the angle between the candidate viewpoint azimuth and the heading angle of the driverless vehicle; distance_from_robot is the absolute distance from the candidate viewpoint to the driverless vehicle.

[0095] The visible target execution module uses a position control algorithm to calculate the control quantity and drive the driverless vehicle to move and converge to the desired position. In this embodiment, the visible target execution module uses a PID controller as the position control algorithm.

[0096] Figure 8 This is an example of the drone hardware system provided by the present invention.

[0097] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in this description can be different and are not limited. Therefore, those skilled in the art of the present invention can modify or equivalently replace the technical solutions recorded in the foregoing embodiments; and these modifications and replacements do not depart from the spirit and technical solutions of the present invention, and shall all fall within the protection scope of the present invention.

Claims

1. An air-ground collaborative exploration system in an unknown environment, characterized in that, Including: Unmanned aerial vehicles (UAVs) and unmanned vehicles; There is a communication link between the UAV and the unmanned vehicle; The UAV is equipped with a downward-looking camera; The unmanned vehicle is equipped with a lidar, a forward-looking camera, and has a mapping and localization module, a motion solution module, a visible target detection module, a visible target execution module, an autonomous exploration module, and an unknown environment path planning module; The lidar collects the point cloud data of the environment; The mapping and localization module constructs a three-dimensional map based on the point cloud data and performs real-time localization; This system has three states: autonomous exploration state, target planning state, and target execution state; In the autonomous exploration state: The autonomous exploration module works, explores the logical path according to the three-dimensional mapping, and determines the target point; The unknown environment path planning module performs local path planning according to the target point determined by the autonomous exploration module to generate an execution path; The motion solution module calculates the control quantity according to the execution path and drives the unmanned vehicle to move; At the same time, the UAV hovers at a specified height above the unmanned vehicle to perform high-altitude monitoring and target recognition, and when the target information is recognized, it is sent to the unmanned vehicle; Target planning state: When the unmanned vehicle receives the target information sent by the UAV, it switches to the target planning state. In an environment where the map is not fully constructed, the unknown environment path planning module performs global path planning and local path planning according to the target point from the UAV to generate an execution path; The motion solution module calculates the control quantity according to the execution path and drives the unmanned vehicle to move; Target execution state: During the movement of the unmanned vehicle, the visible target detection module continuously detects the targets within the visible range through the forward-looking camera; When the target point is recognized, it switches to the target execution state, and the visible target execution module uses the position control algorithm to calculate the control quantity and drives the unmanned vehicle to move and converge to the desired position.

2. The air-ground collaborative exploration system in an unknown environment according to claim 1, characterized in that The autonomous exploration state, the target planning state, and the target execution state have gradually increasing priorities; The high-priority state can interrupt the execution of the low-priority state; When the high-priority task is completed, the system automatically resumes to the previous state until the entire exploration task is completed.

3. The aerial-ground collaborative exploration system in an unknown environment according to claim 2, characterized in that After the system is started, it first enters the autonomous exploration state. In the autonomous exploration state, if the UAV obtains the target information, the exploration is interrupted and it enters the target planning state; If the visible target detection module recognizes a target, the exploration is interrupted and it directly enters the target execution state; In the target planning state or the autonomous exploration state, if the visible target detection module recognizes a target, the current state is interrupted and it enters the target execution state; In the target execution state, if the convergence of the target position is completed but it is not the target point planned in this round, it returns to the target planning state; If the target point planned in this round is completed but there are still target points not completed, it returns to the autonomous exploration state; If the UAV fails to obtain the target information, the unmanned vehicle continues to perform autonomous exploration until all target tasks are completed; When all task targets are completed, the system enters the end state.

4. The air-ground collaborative exploration system in an unknown environment according to claim 1, characterized in that, The UAV is a quadrotor UAV; The unmanned vehicle is a Mecanum wheel unmanned vehicle; The lidar equipped on the unmanned vehicle is a 360-degree lidar; The UAV and the unmanned vehicle perform data communication through the ROS platform.

5. The air-ground collaborative exploration system in an unknown environment as claimed in claim 1, characterized in that: The autonomous exploration module includes an environmental information update unit, a viewpoint generation and screening unit, a traveling salesman problem calculation unit, and a target point determination unit; The entire exploration space obtains global exploration units through grid division. The global exploration units are further divided by grids to obtain local exploration units, and multiple point clouds are included in the local exploration units. The local exploration units are identified as three states: unknown, occupied, and unoccupied according to the statistics of the internal point cloud states. The global exploration units are identified as three states: being explored, unexplored, and explored according to the state statistics of the internal local exploration units. The exploration window is an area composed of multiple global exploration units centered on the unmanned vehicle. The passable area is an area composed of local exploration units determined to be unoccupied. The obstacle area is an area composed of local exploration units determined to be occupied. The unknown area is an area composed of local exploration units determined to be unknown; The environmental information update unit is used to update the environmental information according to the 3D map when the unmanned vehicle crosses the central global exploration unit within the exploration window, including local exploration unit update, global exploration unit update, exploration window sliding update, passable area and obstacle area update. Among them, the local exploration unit update is to update the state according to the point cloud within the unit. The global exploration unit update is to update the states of multiple global exploration units that leave after the exploration window slides, and add the global exploration units updated to the being explored state to the exploration queue; The viewpoint generation and screening unit is used to fill the candidate viewpoints at a set distance interval in the passable area, and screen out a set of optimal viewpoints according to the observable obstacle conditions corresponding to the candidate viewpoints and the cost for the unmanned vehicle to reach the candidate viewpoints; The traveling salesman problem calculation unit is used to calculate the asymmetric traveling salesman problem of the set of optimal viewpoints, obtain the exploration order of the set of optimal viewpoints, that is, obtain the local exploration logical path; calculate the traveling salesman problem of all the global exploration units being explored in the exploration queue, obtain the exploration order of these global exploration units being explored, that is, obtain the global exploration logical path; merge the local exploration logical path and the global exploration logical path to get the overall logical path, and send it to the target point determination unit; The target point determination unit is used to determine the next target point of the unmanned vehicle according to the overall logical path, and send the determined target point to the unknown environment path planning module for local path planning.

6. The air-ground collaborative exploration system in an unknown environment according to claim 5, wherein In the viewpoint generation and screening unit, the method for screening the optimal viewpoints is as follows: Update the parameters of all candidate viewpoints. The parameters include the observable obstacle area Observable_n within the exploration window, the observable boundary size Frontier_n between the passable area and the unknown area within the exploration window, and the cost Inertial_n for the unmanned vehicle to reach the candidate viewpoint; The determination method of the cost Inertial_n for the unmanned vehicle to reach the candidate viewpoint is as follows: Inertial_n = k1·orientation_from_robot + k2·distance_from_robot + k3·orientation_from_path Among them, orientation_from_robot is the angle between the candidate viewpoint orientation and the heading angle of the driverless vehicle; distance_from_robot is the absolute distance from the candidate viewpoint to the driverless vehicle; orientation_from_path is the angle between the candidate viewpoint orientation and the path orientation of the driverless vehicle, where the path orientation is represented by the average value of the heading angles of the driverless vehicle in the past period of time; k1, k2, and k3 are weight values; Perform the first round of screening: If the Observable_n parameter of the candidate viewpoint is less than the area threshold, or the Frontier_n parameter is less than the boundary size threshold, the current candidate viewpoint is eliminated; Perform the second round of screening: For the remaining candidate viewpoints, calculate the weighted sum of the three parameters, sort them, and select a set number of candidate viewpoints with the highest scores as the preferred viewpoints.

7. The air-ground collaborative exploration system in an unknown environment as claimed in claim 6, characterized in that: The target point determination unit determines the next target point of the driverless vehicle according to the overall logical path as follows: Determine the head viewpoint and the tail viewpoint of the overall logical path, calculate the weighted sum of orientation_from_robot and distance_from_robot for the head viewpoint and the tail viewpoint respectively, and select the larger one as the next target point of the driverless vehicle; Among them, orientation_from_robot is the angle between the candidate viewpoint orientation and the heading angle of the driverless vehicle; distance_from_robot is the absolute distance from the candidate viewpoint to the driverless vehicle.

8. The collaborative air-ground exploration system in an unknown environment according to claim 1, wherein, The unknown environment path planning module includes a geometric mapping unit, a global planning unit, and a local planning unit; In the target planning state, the geometric mapping unit transmits target information based on the drone, executes the Far-Planner algorithm, constructs a 2D map based on the 3D point cloud, fits the obstacles into geometric shapes, and constructs a geometric map; the global planning unit plans the global path of the driverless vehicle to the target point using the A* algorithm according to the 2D map; while executing the global path, the local planning unit performs real-time local path planning using the TEB local path planning algorithm according to the global path. In the autonomous exploration state, the local planning unit is also responsible for performing local path planning according to the target point determined by the active exploration module.

9. The air-ground collaborative exploration system in an unknown environment as claimed in claim 1, characterized in that: The 3D mapping module uses the Point-LIO laser odometry calculation method; the visible target execution module uses the PID controller as the position control algorithm.

10. The air-ground collaborative exploration system in an unknown environment as claimed in claim 1, characterized in that: The system further includes a visualization module for providing a visualization interface to display real-time mapping information, drone information, driverless vehicle information, target point information, and path information.

Images