A mission-driven autonomous search method for UAVs

Abstract

The present invention relates to a task-driven autonomous search method for unmanned aerial vehicles (UAVs), comprising: obtaining the UAV's odometer using a binocular RGB image and IMU data acquired by the UAV; obtaining three-dimensional point cloud data in a world coordinate system using a depth map acquired by the UAV and the UAV's odometer, and establishing a probabilistic voxel map through voxelization; performing target detection on the binocular RGB image using a target detection network and obtaining the target pose; establishing and updating a mission scenario map using targets with detection confidence exceeding an upper confidence limit and the probabilistic voxel map; establishing spatial boundaries and potential target boundaries; establishing a UAV search strategy using the spatial boundaries and potential target boundaries and determining a search viewpoint; and autonomously and safely navigating the UAV using the UAV's odometer and search viewpoint. The present invention addresses the difficult problem of fully exploring the mission space while simultaneously detecting mission targets with high precision in communication-denied, unfamiliar, and unknown scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous search for unmanned aerial vehicles (UAVs), and in particular to a task-driven autonomous search method for UAVs. Background Art

[0002] Drones are gaining application in complex scenarios such as field rescue, tunnel inspections, and mine exploration. However, due to the lack of prior maps of the scene and the inability to use GPS, manually operating drones to complete these tasks is extremely difficult. With the rapid advancement of sensor manufacturing and artificial intelligence algorithms, the use of drones to autonomously perform search missions is becoming a future trend. Vision-based positioning and mapping technologies play a key role in autonomous drone search missions, providing the drone's own position and scene map information required for navigation. Object detection technology can be used to search for objects of interest, achieving both detection and localization.

[0003] Autonomous drone search missions require not only rapid exploration of unknown spaces but also accurate detection of mission targets. Published patents and literature on autonomous exploration and search typically establish the area between known and unknown spaces as a common boundary, and the target of interest as a semantic boundary. Exploration is then performed using greedy or hierarchical planning strategies based on these boundaries. These strategies prioritize exploration efficiency and coverage without taking target detection accuracy into account, leading to false or missed detections of mission targets. This limits the practical application of autonomous drone search. Summary of the Invention

[0004] The present invention provides a task-driven UAV autonomous search method to solve the difficult problem of fully exploring the task space and detecting the task target with high precision in communication-denied, unfamiliar and unknown scenarios.

[0005] The technical solution adopted by the present invention to solve the technical problem is to provide a task-driven UAV autonomous search method, comprising the following steps:

[0006] S1, obtains the binocular RGB image, depth map and IMU data obtained by the drone during flight;

[0007] S2, obtaining the drone odometer using the binocular RGB image and IMU data;

[0008] S3, using the depth map and the drone odometer to obtain three-dimensional point cloud data in a world coordinate system, and establishing a probabilistic voxel map through voxelization processing;

[0009] S4, using a target detection network to perform target detection on the binocular RGB image, and calculating a target pixel coordinate set and a pose of the target in a world coordinate system for the detected target to obtain a target pose;

[0010] S5, using the targets whose detection confidence exceeds the upper confidence limit to fuse target information with the probabilistic voxel map, and updating the voxel occupancy probability through Bayesian probability estimation, to complete the establishment and update of the task scene map;

[0011] S6, clustering the regions between the explored free space and its neighboring unknown space in the task scene map using a region growing algorithm to establish a spatial boundary;

[0012] S7, taking the targets whose detection confidence is between the lower confidence limit and the upper confidence limit as potential targets, and establishing the potential target boundary;

[0013] S8, establishing a search strategy for the UAV using the spatial boundary and the potential target boundary, and determining a search viewpoint;

[0014] S9, using the drone odometer and the search viewpoint to perform autonomous and safe navigation of the drone.

[0015] The target pixel coordinate set calculation in step S4 specifically includes:

[0016] (1) Set the lower confidence limit and upper confidence limit;

[0017] (2) obtaining a target detection bounding box whose detection confidence exceeds the lower limit of the confidence level, and calculating a pixel grayscale histogram within the target detection bounding box to obtain a pixel intensity distribution;

[0018] (3) Calculating the center of gravity of the target in the target detection bounding box using a weighted average method based on the pixel intensity distribution;

[0019] (4) The center of gravity is set as a threshold for distinguishing target pixels and background pixels in the target detection bounding box, and pixel coordinates greater than the threshold are established as a target pixel coordinate set.

[0020] The pose calculation of the target in the world coordinate system in step S4 specifically includes:

[0021] (1) Obtain a depth map corresponding to the binocular RGB image and perform scale alignment;

[0022] (2) extracting a depth value set corresponding to the target pixel coordinate set from the aligned depth map, and removing noise by median filtering to obtain a target point cloud;

[0023] (3) obtaining the point cloud centroid by calculating the average value of all point coordinates in the target point cloud, and using the point cloud centroid as the target position in the camera coordinate system;

[0024] (4) Finding the neighborhood points of each point in the target point cloud, performing plane fitting based on the neighborhood points, obtaining the normal vector of the point cloud plane, and converting the normal vector of the point cloud plane from the camera coordinate system to the world coordinate system.

[0025] In step S5, the target information is fused using the target whose detection confidence exceeds the confidence upper limit and the probabilistic voxel map. Specifically, the target point cloud of the target whose detection confidence exceeds the confidence upper limit is mapped to the probabilistic voxel map, and the target category and confidence information are added to the corresponding voxels, and different voxel colors are used to represent targets of different categories; the target posture information is stored in the probabilistic voxel map, and the position is marked with a number at the center of the target voxel, and the posture is marked with an arrow.

[0026] The step S5 of updating the voxel occupancy probability by Bayesian probability estimation is specifically as follows: using the Bayesian probability estimation method to fuse the detection confidence and the prior occupancy probability of the probability voxel map to obtain a posterior occupancy probability.

[0027] The search strategy of the UAV in step S8 includes a global coverage planning part and a local viewpoint optimization part, wherein the global coverage planning part uses the center of each spatial boundary and the center of the potential target boundary as the path point to guide the exploration, and roughly plans a path covering each spatial boundary and the potential target boundary; the local viewpoint optimization part randomly samples the spatial boundaries within the optimization range in their sampling space, and uses the utility function to sort the sampling viewpoints of each spatial boundary and retain the sampling viewpoints with the highest utility value, establishes the retained sampling viewpoints as graph nodes, and constructs a graph search algorithm through a graph search method to determine the local optimization path, and each viewpoint in the local optimization path is the search viewpoint.

[0028] The global coverage planning part specifically includes: using the A* algorithm to calculate the traversable path from the current position of the drone to the position of the center of each spatial boundary and the position of the center of each potential target boundary, using the time cost and flight continuity cost of reaching the center of each spatial boundary and the center of each potential target boundary as the search cost, converting the traversal problem of each spatial boundary and potential target boundary into an asymmetric traveling salesman problem, and solving it to obtain the traversal order of each spatial boundary and potential target boundary.

[0029] The utility function used in the local viewpoint optimization part is a function that is compatible with exploration coverage and target detection confidence, wherein the exploration coverage is represented by the ratio of the coverable volume to the boundary volume; and the target detection confidence is represented by the detection distance and the detection direction.

[0030] The technical solution adopted by the present invention to solve its technical problem is: to provide an electronic device, including a memory, a processor and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the steps of the above-mentioned task-driven drone autonomous search method are implemented.

[0031] The technical solution adopted by the present invention to solve its technical problem is: providing a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned task-driven drone autonomous search method are implemented.

[0032] Beneficial effects

[0033] Due to the adoption of the above-mentioned technical solution, the present invention offers the following advantages and positive effects compared to existing technologies: First, the present invention utilizes a vision-based positioning and mapping algorithm to localize the drone and construct a local scene map. This vision-based positioning method can solve positioning problems in GPS-denied conditions, expanding the application scenarios of drones, particularly in scenarios with terrain obstruction, building occlusion, and when base stations are too far away. Next, a mission scene map is constructed based on a probabilistic voxel map, lightweight target detection, and an efficient pose calculation process, achieving the establishment of spatial boundaries and potential target boundaries. Furthermore, these spatial boundaries and potential target boundaries are used to design the drone's search strategy, including guiding the drone to fully cover the mission scene through global coverage planning, determining a search destination that balances coverage, detection confidence, and search efficiency through local viewpoint optimization, and finally, enabling safe drone search using an autonomous navigation algorithm based on the drone's odometry and search destination. This process is repeated until the search boundary disappears within a preset range. The present invention is particularly suitable for mission scenarios such as communication denial, unfamiliar and unknown field rescue, tunnel inspection, and mine exploration, meeting the requirements of fully exploring the mission space and detecting mission targets with high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a system block diagram of a task-driven autonomous search method for a UAV according to a first embodiment of the present invention;

[0035] Figure 2 is a flow chart of a task-driven autonomous search method for a UAV according to a first embodiment of the present invention;

[0036] Figure 3 is a flow chart for obtaining a target pose in the first embodiment of the present invention;

[0037] Figure 4 Schematic diagram of the drone search strategy in the first embodiment of the present invention. DETAILED DESCRIPTION

[0038] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0039] The first embodiment of the present invention relates to a task-driven autonomous search method for a UAV, which is based on the following Figure 1 The system architecture shown mainly includes sensor data acquisition and preprocessing part, odometer calculation part, mapping and search boundary establishment part, search strategy establishment part, autonomous navigation planning part and flight control part. The sensor data acquisition and preprocessing part mainly collects binocular RGB images, depth maps and IMU data, and performs denoising and downsampling processing; the odometer part uses binocular RGB images and IMU data to realize odometer calculation; the mapping and search boundary establishment part first obtains point cloud data with the odometer and preprocessed depth map as input and establishes a probabilistic voxel map through voxelization, and then establishes the area with known probability and unknown probability around it as the spatial boundary. At the same time, the target detection algorithm is used to detect the target of interest required by the task and calculate the pose. The target that meets the detection confidence requirement is directly marked in the probabilistic voxel map, otherwise it is established as a potential target boundary and needs to be further processed. One-step fine detection; the search strategy establishment part includes global coverage planning and local viewpoint optimization. In global coverage planning, the UAV is considered to fully cover all boundaries and a coarse-grained path is planned to obtain the traversal order of all boundaries. In local viewpoint optimization, the coverage rate and detection confidence are taken into account to sample the viewpoints within the optimization range. Then, the optimal local path is determined by the graph search algorithm with the goal of improving search efficiency. The local path point is the search viewpoint. The autonomous navigation part uses the determined search viewpoint as the navigation destination and obtains control instructions through the path search and trajectory optimization algorithm. The flight control part uses the control instructions as input and obtains the control signals of each rotor motor through the controller solution. During the flight of the UAV, the sensor data, map, search boundary and search viewpoint are dynamically updated. The above process is repeated until the search boundary no longer exists within the preset range.

[0040] like Figure 2 As shown, the task-driven autonomous search method for a UAV of this embodiment includes:

[0041] S1: Use the Intel RealSense D435i camera to collect binocular RGB images, depth maps, and IMU data, and transmit them in real time to the drone host for data preprocessing, including converting the binocular RGB images into grayscale images and filtering out depth and IMU noise.

[0042] S2: Using the binocular RGB image and IMU data preprocessed in S1, this implementation uses the VINS-Fusion algorithm to obtain the drone's odometry. To simplify calculations, the origin of the odometry coordinate system is assumed to be the same as the origin of the world coordinate system. Therefore, the drone's odometry can be used to represent the drone's pose in the world coordinate system.

[0043] S3: The depth map pre-processed in S1 is used to obtain a 3D scene point cloud in the drone's camera coordinate system through back-projection calculation. Combined with the drone's odometry in S2, the coordinate system pose transformation is used to obtain a 3D point cloud in the world coordinate system. To improve the quality of the point cloud, a radius filtering algorithm can be used to remove point cloud noise, and a downsampling algorithm can be used to reduce the number of point clouds to improve real-time mapping. The space is divided into a series of voxel units through the voxelization process, and each voxel unit represents a fixed-size 3D spatial block. The point cloud data is mapped to a voxel grid. For each voxel, statistics are counted to determine whether the point cloud point falls within the voxel. The final occupancy state of the voxel, including occupied, free, and unknown, is calculated based on the point occupancy probability after multiple observations, thereby establishing a probabilistic voxel map.

[0044] S4: In order to reduce power consumption and computing power, this embodiment uses a lightweight Yolo-based improved target detection network to perform target detection on the grayscale image preprocessed in S1, and obtains the target pose through efficient target pixel coordinate set calculation and target pose calculation in the world coordinate system.

[0045] like Figure 3 As shown, in the target pixel coordinate set calculation process, the confidence interval [conf l ,conf h ](To avoid missed detection, set the confidence lower limit conf l is a smaller value, which is set to 0.3 in this embodiment); then, the detection confidence exceeds the confidence lower limit conf l The target detection bounding box is obtained by using OpenCV to calculate the pixel grayscale histogram of the target detection bounding box to obtain the pixel intensity distribution. To determine the target area relatively accurately, the center of gravity of the target in the target detection bounding box is calculated using the weighted average method based on the pixel intensity distribution obtained from the pixel grayscale histogram, that is, the main grayscale value C. The calculation method is:

[0046]

[0047] Where i is the grayscale value, ranging from 0 to 255; H(i) is the number of pixels corresponding to the grayscale value, and N is the number of grayscale levels, which is 256.

[0048] The main grayscale value C is set as the threshold to distinguish the target pixels and background pixels in the bounding box, and the pixel coordinates greater than the threshold are established as the target pixel coordinate set S, which is expressed as:

[0049] S={(x,y)|I(x,y)>C};

[0050] Where I(x,y) is the pixel in the target detection bounding box that meets the threshold requirement, and (x,y) is the corresponding pixel coordinate.

[0051] like Figure 3 As shown in the figure, the target pose calculation process in the world coordinate system includes extracting the depth set corresponding to the target pixels and calculating the target point cloud, calculating the target position and orientation in the camera coordinate system, and calculating the target position and orientation in the world coordinate system, and finally obtaining the target pose in the world coordinate system. The details are as follows:

[0052] (1) Obtain a depth map corresponding to the binocular RGB image and perform scale alignment;

[0053] (2) Extract the depth value set corresponding to the target pixel coordinate set from the aligned depth map, remove the noise through median filtering, and then calculate the target point cloud. The point cloud position calculation process in the camera coordinate system is:

[0054] Assume that the number of target pixels after filtering is n, and the corresponding depth set is {z1, z2, z3, ..., z n}, with a certain depth z i For example, using the camera internal parameters including focal length f x and f y And the offset c of the camera optical axis in the image coordinate system x and c y The position P in the camera coordinate system is obtained by back-projection calculation i , its coordinate value (X i ,Y i ,Z i ) is:

[0055]

[0056] Using the above formula, the point cloud of the target in the camera coordinate system can be obtained as {P1, P2, P3, ..., P i ,...,P n};

[0057] (3) The point cloud center of gravity is obtained by calculating the average value of all point coordinates in the target point cloud and taking it as the target position in the camera coordinate system. c ,Y c ,Z c) is calculated as:

[0058]

[0059] Where m is the total number of point clouds. Further, according to the rigid body transformation between the camera and the drone and drone odometer The position P of the target in the world coordinate system can be calculated w , the expression is:

[0060]

[0061] The first matrix on the right side of the equation converts the four-dimensional vector into a three-dimensional position vector. The homogeneous transformation matrix representing the position and orientation of the drone in the world coordinate system, Represents the fixed transformation matrix between the drone body and the camera;

[0062] (4) Find the neighboring points for each point in the target point cloud, and perform plane fitting based on these points to obtain the normal vector of the point cloud plane, that is, the direction n of the target in the camera coordinate system c =(n x ,n y ,n z ). To transform the target orientation from the camera coordinate system to the world coordinate system, first convert the orientation vector into a homogeneous direction vector V c =[n c ,0] T , and then according to the transformation matrix from the camera to the drone body And the transformation matrix from the drone body to the world coordinate system Calculate the homogeneous direction vector V of the target in the world coordinate system w , the expression is:

[0063]

[0064] According to the above formula, the direction of the target in the world coordinate system can be calculated as n w =(V w [0],V w [1],V w [2]).

[0065] S5: Using the probability voxel map in S3 and the detection confidence in S4 to exceed the confidence limit conf h The goal is to establish and update the task scene map, including target information fusion and occupancy probability update, that is, the probability voxel map in S3 and the detection confidence in S4 exceed the confidence limit conf h (To improve detection accuracy, set the confidence limit confh The target is fused with the voxel occupancy probability of 0.9 and updated through Bayesian probability estimation. At the same time, as the drone's onboard sensor data changes, the mission scene map is updated using the latest probabilistic voxel map and target detection results.

[0066] In the process of target information fusion in this step, the detection confidence exceeds the confidence limit conf h The target point cloud of the target is mapped to the probabilistic voxel map in S3, and the target category and confidence information are added to the corresponding voxels. Different voxel colors are used to represent different categories of targets. At the same time, the target pose information is stored in the voxel map, and the position is marked with a number at the center of the target voxel and the pose is marked with an arrow.

[0067] In this step, the voxel occupancy probability is updated by Bayesian probability estimation. The Bayesian probability estimation method is used to fuse the detection confidence and the prior occupancy probability of the voxel map to obtain a new, more accurate posterior occupancy probability. The expression of the updated occupancy probability P(Occ|Det) is:

[0068]

[0069] Among them, P(Det|Occ) is the probability of detecting an object under the premise that the voxel is occupied, which is given by the object detection confidence; P(Occ) is the prior probability that the voxel is in an occupied state, that is, the initial occupancy probability of the voxel before the object is detected; P(Det) is the total probability of the detection result, which represents the probability of detecting the object in all cases.

[0070] S6: Use a region growing algorithm to cluster the areas (non-target areas) between the explored free space and its neighboring unknown space in the mission scenario map to establish spatial boundaries. At the same time, after the mission scenario map is updated with the latest sensor data in S5, the spatial boundaries also need to be updated. Spatial boundaries where the occupancy status has changed (unknown space is determined to be free or occupied) are removed, and new spatial boundaries are searched for. Principal component analysis is used to divide large-scale boundary clusters into two uniform boundaries to ensure that the drone can fully cover the spatial boundaries during the search process.

[0071] S7: Set the detection confidence level in [conf l ,conf h] are regarded as potential targets and require further fine detection, so they are established as potential target boundaries. The boundary information includes the clustering of the target point cloud after voxelization (the clustering process is similar to the process used in S6, but only the target voxels are clustered), target pose, target category and confidence, and the confidence of the potential target boundary is updated as the drone autonomously searches. Targets that meet the detection confidence requirements (the average confidence value of multiple detections is ≥0.8) during the search process are moved out of the potential target boundary and integrated into the mission scene map in S5; targets that do not meet the confidence requirements are removed from the potential target boundary set, and the related point clouds are ordinary occupied voxels. At the same time, the objects newly detected by the onboard camera continue to be processed by S4 to supplement the potential target boundary set.

[0072] S8: Using the spatial boundary in S6 and the potential target boundary in S7 to establish a UAV search strategy, the UAV search strategy in this embodiment mainly includes a global coverage planning part and a local viewpoint optimization part, such as Figure 4 shown.

[0073] In the global coverage planning phase, a global path traversing each boundary is planned to guide the drone's search of unknown areas and avoid being trapped in the detailed detection of potential targets. To efficiently adapt to changing scenarios, a coarse-grained global coverage path is planned, aiming to guide the drone to areas near the boundary. The boundary traversal and coverage problem is transformed into an asymmetric traveling salesman problem, where the flight time and flight continuity required to reach each boundary from the drone's current position are used as search costs. The search order for each boundary is determined by solving the asymmetric traveling salesman problem.

[0074] In this implementation, the global coverage planning part uses the center of each boundary as a path point to guide exploration, and roughly plans a path covering each space boundary and potential target boundary (see Figure 4 ). In the global coverage path planning, different boundaries are not treated differently, which is conducive to efficiently guiding the drone to cover all boundaries. Specifically, the A* algorithm is used to calculate the pose p from the drone's current pose p0 to the center of each boundary (including the center of the space boundary and the center of the potential target boundary). k The traversable path between the two boundaries is calculated, and the time cost to reach the center of each boundary and the flight continuity cost are used as the search cost, which is expressed as:

[0075] M tsp (0,k)=t(x0,x k )+w c c c (x k,1 );

[0076] where t(x0,x k) is the boundary x0 and the boundary x k The time cost between the two boundaries is determined by respectively calculating the ratio of the path length between the two boundaries to the maximum linear velocity, and the ratio of the rotation angle to the maximum angular velocity, and taking the maximum value; k is the boundary number, satisfying k∈{1,2,…,N cls}; Flight continuity cost c c The expression is:

[0077]

[0078] Then, the traversal problem of each boundary is transformed into an asymmetric traveling salesman problem and solved using the Lin-Hernighan heuristic algorithm to obtain the traversal order of each boundary.

[0079] In local viewpoint optimization, spatial boundaries within the optimization range are randomly sampled within their sampling space. A utility function that considers both coverage and detection confidence is used to rank the sampled viewpoints for each boundary, retaining the top three with the highest utility values. For potential target boundaries, sampling is performed within half of the sampling space to achieve refined detection. The sampled viewpoints are mirrored relative to the normal vector of the potential target boundary, thereby increasing the number of detection viewpoints to guide the drone for refined detection. The retained sampled viewpoints are established as graph nodes, each connected to the graph node of the next boundary via directed edges (target detection viewpoints are directly connected to their mirrored points), forming a directed acyclic graph. The cost between graph nodes is the same as the search cost in global coverage planning. The Dijkstra algorithm is used to search for the local optimization path with the lowest cost. Each viewpoint in the local optimization path is the actual search destination (i.e., search viewpoint) during the drone's autonomous navigation.

[0080] The utility function used in the local viewpoint optimization part is compatible with the exploration coverage and target detection confidence. At the boundary k, viewpoint j k The utility function expression is:

[0081] U(k,j k )=λ1I(k,j k )+λ2D(k,j k );

[0082] Where λ1 and λ2 are the exploration space coverage rates I(k,j k ) and target detection confidence D(k,j k ) is the weight factor of I(k,j k ) through the coverable volume V cover (k,j k ) and the boundary volume V frontier (k) is expressed as:

[0083]

[0084] Among them, V cover (k,j k ) is calculated as follows: in the sampling direction within the camera field of view, from viewpoint j k Extend to the maximum perception depth of the sensor to form a truncated ray, and count the number of unknown voxels intersecting with the truncated ray to calculate V cover (k,j k ); V can be calculated by counting the number of unknown voxels in the process of boundary establishment frontier (k).

[0085] D(k,j k ) is related to the detection distance and detection direction. The farther the camera is from the target, the weaker the target features that can be captured, and the detection confidence will decrease. When the target is facing the camera directly, the features are most obvious and the confidence is the highest. As the target moves away from the camera, the confidence will continue to decrease. A joint model of the impact of detection distance and detection direction on confidence can be established as:

[0086]

[0087] Where D0 is the initial confidence of the target at close range; α is the coefficient that controls the speed of confidence decrease; d is the viewpoint j k The Euclidean distance to the detection target; To detect the target and viewpoint j k The angle between the camera optical axes can be calculated based on the orientation of the target in the camera coordinate system.

[0088] S9: Using the drone odometer in S2 and the search viewpoint determined in S8, the drone performs autonomous and safe navigation, which mainly includes two parts: path search and trajectory optimization. * The path search algorithm plans a safe and collision-free path from the current position of the UAV to the search viewpoint; then, considering the flight smoothness and motion constraints, the planned path is smoothly optimized using the trajectory optimization algorithm based on B-spline curves.

[0089] It is not difficult to find that the present invention first uses a vision-based positioning and mapping algorithm to realize the positioning of the drone and the construction of a local scene map. The vision-based positioning method can solve the positioning problem under GPS denial conditions and expand the application scenarios of drones, especially scenarios with terrain obstruction, building obstruction, and base stations that are too far away; then, based on the probabilistic voxel map, lightweight target detection, and efficient pose calculation process, a mission scene map is established, and the establishment of spatial boundaries and potential target boundaries is realized; then, the spatial boundaries and potential target boundaries are used to design the drone search strategy, including guiding the drone to completely cover the mission scene through global coverage planning, and determining the search destination that takes into account coverage, detection confidence, and search efficiency through local viewpoint optimization; finally, the drone's safe search is realized through an autonomous navigation algorithm based on the drone's odometer and search destination, and the above process is repeated until there is no search boundary within the preset range. The present invention is particularly suitable for mission scenarios such as communication denial, unfamiliar and unknown field rescue, tunnel inspection, and mine exploration, and meets the requirements of complete exploration of mission space and high-precision detection of mission targets.

[0090] A second embodiment of the present invention relates to an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the task-driven drone autonomous search method of the first embodiment are implemented.

[0091] A third embodiment of the present invention relates to a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the task-driven autonomous search method for a drone of the first embodiment are implemented.

[0092] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0093] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction method, which is implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A task-driven autonomous search method for UAVs, characterized in that: The following steps are involved: S1, obtains the binocular RGB image, depth map and IMU data obtained by the drone during flight; S2, obtaining the drone odometer using the binocular RGB image and IMU data; S3, using the depth map and the drone odometer to obtain three-dimensional point cloud data in a world coordinate system, and establishing a probabilistic voxel map through voxelization processing; S4, using a target detection network to perform target detection on the binocular RGB image, and calculating a target pixel coordinate set and a pose of the target in a world coordinate system for the detected target to obtain a target pose; S5, using the targets whose detection confidence exceeds the upper confidence limit to fuse target information with the probabilistic voxel map, and updating the voxel occupancy probability through Bayesian probability estimation, to complete the establishment and update of the task scene map; S6, clustering the regions between the explored free space and its neighboring unknown space in the task scene map using a region growing algorithm to establish a spatial boundary; S7, taking the targets whose detection confidence is between the lower confidence limit and the upper confidence limit as potential targets, and establishing the potential target boundary; S8, establishing a search strategy for the UAV using the spatial boundary and the potential target boundary, and determining a search viewpoint; S9, using the drone odometer and the search viewpoint to perform autonomous and safe navigation of the drone.

2. The task-driven autonomous search method for UAVs according to claim 1, characterized in that: The target pixel coordinate set calculation in step S4 specifically includes: (1) Set the lower confidence limit and upper confidence limit; (2) obtaining a target detection bounding box whose detection confidence exceeds the lower limit of the confidence level, and calculating a pixel grayscale histogram within the target detection bounding box to obtain a pixel intensity distribution; (3) Calculating the center of gravity of the target in the target detection bounding box using a weighted average method based on the pixel intensity distribution; (4) The center of gravity is set as a threshold for distinguishing target pixels and background pixels in the target detection bounding box, and pixel coordinates greater than the threshold are established as a target pixel coordinate set.

3. The task-driven autonomous search method for UAVs according to claim 1, characterized in that: The pose calculation of the target in the world coordinate system in step S4 specifically includes: (1) Obtain a depth map corresponding to the binocular RGB image and perform scale alignment; (2) extracting a depth value set corresponding to the target pixel coordinate set from the aligned depth map, and removing noise by median filtering to obtain a target point cloud; (3) obtaining the point cloud centroid by calculating the average value of all point coordinates in the target point cloud, and using the point cloud centroid as the target position in the camera coordinate system; (4) Finding the neighborhood points of each point in the target point cloud, performing plane fitting based on the neighborhood points, obtaining the normal vector of the point cloud plane, and converting the normal vector of the point cloud plane from the camera coordinate system to the world coordinate system.

4. The task-driven autonomous search method for UAVs according to claim 1, characterized in that: In step S5, the target information is fused using the target whose detection confidence exceeds the confidence upper limit and the probabilistic voxel map. Specifically, the target point cloud of the target whose detection confidence exceeds the confidence upper limit is mapped to the probabilistic voxel map, and the target category and confidence information are added to the corresponding voxels, and different voxel colors are used to represent targets of different categories; the target posture information is stored in the probabilistic voxel map, and the position is marked with a number at the center of the target voxel, and the posture is marked with an arrow.

5. The task-driven autonomous search method for UAVs according to claim 1, characterized in that: The step S5 of updating the voxel occupancy probability by Bayesian probability estimation is specifically as follows: using the Bayesian probability estimation method to fuse the detection confidence and the prior occupancy probability of the probability voxel map to obtain a posterior occupancy probability.

6. The task-driven autonomous search method for UAVs according to claim 1, characterized in that: The search strategy of the UAV in step S8 includes a global coverage planning part and a local viewpoint optimization part, wherein the global coverage planning part uses the center of each spatial boundary and the center of the potential target boundary as the path point to guide the exploration, and roughly plans a path covering each spatial boundary and the potential target boundary; the local viewpoint optimization part randomly samples the spatial boundaries within the optimization range in their sampling space, and uses the utility function to sort the sampling viewpoints of each spatial boundary and retain the sampling viewpoints with the highest utility value, establishes the retained sampling viewpoints as graph nodes, and constructs a graph search algorithm through a graph search method to determine the local optimization path, and each viewpoint in the local optimization path is the search viewpoint.

7. The task-driven autonomous search method for UAVs according to claim 6, characterized in that: The global coverage planning part specifically includes: using the A* algorithm to calculate the traversable path from the current position of the drone to the position of the center of each spatial boundary and the position of the center of each potential target boundary, using the time cost and flight continuity cost of reaching the center of each spatial boundary and the center of each potential target boundary as the search cost, converting the traversal problem of each spatial boundary and potential target boundary into an asymmetric traveling salesman problem, and solving it to obtain the traversal order of each spatial boundary and potential target boundary.

8. The task-driven autonomous search method for UAVs according to claim 6, characterized in that: The utility function used in the local viewpoint optimization part is a function that is compatible with exploration coverage and target detection confidence, wherein the exploration coverage is represented by the ratio of the coverable volume to the boundary volume; and the target detection confidence is represented by the detection distance and the detection direction.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the task-driven autonomous search method for a drone as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the task-driven autonomous search method for a drone as described in any one of claims 1 to 8 are implemented.

Images