Unmanned aerial vehicle autonomous decision-making chasing control method and device, terminal equipment and storage medium

Abstract

The invention discloses a chasing control method and device for autonomous decision making of an unmanned aerial vehicle, terminal equipment and a storage medium, and belongs to the field of unmanned aerial vehicle control. The method comprises the steps that target detection is conducted on a chasing target, the initial coordinate and the motion state of the chasing target are determined, and then a target rasterized map of the environment where the unmanned aerial vehicle is located is constructed in real time; the method comprises the following steps: acquiring real-time coordinates of a chasing target at a target prediction moment, predicting a prediction coordinate of the chasing target at the target prediction moment by adopting an extended Kalman filtering algorithm, performing path planning according to the prediction coordinate and the real-time coordinates by taking the shortest flight path reaching the prediction coordinate as a target, generating a tracking trajectory of the unmanned aerial vehicle, and finally controlling the unmanned aerial vehicle to fly according to the tracking trajectory. Therefore, the problem that the current unmanned aerial vehicle system cannot track the moving target through dynamic path planning and autonomous decision can be solved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control, and more particularly to a method, apparatus, terminal device, and storage medium for autonomous decision-making and pursuit control of UAVs. Background Art

[0002] With the development of drone technology, drones are increasingly used in military and civilian fields, but this also brings security challenges, especially in the low-altitude domain where illegal drone activities pose a threat to public safety. Drones are typical low-altitude, slow-moving, and small targets, characterized by low flight altitude, slow speed, small effective detection area, and difficulty in detection. Compared to traditional air defense systems, using specialized drones, such as those equipped with robotic arms and grasping devices, for low-altitude anti-drone operations is not only more cost-effective but also more flexible and convenient to deploy. Specialized drones, when performing special missions such as pursuing fleeing targets and conducting air-to-air anti-drone operations, typically aim to catch up with and capture moving targets in the shortest possible time.

[0003] However, despite the widespread attention given to the application of drones in the pursuit of moving flying targets, existing drone systems are unable to perform dynamic path planning and autonomous decision-making when carrying out pursuit missions. They usually require skilled operators, which has obvious drawbacks in terms of cost and pursuit success rate. Summary of the Invention

[0004] This invention provides a method, apparatus, terminal device, and storage medium for autonomous decision-making in pursuit and control of unmanned aerial vehicles (UAVs), which can solve the problem that current UAV systems cannot pursue and capture moving targets through dynamic path planning and autonomous decision-making.

[0005] One embodiment of the present invention provides a pursuit and control method for autonomous decision-making of unmanned aerial vehicles (UAVs), comprising:

[0006] Acquire several laser point cloud data and several image data captured by the drone in real time coordinates when it is pursuing the target.

[0007] The image data is used to detect the target and determine the initial coordinates and motion state of the target.

[0008] The laser point cloud data and the image data are fused to generate a target raster map of the environment in which the UAV is located;

[0009] Using the extended Kalman filter algorithm, the predicted coordinates of the target at the target prediction time are predicted based on the target raster map, the initial coordinates, and the motion state.

[0010] With the goal of minimizing the flight path to the predicted coordinates, path planning is performed based on the target raster map, the predicted coordinates, and the real-time coordinates to generate the tracking trajectory of the UAV.

[0011] Control the drone to fly according to the tracking trajectory.

[0012] Further, the step of fusing the laser point cloud data and the image data to generate a target rasterized map of the environment in which the UAV is located includes:

[0013] Dynamic obstacle target detection is performed on the image data to determine the first obstacle coordinates of several dynamic obstacles;

[0014] The laser point cloud data and the image data are fused to generate an initial map;

[0015] Based on the first obstacle coordinates and the initial coordinates, determine the obstacle areas of the obstacles in the initial map and the pursuit coordinates of the target in the initial map;

[0016] The initial map is divided into several grids, and the first grid where the obstacle area and the pursuit coordinates are located is marked as impassable, while the second grid other than the first grid is marked as passable, thus generating an initial rasterized map.

[0017] Determine whether a historical rasterized map exists that records the environment of the drone's historical waypoints;

[0018] If so, the initial rasterized map is matched and fused with the historical rasterized map to generate the target rasterized map, and the marker of each grid in the initial rasterized map is used as the marker of the corresponding grid in the target rasterized map;

[0019] If not, then the initial rasterized map is used as the target rasterized map.

[0020] Further, the step of fusing the laser point cloud data and the image data to generate an initial map includes:

[0021] Feature points are extracted from the image data and the laser point cloud data respectively to generate two-dimensional feature points corresponding to each image data and three-dimensional feature points corresponding to each laser point cloud data.

[0022] The two-dimensional and three-dimensional feature points are matched to generate several feature point pairs. Based on the feature point pairs, the image data and the laser point cloud data are fused to generate a map to be optimized.

[0023] The coordinates of several first obstacles are mapped onto the map to be optimized to determine the initial obstacle region of several obstacles in the initial map;

[0024] The feature point pairs located in the initial obstacle area are deleted, and the map to be optimized is optimized based on the remaining target feature point pairs to generate the initial map.

[0025] Furthermore, the motion state includes: velocity and acceleration;

[0026] Using the extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state, the predicted coordinates of the target at the target prediction time are predicted, including:

[0027] Based on the initial coordinates, determine the first coordinates of the target in the target raster map;

[0028] Based on the first coordinates, velocity, and acceleration, construct the initial state vector of the target being pursued;

[0029] Using a preset nonlinear state transition function, an initial state transition equation and a corresponding first covariance matrix are constructed based on the initial state vector;

[0030] Using a preset nonlinear observation function, an initial observation vector and a corresponding second covariance matrix are constructed based on the initial state vector;

[0031] Construct an initial covariance matrix based on the first covariance matrix and the second covariance matrix;

[0032] Based on the initial state transition equation, the initial observation vector, the initial state vector, and the initial covariance matrix, an iterative prediction operation is performed to generate the predicted coordinates of the target at the target prediction time.

[0033] Further, the step of generating the UAV's tracking trajectory by performing path planning based on the target raster map, the predicted coordinates, and real-time coordinates, with the goal of minimizing the flight path to the predicted coordinates, includes:

[0034] Based on the label of each grid in the target rasterized map, determine the weight of each grid used to represent the difficulty of passage, and calculate the distance value from each grid in the target rasterized map to the nearest first grid.

[0035] An ESDF map is generated based on the distance value and weight; wherein, the nodes in the ESDF map correspond one-to-one with the grids in the target rasterized map;

[0036] Based on the real-time coordinates of the UAV and the predicted coordinates, the A* algorithm is used for path planning to generate several initial path nodes for the UAV to reach the predicted coordinates.

[0037] The path smoothing algorithm is used to generate the tracking trajectory based on several initial path nodes.

[0038] Further, the step of using the A* algorithm to perform path planning based on the real-time coordinates of the UAV and the predicted coordinates to generate several initial path nodes for the UAV to reach the predicted coordinates includes:

[0039] Based on the real-time coordinates of the UAV, the corresponding starting node is determined on the ESDF map, and based on the predicted coordinates, the corresponding ending node is determined on the ESDF map.

[0040] Add the starting node to a preset open list, and repeat the path planning operation according to the open list until the ending node is added to a preset closed list;

[0041] Backtracking is performed based on the parent node bound to the termination node to generate several initial path nodes.

[0042] The path planning operation includes:

[0043] The target node with the minimum cost value is obtained from the open list; initially, the starting node is obtained from the open list as the target node.

[0044] Add the target node to the closed list and calculate the adaptive expansion step size of the target node in the ESDF map;

[0045] Based on the adaptive expansion step size, select several neighboring nodes that are passable and do not exist in the closed list;

[0046] Calculate the shortest movement step length for the target node to reach each neighbor node, and calculate the cost value of each neighbor node based on the movement step length and the weight of each neighbor node.

[0047] Traverse the neighbor nodes, take the currently traversed neighbor node as the target neighbor node, and determine whether the target neighbor node already exists in the open list;

[0048] If not, add the target neighbor node and its corresponding cost to the open list, and record the target node as the parent node of the target neighbor node;

[0049] If so, compare the historical value recorded by the target neighbor node in the open list with the value of the target neighbor node;

[0050] If the value of the target node is less than the historical value of the target node, then the historical value of the target node is updated to the target value, and the historical parent node is updated to the target node.

[0051] If the replacement value is not less than the historical replacement value, then the replacement value of the target neighbor node and its parent node will not be updated.

[0052] When the traversal of several neighbor nodes is completed, it is determined whether the node with the lowest generation value in the open list is the terminating node.

[0053] If so, add the termination node to the close list;

[0054] If not, proceed to the next round of path planning.

[0055] Furthermore, controlling the drone to fly according to the tracking trajectory includes:

[0056] Obtain the initial control parameters of the UAV;

[0057] Monitor the movement position of the drone and calculate the offset distance between the movement position and the tracking trajectory;

[0058] Based on the initial control parameters and the offset distance, control signals are generated to correct the speed and flight direction of the UAV, so that the UAV flies according to the tracking trajectory when it receives the control signals.

[0059] Another embodiment of the present invention provides a pursuit and control device for autonomous decision-making of unmanned aerial vehicles, comprising:

[0060] The detection data acquisition module is used to acquire several laser point cloud data and several image data captured by the UAV in real time coordinates when it is pursuing the target.

[0061] The target detection module is used to detect the target in the image data and determine the initial coordinates and motion state of the target.

[0062] The map building module is used to fuse the laser point cloud data and the image data to generate a target raster map of the environment in which the UAV is located.

[0063] The trajectory prediction module is used to predict the target's coordinates at the target prediction time by employing an extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state.

[0064] The path planning module is used to convert the target raster map into an ESDF map, and with the shortest flight path to the predicted coordinates as the objective, it performs path planning based on the predicted coordinates and real-time coordinates to generate the tracking trajectory of the UAV.

[0065] The drone control module is used to control the drone to fly according to the tracking trajectory.

[0066] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of a drone autonomous decision-making pursuit and control method as described in any of the above embodiments of the present invention.

[0067] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of a drone autonomous decision-making pursuit control method as described in any of the above embodiments of the present invention.

[0068] The following benefits can be obtained by implementing the present invention:

[0069] This invention discloses a method, apparatus, terminal device, and storage medium for autonomous decision-making in pursuit and control of unmanned aerial vehicles (UAVs), belonging to the field of UAV control. The method involves target detection to determine the initial coordinates and motion state of the target, followed by real-time construction of a target-grid map of the UAV's environment. Utilizing the high efficiency of the extended Kalman filter algorithm, the method predicts the target's coordinates at the predicted time. Then, aiming to minimize the flight path to the predicted coordinates, dynamic path planning is performed based on the real-time constructed target-grid map, predicted coordinates, and the UAV's real-time coordinates to generate a tracking trajectory for the UAV, which is then controlled to fly according to this trajectory. Therefore, this invention solves the problem that current UAV systems cannot pursue moving targets through dynamic path planning and autonomous decision-making. Attached Figure Description

[0070] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0071] Figure 1 This is a flowchart illustrating a pursuit and control method for autonomous decision-making by an unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram of the structure of a drone autonomous decision-making pursuit control device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0075] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0076] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0077] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0078] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0079] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0080] See Figure 1 To address the current problem that unmanned aerial vehicle (UAV) systems cannot pursue moving targets through dynamic path planning and autonomous decision-making, an embodiment of the present invention provides a UAV autonomous decision-making pursuit and control method, comprising:

[0081] S1. Acquire several laser point cloud data and several image data captured by the drone in real-time coordinates when it is pursuing the target.

[0082] In a preferred embodiment of the present invention, environmental perception is achieved using a camera and lidar mounted on the drone. It should be noted that in this embodiment, the environmental data collection period can be set, and in each period, the camera and lidar can be configured to acquire a number of image data and lidar point cloud data. Alternatively, the acquisition of image data and lidar can be configured based on the displacement distance of the drone. For example, when the drone is detected to have moved 1 meter, the camera and lidar can be activated to collect environmental data.

[0083] S2. Perform target detection on the image data to determine the initial coordinates and motion state of the target;

[0084] In a preferred embodiment of the present invention, the YOLO v11 target detection model is used to perform real-time target detection on image data captured by the camera to identify and track targets. It is understood that the output of YOLO v11 is a list containing detection boxes and categories, where the detection boxes are represented as follows:

[0085] D={(x i ,y i ,w i ,h i ,c i )};

[0086] Among them, (x i ,y i ) is the center coordinate of the detection box, (w i ,h i ) represents the width and height of the detection box, c i It is the category of the target being detected.

[0087] S3. The laser point cloud data and the image data are fused to generate a target raster map of the environment in which the UAV is located;

[0088] Preferably, the step of fusing the laser point cloud data and the image data to generate a target raster map of the environment in which the UAV is located includes:

[0089] S31. Perform dynamic obstacle target detection on the image data to determine the first obstacle coordinates of several dynamic obstacles;

[0090] S32. The laser point cloud data and the image data are fused to generate an initial map;

[0091] Preferably, the step of fusing the laser point cloud data and the image data to generate an initial map includes:

[0092] S321. Extract feature points from the image data and the laser point cloud data respectively to generate two-dimensional feature points corresponding to each image data and three-dimensional feature points corresponding to each laser point cloud data.

[0093] S322. Match the two-dimensional feature points and the three-dimensional feature points to generate several feature point pairs, and fuse the image data and the laser point cloud data according to the feature point pairs to generate a map to be optimized.

[0094] S323. Map the coordinates of several first obstacles to the map to be optimized, and determine the initial obstacle region of several obstacles in the initial map;

[0095] S324. Delete the feature point pairs located in the initial obstacle area, and optimize the map to be optimized based on the remaining target feature point pairs to generate the initial map.

[0096] S33. Based on the first obstacle coordinates and the initial coordinates, determine the obstacle area of ​​several obstacles in the initial map and the pursuit coordinates of the pursuit target in the initial map;

[0097] S34. Divide the initial map into several grids, and assign an impassable mark to the first grid where the obstacle area and the pursuit coordinates are located, and assign a passable mark to the second grid other than the first grid, thereby generating an initial rasterized map.

[0098] S35. Determine whether a historical rasterized map exists that records the environment of the drone's historical waypoints.

[0099] S36. If so, the initial rasterized map and the historical rasterized map are matched and merged to generate the target rasterized map, and the marker of each grid in the initial rasterized map is used as the marker of the corresponding grid in the target rasterized map.

[0100] S37. If not, then the initial rasterized map shall be used as the target rasterized map.

[0101] In a preferred embodiment of the present invention, the open-source ORB-SLAM3 system is used to fuse laser point cloud data and image data for mapping.

[0102] It should be noted that the image data includes: RGB color images (containing pixel color information) and timestamps (used for synchronization with radar data); the laser point cloud data includes 3D point cloud data (each point contains coordinates x, y, z), timestamps, and scanning angles.

[0103] Specifically, features of local feature points (Oriented Fast and Rotated BRIEF, ORB) are extracted from image data. Point cloud features are extracted from LiDAR point cloud data. ORB local feature points consist of an Oriented Fast keypoint detector and a Rotated BRIEF descriptor, exhibiting rotation and scale invariance. Keypoints contain position, orientation, and scale information, and the descriptor is a 256-bit binary vector. Point cloud features include edge features (such as object contours) and planar features (such as the ground). High-curvature points (edges) or low-curvature regions (planar areas) are selected from the LiDAR point cloud through curvature calculation or normal vector analysis. Then, ORB features are matched with point cloud features to establish a correspondence between the camera and LiDAR. The matching rules between ORB features and point cloud features are mainly achieved through multi-step sensor data fusion. First, hardware synchronization (such as GPS / PPS signals) ensures that the timestamps of the camera and LiDAR are aligned, eliminating timing discrepancies.

[0104] In the feature matching stage, feature points in the laser point cloud data are projected onto the camera image plane. Neighboring ORB feature points are searched around the projected points, and the candidate matches with the highest similarity are selected by comparing the Hamming distance of the binary descriptors. Simultaneously, the ORB feature points are back-projected onto the LiDAR coordinate system, and the 3D feature points with the closest Euclidean distance in the point cloud are searched. Only candidate pairs with consistent bidirectional matching are retained to enhance robustness. Subsequently, geometric verification is performed using a random sampling consensus algorithm. Geometric transformation models are calculated for randomly selected matching pairs, and mismatches with excessive reprojection errors are eliminated. After iterative optimization, geometrically consistent matching results are retained. Successfully matched feature pairs are stored in a joint database, recording their 3D coordinates, descriptors, and timestamps. Interfering feature point pairs in obstacle areas are dynamically filtered in conjunction with real-time target detection results, ultimately establishing a stable correspondence between the camera and LiDAR. Combining the UAV's own pose information, the poses of the camera and LiDAR are estimated using the feature matching results. Based on this, the map is updated, and new keyframes and point clouds are added. Finally, loop closure detection is performed periodically to eliminate accumulated errors.

[0105] After fusing image data and laser point cloud data using the methods described above to generate the map to be optimized, YOLO v11 recognition is also used for dynamic obstacle detection during the construction of the dynamic map (initial map) in ORB-SLAM3. The detected obstacles are then integrated into the map, and feature point pairs are filtered using YOLO v11 recognition results to ensure that dynamic obstacles are not mistaken for part of the static environment, thereby improving the accuracy and robustness of map construction.

[0106] First, the coordinates of several first obstacles are mapped onto the map to be optimized, that is, the two-dimensional coordinates of the YOLO v11 detection boxes (first obstacle coordinates) are converted into three-dimensional coordinates. Assuming the camera's intrinsic parameter matrix is ​​K and the extrinsic parameter matrix is ​​T (the transformation matrix from LiDAR to camera), then:

[0107]

[0108] The center point of the YOLO detection box is back-projected from the 2D image coordinates (xi, yi) to 3D coordinates in the camera coordinate system using the camera intrinsic parameter matrix K, and then transformed to 3D coordinates pi in the LiDAR coordinate system using the extrinsic parameter matrix T. This 3D coordinate is then projected onto the LiDAR point cloud to obtain the corresponding point cloud coordinates (initial obstacle region). Then, feature points located within the initial obstacle region are removed from the feature points extracted from ORB-SLAM3.

[0109] In this embodiment, the identified targets and obstacles are also labeled. Each identified object is assigned a unique ID, and its location and category are recorded, resulting in a dense initial map with semantic annotations.

[0110] When generating the initial map, it needs to be converted into an initial rasterized map. First, rasterization is required, which divides the map into several grids, each representing a small area. Then, based on the annotation information from YOLO v11, grids containing the target and obstacles are marked as impassable.

[0111] Then, the map is updated in real time, that is, the initial rasterized map is matched and merged with the historical rasterized maps, and the state of the raster is adjusted according to the latest initial rasterized map. In this embodiment, a dynamic map database is also maintained by recording the historical state of each raster. Next, the target rasterized map is generated through fusion, and the state of each raster can be represented according to the semantics of the annotation as follows:

[0112]

[0113] S4. Using the extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state, predict the target's coordinates at the target prediction time.

[0114] Preferably, the motion state includes: velocity and acceleration;

[0115] S41. Using the extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state, predict the target's coordinates at the target prediction time, including:

[0116] S42. Based on the initial coordinates, determine the first coordinates of the target in the target raster map;

[0117] S43. Construct the initial state vector of the target based on the first coordinates, velocity, and acceleration;

[0118] S44. Using a preset nonlinear state transition function, construct the initial state transition equation and the corresponding first covariance matrix based on the initial state vector;

[0119] S45. Using a preset nonlinear observation function, construct an initial observation vector and the corresponding second covariance matrix based on the initial state vector;

[0120] S46. Construct an initial covariance matrix based on the first covariance matrix and the second covariance matrix;

[0121] S47. Based on the initial state transition equation, the initial observation vector, the initial state vector, and the initial covariance matrix, perform an iterative prediction operation to generate the predicted coordinates of the target at the target prediction time.

[0122] In a preferred embodiment of the present invention, the three-dimensional coordinate information of the target is obtained from the constructed target raster map, and the extended Kalman filter (EKF) is used to predict the motion state of the target, thereby improving the accuracy of tracking and interception.

[0123] Let the initial state vector of the target be X. k Including the first coordinate, velocity, and acceleration, it is represented as:

[0124]

[0125] Where, x k ,y k ,z k It is the first coordinate of the target at time k. The velocity of the target at time k; It is the acceleration of the target at time k.

[0126] Assuming the motion model of the target being pursued is a linear acceleration model, the initial state transition equation can be expressed as:

[0127] x k+1 =f(x) k u k )+w k ;

[0128] Where, f(x) k u k ) is a nonlinear state transition function, u k It is a control input, w k It is process noise, assumed to be zero-mean Gaussian white noise, and its corresponding first covariance matrix is ​​Qk;

[0129] The specific initial state transition equation can be expressed as:

[0130]

[0131] Let the initial observation vector zk include the initial state vector of the target being pursued, which can be expressed as:

[0132] z k =h(x k )+v k ;

[0133] Where h(x) k ) is a nonlinear observation function, v k It is the observation noise, assumed to be zero-mean Gaussian white noise, and its second covariance matrix is ​​Rk.

[0134] The specific initial state transition equation can then be expressed as:

[0135]

[0136] Next, the extended Kalman filter algorithm is used to predict the motion state and predicted position of the target at various future moments.

[0137] First, construct an initial covariance matrix based on the first and second covariance matrices. Then, perform the following iterative calculations based on the initial state vector and the initial covariance matrix:

[0138] In the forecasting phase:

[0139] Calculate state prediction;

[0140] Calculate the Jacobian matrix:

[0141] Update the predicted covariance matrix:

[0142] During the update phase:

[0143] Calculate the observation Jacobian matrix:

[0144] Calculate the Kalman gain:

[0145] Updated state estimate:

[0146] Update covariance matrix: P k+1 =(IK k+1 H k+1 )P k+1|k ;

[0147] Through the above steps, EKF can effectively predict the movement state and predicted position of the target at various future moments, and then select a target prediction moment from these future moments.

[0148] S5. With the goal of minimizing the flight path to the predicted coordinates, perform path planning based on the target raster map, the predicted coordinates, and the real-time coordinates to generate the tracking trajectory of the UAV.

[0149] In a preferred embodiment of the present invention, the A* algorithm performs well in path planning, but in large-scale complex environments, it suffers from problems such as a large number of nodes to be explored and a long path search time. To improve the efficiency of the algorithm, we introduce an adaptive node expansion step size strategy based on the ESDF (Euclidean Signed Distance Field) map, which reduces redundant nodes in the path search process and smooths the initial path with B-spline curves.

[0150] Preferably, the step of generating the UAV's tracking trajectory by performing path planning based on the target raster map, the predicted coordinates, and real-time coordinates, with the goal of minimizing the flight path to the predicted coordinates, includes:

[0151] S51. Based on the label of each grid in the target rasterized map, determine the weight of each grid used to represent the difficulty of passage, and calculate the distance value from each grid in the target rasterized map to the nearest first grid.

[0152] S52. Generate an ESDF map based on the distance value and weight; wherein, the nodes in the ESDF map correspond one-to-one with the grids in the target rasterized map;

[0153] In a preferred embodiment of the present invention, each grid cell in the target rasterized map is assigned a weight to represent its traversal difficulty. The higher the weight, the greater the traversal difficulty, and it is approximated as a Euclidean distance value.

[0154]

[0155] Secondly, when generating the ESDF map, each grid cell stores the distance value from that grid node to the first grid cell containing the nearest obstacle. It should be noted that a positive distance value indicates the point is in free space, and the distance is the shortest distance to the obstacle. A negative distance value indicates the point is inside an obstacle, and the distance is a negative value to the obstacle's surface. A zero distance value indicates the point is on the obstacle's surface.

[0156] S53. Based on the real-time coordinates of the UAV and the predicted coordinates, the A* algorithm is used to perform path planning and generate several initial path nodes for the UAV to reach the predicted coordinates.

[0157] Preferably, the step of using the A* algorithm to perform path planning based on the real-time coordinates of the UAV and the predicted coordinates to generate several initial path nodes for the UAV to reach the predicted coordinates includes:

[0158] S531. Based on the real-time coordinates of the UAV, determine the corresponding starting node on the ESDF map, and based on the predicted coordinates, determine the corresponding ending node on the ESDF map.

[0159] S532. Add the starting node to a preset open list, and repeat the path planning operation according to the open list until the ending node is added to a preset closed list.

[0160] S533. Backtracking is performed based on the parent node bound to the termination node to generate several initial path nodes.

[0161] The path planning operation includes:

[0162] S01. Obtain the target node with the minimum cost value from the open list; initially, obtain the starting node from the open list as the target node;

[0163] S02. Add the target node to the closed list and calculate the adaptive expansion step size of the target node in the ESDF map;

[0164] S03. Based on the adaptive expansion step size, select several neighboring nodes that are passable and do not exist in the closed list;

[0165] S04. Calculate the shortest movement step length for the target node to reach each neighbor node, and calculate the cost value of each neighbor node based on the movement step length and the weight of each neighbor node.

[0166] S05. Traverse the neighbor nodes, take the currently traversed neighbor node as the target neighbor node, and determine whether the target neighbor node already exists in the open list.

[0167] S06. If not, add the target neighbor node and its corresponding cost to the open list, and record the target node as the parent node of the target neighbor node.

[0168] S07. If so, compare the historical value recorded by the target neighbor node in the open list with the value of the target neighbor node.

[0169] S08. When the generation value is less than the historical generation value, the historical generation value of the target neighbor node is updated to the generation value, and the historical parent node is updated to the target node.

[0170] S09. If the replacement value is not less than the historical replacement value, then the replacement value of the target neighbor node and the parent node shall not be updated.

[0171] S010. When the traversal of several neighbor nodes is completed, determine whether the node with the lowest generation value in the open list is the terminating node.

[0172] S011. If so, add the termination node to the close list;

[0173] S012. If not, proceed to the next round of path planning.

[0174] In a preferred embodiment of the present invention, during the path planning process, the starting node S and the ending node G are first initialized, an open list and a closed list are created, and the starting node is added to the open list.

[0175] During the node expansion phase: Select the target node n with the smallest total cost function f(n) = g(n) + h(n) from the OpenList (the minimum value is efficiently retrieved through a priority queue to ensure the optimal exploration direction that combines the actual cost g(n) and the heuristic estimate h(n), and move it into the ClosedList;

[0176] In the step size calculation stage: the adaptive expansion step size Step(n) is calculated based on the sparsity degree S(n) of the environment in which the target node is located (based on the mean ratio of the neighborhood distance field in the ESDF map). The step size is increased in sparse regions to reduce redundant nodes.

[0177] The neighbor node Ni selection stage: select only the traversable nodes (wi=1) in the ESDF map that are not recorded in the ClosedList, and calculate their cost g(Ni) = g(n) + movement cost (the movement cost is determined by the product of the movement step size and the grid weight wi) and heuristic function h(Ni) (Euclidean distance multiplied by the base weight).

[0178] During the node update phase: If Ni is not in OpenList, add it and set its parent node to the target node n. If it already exists, compare the cost value g(Ni) of the new path with the historical cost value. Only update the parent node and cost value if the new cost value is lower.

[0179] When the target node G enters the ClosedList, the globally optimal path is generated by backtracking the parent node chain; if the OpenList is exhausted and G is still not found, the search is considered to have failed.

[0180] Path optimality is guaranteed by the admissibility heuristic function of the A* algorithm (Euclidean distance does not exceed the actual cost). Combined with the accurate distance field of the ESDF map and the adaptive step size strategy, the shortest path characteristics are maintained while reducing the amount of computation. Finally, the dynamic feasibility of the path is improved by B-spline smoothing without sacrificing global optimality.

[0181] S54. Using a path smoothing algorithm, the tracking trajectory will be generated based on several initial path nodes.

[0182] In a preferred embodiment of the present invention, the initial path node {n} is obtained. i};

[0183] The B-spline algorithm primarily solves for the basis functions and control point coordinates given the curve order. The following B-spline algorithm generates the tracking trajectory:

[0184]

[0185] The odd function of a third-order B-spline curve can be expressed as:

[0186]

[0187] Where n represents the number of control points minus 1, P i N represents the coordinates of the control point. i，k (u) represents the basis function corresponding to Pi, k is the order of the curve, and u is the number of nodes.

[0188] S6. Control the drone to fly according to the tracking trajectory.

[0189] Preferably, controlling the drone to fly according to the tracking trajectory includes:

[0190] S61. Obtain the initial control parameters of the UAV;

[0191] S62. Monitor the movement position of the drone and calculate the offset distance between the movement position and the tracking trajectory;

[0192] S63. Based on the initial control parameters and the offset distance, generate control signals for correcting the speed and flight direction of the UAV, so that the UAV flies according to the tracking trajectory when it receives the control signals.

[0193] In a preferred embodiment of the present invention, the UAV employs a PID controller. The PID controller adjusts the control signal through three parts: proportional (P), integral (I), and derivative (D), to make the system output as close as possible to the target value. For a given tracking trajectory, a position-based PID controller is designed. The target position sequence of the PID controller comes from the initial path nodes generated by the A* algorithm. Based on the error between the UAV's current position and the initial path nodes, the control quantity is dynamically adjusted to ensure flight along the planned path. A* generates a globally optimal path and provides discrete waypoints. The PID controller achieves continuous trajectory tracking, compensates for local deviations after path smoothing, and ensures motion continuity.

[0194] The specific control process is as follows: First, initialize the parameters Kp, Ki, and Kd of the PID controller. Then, read the UAV's movement position Pc. Based on the predetermined tracking trajectory, update the target position Pd in ​​real time. Calculate the position error ep and the position PID control signal up according to the formula. Finally, apply the position control signal up to the UAV's propulsion system to adjust the UAV's speed and direction.

[0195] Specifically, the movement position p is calculated using the following formula. c =(x c ,y c , z c ) and target position p d =(x d ,y d , z d Error of )

[0196] e p =p d -p c =(x d -x c ,y d -y c , z d -z c );

[0197] Calculate the proportional term P using the following formula, which is used to adjust for errors;

[0198] P=K p e p ;

[0199] Where Kp is the proportional gain.

[0200] The integral term I is calculated using the following formula to eliminate steady-state error:

[0201]

[0202] Where Ki is the integral gain.

[0203] The differential term D, used to suppress the rate of change of error, is calculated according to the following formula:

[0204]

[0205] Where Kd is the differential gain.

[0206] Finally, the proportional, integral, and differential terms above are combined to generate control signals for correcting the speed and flight direction of the drone.

[0207]

[0208] like Figure 2 As shown, based on the above method embodiments, corresponding apparatus embodiments are provided;

[0209] An embodiment of the present invention provides a pursuit and control device for autonomous decision-making of unmanned aerial vehicles (UAVs), comprising:

[0210] The detection data acquisition module is used to acquire several laser point cloud data and several image data captured by the UAV in real time coordinates when it is pursuing the target.

[0211] The target detection module is used to detect the target in the image data and determine the initial coordinates and motion state of the target.

[0212] The map building module is used to fuse the laser point cloud data and the image data to generate a target raster map of the environment in which the UAV is located.

[0213] The trajectory prediction module is used to predict the target's coordinates at the target prediction time by employing an extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state.

[0214] The path planning module is used to convert the target raster map into an ESDF map, and with the shortest flight path to the predicted coordinates as the objective, it performs path planning based on the predicted coordinates and real-time coordinates to generate the tracking trajectory of the UAV.

[0215] The drone control module is used to control the drone to fly according to the tracking trajectory.

[0216] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can realize the pursuit and control method for UAV autonomous decision-making provided by any of the above-described method embodiments of the present invention.

[0217] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0218] Based on the above-described embodiment of the drone autonomous decision-making pursuit and control method, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a drone autonomous decision-making pursuit and control method according to any embodiment of the present invention.

[0219] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0220] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0221] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0222] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the drone autonomous decision-making pursuit control method described in any of the above-described method embodiments of the present invention.

[0223] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0224] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A pursuit and control method for autonomous decision-making by unmanned aerial vehicles (UAVs), characterized in that, include: Acquire several laser point cloud data and several image data captured by the drone in real time coordinates when it is pursuing the target. The image data is used to detect the target and determine the initial coordinates and motion state of the target. The laser point cloud data and the image data are fused to generate a target raster map of the environment in which the UAV is located; Using the extended Kalman filter algorithm, the predicted coordinates of the target at the target prediction time are predicted based on the target raster map, the initial coordinates, and the motion state. With the goal of minimizing the flight path to the predicted coordinates, path planning is performed based on the target raster map, the predicted coordinates, and the real-time coordinates to generate the tracking trajectory of the UAV. Control the drone to fly according to the tracking trajectory.

2. The pursuit and control method for autonomous decision-making of unmanned aerial vehicles as described in claim 1, characterized in that, The step of fusing the laser point cloud data and the image data to generate a target rasterized map of the environment in which the UAV is located includes: Dynamic obstacle target detection is performed on the image data to determine the first obstacle coordinates of several dynamic obstacles; The coordinates of the first obstacle, the laser point cloud data, and the image data are fused to generate an initial map; Based on the first obstacle coordinates and the initial coordinates, determine the obstacle areas of the obstacles in the initial map and the pursuit coordinates of the target in the initial map; The initial map is divided into several grids, and the first grid where the obstacle area and the pursuit coordinates are located is marked as impassable, while the second grid other than the first grid is marked as passable, thus generating an initial rasterized map. Determine whether a historical rasterized map exists that records the environment of the drone's historical waypoints; If so, the initial rasterized map is matched and fused with the historical rasterized map to generate the target rasterized map, and the marker of each grid in the initial rasterized map is used as the marker of the corresponding grid in the target rasterized map; If not, then the initial rasterized map is used as the target rasterized map.

3. The pursuit and control method for autonomous decision-making of unmanned aerial vehicles as described in claim 2, characterized in that, The step of fusing the coordinates of the first obstacle, the laser point cloud data, and the image data to generate an initial map includes: Feature points are extracted from the image data and the laser point cloud data respectively to generate two-dimensional feature points corresponding to each image data and three-dimensional feature points corresponding to each laser point cloud data. The two-dimensional and three-dimensional feature points are matched to generate several feature point pairs. Based on the feature point pairs, the image data and the laser point cloud data are fused to generate a map to be optimized. The coordinates of several first obstacles are mapped onto the map to be optimized to determine the initial obstacle region of several obstacles in the initial map; The feature point pairs located in the initial obstacle area are deleted, and the map to be optimized is optimized based on the remaining target feature point pairs to generate the initial map.

4. The pursuit and control method for autonomous decision-making of unmanned aerial vehicles as described in claim 3, characterized in that, The motion state includes: velocity and acceleration; Using the extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state, the predicted coordinates of the target at the target prediction time are predicted, including: Based on the initial coordinates, determine the first coordinates of the target in the target raster map; Based on the first coordinates, velocity, and acceleration, construct the initial state vector of the target being pursued; Using a preset nonlinear state transition function, an initial state transition equation and a corresponding first covariance matrix are constructed based on the initial state vector; Using a preset nonlinear observation function, an initial observation vector and a corresponding second covariance matrix are constructed based on the initial state vector; Construct an initial covariance matrix based on the first covariance matrix and the second covariance matrix; Based on the initial state transition equation, the initial observation vector, the initial state vector, and the initial covariance matrix, an iterative prediction operation is performed to generate the predicted coordinates of the target at the target prediction time.

5. The pursuit and control method for autonomous decision-making of unmanned aerial vehicles as described in claim 4, characterized in that, The process of generating the UAV's tracking trajectory by optimizing the flight path to the predicted coordinates, based on the target raster map, the predicted coordinates, and real-time coordinates, includes: Based on the label of each grid in the target rasterized map, determine the weight of each grid used to represent the difficulty of passage, and calculate the distance value from each grid in the target rasterized map to the nearest first grid. An ESDF map is generated based on the distance value and weight; wherein, the nodes in the ESDF map correspond one-to-one with the grids in the target rasterized map; Based on the real-time coordinates of the UAV and the predicted coordinates, the A* algorithm is used for path planning to generate several initial path nodes for the UAV to reach the predicted coordinates. The path smoothing algorithm is used to generate the tracking trajectory based on several initial path nodes.

6. The pursuit and control method for autonomous decision-making of unmanned aerial vehicles as described in claim 5, characterized in that, The method involves using the A* algorithm to perform path planning based on the real-time coordinates and predicted coordinates of the UAV, generating several initial path nodes for the UAV to reach the predicted coordinates, including: Based on the real-time coordinates of the UAV, the corresponding starting node is determined on the ESDF map, and based on the predicted coordinates, the corresponding ending node is determined on the ESDF map. Add the starting node to a preset open list, and repeat the path planning operation according to the open list until the ending node is added to a preset closed list; Backtracking is performed based on the parent node bound to the termination node to generate several initial path nodes; The path planning operation includes: The target node with the minimum cost value is obtained from the open list; initially, the starting node is obtained from the open list as the target node. Add the target node to the closed list and calculate the adaptive expansion step size of the target node in the ESDF map; Based on the adaptive expansion step size, select several neighboring nodes that are passable and do not exist in the closed list; Calculate the shortest movement step length for the target node to reach each neighbor node, and calculate the cost value of each neighbor node based on the movement step length and the weight of each neighbor node. Traverse the neighbor nodes, take the currently traversed neighbor node as the target neighbor node, and determine whether the target neighbor node already exists in the open list; If not, add the target neighbor node and its corresponding cost to the open list, and record the target node as the parent node of the target neighbor node; If so, compare the historical value recorded by the target neighbor node in the open list with the value of the target neighbor node; If the value of the target node is less than the historical value of the target node, then the historical value of the target node is updated to the target value, and the historical parent node is updated to the target node. If the replacement value is not less than the historical replacement value, then the replacement value of the target neighbor node and its parent node will not be updated. When the traversal of several neighbor nodes is completed, it is determined whether the node with the lowest generation value in the open list is the terminating node. If so, add the termination node to the close list; If not, proceed to the next round of path planning.

7. The pursuit and control method for autonomous decision-making of unmanned aerial vehicles as described in claim 6, characterized in that, Controlling the drone to fly according to the tracking trajectory includes: Obtain the initial control parameters of the UAV; Monitor the movement position of the drone and calculate the offset distance between the movement position and the tracking trajectory; Based on the initial control parameters and the offset distance, control signals are generated to correct the speed and flight direction of the UAV, so that the UAV flies according to the tracking trajectory when it receives the control signals.

8. A pursuit and control device for autonomous decision-making of unmanned aerial vehicles (UAVs), characterized in that, include: The detection data acquisition module is used to acquire several laser point cloud data and several image data captured by the UAV in real time coordinates when it is pursuing the target. The target detection module is used to detect the target in the image data and determine the initial coordinates and motion state of the target. The map building module is used to fuse the laser point cloud data and the image data to generate a target raster map of the environment in which the UAV is located. The trajectory prediction module is used to predict the target's coordinates at the target prediction time by employing an extended Kalman filter algorithm, based on the target raster map, the initial coordinates, and the motion state. The path planning module is used to convert the target raster map into an ESDF map, and with the shortest flight path to the predicted coordinates as the objective, it performs path planning based on the predicted coordinates and real-time coordinates to generate the tracking trajectory of the UAV. The drone control module is used to control the drone to fly according to the tracking trajectory.

9. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a drone autonomous decision-making pursuit control method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, the device containing the computer-readable storage medium is controlled to perform a pursuit control method for autonomous decision-making of an unmanned aerial vehicle as described in any one of claims 1-7.

Images