Trajectory tracking method, device and equipment of unmanned aerial vehicle group and medium

Abstract

The application discloses a kind of unmanned aerial vehicle group's track tracking method, device, equipment and medium, comprising: first data preprocessing and principal component analysis, obtain the first principal component of source point cloud in key feature point and the second principal component of target point cloud in key feature point. Then rough registration is carried out, and further fine registration is carried out, including based on the rough registration result between source point cloud and target point cloud, determine all the nearest neighbor points in target point cloud according to the coordinates of source point cloud, calculate registration loss value according to all the nearest neighbor points, and based on registration loss value iterative adjustment rough registration result, until the preset termination condition is satisfied, the fine registration result between the source point cloud and target point cloud obtained is obtained. Finally, in the same coordinate system determined based on fine registration result, connect first centroid coordinate and second centroid coordinate, to obtain the connecting line between source point cloud and target point cloud, obtain the connecting line between point cloud at multiple time as the track of unmanned aerial vehicle group.

Description

[0001] Technology Neighborhood

[0002] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to methods, apparatuses, devices, and media for tracking the flight paths of UAV swarms. Background Technology

[0003] In recent years, with the development of drone technology, drones have been increasingly widely used in fields such as industrial and agricultural production and disaster relief. While a single drone can complete some complex tasks, its success rate and ability to withstand emergencies are relatively weak. Drone swarms, on the other hand, can fully leverage the advantages of teamwork, enabling formation and collaborative operations. Furthermore, drone swarms can adapt to diverse environments and working conditions, all of which make it possible to complete complex tasks efficiently and with a high success rate.

[0004] In collaborative drone swarm operations, trajectory tracking is crucial for successful collaboration, and its accuracy determines the swarm's efficiency and task completion. However, compared to accurate trajectory tracking of a single drone, current methods for accurate trajectory tracking of drone swarms are more challenging and their efficiency falls short of practical requirements. Summary of the Invention

[0005] Therefore, it is necessary to provide methods, devices, equipment, and media for tracking the flight paths of drone swarms to solve the above problems.

[0006] A method for tracking the flight paths of a swarm of unmanned aerial vehicles (UAVs), the method comprising:

[0007] Acquire point clouds of the drone swarm at multiple time points, and extract key feature points within the point cloud at each time point;

[0008] Principal component analysis is used to obtain the first principal component of key feature points in the source point cloud and the second principal component of key feature points in the target point cloud. The target point cloud is the point cloud at any time other than the first time, and the source point cloud is the point cloud at the previous time of the target point cloud.

[0009] The source point cloud and the target point cloud are coarsely registered according to the transformation relationship between the first principal component and the second principal component to obtain the coarse registration result between the source point cloud and the target point cloud; wherein, the registration result is used to transform the source point cloud and the target point cloud to the same coordinate system;

[0010] Based on the coarse registration result between the source point cloud and the target point cloud, all nearest points in the target point cloud are determined according to the coordinates of the source point cloud. The registration loss value is calculated based on all nearest points, and the coarse registration result is iteratively adjusted based on the registration loss value until the preset termination condition is met, so as to obtain the fine registration result between the source point cloud and the target point cloud.

[0011] Within the same coordinate system determined based on the fine registration results, the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud are determined. The first centroid coordinates and the second centroid coordinates are connected to obtain the line connecting the source point cloud and the target point cloud. The line connecting the point clouds at multiple times is obtained as the track of the UAV swarm.

[0012] In one embodiment, the extraction of key feature points within the point cloud at each time step includes:

[0013] In the point cloud to be extracted, a neighborhood of the first point to be confirmed is constructed; wherein, the first point to be confirmed is a point in the point cloud to be extracted that has not been confirmed as a feature point, the point cloud to be extracted is a point cloud at any time in multiple time periods, and the neighborhood of the point to be confirmed indicates a spherical space with the point to be confirmed as the center point and extending a preset radius.

[0014] Obtain the number of neighborhood points of all neighboring points within the neighborhood of the first point to be confirmed; where neighborhood points refer to all points within the neighborhood except the center point;

[0015] If the number of neighborhood points is greater than the preset threshold for the number of neighborhood points, then all points in the neighborhood of the first point to be confirmed are determined as feature points in the point cloud to be extracted.

[0016] Based on the curvature criterion, key feature points are extracted from all feature points in the point cloud to be extracted, so as to obtain the key feature points in the point cloud at each time.

[0017] In one embodiment, the extraction of key feature points from all feature points within the point cloud to be extracted based on the curvature criterion includes:

[0018] A quadratic surface function is constructed based on the second point to be confirmed; wherein, the second point to be confirmed is any one of all feature points in the point cloud to be extracted;

[0019] The coordinates of all neighborhood points within the target domain are substituted into the quadratic surface function for fitting, and the curvature values ​​of all feature points within the target domain are calculated based on the fitted quadratic surface function; wherein, the target domain is the neighborhood of the second point to be confirmed;

[0020] Calculate the local average curvature based on the curvature of all feature points in the target domain, and select feature points in the point cloud whose curvature is greater than the local average curvature as key feature points.

[0021] In one embodiment, obtaining the first principal component of key feature points in the source point cloud and the second principal component of key feature points in the target point cloud through principal component analysis includes:

[0022] The coordinates of the centroid of the point cloud to be analyzed are calculated based on the coordinates of all key feature points within the point cloud to be analyzed; wherein, the point cloud to be analyzed includes the source point cloud and the target point cloud;

[0023] The covariance matrix is ​​calculated based on the coordinates of the centroid and all key feature points of the point cloud to be analyzed. Singular value decomposition is performed on the covariance matrix to obtain multiple eigenvalues. A preset number of eigenvalues ​​with the largest variance are selected from the multiple eigenvalues ​​to form the feature vector of the point cloud to be analyzed.

[0024] The centroid coordinates and eigenvectors of the source point cloud are used as the first principal component, and the centroid coordinates and eigenvectors of the target point cloud are used as the second principal component.

[0025] In one embodiment, the step of performing coarse registration of the source point cloud and the target point cloud based on the transformation relationship between the first principal component and the second principal component to obtain a coarse registration result between the source point cloud and the target point cloud includes:

[0026] Establish an initial rotation matrix based on the transformation relationship between the feature vectors of the source point cloud and the feature vectors of the target point cloud;

[0027] An initial translation matrix is ​​established based on the transformation relationship between the centroid coordinates of the source point cloud and the centroid coordinates of the target point cloud.

[0028] The coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud based on the initial rotation matrix and the initial translation matrix to obtain the coarse registration result between the source point cloud and the target point cloud.

[0029] In one embodiment, the process of determining all nearest neighbors in the target point cloud based on the coarse registration result between the source point cloud and the target point cloud, calculating the registration loss value based on all nearest neighbors, and iteratively adjusting the coarse registration result based on the registration loss value until a preset termination condition is met, thereby obtaining the fine registration result between the source point cloud and the target point cloud, includes:

[0030] The coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud according to the initial rotation matrix and the initial translation matrix to obtain the transformed point cloud. The nearest neighbor point of the transformed point is determined in the target point cloud to obtain all the nearest neighbor points of the transformed point cloud. The transformed point is any point in the transformed point cloud, and the nearest neighbor point is the point in the target point cloud that is closest to the transformed point.

[0031] Calculate the current average distance between all nearest neighbors of the transformed point cloud and the transformed point cloud, use the current average distance as the registration loss value and adjust it based on the least squares method to update the initial rotation matrix and the initial translation matrix, and obtain the updated rotation matrix and the updated translation matrix.

[0032] The coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud according to the updated rotation matrix and the updated translation matrix to obtain the updated transformed point cloud. The nearest neighbor of the transformed point is determined in the target point cloud to obtain all the nearest neighbor points of the updated transformed point cloud.

[0033] Calculate the updated average distance between all nearest neighbors of the updated transformed point cloud and the updated transformed point cloud, determine whether the updated average distance is less than or equal to a preset distance threshold, if the updated average distance is greater than the preset distance threshold, then take the updated average distance as the current average distance, return to execute the step of taking the current average distance as the registration loss value and adjusting it based on the least squares method and subsequent steps.

[0034] If the updated average distance is less than or equal to a preset distance threshold, then the preset termination condition is met. The updated rotation matrix and the updated translation matrix are obtained. Based on the updated rotation matrix and the updated translation matrix, the coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud to obtain the fine registration result between the source point cloud and the target point cloud.

[0035] In one embodiment, determining the nearest neighbor point of the transform point within the target point cloud includes:

[0036] Input the transformation point and the target point cloud into a 3D decision tree. In the i-th root node of the 3D decision tree, the target point cloud and the transformation point are divided into two i-th child nodes in the target dimension based on a set partitioning criterion. Here, the initial value of i is 1. At the i-th partition, the median value of all coordinates of the transformation point and the target point cloud in the target dimension is the partitioning criterion of the i-th root node. The target dimension is any one of x-dimensional, y-dimensional and z-dimensional.

[0037] Determine whether the number of current nearest neighbors of the transformation point is less than or equal to a preset number of n; wherein, the current nearest neighbors of the transformation point and the transformation point are all assigned to the same child node;

[0038] If the number of current nearest neighbors of the transformation point is greater than the preset n, then the i-th child node containing the transformation point is taken as the (i+1)-th root node, let i = i+1, and return to execute the step of dividing the target point cloud and the transformation point into two i-th child nodes in the target dimension based on the set division criterion within the i-th root node of the three-dimensional decision tree, and subsequent steps.

[0039] If the number of current nearest neighbors of the transformation point is less than or equal to the preset n, then all current nearest neighbors of the transformation point are taken as the nearest neighbor.

[0040] A flight path tracking device for a swarm of unmanned aerial vehicles (UAVs), the device comprising:

[0041] The data preprocessing module is used to acquire point clouds of the UAV swarm at multiple time points and extract key feature points in the point cloud at each time point; through principal component analysis, the first principal component of the key feature points in the source point cloud and the second principal component of the key feature points in the target point cloud are obtained; wherein, the target point cloud is the point cloud at any time point other than the first time point among the point clouds at multiple time points, and the source point cloud is the point cloud at the time point before the target point cloud.

[0042] The coarse registration module is used to perform coarse registration between the source point cloud and the target point cloud according to the transformation relationship between the first principal component and the second principal component, so as to obtain the coarse registration result between the source point cloud and the target point cloud; wherein, the registration result is used to transform the source point cloud and the target point cloud to the same coordinate system;

[0043] The fine registration module is used to determine all nearest points in the target point cloud based on the coarse registration result between the source point cloud and the target point cloud, calculate the registration loss value based on all nearest points, and iteratively adjust the coarse registration result based on the registration loss value until a preset termination condition is met, so as to obtain the fine registration result between the source point cloud and the target point cloud.

[0044] The trajectory determination module is used to determine the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud within the same coordinate system determined based on the fine registration results, connect the first centroid coordinates and the second centroid coordinates to obtain the connection between the source point cloud and the target point cloud, and obtain the connection between the point clouds at multiple times as the trajectory of the UAV swarm.

[0045] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the above-described method for tracking the flight path of an unmanned aerial vehicle (UAV) swarm.

[0046] A flight path tracking device for a swarm of unmanned aerial vehicles (UAVs) includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the aforementioned flight path tracking method for UAVs.

[0047] This invention provides a method, apparatus, device, and medium for tracking the flight paths of unmanned aerial vehicle (UAV) swarms. First, data preprocessing is performed, including acquiring point clouds of the UAV swarm at multiple time points and extracting key feature points within each point cloud. Principal component analysis (PCA) is then used to obtain the first principal components of the key feature points in the source point cloud and the second principal components of the key feature points in the target point cloud. This approach considers local point cloud features while reducing the impact of outlier noise and improves the efficiency of subsequent data processing. Next, coarse registration is performed on the source and target point clouds based on the transformation relationship between the first and second principal components to obtain a coarse registration result. Further fine registration is then performed, including determining all nearest neighbors in the target point cloud based on the coarse registration result and the coordinates of the source point cloud. A registration loss value is calculated based on all nearest neighbors, and the coarse registration result is iteratively adjusted based on the registration loss value until a preset termination condition is met, resulting in a fine registration result between the source and target point clouds. This further improves the registration accuracy. Finally, the flight paths between point clouds are determined. This includes determining the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud within the same coordinate system determined based on the fine registration results. Connecting the first and second centroid coordinates yields a line between the source and target point clouds. These lines at multiple time points are then used as the flight paths of the UAV swarm. The UAV swarm flight path tracking method proposed in this invention not only has good registration accuracy but also good registration efficiency. The combination of these two aspects enables effective UAV swarm flight path tracking. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] in:

[0050] Figure 1 This is a flowchart illustrating a method for tracking the flight path of a drone swarm in one embodiment;

[0051] Figure 2 This is a flowchart illustrating the process of extracting key feature points in a specific embodiment.

[0052] Figure 3 This is a flowchart illustrating the process of selecting key feature points from feature points in a specific embodiment.

[0053] Figure 4 This is a flowchart illustrating the principal component analysis process in a specific embodiment.

[0054] Figure 5 This is a schematic diagram of the point cloud coarse registration process in a specific embodiment;

[0055] Figure 6 This is a flowchart illustrating the point cloud fine registration process in a specific embodiment;

[0056] Figure 7 This is a flowchart illustrating the process of determining the nearest neighbor point in a specific embodiment.

[0057] Figure 8 This is a schematic diagram of the line connecting the source point cloud and the target point cloud in a specific embodiment.

[0058] Figure 9 This is a schematic diagram of the flight path of a drone swarm in a specific embodiment;

[0059] Figure 10 This is a schematic diagram of the trajectory tracking device for a drone swarm in one embodiment;

[0060] Figure 11 This is a structural block diagram of a drone swarm tracking device in one embodiment. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for tracking the flight path of a drone swarm in one embodiment. The steps provided by the method for tracking the flight path of a drone swarm in this embodiment include:

[0063] Step 102: Obtain point clouds of the drone swarm at multiple time points, and extract key feature points within the point cloud at each time point.

[0064] In this embodiment, during the flight of the drone swarm, a point data set of the drone swarm is obtained in real time through measuring instruments. This point data set contains the three-dimensional coordinates of each drone and is also called a point cloud. Point clouds at multiple time intervals can be obtained at equal time intervals, such as every 5 seconds or every 10 seconds, or other forms are also possible, and no specific limitation is made here.

[0065] Considering that individual drones in a drone swarm often experience unstable flight, leading to uneven point cloud distribution and significant positioning noise, this embodiment addresses this issue by employing feature point extraction. This method can reduce the impact of noise points while preserving as much point cloud feature information as possible.

[0066] In one specific embodiment, such as Figure 2 As shown, the specific steps for extracting key feature points include:

[0067] Step 1022: Construct the neighborhood of the first point to be confirmed in the point cloud to be extracted.

[0068] Among them, the first point to be confirmed is a point in the point cloud to be extracted that has not been confirmed as a feature point, the point cloud to be extracted is a point cloud at any time in multiple time periods, and the neighborhood indicator of the point to be confirmed is a spherical space with the point to be confirmed as the center point and extending a preset radius.

[0069] For example, the point cloud at time 1 is taken as the point cloud to be extracted. This point cloud at time 1 contains a point pi, which has not yet been confirmed as a feature point. Then, a spherical space SPi with a preset radius of ε is constructed with pi as the center point.

[0070] Step 1024: Obtain the number of neighborhood points of all neighboring points in the neighborhood of the first point to be confirmed.

[0071] Here, a neighboring point refers to all points within a neighborhood except for the center point. That is, within the spherical space SPi, all points except point pi are neighboring points. This specific step requires counting the number ni of neighboring points within the spherical space SPi of a preset radius ε.

[0072] Step 1026: If the number of neighborhood points is greater than the preset threshold for the number of neighborhood points, then all points in the neighborhood of the first point to be confirmed are determined as feature points in the point cloud to be extracted.

[0073] For example, if the threshold for the number of neighborhood points is n, and ni > n, then all points in the spherical space SPi are determined as feature points. Of course, by continuously repeating steps 1022-1026 above, all feature points within the point cloud at the first time step can be selected.

[0074] Step 1028: Extract key feature points from all feature points in the point cloud to be extracted based on the curvature criterion to obtain the key feature points in the point cloud at each time step.

[0075] Specifically, such as Figure 3 As shown, the specific steps for selecting key feature points from feature points include:

[0076] Step 1028a: Construct a quadratic surface function based on the second point to be confirmed.

[0077] The second point to be confirmed is any one of the feature points in the point cloud to be extracted, that is, any one of the points in the output result of step 1026. A quadratic surface function is constructed based on this second point to be confirmed, and this quadratic surface function is expressed as:

[0078] C(x,y)=ax 2 +bxy+cy 2 (1)

[0079] Step 1028b: Substitute the coordinates of all neighboring points in the target domain into the quadratic surface function for fitting, and calculate the curvature values ​​of all feature points in the target domain based on the fitted quadratic surface function.

[0080] The target domain is the neighborhood of the second point to be confirmed.

[0081] First, construct the fitting objective function as follows:

[0082]

[0083] In the above formula, i indicates the i-th neighboring point in the target domain, and x i y i , z i Let X, Y, and Z be the coordinates of the i-th neighboring point. Substituting all neighboring points within the target area into the above quadratic surface function, and by taking the partial derivative of equation (2) and setting the partial derivative to 0, the values ​​of the parameters a, b, and c of the best-fit surface can be obtained. The calculation process is as follows:

[0084]

[0085] After solving for the parameters a, b, c, the fitting of equation (2) is completed. Finally, the coordinates of all feature points in the target area are substituted into the fitted quadratic surface function to obtain the curvature values ​​of all feature points in the target area.

[0086] Step 1028c: Calculate the local average curvature based on the curvature of all feature points in the target domain, and take the feature points in the point cloud to be extracted whose curvature is greater than the local average curvature as key feature points.

[0087] Step 104: Obtain the first principal component of the key feature points in the source cloud and the second principal component of the key feature points in the target cloud through principal component analysis.

[0088] The target point cloud is the point cloud at any point other than the first point, and the source point cloud is the point cloud at the previous point. For example, if the source point cloud is the point cloud at the first point, then the target point cloud is the point cloud at the second point.

[0089] Considering that the location points of individual drones in a drone swarm may overlap and are susceptible to location interference, this step utilizes principal component analysis to extract feature vectors from the swarm's location feature points. This reduces the impact of location noise and minimizes positioning errors caused by overlapping location points. Furthermore, using the feature vectors as coarse positioning inputs ensures positioning efficiency.

[0090] In one specific embodiment, such as Figure 4 As shown, the specific steps of principal component analysis are as follows:

[0091] Step 1042: Calculate the coordinates of the centroid of the point cloud to be analyzed based on the coordinates of all key feature points within the point cloud.

[0092] The point cloud to be analyzed includes a source point cloud and a target point cloud; that is, in this specific embodiment, the same operation is performed on both the source and target point clouds. The point cloud to be analyzed can be represented in matrix form as follows:

[0093]

[0094] In the above formula, each column represents the coordinates of the i-th key feature point, and m is the total number of key feature points. Therefore, the centroid coordinates of the point cloud to be analyzed can be expressed as: M0 = (x0, y0, z0) T ,in:

[0095]

[0096] Step 1044: Calculate the covariance matrix based on the coordinates of the centroid of the point cloud to be analyzed and the coordinates of all key feature points. Perform singular value decomposition on the covariance matrix to obtain multiple eigenvalues. Select a preset number of eigenvalues ​​with the largest variance from the multiple eigenvalues ​​to form the feature vector of the point cloud to be analyzed.

[0097] The formula for calculating the covariance matrix can be expressed as:

[0098] cov=(X-M0)(X-M0) T (6)

[0099] Next, singular value decomposition is performed on the covariance matrix to obtain multiple eigenvalues. For example, if the first three eigenvalues ​​with the largest variance (λ1, λ2, λ3) are taken, the resulting eigenvector is denoted as:

[0100] V=[η1,η2,η3] (7)

[0101] Step 1046: The centroid coordinates and feature vectors of the source point cloud are used as the first principal components, and the centroid coordinates and feature vectors of the target point cloud are used as the second principal components.

[0102] After steps 1042-1044 above, the centroid coordinates and eigenvectors of the point cloud to be analyzed can be obtained. Optionally, the centroid coordinates of the source point cloud are represented as M0 = (x0, y0, z0). T If the eigenvectors of the source point cloud are represented as V = [η1, η2, η3], then M0 and V together form the first principal component.

[0103] The centroid coordinates of the target point cloud are represented as M'0 = (x'0, y'0, z'0). T If the feature vector of the target point cloud is represented as W = [ξ1, ξ2, ξ3], then M'0 and W together form the second principal component.

[0104] Step 106: Perform coarse registration between the source point cloud and the target point cloud based on the transformation relationship between the first principal component and the second principal component to obtain the coarse registration result between the source point cloud and the target point cloud.

[0105] To improve the efficiency of UAV swarm trajectory tracking, this invention employs a registration method combining coarse and fine registration to achieve UAV trajectory tracking. The registration results are used to transform the source and target point clouds to the same coordinate system. The main purpose of the coarse registration stage is to provide relatively good initial values ​​for fine registration.

[0106] In one specific embodiment, such as Figure 5 As shown, the specific steps for coarse registration of point clouds are as follows:

[0107] Step 1062: Establish an initial rotation matrix based on the transformation relationship between the feature vectors of the source point cloud and the feature vectors of the target point cloud.

[0108] Specifically, the initial rotation matrix is ​​represented as follows:

[0109] R0 = VW -1 (8)

[0110] Step 1064: Establish an initial translation matrix based on the transformation relationship between the centroid coordinates of the source point cloud and the centroid coordinates of the target point cloud.

[0111] Specifically, the initial translation matrix is ​​represented as follows:

[0112] T0 = ​​M0' - M0 (9)

[0113] Step 1066: Transform the coordinate system of all points in the source point cloud to the coordinate system of the target point cloud based on the initial rotation matrix and the initial translation matrix to obtain the coarse registration result between the source point cloud and the target point cloud.

[0114] Specifically, the conversion process is represented as follows:

[0115] P'=R0*P+T0 (10)

[0116] Where P is the source point cloud, and P' is the point cloud after rotation and translation of the source point cloud.

[0117] Step 108: Based on the coarse registration result between the source point cloud and the target point cloud, determine all nearest points in the target point cloud according to the coordinates of the source point cloud, calculate the registration loss value based on all nearest points, and iteratively adjust the coarse registration result based on the registration loss value until the preset termination condition is met, and obtain the fine registration result between the source point cloud and the target point cloud.

[0118] To improve the accuracy of key feature point cloud registration for UAV swarms, fine registration is performed after coarse registration.

[0119] In one specific embodiment, such as Figure 6 As shown, the specific steps for fine registration of point clouds include:

[0120] Step 1082: Transform the coordinate system of all points in the source point cloud to the coordinate system of the target point cloud according to the initial rotation matrix and the initial translation matrix to obtain the transformed point cloud. Determine the nearest neighbor of the transformed point in the target point cloud to obtain all the nearest neighbor points of the transformed point cloud.

[0121] In this context, the transform point is any point in the transformed point cloud, and the nearest neighbor point is the point in the target point cloud that is closest to the transform point. That is, if the transformed point cloud is P', one of its transform points is P'. i The target point cloud is Q, and this target point cloud Q contains multiple points Q. i Then it is necessary to use these Qs. i Find P' i The nearest point is selected as the nearest neighbor. By repeating this process, all nearest neighbors of the transformed point cloud can be obtained.

[0122] Optional, such as Figure 7 As shown, the specific steps for determining the nearest neighbor point include:

[0123] Step 1082a: Input the transformation points and target point cloud into the 3D decision tree.

[0124] Specifically, since the coordinates of the UAV include coordinates on the x, y, and z axes, a 3D decision tree is used here to divide the transformation point and the target point cloud.

[0125] Step 1082b: In the i-th root node of the 3D decision tree, the target point cloud and transformation points are divided into two i-th child nodes in the target dimension based on the set partitioning criterion.

[0126] Where i is initially 1, and at the i-th division, the median of all coordinates of the transformation point and the target point cloud in the target dimension is the division criterion of the i-th root node. The target dimension is any one of x, y and z dimensions.

[0127] For example, at the first root node, the target dimension needs to be determined first. Optionally, the target dimension can be any one of the x, y, and z dimensions. Alternatively, the variance of each data point can be calculated along each dimension, and the dimension with the largest variance is the most imbalanced dimension, resulting in the best partitioning effect; this dimension is then selected as the target dimension. For instance, if the x dimension is selected as the target dimension, the median of all coordinates of the transformed point and the target point cloud along the x dimension is set as the partitioning criterion for the first root node. Partitioning is then performed at the first root node based on this criterion, thus evenly dividing the transformed point and the target point cloud into two first child nodes.

[0128] Step 1082c: Determine if the number of current nearest neighbors of the transformed point is less than or equal to a preset number of n. If the number of current nearest neighbors of the transformed point is greater than the preset number of n, then take the i-th child node containing the transformed point as the (i+1)-th root node, let i = i+1, and return to step 1082b and subsequent steps. If the number of current nearest neighbors of the transformed point is less than or equal to the preset number of n, then execute step 1082d, taking all current nearest neighbors of the transformed point as the nearest neighbor.

[0129] In this context, the current nearest neighbor of the transformation point and the current transformation point are both assigned to the same child node.

[0130] For example, after step 1082b, some points and transformed points within the target point cloud are assigned to the i-th child node on the left, while the remaining points are assigned to the i-th child node on the right. Therefore, the current nearest neighbors counted here are all points within the i-th child node on the left, excluding the transformed points, and the number of current nearest neighbors is denoted as n0. Thus, this step determines whether n0 is less than or equal to n.

[0131] If the condition in step 1082b is not met, it is considered that the nearest neighbor has not yet been found. In this case, steps 1082b-1082c are repeated to continue the partitioning. If the condition in step 1082b is met, all current nearest neighbors of the transformed point are taken as the nearest neighbors.

[0132] By adopting the above specific embodiments, the efficiency of matching the nearest neighbor point of the transformation point can be effectively improved, which in turn further improves the efficiency of fine registration.

[0133] Step 1084: Calculate the current average distance between all nearest neighbors of the transformed point cloud and the transformed point cloud.

[0134] The formula for calculating the current average distance can be expressed as:

[0135]

[0136] In the above formula, S represents the number of clusters, which is the total number of points in the point cloud; R t-1 T is the rotation matrix at the t-th iteration; t-1 Let be the displacement matrix at the t-th iteration.

[0137] Step 1086: Use the current average distance as the registration loss value and adjust it based on the least squares method to update the initial rotation matrix and the initial translation matrix, and obtain the updated rotation matrix and the updated translation matrix.

[0138] For example, in the first iteration, the initial rotation matrix R0 and the initial translation matrix T0 are used, and after updating, R1 and T1 are obtained. This fine registration process is actually updating R... t-1 and T t-1 The process here uses the least squares method for adjustment, but other methods can also be used for adjustment.

[0139] Step 1088: Transform the coordinate system of all points in the source point cloud to the coordinate system of the target point cloud according to the updated rotation matrix and the updated translation matrix to obtain the updated transformed point cloud. Determine the nearest neighbor of the transformed point in the target point cloud to obtain all the nearest neighbor points of the updated transformed point cloud.

[0140] Step 1090: Calculate the updated average distance between all nearest neighbors of the updated transformed point cloud and the updated transformed point cloud.

[0141] Steps 1088-1090 above are basically the same as steps 1082-1084. The main difference is that here we use the updated rotation matrix and the updated translation matrix instead of the initial rotation matrix and the initial translation matrix.

[0142] Step 1092: Determine whether the updated average distance is less than or equal to a preset distance threshold. If the updated average distance is greater than the preset distance threshold, use the updated average distance as the current average distance and return to step 1086 and subsequent steps. If the updated average distance is less than or equal to the preset distance threshold, proceed to step 1094.

[0143] Step 1094: Determine if the preset termination condition is met, obtain the updated rotation matrix and the updated translation matrix, and transform the coordinate system of all points in the source point cloud to the coordinate system of the target point cloud based on the updated rotation matrix and the updated translation matrix to obtain the fine registration result between the source point cloud and the target point cloud.

[0144] In steps 1092-1094 above, if the condition in step 1092 is not met, the process of updating the rotation and translation matrices is repeated. If the condition in step 1092 is met, registration is performed based on the last updated rotation and translation matrices to obtain a fine registration result.

[0145] Step 110: Within the same coordinate system determined based on the fine registration results, determine the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud, connect the first centroid coordinates and the second centroid coordinates to obtain the line connecting the source point cloud and the target point cloud, and obtain the line connecting the point clouds at multiple times as the track of the UAV swarm.

[0146] Among them, the centroid coordinates The formula for determining it is:

[0147]

[0148]

[0149] Where, x m Let x be the x-axis coordinate of the m-th key feature point in the cluster, and y be the x-axis coordinate of the m-th key feature point in the cluster. m Let y be the y-coordinate of the m-th key feature point in the cluster.

[0150] For example, such as Figure 8 As shown, the area within the dashed circle on the left is the source point cloud, and the area within the dashed circle on the right is the target point cloud. After registering the source and target point clouds to the same coordinate system based on the fine registration result, the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud can be determined based on equation (12). Then, connecting these two points yields the line connecting the source and target point clouds. Similarly, as... Figure 9 As shown, by repeating the above steps, we can obtain the connections between point clouds at multiple times, which is the flight path of the drone swarm.

[0151] In one embodiment, such as Figure 10 As shown, a trajectory tracking device for a swarm of unmanned aerial vehicles (UAVs) is proposed, the device comprising:

[0152] The data preprocessing module 1002 is used to acquire point clouds of the UAV swarm at multiple times and extract key feature points in the point cloud at each time. Through principal component analysis, the first principal component of the key feature points in the source point cloud and the second principal component of the key feature points in the target point cloud are obtained. The target point cloud is the point cloud at any time except the first time, and the source point cloud is the point cloud at the time before the target point cloud.

[0153] The coarse registration module 1004 is used to perform coarse registration between the source point cloud and the target point cloud according to the transformation relationship between the first principal component and the second principal component, so as to obtain the coarse registration result between the source point cloud and the target point cloud; wherein, the registration result is used to transform the source point cloud and the target point cloud to the same coordinate system;

[0154] The fine registration module 1006 is used to determine all nearest points in the target point cloud based on the coarse registration result between the source point cloud and the target point cloud, calculate the registration loss value based on all nearest points, and iteratively adjust the coarse registration result based on the registration loss value until the preset termination condition is met, so as to obtain the fine registration result between the source point cloud and the target point cloud.

[0155] The track determination module 1006 is used to determine the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud in the same coordinate system determined based on the fine registration result, connect the first centroid coordinates and the second centroid coordinates to obtain the connection between the source point cloud and the target point cloud, and obtain the connection between the point clouds at multiple times as the track of the UAV swarm.

[0156] Figure 11 An internal structural diagram of the trajectory tracking device for a drone swarm in one embodiment is shown. Figure 11 As shown, the flight path tracking device for this UAV swarm includes a processor, memory, and network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement the UAV swarm flight path tracking method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to implement the UAV swarm flight path tracking method. Those skilled in the art will understand that... Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the trajectory tracking device of the UAV swarm to which the present application is applied. The specific UAV swarm trajectory tracking device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0157] A flight path tracking device for a swarm of unmanned aerial vehicles (UAVs) includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: acquiring point clouds of the UAV swarm at multiple time points, and extracting key feature points within the point cloud at each time point; obtaining a first principal component of the key feature points in the source point cloud and a second principal component of the key feature points in the target point cloud through principal component analysis; performing coarse registration between the source point cloud and the target point cloud based on the transformation relationship between the first and second principal components to obtain a coarse registration result between the source point cloud and the target point cloud; and performing coarse registration based on the source point cloud and the target point cloud. The coarse registration result between the source and target point clouds is obtained by determining all nearest neighbors in the target point cloud based on the coordinates of the source point cloud, calculating the registration loss value based on all nearest neighbors, and iteratively adjusting the coarse registration result based on the registration loss value until a preset termination condition is met, thus obtaining the fine registration result between the source and target point clouds. In the same coordinate system determined based on the fine registration result, the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud are determined, and the first centroid coordinates and the second centroid coordinates are connected to obtain the line between the source and target point clouds. The line between the point clouds at multiple time points is obtained as the flight path of the UAV swarm.

[0158] A computer-readable storage medium storing a computer program, which, when executed by a processor, performs the following steps: acquiring point clouds of a UAV swarm at multiple time points and extracting key feature points within the point cloud at each time point; obtaining, through principal component analysis, a first principal component of the key feature points in the source point cloud and a second principal component of the key feature points in the target point cloud; performing coarse registration of the source point cloud and the target point cloud based on the transformation relationship between the first and second principal components to obtain a coarse registration result between the source point cloud and the target point cloud; and performing coarse registration based on the coarse registration result between the source point cloud and the target point cloud. For the coarse registration result, all nearest points in the target point cloud are determined based on the coordinates of the source point cloud. The registration loss value is calculated based on all nearest points, and the coarse registration result is iteratively adjusted based on the registration loss value until a preset termination condition is met, thus obtaining the fine registration result between the source point cloud and the target point cloud. In the same coordinate system determined based on the fine registration result, the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud are determined. The first centroid coordinates and the second centroid coordinates are connected to obtain the line between the source point cloud and the target point cloud. The line between the point clouds at multiple time points is obtained as the track of the UAV swarm.

[0159] It should be noted that the above-mentioned drone swarm tracking method, apparatus, device, and computer-readable storage medium belong to a general inventive concept, and the contents of the embodiments of the drone swarm tracking method, apparatus, device, and computer-readable storage medium are applicable to each other.

[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for tracking the flight paths of a swarm of unmanned aerial vehicles (UAVs), characterized in that, The method includes: Acquire point clouds of the drone swarm at multiple time points, and extract key feature points within the point cloud at each time point; Principal component analysis is used to obtain the first principal component of key feature points in the source point cloud and the second principal component of key feature points in the target point cloud. The target point cloud is the point cloud at any time other than the first time, and the source point cloud is the point cloud at the previous time of the target point cloud. The source point cloud and the target point cloud are coarsely registered according to the transformation relationship between the first principal component and the second principal component to obtain the coarse registration result between the source point cloud and the target point cloud; wherein, the registration result is used to transform the source point cloud and the target point cloud to the same coordinate system; Based on the coarse registration result between the source point cloud and the target point cloud, all nearest points in the target point cloud are determined according to the coordinates of the source point cloud. The registration loss value is calculated based on all nearest points, and the coarse registration result is iteratively adjusted based on the registration loss value until the preset termination condition is met, so as to obtain the fine registration result between the source point cloud and the target point cloud. Within the same coordinate system determined based on the fine registration results, the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud are determined. The first centroid coordinates and the second centroid coordinates are connected to obtain the line between the source point cloud and the target point cloud. The line between the point clouds at multiple times is obtained as the track of the UAV swarm. The extraction of key feature points within the point cloud at each time step includes: In the point cloud to be extracted, a neighborhood of the first point to be confirmed is constructed; wherein, the first point to be confirmed is a point in the point cloud to be extracted that has not been confirmed as a feature point, the point cloud to be extracted is a point cloud at any time in multiple time periods, and the neighborhood of the point to be confirmed indicates a spherical space with the point to be confirmed as the center point and extending a preset radius. Obtain the number of neighborhood points of all neighboring points within the neighborhood of the first point to be confirmed; where neighborhood points refer to all points within the neighborhood except the center point; If the number of neighborhood points is greater than the preset threshold for the number of neighborhood points, then all points in the neighborhood of the first point to be confirmed are determined as feature points in the point cloud to be extracted. Based on the curvature criterion, key feature points are extracted from all feature points in the point cloud to be extracted, so as to obtain the key feature points in the point cloud at each time.

2. The method according to claim 1, characterized in that, The extraction of key feature points from all feature points within the point cloud to be extracted based on the curvature criterion includes: A quadratic surface function is constructed based on the second point to be confirmed; wherein, the second point to be confirmed is any one of all feature points in the point cloud to be extracted; The coordinates of all neighborhood points within the target domain are substituted into the quadratic surface function for fitting, and the curvature values ​​of all feature points within the target domain are calculated based on the fitted quadratic surface function; wherein, the target domain is the neighborhood of the second point to be confirmed; Calculate the local average curvature based on the curvature of all feature points in the target domain, and select feature points in the point cloud whose curvature is greater than the local average curvature as key feature points.

3. The method according to claim 1, characterized in that, The process of obtaining the first principal components of key feature points within the source point cloud and the second principal components of key feature points within the target point cloud through principal component analysis includes: The coordinates of the centroid of the point cloud to be analyzed are calculated based on the coordinates of all key feature points within the point cloud to be analyzed; wherein, the point cloud to be analyzed includes the source point cloud and the target point cloud; The covariance matrix is ​​calculated based on the coordinates of the centroid and all key feature points of the point cloud to be analyzed. Singular value decomposition is performed on the covariance matrix to obtain multiple eigenvalues. A preset number of eigenvalues ​​with the largest variance are selected from the multiple eigenvalues ​​to form the feature vector of the point cloud to be analyzed. The centroid coordinates and eigenvectors of the source point cloud are used as the first principal component, and the centroid coordinates and eigenvectors of the target point cloud are used as the second principal component.

4. The method according to claim 3, characterized in that, The step of performing coarse registration of the source point cloud and the target point cloud based on the transformation relationship between the first principal component and the second principal component to obtain the coarse registration result between the source point cloud and the target point cloud includes: Establish an initial rotation matrix based on the transformation relationship between the feature vectors of the source point cloud and the feature vectors of the target point cloud; An initial translation matrix is ​​established based on the transformation relationship between the centroid coordinates of the source point cloud and the centroid coordinates of the target point cloud. The coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud based on the initial rotation matrix and the initial translation matrix to obtain the coarse registration result between the source point cloud and the target point cloud.

5. The method according to claim 4, characterized in that, The coarse registration result between the source point cloud and the target point cloud involves determining all nearest neighbors in the target point cloud based on the coordinates of the source point cloud, calculating the registration loss value based on all nearest neighbors, and iteratively adjusting the coarse registration result based on the registration loss value until a preset termination condition is met, thereby obtaining the fine registration result between the source point cloud and the target point cloud, including: The coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud according to the initial rotation matrix and the initial translation matrix to obtain the transformed point cloud. The nearest neighbor point of the transformed point is determined in the target point cloud to obtain all the nearest neighbor points of the transformed point cloud. The transformed point is any point in the transformed point cloud, and the nearest neighbor point is the point in the target point cloud that is closest to the transformed point. Calculate the current average distance between all nearest neighbors of the transformed point cloud and the transformed point cloud, use the current average distance as the registration loss value and adjust it based on the least squares method to update the initial rotation matrix and the initial translation matrix, and obtain the updated rotation matrix and the updated translation matrix. The coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud according to the updated rotation matrix and the updated translation matrix to obtain the updated transformed point cloud. The nearest neighbor of the transformed point is determined in the target point cloud to obtain all the nearest neighbor points of the updated transformed point cloud. Calculate the updated average distance between all nearest neighbors of the updated transformed point cloud and the updated transformed point cloud, determine whether the updated average distance is less than or equal to a preset distance threshold, if the updated average distance is greater than the preset distance threshold, then take the updated average distance as the current average distance, return to execute the step of taking the current average distance as the registration loss value and adjusting it based on the least squares method and subsequent steps. If the updated average distance is less than or equal to a preset distance threshold, then the preset termination condition is met. The updated rotation matrix and the updated translation matrix are obtained. Based on the updated rotation matrix and the updated translation matrix, the coordinate system of all points in the source point cloud is transformed to the coordinate system of the target point cloud to obtain the fine registration result between the source point cloud and the target point cloud.

6. The method according to claim 5, characterized in that, Determining the nearest neighbor point of the transformation point within the target point cloud includes: Input the transformation point and the target point cloud into a 3D decision tree. In the i-th root node of the 3D decision tree, the target point cloud and the transformation point are divided into two i-th child nodes in the target dimension based on a set partitioning criterion. Here, the initial value of i is 1. At the i-th partition, the median value of all coordinates of the transformation point and the target point cloud in the target dimension is the partitioning criterion of the i-th root node. The target dimension is any one of x-dimensional, y-dimensional and z-dimensional. Determine whether the number of current nearest neighbors of the transformation point is less than or equal to a preset number of n; wherein, the current nearest neighbors of the transformation point and the transformation point are all assigned to the same child node; If the number of current nearest neighbors of the transformation point is greater than the preset n, then the i-th child node containing the transformation point is taken as the (i+1)-th root node, i=i+1, and the process returns to the step of dividing the target point cloud and the transformation point into two i-th child nodes in the target dimension based on the set division criterion within the i-th root node of the three-dimensional decision tree, and subsequent steps. If the number of current nearest neighbors of the transformation point is less than or equal to the preset n, then all current nearest neighbors of the transformation point are taken as the nearest neighbor.

7. A trajectory tracking device for a swarm of unmanned aerial vehicles (UAVs), characterized in that, The device includes: The data preprocessing module is used to acquire point clouds of the UAV swarm at multiple time points and extract key feature points in the point cloud at each time point; through principal component analysis, the first principal component of the key feature points in the source point cloud and the second principal component of the key feature points in the target point cloud are obtained; wherein, the target point cloud is the point cloud at any time point other than the first time point among the point clouds at multiple time points, and the source point cloud is the point cloud at the time point before the target point cloud. The coarse registration module is used to perform coarse registration between the source point cloud and the target point cloud according to the transformation relationship between the first principal component and the second principal component, so as to obtain the coarse registration result between the source point cloud and the target point cloud; wherein, the registration result is used to transform the source point cloud and the target point cloud to the same coordinate system; The fine registration module is used to determine all nearest points in the target point cloud based on the coarse registration result between the source point cloud and the target point cloud, calculate the registration loss value based on all nearest points, and iteratively adjust the coarse registration result based on the registration loss value until a preset termination condition is met, so as to obtain the fine registration result between the source point cloud and the target point cloud. The trajectory determination module is used to determine the first centroid coordinates of the source point cloud and the second centroid coordinates of the target point cloud within the same coordinate system determined based on the fine registration results, connect the first centroid coordinates and the second centroid coordinates to obtain the connection between the source point cloud and the target point cloud, and obtain the connection between the point clouds at multiple times as the trajectory of the UAV swarm. The data preprocessing module is specifically used to acquire point clouds of the drone swarm at multiple times, and to construct a neighborhood of a first point to be confirmed in the point cloud to be extracted; wherein, the first point to be confirmed is a point in the point cloud to be extracted that has not been confirmed as a feature point, the point cloud to be extracted is a point cloud at any time in the multiple time points, and the neighborhood of the point to be confirmed indicates a spherical space with the point to be confirmed as the center point and extending a preset radius. Obtain the number of neighborhood points of all neighboring points within the neighborhood of the first point to be confirmed; where neighborhood points refer to all points within the neighborhood except the center point; If the number of neighborhood points is greater than the preset threshold for the number of neighborhood points, then all points in the neighborhood of the first point to be confirmed are determined as feature points in the point cloud to be extracted. Based on the curvature criterion, key feature points are extracted from all feature points in the point cloud to be extracted, so as to obtain the key feature points in the point cloud at each time.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the processor performs the steps of the method as described in any one of claims 1 to 6.

9. A flight path tracking device for a swarm of unmanned aerial vehicles (UAVs), comprising a memory and a processor, characterized in that, The memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.

Images