Cataloging method of uav swarm based on local trajectory curvature

Abstract

The present application relates to the technical field of unmanned aerial vehicle cataloging, and particularly relates to a cataloging method for unmanned aerial vehicle swarms based on local trajectory curvature, comprising the following steps: S1: collecting image sequences of the unmanned aerial vehicle swarms to generate a set of unmanned aerial vehicle positions; S2: in the initial frame, assigning a unique number to each target; S3: based on a multi-frame target association method, forming a preliminary cataloging and a preliminary target trajectory; S4: using a trajectory curvature discrimination algorithm to determine the correct attribution of the interlaced trajectory segments, correcting the erroneous association in the preliminary target trajectory, and obtaining a corrected target cataloging; S5: for all targets with interlaced trajectories, repeating step S4 until the final cataloging of all targets in all frames of images is completed. The present application has the advantage that the trajectory curvature discrimination algorithm is reasonably applied to multi-target cataloging, which can well classify and catalog the positions of each target and accurately handle the interlaced situation of two target trajectories, effectively solving the difficult problems in multi-target cataloging.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle cataloging, and particularly relates to a cataloging method for unmanned aerial vehicle swarms based on local trajectory curvature. BACKGROUND

[0002] Traditional measurement and control tasks have mature processing methods for trajectory measurement and tracking of a single target, but when the measurement and control scene extends to unmanned aerial vehicle swarms, the increase in degrees of freedom brings a geometric growth in calculation amount, such as the increase in the number of traversal rendezvous calculations from n times to n 2 times. Various trajectory intersection scenarios between targets introduce a problem of explosive growth in trajectory combination complexity, such as the need to distinguish the attribution of trajectory segments after the intersection of two target trajectories. In addition, real-time trajectory changes of a maneuvering target can cause a tracking algorithm based on inertial targets to fail, such as a sudden jump in motion parameters such as the speed and acceleration of a target in an unmanned aerial vehicle swarm. Therefore, efficient and accurate cataloging of the swarm targets is a prerequisite and key to realizing trajectory measurement and tracking of unmanned aerial vehicle swarms.

[0003] Unmanned aerial vehicle swarms belong to multiple targets, and there are two major difficulties in cataloging the targets: one is that there are dozens of targets to be processed, and it is inevitable that two target trajectories will intersect in a continuous frame of video, and the minimum coplanar difference algorithm used in rendezvous calculations is invalid at the intersection point of the two trajectories, so that the target cataloging of subsequent frames after the intersection of the two trajectories cannot be accurately completed; the other is that the evaluation function method based on three positions of the same target in the past target cataloging, which is mainly suitable for inertial targets with relatively stable trajectory changes, but unmanned aerial vehicles belong to high-maneuvering targets that can change trajectory in real time, and the motion quantities such as speed and acceleration can suddenly change at a certain moment, which can cause the error rate of the evaluation function method to suddenly increase, resulting in a decrease in the accuracy of target cataloging. SUMMARY

[0004] Therefore, the cataloging method for unmanned aerial vehicle swarms based on local trajectory curvature reasonably applies a trajectory curvature discrimination algorithm to multi-target cataloging, can well classify and catalog the positions of each target and accurately process the intersection of two target trajectories, and effectively solves the difficult problems in multi-target cataloging.

[0005] To achieve the above object, the technical scheme created by the present application is as follows: a cataloging method for unmanned aerial vehicle swarms based on local trajectory curvature, comprising the following steps:

[0006] S1: collecting an image sequence of a target group, extracting the center positions of all targets in each frame of image through image processing, and generating a target position set;

[0007] S2: assigning a unique number to each target in the initial frame, and initializing the target variable corresponding to the target;

[0008] S3: Based on the multi-frame target association method, the pixel distance of each target in the next frame from all targets in the current frame is calculated, and the target position in the next frame is associated with the corresponding numbered target variable in the current frame according to the principle of minimum pixel distance, until the target association of all frames is completed, forming a preliminary cataloging and a preliminary target trajectory;

[0009] S4: During the formation of the preliminary target trajectory, the target number and the corresponding frame number of the trajectory intersection are recorded; the correct attribution of the intersected trajectory segment is determined using a trajectory curvature determination algorithm, and the erroneous association in the preliminary target trajectory is corrected to obtain a corrected target catalog;

[0010] S5: For all targets with trajectory intersection, step S4 is repeated until the final cataloging of all targets in all frames of images is completed.

[0011] Further, step S4 includes the following steps:

[0012] S41: The trajectories of the two targets with trajectory intersection are divided into two segments before and after the intersection point, forming two sets of trajectory combinations;

[0013] The trajectory combination of the original multi-frame target association is defined as the first set of trajectory combinations; the trajectory combination formed after the exchange of the trajectory segments after the intersection point is defined as the second set of trajectory combinations; each set of trajectory combinations includes two target trajectories;

[0014] S42: The trajectory point data of the two target trajectories in each of the t frames before and after the intersection point is extracted, and parabolic fitting is performed based on the trajectory point data to obtain four fitted parabolas;

[0015] S43: The curvature value of each fitted parabola at the intersection point is calculated;

[0016] S44: The curvature values of each target trajectory at the intersection point are compared, and the trajectory combination in which the target trajectory with the smallest curvature is located is determined as the true trajectory;

[0017] S45: If the trajectory combination corresponding to the smallest curvature is the trajectory of multi-frame target association, the original trajectory is maintained;

[0018] If the trajectory combination corresponding to the smallest curvature is the exchanged trajectory combination, the trajectory segments of the two targets after the intersection point are interchanged.

[0019] Further, t in step S42 is greater than or equal to 5.

[0020] Further, the parabola and its first and second derivative formulas are as follows:

[0021] ;

[0022] wherein, , and respectively represent the quadratic term coefficient, the linear term coefficient and the zero term coefficient of the parabola, which are determined by the least square fitting; X and Y respectively represent the X-axis coordinate and the Y-axis coordinate in the image coordinate system; X respectively represent the X-axis coordinate and the Y-axis coordinate in the image coordinate system; Y respectively represent the X-axis coordinate and the Y-axis coordinate in the image coordinate system; and respectively represent the first derivative and the second derivative of Y .

[0023] Further, the calculation formula of the trajectory curvature at the intersection point is:

[0024] .

[0025] wherein K represents the curvature of the trajectory at the intersection point; represents the image horizontal coordinate of the intersection point of the two trajectories.

[0026] Further, in step S2: in the initial frame, all unmanned aerial vehicles are sorted according to the same axis of the image coordinate system, and the target positions are preliminarily cataloged.

[0027] Further, in step S3: the calculation formula of the pixel distance of each target position in the next frame relative to all targets in the current frame is:

[0028] .

[0029] wherein, represents the pixel distance of the nth frame relative to the previous frame; , respectively represent the X-axis coordinate and the Y-axis coordinate of the target i to be cataloged in the nth frame under the image coordinate system; , respectively represent the X-axis coordinate and the Y-axis coordinate of the target j cataloged in the nth-1 frame under the image coordinate system.

[0030] The present application can achieve the following beneficial effects: the present application combines the trajectory characteristics of the unmanned aerial vehicle swarm, reasonably applies the trajectory curvature discrimination algorithm, can effectively catalog each target position, and accurately calculates the curvature at the intersection point of the two trajectories to determine the attribution of the trajectory segment after the trajectory intersection, effectively solves the difficult problems in the multi-target cataloging, and has high algorithm sensitivity and is easy to be applied in practice. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which form a part of the present application, are included to provide a further understanding of the application and are incorporated herein for purposes of illustration. The embodiments of the present application, together with its advantages, can best be understood by referring to the following description taken in connection with the accompanying drawings, in which:

[0032] Figure 1 is a flowchart of a cataloging method of a UAV swarm based on local trajectory curvature according to an embodiment of the present application;

[0033] Figure 2 is a preliminary target trajectory processed by a multi-frame target association method according to an embodiment of the present application;

[0034] Figure 3 is a schematic diagram of a fitted parabola at the intersection of two target trajectories according to an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of a corrected target trajectory after trajectory curvature discrimination according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not constitute a limitation on the present application.

[0037] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0038] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only used for description purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0039] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "linking" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0040] The present application will be described in detail below with reference to the embodiments.

[0041] As shown in the figure, the cataloging method of the UAV swarm based on local trajectory curvature provided by the embodiment of the present application comprises the following steps: Figures 1 to 4

[0042] S1: The phototheodolite collects the image sequence of the target group, extracts the center position of all target images in each frame of image through image processing, and generates the target position set.

[0043] S2: In the initial frame, a unique number is assigned to each target, and the target variable of each target is initialized.

[0044] Specifically, the target trajectory of the continuous frame (taking a trajectory data containing 30 UAVs as an example) is input, the n targets in the initial frame (i.e. the first frame) are sorted according to the image coordinate system axis direction, and the target position (i.e. the image coordinates) of the n targets is obtained. A unique number is assigned to each target, which is 1, 2, 3...n respectively, and the target variable is initialized. In this embodiment, the target variable is the image coordinates of the target.

[0045] Wherein, represents the X-axis coordinate and Y-axis coordinate of target 1 under the image coordinate system of the first frame; represents the X-axis coordinate and Y-axis coordinate of target 2 under the image coordinate system of the first frame; represents the X-axis coordinate and Y-axis coordinate of target n under the image coordinate system of the first frame.

[0046] S3: Based on the multi-frame target association method, the pixel distance of each target in the next frame from all targets in the current frame is calculated, and according to the principle of minimum pixel distance, the target in the next frame is associated to the corresponding numbered target variable in the current frame, until the target association of all frames is completed, forming the preliminary cataloging and preliminary target trajectory.

[0047] The calculation formula of the pixel distance of each target in the next frame from all targets in the current frame is: ​​

[0048] ;

[0049] wherein, represents the pixel distance of the nth frame relative to the previous frame; 、 respectively represent the X-axis coordinate and Y-axis coordinate of the target i to be cataloged in the n-th frame image coordinate system; X 、 Y respectively represent the X-axis coordinate and Y-axis coordinate of the cataloged target i in the n-th frame image coordinate system. 、 respectively represent the X-axis coordinate and Y-axis coordinate of the cataloged target i in the n-th frame image coordinate system. j 、 n-1 respectively represent the X-axis coordinate and Y-axis coordinate of the cataloged target i in the n-th frame image coordinate system. X 、

[0050] For each target in the n-th frame, find the minimum pixel distance between each target in this frame and each target in the n-1-th frame n , and thus the target variable of the i-th target in the n-th frame can be stored in the target variable of the cataloged target in the n-1-th frame. 、 n-1 For each target in the n-th frame, find the minimum pixel distance between each target in this frame and each target in the n-1-th frame j , and thus the target variable of the i-th target in the n-th frame can be stored in the target variable of the cataloged target in the n-1-th frame.

[0051] S4: In the process of forming the preliminary target trajectory, record the target number and corresponding frame number of the trajectory intersection; use the trajectory curvature discrimination algorithm to determine the correct attribution of the intersected trajectory segment, correct the wrong association in the preliminary target trajectory, and obtain the corrected target catalog. It includes the following steps:

[0052] S41: Divide the trajectories of the two targets intersected in the trajectory into two segments according to the intersection point, forming two groups of trajectory combinations;

[0053] The original trajectory combination of the target association of multiple frames is defined as the first group of trajectory combinations; the trajectory combination formed after exchanging the trajectory segments after the intersection point is defined as the second group of trajectory combinations; each group of trajectory combinations contains two target trajectories.

[0054] It should be noted that if one of the target trajectories contains multiple intersection points, the target trajectory is segmented by taking each intersection point as a dividing point.

[0055] S42: Extract the trajectory point data of the two target trajectories in each t frame before and after the intersection point, and perform parabolic fitting based on the trajectory point data to obtain four fitted parabolas;

[0056] S43: Calculate the curvature value of each fitted parabola at the intersection point;

[0057] S44: Compare the curvature values of each target trajectory at the intersection point, and determine the trajectory combination in which the target trajectory with the smallest curvature is the true trajectory;

[0058] S45: If the trajectory combination corresponding to the minimum curvature is the trajectory combination of the multi-frame target association, the original trajectory is maintained;

[0059] If the trajectory combination corresponding to the minimum curvature is the trajectory combination after the exchange, the trajectory segments of the two targets after the intersection point are exchanged.

[0060] In actual measurement, the target trajectory is not a continuous smooth curve, but a series of discrete pixel points. Therefore, the trajectory points of the previous and subsequent frames can be taken first, and the least square method is used to fit the trajectory curve, and then the curvature is calculated. When the curvature is small, the trajectory curve tends to be flat, and the greater the curvature, the more acute the trajectory curve. A pair of trajectory curves produces two possible trajectory groups after the intersection point, and the trajectory group with the minimum curvature is the true trajectory group.

[0061] Specifically, as shown in Figure 2 , in the preliminary cataloging process, the target numbers and corresponding frame numbers of the trajectory intersections are recorded ; Wherein m and i represent the mth frame and the ith intersection point respectively; X and Y represent the X axis coordinates and Y axis longitudinal coordinates of the i point in the image coordinate system.

[0062] The two target trajectories of the trajectory intersection are divided into two segments according to the intersection point, forming two trajectory combinations. The front and rear segments of one of the target trajectories are defined as and , and the front and rear segments of the other target trajectory are defined as and , and belong to the front segment of the intersection point, and belong to the rear segment of the intersection point.

[0063] For the mth frame of the effective intersection point, take the trajectory point data of the two target trajectories of the previous and subsequent 5 frames of the trajectory intersection frame, a total of 21 trajectory points, and fit four parabolas for the 21 trajectory points: , , , .

[0064] Among them, and are the first trajectory combination of the multi-frame target association in step S3, and are the trajectory combination of the trajectory segment after the exchange of the intersection point.

[0065] Considering that parabolic functions have few undetermined parameters (only three) and simple first and second derivative forms, parabolic fitting can significantly simplify curvature calculation near the intersection of two trajectories.

[0066] The formulas for a parabola and its first and second derivatives are as follows:

[0067] ;

[0068] in, , and Let represent the coefficients of the quadratic term, the linear term, and the zeroth-order term of the parabola, respectively, which are determined by least squares fitting. X and Y In the image coordinate system X axis coordinates and Y Axis coordinates; and They represent Y The first and second derivatives.

[0069] Considering the continuity of the target's motion, the formula for calculating the curvature of a continuous, smooth, and differentiable function is:

[0070] ;

[0071] Where K represents curvature; The x-coordinate of the image representing the intersection point of the two trajectories.

[0072] Find the minimum curvature ,like In the first set of trajectory combinations, no modification to the existing trajectory is required. In the second set of trace combinations, the second set of trace combinations is considered the true trajectory. Among them, , These represent the curvatures of the two parabolas in the first set of trajectory combinations associated with the target across multiple frames in step S3; , Let represent the curvatures of the two parabolas formed by the trajectory combination of the trajectory segments after exchanging intersection points.

[0073] The curvature discrimination method can effectively solve the trajectory extension problem when two trajectories intersect. The first derivative reflects the velocity characteristics, while the second derivative reflects the direction and acceleration characteristics, thus the algorithm has high sensitivity.

[0074] It should be noted that when performing curve fitting, taking 5 frames before and 5 frames after the intersection point of the trajectories, for a total of 21 sampling points, is sufficient to reflect the curvature difference between the two curves. Increasing the number of sampling points generally will not affect the discrimination result.

[0075] S5: repeat steps S3 and S4 for all targets with trajectory intersection until the final cataloging of all targets in all frame images is completed.

[0076] The step S4 is described below with a specific embodiment:

[0077] The frame number and the curvature calculation value corresponding to each valid intersection point are shown in Table 1, which includes:

[0078] The curvature value at the trajectory intersection of the 21st target and the 23rd target in the 1318th frame.

[0079] The curvature value at the trajectory intersection of the 21st target and the 23rd target in the 1319th frame.

[0080] The curvature value at the trajectory intersection of the 19th target and the 23rd target in the 1391st frame.

[0081] The curvature value at the trajectory intersection of the 12th target and the 23rd target in the 1740th frame.

[0082] Wherein, the curvature of the two parabolas of the first group of trajectory combinations after the step S3 multi-frame target association is marked as 、 The curvature of the two parabolas of the trajectory combination of the trajectory segment after the exchange of the intersection point is marked as and .

[0083] Table 1: Curvature calculation value at each valid intersection point

[0084]

[0085] It is pointed out in Table 1 that the minimum curvature of the 1318th frame and the 1319th frame appears in the second group of trajectory groups (the trajectory combination formed after the exchange of the trajectory segment from this intersection point to the next intersection point), so the position of the two trajectory intersection targets of the 1318th frame needs to be exchanged, and the position of the same target of the two trajectory intersection of the 1319th frame to the 1391st frame needs to be exchanged, and after the exchange, the corresponding point traces are stored into the corresponding cataloged targets. The minimum curvature of the 1391st frame and the 1740th frame appears in the first group of trajectory groups, so the preliminary target trajectory from the 1391st frame is retained.

[0086] After the cataloging of all targets is processed by the above steps, the target trajectory consistent with the theoretical trajectory is obtained, as shown in Figure 4 , which verifies the effectiveness of the method in this paper in the complex scene of 30 targets, multiple target trajectory intersections, and maneuvering variable trajectory.

[0087] For example, the 21st target and the 23rd target, the trajectory curvature at the intersection point of the 1319th frame is visualized by fitting a parabola, asFigure 3 As shown, the curvature of the trajectory segment before the exchange is visualized using a red parabola, and the curvature after the exchange is indicated by green. Clearly, the solid green line is straight and has the minimum curvature.

[0088] It should be noted that this method can be used for both post-processing and near-real-time measurement. When applied to near-real-time measurement, since fitting the parabola requires taking 5 frames before and after the trajectory intersection frame, at least 5 frames of data need to be accumulated. Therefore, target cataloging has some drawbacks compared to real-time processing. seconds (in the formula) This refers to the time delay (which is the camera's sampling rate or frame rate). For example, for a common 60Hz sampling rate camera, the corresponding time delay is only... This delay is relatively small, so it can be considered a near real-time measurement.

[0089] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A cataloging method for UAV swarms based on local trajectory curvature, characterized in that, Includes the following steps: S1: Collect image sequences of drone swarms, extract the center position of all drones in each frame of the image through image processing, and generate a drone position set; S2: In the initial frame, each drone is defined as a target, each target is assigned a unique number, and the target variables for each target are initialized; S3: Based on the multi-frame target association method, calculate the pixel distance between each target in the next frame and all targets in the current frame. According to the principle of minimum pixel distance, associate the target position in the next frame with the corresponding numbered target variable in the current frame until the target association of all frames is completed, forming a preliminary catalog and a preliminary target trajectory. S4: During the formation of the initial target trajectory, record the target number and corresponding frame number of the intersecting trajectories; use the trajectory curvature discrimination algorithm to determine the correct attribution of the intersecting trajectory segments, correct the erroneous associations in the initial target trajectory, and obtain the corrected target catalog; including the following steps: S41: Divide the trajectories of two intersecting targets into two segments based on the intersection point, forming two sets of trajectory combinations; The trajectory combination associated with the original multi-frame targets is defined as the first trajectory combination; the trajectory combination formed by exchanging the trajectory segments after the intersection is defined as the second trajectory combination; each of the trajectory combinations contains two target trajectories; S42: Extract the trajectory point data of the two target trajectories in each t frames before and after the intersection point, and perform parabolic fitting based on the trajectory point data to obtain four fitted parabolas; S43: Calculate the curvature value of each fitted parabola at the intersection point; S44: Compare the curvature values ​​of each target trajectory at the intersection point, and determine the combination of trajectories containing the target trajectory with the minimum curvature as the true trajectory; S45: If the trajectory combination corresponding to the minimum curvature is a trajectory associated with multiple target frames, then the original trajectory remains unchanged; If the trajectory combination corresponding to the minimum curvature is the same as the swapped trajectory combination, then the trajectory segments of the two targets after the intersection point are swapped. S5: Repeat step S4 for all targets with intersecting trajectories until the final cataloging of all targets in all frame images is completed.

2. The cataloging method for UAV swarms based on local trajectory curvature according to claim 1, characterized in that, In step S42, t is greater than or equal to 5.

3. The cataloging method for UAV swarms based on local trajectory curvature according to claim 1, characterized in that, The formulas for the parabola and its first and second derivatives are as follows: ； in, , and Let represent the coefficients of the quadratic term, the linear term, and the zeroth-order term of the parabola, respectively, which are determined by least squares fitting. X and Y In the image coordinate system X Axis coordinates and Y Axis coordinates; and They represent Y The first and second derivatives.

4. The cataloging method for UAV swarms based on local trajectory curvature according to claim 3, characterized in that, The formula for calculating the curvature of the trajectory at the intersection point is: ； Where K represents curvature; The x-coordinate of the image representing the intersection point of the two trajectories.

5. The cataloging method for UAV swarms based on local trajectory curvature according to claim 1, characterized in that, In step S2: In the initial frame, all UAVs are sorted along the same axis of the image coordinate system to perform preliminary cataloging of the target positions.

6. The cataloging method for UAV swarms based on local trajectory curvature according to claim 1, characterized in that, In step S3: the formula for calculating the pixel distance between each target position in the next frame and all targets in the current frame is: ； in, This represents the pixel distance of the nth frame relative to the previous frame; , These represent the X-axis coordinates and Y-axis coordinates of the target i to be cataloged in the nth frame in the image coordinate system, respectively. , These represent the X-axis and Y-axis coordinates of the cataloged target j in the (n-1)th frame in the image coordinate system, respectively.

Images