Unmanned aerial vehicle swarm cataloguing method based on local trajectory curvature

Abstract

The invention relates to the technical field of unmanned aerial vehicle cataloguing, in particular to an unmanned aerial vehicle swarm cataloguing method based on local trajectory curvature, comprising the following steps: S1, collecting an image sequence of an unmanned aerial vehicle swarm, and generating an unmanned aerial vehicle position set; s2, distributing a unique number for each target in the initial frame; s3, based on a multi-frame target association method, forming a preliminary catalogue and a preliminary target trajectory; s4, determining the correct attribution of the staggered track segments by using a track curvature discrimination algorithm, and correcting the wrong association in the initial target track to obtain a corrected target catalogue; and S5, repeatedly executing the step S4 for all the targets with staggered tracks until the final cataloguing of all the targets in all the frame images is completed. The multi-target cataloguing method has the advantages that the trajectory curvature discrimination algorithm is reasonably applied to multi-target cataloguing, the target positions can be classified and catalogued well, the trajectory staggering situation of the two targets can be accurately processed, and the difficult problems in multi-target cataloguing are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone cataloging, and in particular to a drone swarm cataloging method based on local trajectory curvature. Background Art

[0002] Traditional measurement and control tasks have mature processing methods for trajectory measurement and tracking of single targets. However, when the measurement and control scenario is extended to drone swarms, the increase in degrees of freedom leads to a geometric increase in the amount of calculation. For example, the number of traversal intersection calculations increases from n to n. 2 Various intersecting trajectory scenarios between targets introduce a significant increase in trajectory combinatorial complexity. For example, after two target trajectories intersect, it is necessary to determine the ownership of the trajectory segments. In addition, the real-time trajectory changes of maneuvering targets can invalidate inertial target tracking algorithms. For example, if a drone in a swarm suddenly changes its trajectory, its motion parameters such as speed and acceleration will experience discontinuous jumps. Therefore, efficient and accurate cataloging of swarm targets is the prerequisite and key to achieving drone swarm trajectory measurement and tracking.

[0003] Drone swarms are multi-targets, and there are two main difficulties in cataloging them: first, there are dozens of targets to be processed, and it is inevitable that two target trajectories will intersect in consecutive frames of the video. The minimum coplanarity difference algorithm used in the intersection calculation fails at the intersection of the two trajectories, making it impossible to accurately complete the target cataloging in the subsequent frames where the two trajectories intersect; second, the previous target cataloging method is based on the evaluation function method of the three positions of the same target. This method is mainly suitable for inertial targets with relatively stable trajectory changes. Drones are highly maneuverable targets that can change trajectories in real time. Their motion quantities such as speed and acceleration may suddenly change at a certain moment, which will cause the error probability of the evaluation function method to suddenly increase, resulting in a decrease in the accuracy of target cataloging. Summary of the Invention

[0004] Based on this, the cataloging method of drone swarms based on local trajectory curvature proposed in the present invention rationally applies the trajectory curvature discrimination algorithm to multi-target cataloging, which can well classify and catalog the target positions and accurately handle the intersection of two target trajectories, effectively solving the difficult problems in multi-target cataloging.

[0005] To achieve the above-mentioned purpose, the technical solution created by the present invention is implemented as follows: a method for cataloging drone swarms based on local trajectory curvature, comprising the following steps: S1: Collect image sequences of the target group, extract the center positions of all targets in each frame of the image through image processing, and generate a target position set; S2: In the initial frame, assign a unique number to each target and initialize the target variable corresponding to the target; S3: Based on the multi-frame target association method, the pixel distance between each target in the next frame and all targets in the current frame is calculated. According to the minimum pixel distance principle, the target position in the next frame is associated with the corresponding numbered target variable in the current frame until the target association of all frames is completed, forming a preliminary catalog and preliminary target trajectory; S4: In the process of forming the preliminary target trajectory, the target number and the corresponding frame number of the trajectory interleaving are recorded; the trajectory curvature discrimination algorithm is used to determine the correct attribution of the interleaved trajectory segment, and the incorrect association in the preliminary target trajectory is corrected to obtain the corrected target catalog; S5: Repeat step S4 for all targets whose trajectories intersect, until the final cataloging of all targets in all frame images is completed.

[0006] Furthermore, step S4 includes the following steps: S41: dividing the trajectories of the two targets whose trajectories intersect into two segments according to the intersection point, thereby forming two sets of trajectory combinations; The trajectory combination associated with the original multi-frame target is defined as the first group of trajectory combinations; the trajectory combination formed by exchanging the trajectory segments after the intersection is defined as the second group of trajectory combinations; each group of the trajectory combinations contains two target trajectories; S42: extracting the trajectory point data of the two target trajectories in each t frame before and after the intersection, performing parabola fitting based on the trajectory point data, and obtaining four fitting parabolas; S43: Calculate the curvature value of each fitting parabola at the intersection point; S44: comparing the curvature values ​​of each target trajectory at the intersection point, and determining the trajectory combination containing the target trajectory with the minimum curvature as the true trajectory; S45: If the trajectory combination corresponding to the minimum curvature is a trajectory associated with multiple frame targets, the original trajectory is maintained unchanged; If the trajectory combination corresponding to the minimum curvature is the exchanged trajectory combination, the trajectory segments of the two targets after the intersection are exchanged.

[0007] Furthermore, t in step S42 is greater than or equal to 5.

[0008] Furthermore, the formulas for the parabola and its first-order and second-order derivatives are as follows: ; in, 、 and They represent the quadratic term coefficient, linear term coefficient and zero-order term coefficient of the parabola respectively, and are determined by least squares fitting; X and Y are respectively in the image coordinate system X Axis coordinates and Y axis coordinates; and respectively represent the first derivative and the second derivative. Y

[0009] Further, the calculation formula of the trajectory curvature at the intersection point is: ; Wherein, K represents the curvature of the trajectory at the intersection point; represents the image abscissa of the intersection point of the two trajectories.

[0010] Further, in step S2: in the initial frame, all unmanned aerial vehicles are sorted according to the same axis of the image coordinate system, and the target position is preliminarily cataloged.

[0011] Further, in step S3: the calculation formula of the pixel distance of each target position in the next frame from all targets in the current frame is: ; Wherein, represents the pixel distance of the nth frame relative to the previous frame; , represents the X-axis coordinate and Y-axis coordinate of the nth frame to be cataloged target i under the image coordinate system respectively; , represents the X-axis coordinate and Y-axis coordinate of the nth-1 frame cataloged target j under the image coordinate system respectively.

[0012] The present application can achieve the following beneficial effects: the present application combines the trajectory characteristics of the unmanned aerial vehicle swarm, reasonably applies the trajectory curvature discrimination algorithm, can effectively classify and catalog each target position, accurately calculates the curvature at the intersection point of the two trajectories to determine the attribution of the trajectory segment after the intersection of the trajectories, effectively solves the difficult problem in multi-target cataloging, and has high algorithm sensitivity and is easy to apply in practice. BRIEF DESCRIPTION OF DRAWINGS

[0013] The drawings constituting a part of the present application are used to provide a further understanding of the present application, the schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of the cataloging method of the unmanned aerial vehicle swarm based on local trajectory curvature provided by the embodiment of the present application; Figure 2 is a preliminary target trajectory processed by the multi-frame target correlation method according to the embodiment of the present application; Figure 3 is a schematic diagram of the fitted parabola at the intersection point of the two target trajectories according to the embodiment of the present application; Figure 4 ​3 is a schematic diagram of a target trajectory corrected after trajectory curvature determination according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0015] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0016] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0017] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0018] The present invention will be described in detail below with reference to the embodiments.

[0019] like Figures 1 to 4 As shown, an embodiment of the present invention provides a method for cataloging a drone swarm based on local trajectory curvature, comprising the following steps: S1: The photoelectric theodolite collects an image sequence of the target group, extracts the center position of all target images in each frame through image processing, and generates a target position set.

[0020] S2: In the initial frame, each target is assigned a unique number and the target variable of each target is initialized.

[0021] Specifically, input continuous The target trajectory of the frame (taking a segment containing 30 drones as an example), the n targets in the initial frame (i.e., the first frame) are calculated according to the image coordinate system. Sort by axis direction, and get The target positions (i.e., image coordinates) of the targets are assigned unique numbers to each target, 1, 2, 3...n, and their target variables are initialized: In this embodiment, the target variable is the image coordinate of the target.

[0022] in, Indicates the X-axis coordinate and Y-axis coordinate of target 1 in the image coordinate system of the first frame; Indicates the X-axis coordinate and Y-axis coordinate of target 2 in the image coordinate system of frame 1; Indicates the X-axis and Y-axis coordinates of target n in the image coordinate system of the first frame.

[0023] S3: Based on the multi-frame target association method, the pixel distance between each target in the next frame and all targets in the current frame is calculated. According to the principle of minimum pixel distance, the target in the next frame is associated with the corresponding numbered target variable in the current frame until the target association of all frames is completed, forming a preliminary catalog and preliminary target trajectory.

[0024] The calculation formula for the pixel distance between each target in the next frame and all targets in the current frame is: ; in, Indicates the pixel distance of the nth frame relative to the previous frame; 、 They represent the coordinates of the target i to be cataloged in the image coordinate system of the nth frame. X Axis coordinates and Y axis coordinates; 、 Represents cataloged targets j In the n-1 Frame image coordinate system X Axis coordinates and Y-axis coordinates.

[0025] For the first n For each target in the frame, find the minimum pixel distance between each target in the frame and each target in the n-1th frame , so that the target variable of the i-th target in the n-th frame can be stored in the n-1 Frame cataloged target j of the target variables.

[0026] S4: In the process of forming the preliminary target trajectory, the target number and the corresponding frame number of the interleaved trajectory are recorded; the trajectory curvature discrimination algorithm is used to determine the correct attribution of the interleaved trajectory segment, correct the incorrect association in the preliminary target trajectory, and obtain the corrected target catalog. The steps include the following: S41: dividing the trajectories of the two targets whose trajectories intersect into two segments according to the intersection point, thereby forming two sets of trajectory combinations; The trajectory combination associated with the original multi-frame targets is defined as the first group of trajectory combinations; the trajectory combination formed by exchanging the trajectory segments after the intersection is defined as the second group of trajectory combinations; each group of the trajectory combinations contains two target trajectories.

[0027] It should be noted that if one of the target trajectories contains multiple intersection points, the target trajectory is segmented using each intersection point as a dividing point.

[0028] S42: extracting the trajectory point data of the two target trajectories in each t frame before and after the intersection, performing parabola fitting based on the trajectory point data, and obtaining four fitting parabolas; S43: Calculate the curvature value of each fitting parabola at the intersection point; S44: comparing the curvature values ​​of each target trajectory at the intersection point, and determining the trajectory combination containing the target trajectory with the minimum curvature as the true trajectory; S45: If the trajectory combination corresponding to the minimum curvature is a trajectory associated with multiple frame targets, the original trajectory is maintained unchanged; If the trajectory combination corresponding to the minimum curvature is the exchanged trajectory combination, the trajectory segments of the two targets after the intersection are exchanged.

[0029] In actual measurement, the target trajectory is not a continuous, smooth curve, but rather a series of discrete pixel points. Therefore, we first use the least squares method to fit the trajectory curve using the interleaved frames. Curvature calculation is then performed. Smaller curvatures result in a flatter trajectory, while larger curvatures indicate a sharper trajectory. A pair of trajectory curves intersects to produce two possible trajectory groups. The group with the smallest curvature is the true trajectory group.

[0030] Specifically, such as Figure 2 As shown, during the preliminary cataloging process, record the target numbers and corresponding frame numbers of all track intersections ; Where m and i represent the i-th intersection point in the m-th frame respectively; X and Y Respectively represent the i point in the image coordinate system X Axis coordinates and Y Axis vertical coordinate.

[0031] The two target trajectories with staggered trajectories are divided into two sections before and after the intersection point, forming two groups of trajectory combinations. The front and back sections of one of the target trajectories are defined as and , and the front and back sections of the other target trajectory are defined as and , and belong to the front section of the intersection point, and belong to the back section of the intersection point.

[0032] For the mth frame of the effective intersection point, the trajectory point data of the two target trajectories in the 5 frames before and after the trajectory intersection frame are taken, a total of 21 trajectory points, and four parabolas are fitted for the 21 trajectory points as follows: , , , .

[0033] Among them, and are the first group of trajectory combinations of the target correlation of multiple frames in step S3, and are the trajectory combinations of the trajectory sections after the exchange of the intersection point.

[0034] Considering that the parabola function has few undetermined parameters (only three undetermined parameters) and simple mathematical properties of first and second derivatives, parabola fitting can significantly simplify curvature calculation near the intersection of two trajectories.

[0035] The parabola and its first and second derivatives are as follows: ; Among them, , and represent the quadratic term coefficient, the linear term coefficient and the zero term coefficient of the parabola respectively, which are determined by least squares fitting; X and Y are the X axis coordinates and Y axis coordinates in the image coordinate system respectively; and represent the first derivative and the second derivative of Y respectively.

[0036] Considering the continuity of the target motion process, for a continuous and smooth derivable function, the curvature calculation formula is: ; Among them, K represents the curvature; represents the image horizontal coordinate of the intersection point of the two trajectories.

[0037] Find the minimum curvature ,like In the first set of trajectory combinations, there is no need to modify the existing trajectory. In the second set of trace combinations, the second set of trace combinations is taken as the true trace. 、 Respectively represent the curvatures of the two parabolas of the first set of trajectory combinations associated with the multi-frame target in step S3; 、 They represent the curvatures of the two parabolas of the trajectory combination after exchanging the intersection points.

[0038] The curvature discrimination method can effectively solve the trajectory extension problem when two trajectories intersect. The first-order derivative reflects the velocity characteristics, while the second-order derivative reflects the direction and acceleration characteristics, so the algorithm is highly sensitive.

[0039] It should be noted that when performing curve fitting, taking the trajectory intersection point as the center and taking 5 frames before and after, a total of 21 sampling points, can fully reflect the curvature difference between the two curves. Continuing to increase the number of sampling points usually does not affect the discrimination results.

[0040] S5: Repeat steps S3 and S4 for all targets whose trajectories intersect, until the final cataloging of all targets in all frame images is completed.

[0041] The following describes step S4 using a specific embodiment: The frame numbers and curvature calculation values ​​corresponding to each valid intersection are shown in Table 1. The data in the table include: The curvature value at the intersection of the trajectories of targets 21 and 23 in frame 1318.

[0042] The curvature value at the intersection of the trajectories of targets 21 and 23 in frame 1319.

[0043] The curvature value at the intersection of the trajectories of targets 19 and 23 in frame 1391.

[0044] The curvature value at the intersection of the trajectories of targets 12 and 23 in frame 1740.

[0045] The curvature of the two parabolas of the first set of trajectory combinations associated with the multi-frame target in step S3 is marked as 、 , the curvature of the two parabolas of the trajectory combination of the trajectory segments after exchanging the intersection points is marked as and .

[0046] Table 1 Calculated curvature values ​​at each effective intersection point

[0047] Table 1 indicates that the minimum curvature for frames 1318 and 1319 occurs in the second trajectory group (a trajectory group formed by swapping the trajectory segments from this intersection to the next intersection). Therefore, the target location of the two trajectories intersecting in frame 1318 needs to be swapped, and the locations of the two trajectories intersecting the same target from frames 1319 to 1391 need to be swapped. After the swap, the corresponding points are re-stored to the corresponding cataloged targets. The minimum curvature for frames 1391 and 1740 occurs in the first trajectory group, so the preliminary target trajectory starting from frame 1391 is retained.

[0048] After the above steps are completed to catalog all the targets, the target trajectory that is consistent with the theoretical trajectory is obtained, such as Figure 4 As shown in the figure, the effectiveness of the proposed method in cataloging UAV swarm targets in complex scenarios such as 30 targets, multiple target trajectory intersections, and maneuverable trajectory changes is verified.

[0049] For example, for target No. 21 and target No. 23, the trajectory curvature is visualized using a fitted parabola at the intersection of frame 1319, as shown in Figure 3 As shown in Figure 2, the curvature of the trajectory segments before the exchange is visualized using red parabolas, and the curvature after the exchange is visualized using green parabolas. Obviously, the green solid line is straight and has the smallest curvature.

[0050] It should be noted that this method can be used for both post-processing and quasi-real-time measurement. When this method is applied to quasi-real-time measurement, since fitting the parabola requires taking 5 frames before and after the trajectory interleaving frame, at least 5 frames of data need to be accumulated. Therefore, target cataloging is more complex than real-time processing. Seconds (where is the sampling rate or frame rate of the camera). For example, for a common camera with a sampling rate of 60Hz, the corresponding time delay is only , this delay is small and can be regarded as quasi-real-time measurement.

[0051] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for cataloging drone swarms based on local trajectory curvature, characterized in that: The steps include: S1: Collect image sequences of drone swarms, extract the center positions of all drones in each frame through image processing, and generate a drone position set; S2: In the initial frame, each UAV is defined as a target, a unique number is assigned to each target, and the target variable of each target is initialized; S3: Based on the multi-frame target association method, the pixel distance between each target in the next frame and all targets in the current frame is calculated. According to the minimum pixel distance principle, the target position in the next frame is associated with the corresponding numbered target variable in the current frame until the target association of all frames is completed, forming a preliminary catalog and preliminary target trajectory; S4: In the process of forming the preliminary target trajectory, the target number and the corresponding frame number of the trajectory interleaving are recorded; the trajectory curvature discrimination algorithm is used to determine the correct attribution of the interleaved trajectory segment, and the incorrect association in the preliminary target trajectory is corrected to obtain the corrected target catalog; S5: Repeat step S4 for all targets whose trajectories intersect, until the final cataloging of all targets in all frame images is completed.

2. The method for cataloging drone swarms based on local trajectory curvature according to claim 1, characterized in that: Step S4 includes the following steps: S41: dividing the trajectories of the two targets whose trajectories intersect into two segments according to the intersection point, thereby forming two sets of trajectory combinations; The trajectory combination associated with the original multi-frame target is defined as the first group of trajectory combinations; the trajectory combination formed by exchanging the trajectory segments after the intersection is defined as the second group of trajectory combinations; each group of the trajectory combinations contains two target trajectories; S42: extracting the trajectory point data of the two target trajectories in each t frame before and after the intersection, performing parabola fitting based on the trajectory point data, and obtaining four fitting parabolas; S43: Calculate the curvature value of each fitting parabola at the intersection point; S44: comparing the curvature values ​​of each target trajectory at the intersection point, and determining the trajectory combination containing the target trajectory with the minimum curvature as the true trajectory; S45: If the trajectory combination corresponding to the minimum curvature is a trajectory associated with multiple frame targets, the original trajectory is maintained unchanged; If the trajectory combination corresponding to the minimum curvature is the exchanged trajectory combination, the trajectory segments of the two targets after the intersection are exchanged.

3. The method for cataloging drone swarms based on local trajectory curvature according to claim 2, characterized in that: In step S42 , t is greater than or equal to 5.

4. The method for cataloging drone swarms based on local trajectory curvature according to claim 2, characterized in that: The formulas for the parabola and its first-order and second-order derivatives are as follows: ； in, 、 and They represent the quadratic term coefficient, linear term coefficient and zero-order term coefficient of the parabola respectively, and are determined by least squares fitting; X and Y are respectively in the image coordinate system X Axis coordinates and Y axis coordinates; and Respectively Y The first and second derivatives of .

5. The method for cataloging drone swarms based on local trajectory curvature according to claim 4, characterized in that: The calculation formula of the trajectory curvature at the intersection point is: ； Where K represents the curvature; The image horizontal coordinate representing the intersection of the two trajectories.

6. The method for cataloging drone swarms based on local trajectory curvature according to claim 1, characterized in that: In step S2: In the initial frame, all UAVs are sorted according to the same axis of the image coordinate system to perform a preliminary cataloging of the target positions.

7. The method for cataloging drone swarms based on local trajectory curvature according to claim 1, characterized in that: In step S3: the calculation formula for the pixel distance between each target position in the subsequent frame and all targets in the current frame is: ； in, Indicates the pixel distance of the nth frame relative to the previous frame; 、 They represent the X-axis coordinate and Y-axis coordinate of the target i to be cataloged in the nth frame in the image coordinate system; 、 They represent the X-axis coordinate and Y-axis coordinate of the cataloged target j in the image coordinate system of the n-1th frame respectively.

Images