Group separation and combination detection algorithm based on envelope characteristics of photoelectric detection unmanned aerial vehicle group

Abstract

The invention relates to the field of unmanned aerial vehicle group detection, and provides a group separation and combination detection algorithm based on a photoelectric detection unmanned aerial vehicle group envelope characteristic in order to solve the technical problems that an existing detection method is low in detection resolution, cannot perform overall situation estimation and the like. Extracting a clustering center coordinate, a group ellipse envelope edge coordinate, a group ellipse envelope main axis direction, a long axis length, a short axis length and an eccentricity of each unmanned aerial vehicle group; respectively calculating a center track motion direction included angle of each group, a two-side edge track motion direction included angle, an included angle of two edge tracks relative to the center track motion direction, an eccentricity difference value of front and back three frames of each group and a center distance between every two groups; and taking the data as input to perform separation or combination judgment processing, extracting a combined key frame and a separated key frame, and completing grouping or combination detection of the unmanned aerial vehicle group target.

Description

Technical Field

[0001] The present invention relates to the field of drone swarm detection, and in particular to a detection algorithm for group separation and merging based on the envelope characteristics of a photoelectric detection drone swarm. Background Art

[0002] Group target tracking technology is one of the important research contents in the field of machine vision, and it has wide applications in military and civilian fields. With the scale of drone applications and the diversification of scenarios, simply detecting multiple drones in the field of view can no longer meet the application scenarios and actual combat needs based on group attributes. Real-time tracking of group targets and obtaining the spatial situation of multiple drone groups to meet actual application needs are the main development trends of group target tracking technology in the future.

[0003] Group target tracking is to establish time correlation at different time states to form a continuous track in time and space, including: track initiation, maintenance, termination, separation and merging, etc.; among them, group target separation and merging detection is the key process feature that reflects the overall situation of the group target.

[0004] However, there are two main means for detecting the separation and merging of group targets at present: one is to consider the structural model of the group, and for distinguishable group targets, the evolutionary graph network model is used to detect the overall situation of the group targets; the other is to detect the separation and merging of group targets based on information such as position, speed direction, trajectory history, etc., based on clustering, pattern space, PDA and other methods; for low-altitude, slow-flying, and densely dispersed drone clusters, their low-altitude flying blind spots are large, and the overall characteristics of the group are not easy to be detected, tracked and identified; and the above two detection methods both use radar signals as input. Due to the insufficient detection resolution of radar for low-speed and small drones, only individual detection can be formed for dense drone group targets, and the overall situation estimation of the group targets cannot be formed, and the key frames of the movement trend changes of the group targets in the image sequence cannot be detected; therefore, it is urgent to seek a detection method that can image the group targets in real time, detect and track individual targets in the field of view, and associate and manage multi-frame state estimates based on video information, distinguish the birth, splitting and merging of trajectories, and form an overall situation estimation of the group targets. Summary of the invention

[0005] The purpose of the present invention is to solve the technical problems that the existing group target separation and merging detection method using radar signals as input has insufficient detection resolution for low-altitude and slow-flying UAV clusters with large scatter density and large blind spots, and can only form individual detections, but cannot form an overall situation estimation of the group targets, nor can it detect key frames of changes in the motion trend of the group targets in the image sequence. A group separation and merging detection algorithm based on the envelope characteristics of the photoelectric detection UAV group is proposed.

[0006] The design idea of ​​the present invention is: in the process stage of UAV swarm target tracking, the group envelope characteristic parameters of the UAV swarm targets are obtained through sequential single-frame images, multiple group motion trajectories are established, and the separation and merging detection in the group target tracking process is carried out jointly through parameters such as the group motion trajectory direction, eccentricity, and group cluster center distance, so as to provide accurate real-time group situation parameter estimation for anti-UAV; by determining the key frame of separation or merging, the trajectory splitting or merging can be determined, which provides accurate criteria for updating the group target trajectory and improving the stability of group target tracking.

[0007] Taking photoelectric detection observation data as input, the separation and merging of group targets are detected through information such as position, speed direction, and trajectory history; the key frames of the change of motion trend of group targets in the video sequence are detected to provide prior information for correcting data association and tracking results.

[0008] To achieve the above objectives, the technical solution proposed by the present invention is:

[0009] A detection algorithm for group separation and merging based on the envelope characteristics of photoelectric detection drone groups is special in that it includes the following steps:

[0010] S1. Collect a single-frame image of the drone group target through the photoelectric detector, and extract the cluster center coordinates (Ti_x0, Ti_y0) of each drone group Ti in the single-frame image, the group ellipse envelope edge coordinates (Ti_x1, Ti_y1), (Ti_x2, Ti_y2), and obtain the main axis direction b1, major axis length λ1, minor axis length λ2 and eccentricity of the group ellipse envelope

[0011] S2, respectively calculate the central track motion direction angle Ti_θ0, the two side edge track motion direction angles Ti_θ1, Ti_θ2, and the angle Ti_Δθ(t) between the two edge tracks and the central track motion direction at time t for each group in a single frame image;

[0012] Calculate the eccentricity difference Ti_Δe(t) of the three frames before and after each group at time t;

[0013] Calculate the distance between the centers of each group at time t Tab_Δd(t);

[0014] S3. The included angle of the trajectory motion direction of each group target, the eccentricity difference, and the center distance between each group obtained in step S2 are used as input for separation or merging judgment processing, and the merged key frame and the separated key frame are extracted. The group target tracking parameters and the group situation estimation parameters are corrected, thereby completing the grouping and merging detection of the drone group targets.

[0015] Furthermore, step S2 is specifically as follows:

[0016] S2.1, the central trajectory motion direction angle Ti_θ0 of each group is calculated by the following formula:

[0017]

[0018] The calculation formulas for the edge trajectory motion direction angles Ti_θ1 and Ti_θ2 are as follows:

[0019]

[0020]

[0021] The calculation formula of the angle Ti_Δθ(t) between the two edge tracks and the moving direction of the center track is:

[0022] Ti_Δθ(t)=|Ti_θ0(t)-Ti_θ1(t)|+|Ti_θ0(t)-Ti_θ2(t)|

[0023] Among them, (Ti_x0(t), Ti_y0(t)), (Ti_x0(t-1), Ti_y0(t-1)) are the central trajectory positions of the drone group Ti at time t and time t-1 respectively, (Ti_x1(t), Ti_y1(t)), (Ti_x2(t), Ti_y2(t)) are the two edge trajectory positions of the group ellipse envelope of the drone group Ti at time t respectively, (Ti_x1(t-1), Ti_y1(t-1)), (Ti_x2(t-1), Ti_y2(t-1)) are the two edge trajectory positions of the group ellipse envelope of the drone group Ti at time t-1 respectively, Ti_θ0(t) is the moving direction angle of the central trajectory of the drone group Ti at time t, Ti_θ1(t) and Ti_θ2(t) are the moving direction angles of the two edge trajectories of the group ellipse envelope at time t;

[0024] S2.2, the calculation formula of the eccentricity difference Ti_Δe(t) of the three frames before and after each group is:

[0025] Ti_Δe(t)=Ti_e(t)-Ti_e(t-3)

[0026] Where: Ti_e(t) is the eccentricity of the group ellipse shape of the drone group Ti at time t, Ti_e(t-3) is the eccentricity of the group ellipse shape of the drone group Ti at time t-3;

[0027] S2.3, the calculation formula of the center distance Tab_Δd(t) between two groups at time t is:

[0028]

[0029] Among them: (Ta_x0(t), Ta_y0(t)) is the center trajectory position of group a at time t, and (Tb_x0(t), Tb_y0(t)) is the center trajectory position of group b at time t.

[0030] Further, step S3 specifically includes: group separation detection and group merging detection;

[0031] The group separation detection comprises the following steps:

[0032] S3A.1. Compare the calculated result of the angle Ti_Δθ(t) between the two edge tracks and the moving direction of the center track at time t in step S2.1 with the set separation angle threshold θ th Compare; if Ti_Δθ(t)≥θ th , then execute step S3A.2; if Ti Δθ (t)<θ th , it is determined that the frame has not been separated, and the process returns to step S2 to process the next frame of image;

[0033] S3A.2, compare the calculated result of the eccentricity difference Ti_Δe(t) of the three frames before and after each group in step S2.2 with the set eccentricity threshold e th Compare; if Ti_Δe(t)≥e th , it is determined that the group Ti is separated at time t, and the image frame at this time is the separation starting key frame Ti_N(t), thereby completing the separation detection and judgment of the drone group target, confirming the addition of a new group target tracking trajectory, and updating the tracking filter parameters; if Ti Δe (t)<e th , it is determined that the frame has not been separated, and the process returns to step S2 to process the next frame of image;

[0034] The group merging detection comprises the following steps:

[0035] S3B.1. Compare the calculated result of the group center distance Tab_Δd(t) at time t in step S2.3 with the set group center distance threshold d th For comparison, if Tab Δd (t)≤d th , then execute step S3B.2; if Tab Δd (t)>d th , it is determined that the frame has not been merged, and the process returns to step S2 to process the next frame of image;

[0036] S3B.2, compare the calculated result of the merged trajectory center angle threshold Tab_Δθ(t) in step S2.4 with the set group merged trajectory center angle threshold ab_θ thCompare; if 5 consecutive frames all satisfy Tab_Δθ(t)≤ab_θ th , it is determined that the merger of group a and group b occurs at time t, and the image frame at this time is the merger start key frame Tab_N(t), thereby completing the merger detection and judgment of group a and group b, the two trajectories are merged into one trajectory, and the tracking filter parameters are updated; if one of the five consecutive frames does not satisfy Tab_Δθ(t)≤ab_θ th , it is determined that the frame has not been merged, and the process returns to step S2 to proceed to the next frame image for processing.

[0037] Furthermore, in step S3B.2, the group merge trajectory center angle threshold ab_θ th Determined by the following formula:

[0038]

[0039] in:

[0040] (Ta_x0(t), Ta_y0(t)) represents the center trajectory position of group a at time t, (Ta_x0(t-1), Ta_y0(t-1)) represents the center trajectory position of group a at time t-1;

[0041] (Tb_x0(t), Tb_y0(t)) represents the center trajectory position of group b at time t, and (Tb_x0(t-1), Tb_y0(t-1)) represents the center trajectory position of group b at time t-1.

[0042] Beneficial effects of the present invention:

[0043] [1] The detection algorithm of group separation and merging based on the envelope characteristics of photoelectric detection of drone groups in the present invention can solve the problems of large blind spots, slow flight and high scatter density of drone clusters at low altitudes, and the inability to identify the overall characteristics of the group. By obtaining the key frames for group target separation and merging, accurate prior information is provided for correcting data association and tracking results, stable tracking of group targets is achieved, and the detection accuracy of group separation and merging is effectively improved. Not only can real-time imaging of group targets be performed, but also single targets in the tracking field of view can be detected and tracked. In addition, multi-frame state estimation can be associated and managed based on video information, and the generation, splitting and merging of trajectories can be distinguished to form an overall situation estimation of the group targets, effectively improving the stability and reliability of group target tracking.

[0044] [2] The present invention can track group targets through two-dimensional observation data, and can effectively estimate the movement status of group targets through a simple and effective algorithm, thereby eliminating the misjudgment of situations such as the overall expansion of group targets and the intersection of group targets, and effectively improving the detection accuracy of group separation and merging.

[0045] [3] The present invention associates and manages the multi-frame state estimation of group targets and distinguishes the splitting and merging of trajectories, thereby obtaining the overall situation distribution estimation and group correlation of group targets, providing intuitive visual data results for monitoring personnel, enabling the drone group to achieve intuitive visual observation and situation estimation, and adopts dual judgment criteria to detect group targets, realizing independent estimation of group separation and group merging, effectively improving the detection accuracy of group targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of group target separation and determination in an embodiment of a detection algorithm for group separation and merging based on the envelope characteristics of a group of photoelectric detection drones of the present invention;

[0047] Figure 2 Schematic diagram of group target merging determination in an embodiment of the present invention;

[0048] Figure 3 The simulation result of key frame detection for group target separation in an embodiment of the present invention;

[0049] Figure 4 This is the simulation result of group target merging key frame detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] A detection algorithm for group separation and merging based on the envelope characteristics of photoelectric detection of drone groups includes the following steps:

[0051] S1. Collect a single-frame image of the drone group target through the photoelectric detector, and extract the cluster center coordinates (Ti_x0, Ti_y0) of each drone group Ti in the single-frame image, the group ellipse envelope edge coordinates (Ti_x1, Ti_y1), (Ti_x2, Ti_y2), and obtain the main axis direction b1, major axis length λ1, minor axis length λ2 and eccentricity of the group ellipse envelope

[0052] S2, respectively calculate the central track motion direction angle Ti_θ0, the two side edge track motion direction angles Ti_θ1, Ti_θ2, and the angle Ti_Δθ(t) between the two edge tracks and the central track motion direction at time t for each group in a single frame image;

[0053] Calculate the eccentricity difference Ti_Δe(t) of the three frames before and after each group at time t;

[0054] Calculate the distance between the centers of each group at time t Tab_Δd(t);

[0055] S2.1. The included angle Ti_θ0 of the center trajectory motion direction of each group is calculated by the following formula:

[0056]

[0057] The calculation formula of the edge trajectory motion direction angles Ti_θ1 and Ti_θ2 is:

[0058]

[0059]

[0060] The calculation formula of the angle Ti_Δθ(t) between the two edge tracks and the moving direction of the center track is:

[0061] Ti_Δθ(t)=|Ti_θ0(t)-Ti_θ1(t)|+|Ti_θ0(t)-Ti_θ2(t)|

[0062] Among them, (Ti_x0(t), Ti_y0(t)), (Ti_x0(t-1), Ti_y0(t-1)) are the central trajectory positions of the drone group Ti at time t and time t-1 respectively, (Ti_x1(t), Ti_y1(t)), (Ti_x2(t), Ti_y2(t)) are the two edge trajectory positions of the group ellipse envelope of the drone group Ti at time t respectively, (Ti_x1(t-1), Ti_y1(t-1)), (Ti_x2(t-1), Ti_y2(t-1)) are the two edge trajectory positions of the group ellipse envelope of the drone group Ti at time t-1 respectively, Ti_θ0(t) is the moving direction angle of the central trajectory of the drone group Ti at time t, Ti_θ1(t) and Ti_θ2(t) are the moving direction angles of the two edge trajectories of the group ellipse envelope at time t;

[0063] S2.2. The calculation formula of the eccentricity difference Ti_Δe(t) of the three frames before and after each group is:

[0064] Ti_Δe(t)=Ti_e(t)-Ti_e(t-3)

[0065] Where: Ti_e(t) is the eccentricity of the group ellipse shape of the drone group Ti at time t, Ti_e(t-3) is the eccentricity of the group ellipse shape of the drone group Ti at time t-3;

[0066] S2.3. The calculation formula of the distance between the centers of the two groups at time t is Tab_Δd(t):

[0067]

[0068] Among them: (Ta_x0(t), Ta_y0(t)) is the center trajectory position of group a at time t, and (Tb_x0(t), Tb_y0(t)) is the center trajectory position of group b at time t.

[0069] S3, taking the motion direction angle of each trajectory of the group target, the eccentricity difference, and the center distance between each group obtained in step S2 as input for separation or merging determination processing, extracting the merged key frame and the separated key frame, correcting the group target tracking parameters and the group situation estimation parameters, thereby completing the grouping and merging detection of the drone group target;

[0070] When performing group separation or merging determination processing, it specifically includes group separation detection and group merging detection;

[0071] The group separation test includes the following steps:

[0072] S3A.1, if Figure 1 As shown, the calculated result of the angle Ti_Δθ(t) between the two edge tracks and the moving direction of the center track at time t in step S2.1 is compared with the set separation angle threshold θ th Compare; if Ti_Δθ(t)≥θ th , then execute step S3A.2; if Ti Δθ (t)<θ th , it is determined that the frame has not been separated, and the process returns to step S2 to process the next frame of image;

[0073] S3A.2, if Figure 3 As shown, the calculation result of the eccentricity difference Ti_Δe(t) of the three frames before and after each group in step S2.2 is compared with the set eccentricity threshold e th Compare; if Ti_Δe(t)≥e th , it is determined that the group Ti is separated at time t, and the image frame at this time is the separation starting key frame Ti_N(t), thereby completing the separation detection and judgment of the drone group target, confirming the addition of a new group target tracking trajectory, and updating the tracking filter parameters; if Ti Δe (t)<e th , it is determined that the frame has not been separated, and the process returns to step S2 to process the next frame of image;

[0074] The group merge detection includes the following steps:

[0075] S3B.1, if Figure 2 As shown, the calculation result of the group center distance Tab_Δd(t) at time t in step S2.3 is compared with the set group center distance threshold d th For comparison, if Tab Δd (t)≤d th , then execute step S3B.2; if Tab Δd (t)>d th , it is determined that the frame has not been merged, and the process returns to step S2 to process the next frame of image;

[0076] S3B.2, such as Figure 4 As shown, the calculation result of the merged trajectory center angle threshold Tab_Δθ(t) in step S2.4 is compared with the set group merged trajectory center angle threshold ab_θ th Compare; if 5 consecutive frames all satisfy Tab_Δθ(t)≤ab_θ th , it is determined that the merger of group a and group b occurs at time t, and the image frame at this time is the merger start key frame Tab_N(t), thereby completing the merger detection and judgment of group a and group b, the two trajectories are merged into one trajectory, and the tracking filter parameters are updated; if one of the five consecutive frames does not satisfy Tab_Δθ(t)≤ab_θ th , it is determined that the frame has not been merged, and the process returns to step S2 to proceed to the next frame image for processing.

[0077] Among them, the group merge trajectory center angle threshold ab_θ th Determined by the following formula:

[0078]

[0079] in:

[0080] (Ta_x0(t), Ta_y0(t)) represents the center trajectory position of group a at time t, (Ta_x0(t-1), Ta_y0(t-1)) represents the center trajectory position of group a at time t-1;

[0081] (Tb_x0(t), Tb_y0(t)) represents the center trajectory position of group b at time t, and (Tb_x0(t-1), Tb_y0(t-1)) represents the center trajectory position of group b at time t-1.

Claims

1. A detection algorithm for group separation and merging based on the envelope characteristics of photoelectric detection of drone groups, characterized in that: The following steps are involved: S1. Collect a single-frame image of the drone group target through the photoelectric detector, and extract the cluster center coordinates (Ti_x0, Ti_y0) and the edge coordinates (Ti_x1, Ti_y1) and (Ti_x2, Ti_y2) of each drone group Ti in the single-frame image, and obtain the main axis direction b1, major axis length λ1, minor axis length λ2 and distance of the group ellipse envelope. S2, respectively calculate the central track motion direction angle Ti_θ0, the two side edge track motion direction angles Ti_θ1, Ti_θ2, and the angle Ti_Δθ(t) between the two edge tracks and the central track motion direction at time t for each group in a single frame image; Calculate the eccentricity difference Ti_Δe(t) of the three frames before and after each group at time t; Calculate the distance between the centers of each group at time t Tab_Δd(t); S3. The included angle of the trajectory motion direction of each group target, the eccentricity difference, and the center distance between each group obtained in step S2 are used as input for separation or merging judgment processing, and the merged key frame and the separated key frame are extracted. The group target tracking parameters and the group situation estimation parameters are corrected, thereby completing the grouping and merging detection of the drone group targets.

2. The detection algorithm for group separation and merging based on the envelope characteristics of photoelectric detection drone groups according to claim 1 is characterized in that: Step S2 is specifically as follows: S2.1, the central trajectory motion direction angle Ti_θ0 of each group is calculated by the following formula: The calculation formulas for the edge track motion direction angles Ti_θ1 and Ti_θ2 are as follows: The calculation formula of the angle Ti_Δθ(t) between the two edge tracks and the moving direction of the center track is: Ti_Δθ(t)=|Ti_θ0(t)-Ti_θ1(t)|+|Ti_θ0(t)-Ti_θ2(t)| Among them, (Ti_x0(t), Ti_y0(t)), (Ti_x0(t-1), Ti_y0(t-1)) are the central trajectory positions of the drone group Ti at time t and time t-1 respectively, (Ti_x1(t), Ti_y1(t)), (Ti_x2(t), Ti_y2(t)) are the two edge trajectory positions of the group ellipse envelope of the drone group Ti at time t respectively, (Ti_x1(t-1), Ti_y1(t-1)), (Ti_x2(t-1), Ti_y2(t-1)) are the two edge trajectory positions of the group ellipse envelope of the drone group Ti at time t-1 respectively, Ti_θ0(t) is the moving direction angle of the central trajectory of the drone group Ti at time t, Ti_θ1(t) and Ti_θ2(t) are the moving direction angles of the two edge trajectories of the group ellipse envelope at time t; S2.2, the calculation formula of the eccentricity difference Ti_Δe(t) of the three frames before and after each group is: Ti_Δe(t)=Ti_e(t)-Ti_e(t-3) Where: Ti_e(t) is the eccentricity of the group ellipse shape of the drone group Ti at time t, Ti_e(t-3) is the eccentricity of the group ellipse shape of the drone group Ti at time t-3; S2.3, the calculation formula of the center distance Tab_Δd(t) between two groups at time t is: Among them: (Ta_x0(t), Ta_y0(t)) is the center trajectory position of group a at time t, and (Tb_x0(t), Tb_y0(t)) is the center trajectory position of group b at time t.

3. The detection algorithm for group separation and merging based on the envelope characteristics of photoelectric detection drone groups according to claim 2 is characterized in that: Step S3 specifically includes: group separation detection and group merging detection; The group separation detection comprises the following steps: S3A.

1. Compare the calculated result of the angle Ti_Δθ(t) between the two edge tracks and the moving direction of the center track at time t in step S2.1 with the set separation angle threshold θ th Compare; if Ti_Δθ(t)≥θ th , then execute step S3A.2; if Ti Δθ (t)<θ th , it is determined that the frame has not been separated, and the process returns to step S2 to process the next frame of image; S3A.2, compare the calculated result of the eccentricity difference Ti_Δe(t) of the three frames before and after each group in step S2.2 with the set eccentricity threshold e th Compare; if Ti_Δe(t)≥e th , it is determined that the group Ti is separated at time t, and the image frame at this time is the separation starting key frame Ti_N(t), thereby completing the separation detection and judgment of the drone group target, confirming the addition of a new group target tracking trajectory, and updating the tracking filter parameters; if Ti Δe (t)<e th , it is determined that the frame has not been separated, and the process returns to step S2 to process the next frame of image; The group merging detection comprises the following steps: S3B.

1. Compare the calculated result of the group center distance Tab_Δd(t) at time t in step S2.3 with the set group center distance threshold d th For comparison, if Tab Δd (t)≤d th , then execute step S3B.2; if Tab Δd (t)>d th , it is determined that the frame has not been merged, and the process returns to step S2 to process the next frame of image; S3B.2, compare the calculated result of the merged trajectory center angle threshold Tab_Δθ(t) in step S2.4 with the set group merged trajectory center angle threshold ab_θ th Compare; if 5 consecutive frames all satisfy Tab_Δθ(t)≤ab_θ th , it is determined that the merger of group a and group b occurs at time t, and the image frame at this time is the merger start key frame Tab_N(t), thereby completing the merger detection and judgment of group a and group b, the two trajectories are merged into one trajectory, and the tracking filter parameters are updated; if one of the five consecutive frames does not satisfy Tab_Δθ(t)≤ab_θ th , it is determined that the frame has not been merged, and the process returns to step S2 to proceed to the next frame image for processing.

4. The detection algorithm for group separation and merging based on the envelope characteristics of photoelectric detection drone groups according to claim 3 is characterized by: In step S3B.2, the group merge trajectory center angle threshold ab_θ th Determined by the following formula: in: (Ta_x0(t), Ta_y0(t)) represents the center trajectory position of group a at time t, (Ta_x0(t-1), Ta_y0(t-1)) represents the center trajectory position of group a at time t-1; (Tb_x0(t), Tb_y0(t)) represents the center trajectory position of group b at time t, and (Tb_x0(t-1), Tb_y0(t-1)) represents the center trajectory position of group b at time t-1.

Images