Two-stage video target detection box association method and system based on density clustering

Abstract

The application discloses a two-stage video target detection frame association method and system based on density clustering, which is applied to unmanned aerial vehicle (UAV) inspection of power equipment, based on complete video collected by the UAV, two-stage steps are used to associate video target detection frames, in the first stage, a density clustering method with improved distance measurement is used to realize preliminary association of interframe target detection frames, in the second stage, an interpolation method is used to solve the problem of re-counting caused by short-time loss of the video target detection frames, association of the video target detection frames is completed, and then the number of targets contained in the video is accurately detected; the application can use future information to assist in association of the target detection frames based on complete video, shows good association accuracy, and can accurately detect the number of targets contained in the video, and shows better robustness in comparison with a traditional method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of associating video target detection boxes, in particular to a two-stage video target detection box association method and system based on density clustering. BACKGROUND

[0002] Using a UAV to conduct inspection on power equipment can save a lot of manpower cost, and has been more and more widely used in recent years. In order to automatically detect the video collected by the UAV, it is necessary to first identify and match the electrical equipment objects existing in the video. The traditional target detection algorithm can only detect each frame of the video, and return the position and size of the electrical equipment target in each frame, so an algorithm is needed to associate the detection boxes belonging to the same target, so as to facilitate subsequent state analysis of each identification.

[0003] In the traditional technology, researchers often use target tracking algorithms to complete such tasks, such as the SORT algorithm, which is a real-time target tracking algorithm. Due to the emphasis on real-time performance, the phenomenon of re-counting and mis-association is prone to occur, and the re-counting phenomenon refers to the misidentification of the target detection box belonging to an object as multiple objects. The re-counting phenomenon has no significant effect on judging the existence of the target, but in power inspection, if the video target can be accurately corresponded to the real target, and then the health status of the real equipment can be judged through the video information, a more suitable target detection box association algorithm needs to be researched according to the characteristics of the UAV inspection task. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art, and a two-stage video target detection box association method and system based on density clustering are proposed. In the process of UAV video inspection task, due to the limitation of computing power and power consumption, the UAV usually only completes the video collection task, and the video analysis is realized using other devices after the UAV returns, which means that the task is usually offline. According to this characteristic, the present application does not need to design a complex predictor to estimate the future state of the target like the SORT algorithm, but can use future information to assist in target detection box association based on the complete video, and then obtain more accurate and robust results.

[0005] The purpose of the present application is achieved by the following technical scheme: a two-stage video target detection box association method based on density clustering, which is applied to the inspection of power equipment by a UAV, based on the complete video collected by the UAV, using two stages to associate the video target detection box, in the first stage, using the improved distance measurement density clustering method to realize the preliminary association of the inter-frame target detection box, in the second stage, using the interpolation method to solve the re-counting problem caused by the short-time loss of the video target detection box, complete the association of the video target detection box, and then realize the accurate detection of the number of targets contained in the video.

[0006] Further, the density clustering method of improved distance metric includes:

[0007] The length and width information of the detection frame given by the target detection algorithm target is combined, and the size and position of the target are considered to calculate a reasonable distance estimate d ns , Distance estimate d ns is composed of two parts, which are normalized distance d n and shape distance d s , as shown in the following formula:

[0008] d ns = d n * d s ;

[0009] Among them, the normalized distance d n is to use the length and width data of two video target detection frames as a unit to measure the distance between two video target detection frames. It is assumed that the two targets are located in the same plane, and in the process of approaching the unmanned aerial vehicle, even if the proportion of the target in the overall picture gradually increases, the distance between the two targets is relatively fixed compared to the size of the proportion. Therefore, we give the normalized distance d n calculation formula:

[0010]

[0011] Among them, h1 is the length of the first video target detection frame, w1 is the width of the first video target detection frame, h2 is the length of the second video target detection frame, w2 is the width of the second video target detection frame, (x1, y1) is the first target point coordinate, and (x2, y2) is the second target point coordinate.

[0012] When two video target detection frames have one side completely coinciding, the normalized distance d n is always 1, and when two video target detection frames have one vertex completely coinciding, the normalized distance is always

[0013] In addition to the size of the detection frame, the height-width ratio of the detection frame of the same object between adjacent frames will usually remain relatively constant. If the two detection frames belong to the same target, then is close to 1, and the shape distance d s is defined as:

[0014]

[0015] wherein h1 is the length of the first video target bounding box, w1 is the width of the first video target bounding box, h2 is the length of the second video target bounding box, and w2 is the width of the second video target bounding box; the minimum value 1 is obtained only when the length-width ratios of the two target boxes are exactly the same, so that for two pairs of video target bounding boxes with the same normalized distance, the comprehensive distance of the pair of video target bounding boxes with more similar shapes is smaller.

[0016] Finally, the frame range parameter FrameSpan is introduced, and if the frame difference of the two video target bounding boxes is greater than the frame range parameter FrameSpan, distance calculation is not performed, and it is considered not to be in the neighborhood.

[0017] Further, the use of the interpolation method to solve the re-counting problem caused by short-term loss of the target bounding box comprises:

[0018] The missing frames between two adjacent clusters are completed using the linear interpolation method, and the maximum interpolation number MaxIF is defined, if the number of missing frames between the adjacent two clusters is greater than the maximum interpolation number, interpolation calculation is not performed, and two different targets are directly considered; in addition, if two clusters have intersection in the time dimension, interpolation calculation is also not performed; for any cluster, the adjacent clusters that meet the interpolation condition may not be unique, therefore, the calculation priority needs to be determined, the improved distance between the two most adjacent points of the two clusters is taken as the measurement index, the interpolation calculation is preferentially performed on the pair with smaller distance, if the merging condition is not met, the interpolation calculation is performed on the pair with larger distance, and the process is repeated until all adjacent clusters that meet the interpolation condition are traversed.

[0019] The two-stage video target bounding box association system based on density clustering is used to implement the two-stage video target bounding box association method based on density clustering, comprising:

[0020] The clustering module realizes preliminary association of inter-frame target bounding boxes according to the density clustering method of improved distance measurement;

[0021] The interpolation module solves the re-counting problem caused by short-term loss of the video target bounding box according to the interpolation method.

[0022] Further, the clustering module comprises:

[0023] The length-width information of the detection box given by the target detection algorithm is combined, and the size and position of the target are considered, so that a reasonable distance estimate d ns is calculated. ns The distance estimate d n is composed of two parts, which are normalized distance d s and shape distance d ns , as shown in the following formula:

[0024] d n * ds ;

[0025] wherein, the normalized distance d n is to measure the distance between two video target bounding boxes using the length and width data of the two video target bounding boxes as units, and it is assumed that there are two targets in the same plane, and in the process of approaching the UAV, even if the proportion of the target in the overall picture gradually increases, the distance between the two targets is relatively fixed compared to the proportion of their own size, and thus we give the normalized distance d n The calculation formula is:

[0026]

[0027] wherein, h1 is the length of the first video target bounding box, w1 is the width of the first video target bounding box, h2 is the length of the second video target bounding box, and w2 is the width of the second video target bounding box; (x1, y1) is the first target point coordinate, and (x2, y2) is the second target point coordinate;

[0028] When two video target bounding boxes have one side completely coinciding, the normalized distance d n is always 1, and when two video target bounding boxes have one vertex completely coinciding, the normalized distance is always

[0029] In addition to the size of the detection box, the height-width ratio of the detection box of the same object between adjacent frames will usually remain relatively constant, and if the two detection boxes belong to the same target, then is close to 1, and the shape distance d s is defined:

[0030]

[0031] wherein, h1 is the length of the first video target bounding box, w1 is the width of the first video target bounding box, h2 is the length of the second video target bounding box, and w2 is the width of the second video target bounding box; and only when the length-width ratio of the two target boxes is exactly the same, the minimum value 1 is obtained, so that for two pairs of video target bounding boxes with the same normalized distance, the comprehensive distance of the pair of video target bounding boxes with more similar shapes is smaller;

[0032] Finally, the frame range parameter FrameSpan is introduced, and if the frame difference of the two video target bounding boxes is greater than the frame range parameter FrameSpan, the distance calculation is not performed, and it is considered not to be in the neighborhood.

[0033] Further, the frame insertion module comprises:

[0034] The missing frame between two adjacent clusters is completed by using a linear interpolation method, a maximum interpolation frame number MaxIF is defined, if the number of missing frames between the two adjacent clusters is greater than the maximum interpolation frame number, interpolation frame calculation is not performed, and two different targets are directly considered; in addition, if two clusters have intersection in the time dimension, interpolation frame calculation is also not performed; for any cluster, the adjacent clusters that meet the interpolation frame condition may not be unique, therefore, the calculation priority needs to be determined, the improved distance between the two most adjacent points of the two clusters is taken as the measurement index, the pairing with smaller distance is preferentially calculated, if the merging condition is not met, the pairing with larger distance is calculated, and all adjacent clusters that meet the interpolation frame condition are traversed.

[0035] A non-transitory computer readable medium storing instructions, when executed by a processor, performs the steps of the two-stage video target bounding box association method based on density clustering.

[0036] A computing device includes a processor and a memory for storing a processor-executable program, when the processor executes the program stored in the memory, the two-stage video target bounding box association method based on density clustering is implemented.

[0037] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0038] 1、The present application uses the density clustering method with improved distance measurement in the first stage to realize the preliminary association of inter-frame target detection boxes, uses the interpolation method in the second stage to solve the re-counting problem caused by short-term loss of target detection boxes, and shows good association accuracy in experiments, can accurately detect the number of targets contained in the video, and shows better robustness in comparison with traditional methods.

[0039] 2、The density clustering method with improved distance measurement is used in the first stage of the present application, the length and width information of the detection box given by the target detection algorithm can be combined, the size and position of the target are considered comprehensively, and therefore more reasonable distance estimation is given, and the problem that the target cannot be correctly associated due to the large distance of the target moving in unit time in the process of the UAV gradually approaching the target due to image perspective in the video frame is solved.

[0040] 3、The interpolation method is used in the second stage of the present application, the short-term loss of the target in the video frame can be solved when the object is blocked and the surrounding environment of the object is changed dramatically, and since the present application is based on the complete video collected by the UAV for calculation, the future frame calculation in the video can be effectively used to solve the re-counting problem, and simple linear extrapolation prediction is not needed. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1A schematic diagram of a clustering process of a density clustering algorithm.

[0042] Figure 2 A schematic diagram of a target bounding box that cannot be associated due to image perspective.

[0043] Figure 3 A schematic diagram of a normalized distance.

[0044] Figure 4 A schematic diagram of a shape distance.

[0045] Figure 5 A process diagram for handling target loss.

[0046] Figure 6 A flowchart of an algorithm of a two-stage video target bounding box association method based on density clustering. DETAILED DESCRIPTION

[0047] The application will be further described below with reference to specific embodiments.

[0048] Embodiment 1

[0049] The two-stage video target bounding box association method based on density clustering provided in the embodiment is applied to unmanned aerial vehicle inspection of power equipment. Based on complete videos collected by the unmanned aerial vehicle, two stages are used to associate the video target bounding boxes. In the first stage, a density clustering method with improved distance measurement is used to realize preliminary association of inter-frame target bounding boxes. In the second stage, an interpolation method is used to solve the re-counting problem caused by short-term loss of the video target bounding boxes, complete the association of the video target bounding boxes, and accurately detect the number of targets contained in the video.

[0050] Referring to Figure 1 DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density clustering method that can discover clusters of arbitrary shape in spatial data sets with noise and does not need to specify the number of classes in advance. It divides the data into core points and boundary points according to the density threshold, and has two parameters, ε and MinPts. ε refers to the neighborhood radius, and MinPts refers to the minimum number of points in the neighborhood of a core point, that is, the density threshold of DBSCAN. DBSCAN finds a cluster by randomly selecting a core point and traversing all points density-reachable from it, judging whether they are core objects, and expanding through the propagation of core objects. DBSCAN can discover all clusters one by one and mark unclassified points as noise points.

[0051] If the center coordinates of continuously moving objects are laid flat under the same coordinate system, the discrete points belonging to the same object form a cluster of trajectories. This allows us to use clustering algorithms to discover data clusters. Considering that the target detection boxes in video inspection tasks always present a banded distribution and we cannot predict the total number of clusters, the DBSCAN algorithm, which can discover clusters of arbitrary shapes and does not require pre-specifying the number of clusters, becomes the preferred choice.

[0052] Using DBSCAN for target association must first address the perspective problem in the image. During UAV inspection, targets always change from far to near and from small to large, meaning the distance the target bounding box moves between adjacent frames also changes gradually. The DBSCAN algorithm requires a fixed ε parameter, and this parameter setting has a decisive impact on the detection results. See also Figure 2 As shown, if the ε parameter is set too small, the target's movement distance per unit time will be too large as the drone gradually approaches the target, making it impossible to correctly associate the target with the drone. If it is set too large, multiple targets may be incorrectly classified as a single target.

[0053] To address the problems caused by perspective, the improved distance metric density clustering method includes:

[0054] By combining the width and height information of the detection bounding box provided by the object detection algorithm, and considering the size and position of the object, a reasonable distance estimate d is calculated. ns Distance estimation d ns It consists of two parts, namely the normalized distance d n and shape distance d s As shown in the formula below:

[0055] d ns =d n *d s ;

[0056] Among them, see Figure 3 As shown, the normalized distance d n To measure the distance between two video object detection boxes using their width and height as units, we assume that two targets are located on the same plane. As the drone approaches, even though the proportion of the target in the overall frame gradually increases, the ratio of the distance between the two targets to their own size remains relatively constant. Therefore, we derive the normalized distance d. n Calculation formula:

[0057]

[0058] Where h1 is the length of the first video target detection box, w1 is the width of the first video target detection box, h2 is the length of the second video target detection box, w2 is the width of the second video target detection box, (x1, y1) are the coordinates of the first target point, and (x2, y2) are the coordinates of the second target point;

[0059] Compared to Euclidean distance, normalized distance does not require prior estimation of the approximate range of the distance between object boxes in the image. Regardless of the specific size of the object boxes, when two video object detection boxes have one edge that completely overlaps, the normalized distance d... n The normalized distance is always 1 when two video object detection boxes have a vertex that completely overlaps.

[0060] Aside from the size of the detection box, the aspect ratio of the detection box for the same object typically remains relatively constant between adjacent frames. See [link / reference] Figure 4 As shown, if two detection boxes belong to the same target, then If the distance metric is close to 1, but the distance index needs to be reciprocal, then the shape distance d is defined. s :

[0061]

[0062] Where h1 is the length of the first video target detection box, w1 is the width of the first video target detection box, h2 is the length of the second video target detection box, and w2 is the width of the second video target detection box; the minimum value of 1 is obtained only when the aspect ratios of the two target boxes are exactly the same, so that for two pairs of video target detection boxes with the same normalized distance, the combined distance of the pair of video target detection boxes with more similar shapes is smaller.

[0063] Finally, the frame range parameter FrameSpan is introduced. If the difference between the frames of two video target detection boxes is greater than the frame range parameter FrameSpan, no distance calculation is performed, and they are considered not to be in the neighborhood. This operation will not affect the density clustering. For three target detection boxes A, B, and C belonging to the same object, even if the difference between the frames of A and C is greater than FrameSpan, if B is located in the neighborhood of both A and C, then the three detection boxes can still be classified into one class.

[0064] The issue of short-term target loss during the target detection stage also needs to be addressed. Target loss typically occurs when objects are occluded or when the surrounding environment changes drastically. The SORT algorithm can predict the position of the target box using a Kalman filter, thus alleviating the recounting problem caused by target loss to some extent. However, this prediction is merely a simple linear extrapolation. Considering that the video is fully available in UAV inspection missions, it is necessary to consider how to effectively utilize information from future frames.

[0065] Referring to Figure 5 As shown in Figure 5 In (a), due to the short-term loss of the target during shooting, the density clustering method will identify the same entity as two different targets, while the traditional SORT method will use the last state of the target box for linear extrapolation, as shown by the line in Figure 5 In (b), the line shows that this method will have a large error when the object moves nonlinearly; considering that the target detection box in the future frame is known, it means that the missing frame between the two adjacent clusters can be completed by linear interpolation, as shown by the completion point between the two adjacent clusters in Figure 5 (b), on this basis, we run the density clustering algorithm again for all points in Figure 5 (c), if all points can be attributed to a class, it means that the two cluster data points belong to the same target.

[0066] The use of the interpolation method to solve the re-counting problem caused by the short-term loss of the target detection box includes:

[0067] The missing frame between the two adjacent clusters is completed by using the linear interpolation method, and the maximum interpolation number MaxIF is defined, if the number of missing frames between the two adjacent clusters is greater than the maximum interpolation number, interpolation calculation is not performed, and it is directly regarded as two different targets; in addition, if the two clusters have intersection in the time dimension, interpolation calculation is also not performed; for any cluster, the adjacent clusters that meet the interpolation condition may not be unique, therefore, the calculation priority needs to be determined, the improved distance between the two most adjacent points of the two clusters is taken as the measurement index, the pairing with smaller distance is preferentially calculated, if the merging condition is not met, the pairing with larger distance is calculated, and the process is repeated until all adjacent clusters that meet the interpolation condition are traversed.

[0068] The overall flow of the algorithm is as follows:

[0069] Referring to Figure 6 As shown, the first to ninth lines are the first stage of the algorithm, and the steps are exactly the same as DBSCAN, the difference is that the improved distance function is used to search the neighborhood of the core point at line 5. In addition, it should be pointed out that each round of clustering is only for a single category of object, because objects of different categories cannot belong to the same object. The tenth to nineteenth lines are the second stage of the algorithm, the process is to traverse the clustering obtained in the first step, search for other clusters that meet the interpolation condition through the getInterpolatableClusters function, sort these clusters that meet the interpolation condition according to the distance as shown in line 12, then use the getInterpolatedPoints function for interpolation, and finally judge whether the two clusters after interpolation can be combined into one, if they can be combined into one, update the clustering result, and repeat the process until there is no other object that meets the interpolation condition for the cluster.

[0070] The efficiency of this algorithm is similar to that of DBSCAN because the second stage only requires simple calculations based on the finite clusters provided in the first stage. Furthermore, for noisy points in the data, only small clusters need to be filtered out by setting a reasonable threshold.

[0071] Example 2

[0072] This embodiment provides a two-stage video object detection bounding box association system based on density clustering, used to implement the two-stage video object detection bounding box association method based on density clustering described in Embodiment 1, including:

[0073] The clustering module performs preliminary association of target detection boxes between frames based on density clustering with improved distance metric.

[0074] The frame interpolation module solves the recounting problem caused by the short-term loss of video target detection boxes using the frame interpolation method.

[0075] The clustering module includes:

[0076] By combining the width and height information of the detection bounding box provided by the object detection algorithm, and considering the size and position of the object, a reasonable distance estimate d is calculated. ns Distance estimation d ns It consists of two parts, namely the normalized distance d n and shape distance d s As shown in the formula below:

[0077] d ns =d n *d s ;

[0078] Wherein, the normalized distance d n To measure the distance between two video object detection boxes using their width and height as units, we assume that two targets are located on the same plane. As the drone approaches, even though the proportion of the target in the overall frame gradually increases, the ratio of the distance between the two targets to their own size remains relatively constant. Therefore, we derive the normalized distance d. n Calculation formula:

[0079]

[0080] Where h1 is the length of the first video target detection box, w1 is the width of the first video target detection box, h2 is the length of the second video target detection box, w2 is the width of the second video target detection box, (x1, y1) are the coordinates of the first target point, and (x2, y2) are the coordinates of the second target point;

[0081] When two video object detection boxes have one edge that completely overlaps, the normalized distance d nTotal is 1 when two video target bounding boxes have one vertex completely coincident, and the normalized distance is always

[0082] In addition to the size of the bounding box, the height-to-width ratio of the bounding box of the same object between adjacent frames is usually relatively constant, and if two bounding boxes belong to the same target, then Close to 1, define shape distance d s :

[0083]

[0084] where h1 is the length of the first video target bounding box, w1 is the width of the first video target bounding box, h2 is the length of the second video target bounding box, and w2 is the width of the second video target bounding box; and the minimum value 1 is obtained only when the length-to-width ratios of the two target boxes are exactly the same, so that for two pairs of video target bounding boxes with the same normalized distance, the pair of video target bounding boxes with a more similar shape has a smaller comprehensive distance;

[0085] Finally, the frame range parameter FrameSpan is introduced, and if the frame difference between the two video target bounding boxes is greater than the frame range parameter FrameSpan, no distance calculation is performed, and it is considered not to be in the neighborhood.

[0086] The interpolation module comprises:

[0087] The missing frames between two adjacent clusters are completed using the linear interpolation method, and the maximum number of interpolation frames MaxIF is defined, and if the number of missing frames between the two adjacent clusters is greater than the maximum number of interpolation frames, no interpolation calculation is performed, and it is directly considered as two different targets; in addition, if there is an intersection between the two clusters in the time dimension, no interpolation calculation is also performed; for any cluster, the adjacent clusters that meet the interpolation condition may not be unique, so the calculation priority needs to be determined, and the improved distance between the two most adjacent points between the two clusters is taken as the measurement index, and the interpolation calculation is preferentially performed on the pair with smaller distance, and if the merging condition is not met, the pair with slightly larger distance is calculated, and the process is repeated until all adjacent clusters that meet the interpolation condition are traversed.

[0088] Embodiment 3

[0089] The embodiment discloses a non-transitory computer readable medium storing instructions, when the instructions are executed by a processor, the steps of the two-stage video target bounding box association method based on density clustering according to embodiment 1 are performed.

[0090] The non-transitory computer readable medium in the embodiment can be a disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), a U disk, a mobile hard disk, and the like.

[0091] Embodiment 4

[0092] The embodiment discloses a computing device, comprising a processor and a memory for storing a processor-executable program, when the processor executes the program stored in the memory, the two-stage video target detection box association method based on density clustering in the embodiment 1 is realized.

[0093] The computing device in the embodiment can be a desktop computer, a notebook computer, a smart phone, a PDA handheld terminal, a tablet computer, a programmable logic controller (PLC), or other terminal devices with processor functions.

[0094] The above-mentioned embodiments are only the preferred embodiments of the present application, and are not intended to limit the scope of the present application. Any changes made according to the shape and principle of the present application should be covered within the scope of protection of the present application.

Claims

1. A two-stage video object detection bounding box association method based on density clustering, characterized in that: The method is applied to unmanned aerial vehicle inspection of power equipment, based on complete video collected by the unmanned aerial vehicle, two-stage steps are used to associate video target detection boxes, in the first stage, an improved distance metric density clustering method is used to realize preliminary association of inter-frame target detection boxes, in the second stage, an interpolation method is used to solve the problem of re-counting caused by short-time loss of the video target detection box, the association of the video target detection box is completed, and the number of targets contained in the video is accurately detected; The improved distance metric density clustering method comprises: Combine the target detection algorithm given target detection frame length and width information, and consider the size and position of the target, so as to calculate the reasonable distance estimation d ns , distance estimation d ns Composed of two parts, respectively, normalized distance d n And shape distance d s As shown in the following formula: d ns = d n * d s ; wherein the normalized distance d n In order to measure the distance between two video target detection frames using the length and width data of the two video target detection frames as units, it is assumed that the two targets are located on the same plane, and in the process of approaching the UAV, even if the proportion of the targets in the overall picture gradually increases, the proportion of the distance between the two targets compared to the size of the targets is relatively fixed, and thus the normalized distance d n The calculation formula is: Wherein, h1 is the length of the first video target detection box, w1 is the width of the first video target detection box, h2 is the length of the second video target detection box, w2 is the width of the second video target detection box, (x1, y1) is the first target point coordinate, (x2, y2) is the second target point coordinate; When two video object bounding boxes have one edge completely coinciding, the normalized distance d n When two video object bounding boxes have one vertex completely coinciding, the normalized distance d In addition to the size of the detection frame, the height-to-width ratio of the detection frame of the same object between adjacent frames will generally remain relatively constant, and if the two detection frames belong to the same target, then Approaching 1, define shape distance d s : Wherein, h1 is the length of the first video target detection box, w1 is the width of the first video target detection box, h2 is the length of the second video target detection box, w2 is the width of the second video target detection box; Only when the length-width ratio of the two target boxes is exactly the same, the minimum value 1 is obtained, so that for two pairs of video target detection boxes with the same normalized distance, the comprehensive distance of the pair of video target detection boxes with more similar shape is smaller. Finally, the frame range parameter FrameSpan is introduced, if the frame difference value of the two video target detection boxes is greater than the frame range parameter FrameSpan, the distance calculation is not performed, and it is considered not in the neighborhood.

2. The two-stage video object detection box association method based on density clustering according to claim 1, wherein, The use of the interpolation method to solve the problem of re-counting caused by short-time loss of the target detection box comprises: The missing frames between two adjacent clusters are completed by using linear interpolation method, the maximum interpolation number MaxIF is defined, if the number of missing frames between the two adjacent clusters is greater than the maximum interpolation number, interpolation calculation is not performed, and two different targets are directly considered; In addition, if two clusters have intersection in time dimension, interpolation calculation is also not performed; For any cluster, the adjacent clusters that meet the interpolation condition may not be unique, therefore, the calculation priority needs to be determined, the improved distance between the two most adjacent points of the two clusters is used as the measurement index, the interpolation calculation is preferentially performed on the pair with smaller distance, if the merging condition is not met, the interpolation calculation is performed on the pair with larger distance, and all adjacent clusters that meet the interpolation condition are traversed.

3. A two-stage video object detection bounding box association system based on density clustering, characterized in that, The method for realizing the two-stage video target detection box association method based on density clustering in any one of claims 1-2 comprises: The clustering module realizes preliminary association of inter-frame target detection boxes according to the improved distance metric density clustering method, comprising: Combine the target detection algorithm given target detection frame length and width information, and consider the size and position of the target, so as to calculate the reasonable distance estimation d ns , Distance estimation d ns Composed of two parts, respectively, normalized distance d n And shape distance d s As shown in the following formula: d ns = d n * d s ; wherein the normalized distance d n In order to measure the distance between two video target detection frames using the length and width data of the two video target detection frames as units, it is assumed that the two targets are located on the same plane, and in the process of approaching the UAV, even if the proportion of the targets in the overall picture gradually increases, the proportion of the distance between the two targets compared to their own size is relatively fixed, and thus the normalized distance d n The calculation formula is: Wherein, h1 is the length of the first video target detection box, w1 is the width of the first video target detection box, h2 is the length of the second video target detection box, w2 is the width of the second video target detection box, (x1, y1) is the first target point coordinate, (x2, y2) is the second target point coordinate; When two video object bounding boxes have one edge completely coinciding, the normalized distance d n When two video object bounding boxes have one vertex completely coinciding, the normalized distance d In addition to the size of the detection frame, the height-to-width ratio of the detection frame of the same object between adjacent frames will generally remain relatively constant, and if the two detection frames belong to the same target, then Approaching 1, define shape distance d s : Wherein, h1 is the length of the first video target detection frame, w1 is the width of the first video target detection frame, h2 is the length of the second video target detection frame, w2 is the width of the second video target detection frame; the minimum value 1 is obtained only when the length-width ratios of the two target frames are exactly the same, so that for two pairs of video target detection frames with the same normalized distance, the comprehensive distance of the pair of video target detection frames with more similar shapes is smaller; Finally, the frame range parameter FrameSpan is introduced, if the frame difference of the two video target detection frames is greater than the frame range parameter FrameSpan, the distance calculation is not performed, and it is considered not in the neighborhood; The frame insertion module solves the re-counting problem caused by short-time loss of the video target detection frame by using the frame insertion method.

4. The two-stage video object detection bounding box association system based on density clustering of claim 3, wherein, The frame insertion module comprises: The missing frames between two adjacent clusters are completed by using the linear interpolation method, the maximum frame insertion number MaxIF is defined, if the number of missing frames between the two adjacent clusters is greater than the maximum frame insertion number, the frame insertion calculation is not performed, and it is directly considered as two different targets; in addition, if there is an intersection between the two clusters in the time dimension, the frame insertion calculation is also not performed; for any cluster, the adjacent clusters that meet the frame insertion condition may not be unique, therefore, the calculation priority needs to be determined, the improved distance between the two most adjacent points between the two clusters is taken as the measurement index, the frame insertion calculation is preferentially performed on the pair with smaller distance, if the merging condition is not met, the frame insertion calculation is performed on the pair with larger distance, and the calculation is performed on all adjacent clusters that meet the frame insertion condition.

5. A non-transitory computer-readable medium storing instructions, the method comprising: When the instructions are executed by the processor, the steps of the two-stage video target detection frame association method based on density clustering according to any one of claims 1-2 are executed.

6. A computing device comprising a processor and a memory for storing processor-executable programs, characterized in that When the processor executes the program stored in the memory, the two-stage video target detection frame association method based on density clustering according to any one of claims 1-2 is realized.

Images