Multi-target cataloguing tracking method based on data association

Abstract

The invention belongs to the technical field of optical measurement, and particularly relates to a multi-target data association cataloguing method. According to the method, in combination with association characteristics of a Deep Sort algorithm and time sequence multi-target data, target appearance characteristics are extracted in a real-time target tracking process by introducing a CNN model to perform nearest neighbor matching, track smoothness verification and confirmation are performed, the target tracking problem and the problem of frequent jump of a target number ID under a shielding condition are solved, and the target tracking efficiency is improved. And the serial number stability during multi-target tracking is ensured. After a multi-target two-dimensional track in a single measuring station is obtained, intersection calculation can be carried out on the two-dimensional tracks of two or more measuring stations to obtain a three-dimensional space track.

Description

Technical Field

[0001] This invention belongs to the field of optical measurement technology, and in particular relates to a multi-target cataloging and tracking method based on data association. Background Technology

[0002] In traditional optical measurement equipment tracking, multi-target tracking generally refers to tracking multiple secondary targets while simultaneously tracking the primary target. A key characteristic of multi-target tracking is the need to track the size and position of multiple targets simultaneously. Currently, most multi-target tracking algorithms are based on a Tracking-by-Detection strategy, meaning they track targets based on target recognition results. The basic idea of ​​multi-target tracking is to use target recognition algorithms to identify targets in the acquired raw video frames, extract feature information from the bounding boxes corresponding to the detected targets, including appearance and motion features, and then calculate the matching degree between adjacent frames to catalog the tracked targets.

[0003] Currently, the following problems exist in the cataloging of tracked targets in multi-target tracking tasks: (1) the catalog cannot be updated in time when new targets appear or the original targets disappear; (2) the motion prediction and similarity judgment of tracked targets are not accurate, that is, the target cataloging of the previous frame image and the next frame image do not match; (3) in overlapping and occlusion scenarios, the cataloging is prone to errors or interruptions. Summary of the Invention

[0004] In view of this, the present invention aims to provide a multi-target cataloging and tracking method based on data association to solve the multi-target tracking problem under occlusion conditions and the problem of frequent changes in target ID in the prior art. The present invention can improve the stability of optical measurement equipment for real-time cataloging and tracking of multiple targets.

[0005] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0006] A multi-target cataloging and tracking method based on data association specifically includes the following steps: S1: Obtain the target trajectory set. The optical measurement device tracks multiple targets in real time with the main target as the field of view center and obtains the k-th frame image. k starts from 3. Based on the deep learning target detection algorithm, target detection is performed on the k-th frame image to obtain the position information of each target. S2: Based on the trajectory of each target in the (k-1)th frame image, predict the trajectory points of each target in the kth frame image. Based on the position information of each target in the previous (k-1)th frames image, determine whether the trajectory of each target in the kth frame image is a definite trajectory. Execute step S3 based on the definite trajectory and execute step S4 based on the uncertain trajectory. S3: Perform concatenated matching on the position information of each target in the k-th frame image and the trajectory point prediction results of each target in the k-th frame image. Execute step S6 based on each target that is successfully matched and the trajectory corresponding to each target. Execute step S4 based on each target that fails to be matched. S4: Perform IOU matching on each target and each trajectory that failed to match, and perform IOU matching on each target and the uncertain trajectory that failed to match. If either of them matches successfully, then execute step S6 based on each target that successfully matches and the trajectory corresponding to each target. Otherwise, update the number of failed matches for each target and trajectory that failed to match, and execute step S5. The position information of each target that failed to match is used as the first trajectory point of each new target and added to the target trajectory set. S5: Determine if there are any targets and trajectories whose number of failed matches exceeds the threshold. If so, delete the targets and trajectories whose number of failed matches exceeds the threshold and update the target trajectory set. Otherwise, proceed to step S6. S6: Construct an evaluation function, use the evaluation function to verify the trajectory smoothness of each target corresponding to the cascaded matching or the target corresponding to the IOU matching, update the position information of each target that has successfully verified the trajectory smoothness to the trajectory data corresponding to each target, and update the Kalman filter based on the updated trajectory data corresponding to each target, and add each target that fails the trajectory smoothness verification as the first trajectory point of each new target to the target trajectory set. S7: Determine whether the target tracking task has ended. If not, output the multi-target data association cataloging result based on the target trajectory set corresponding to the k-th frame image, and replace the k-th frame target image with the (k+1)-th frame target image to execute steps S1-S6 until the target tracking task ends. If yes, output the multi-target data association cataloging result based on the target trajectory set corresponding to the k-th frame image.

[0007] Furthermore, in step S1, the method for obtaining the target trajectory set is as follows: The optical measurement equipment tracks multiple targets in real time with the main target as the center of the field of view, obtains images arranged in the shooting sequence, performs target detection on the first frame image based on a deep learning target detection algorithm, obtains the position information of each target, obtains the first trajectory point of each target based on the position information of each target, generates the corresponding target number, and constructs an initial target trajectory set. Obtain the second frame image, perform target detection on the second frame image based on the deep learning target detection algorithm, and predict the trajectory points of each target in the second frame image based on the initial target trajectory set. Match the position information of each target in the second frame image with the trajectory point prediction results of each target in the second frame image. Add the position information of each successfully matched target to the trajectory data of the corresponding target as the second trajectory point of each target. Use the position information of each unmatched target as the first trajectory point of each new target. Create a new target number and add it to the current target trajectory set. Obtain the target trajectory set obtained in step S1, and update the number of unmatched targets and trajectories.

[0008] Furthermore, in step S1, the deep learning object detection algorithm is either the YOLO algorithm or the SSD algorithm.

[0009] Furthermore, in step S2, a Kalman filter is used to predict the trajectory points of each target in the k-th frame image, where the trajectory numbered i in the k-th frame image is... The trajectory point prediction results are as follows: ; in, For the trajectory numbered i in the k-th frame of the image. The trajectory point prediction results For the trajectory numbered i in the (k-1)th frame of the image The prediction results For the trajectory numbered i in the (k-1)th frame of the image The target predicted velocity change, This represents the time interval between two adjacent frames.

[0010] Furthermore, in step S2, the specific method for determining whether the trajectory corresponding to each target is a determined trajectory based on the target trajectory set of the previous k-1 frames is as follows: for the trajectory numbered i in the current frame image... Statistical analysis of the trajectory with ID i The location information corresponding to the first k-1 frames of the image is recorded at least 3 times. If the number of records is not less than 3 times, it is determined as a definite trajectory; otherwise, it is determined as an uncertain trajectory.

[0011] Furthermore, in step S3, a traversal approach is adopted, and the position information of the j-th target in the k-th frame image is obtained using the following formula. and the trajectory numbered i in the k-th frame image Perform cascading matching: ; in, As a weighting factor, The Mahalanobis distance metric. This is a cosine distance metric. This is the cascaded matching value.

[0012] Furthermore, the formula for calculating the Mahalanobis distance metric is as follows:

[0013] in, For the k-th frame of the image The detection box corresponding to each target For the trajectory numbered i in the k-th frame image The trajectory point prediction results Let be the covariance matrix between the location information corresponding to trajectory number i in the k-th frame and the average location information corresponding to trajectory number i in the previous k-1 frames.

[0014] Furthermore, the formula for calculating the cosine distance metric is as follows: ; in, For the detection box corresponding to the j-th target in the k-th frame image The descriptive vector of the extracted appearance features. This represents the set of descriptive vectors representing the appearance features of each target successfully associated with trajectory number i. Belongs to set The appearance feature description vector.

[0015] Furthermore, in step S6, the evaluation function... The calculation formula is: ; in, and As a weighting factor, For the trajectory numbered i in the (k-2)th frame image The center coordinates, For the trajectory numbered i in the (k-1)th frame of the image The center coordinates, For the k-th frame of the image The center coordinates of each target; If the evaluation function value is less than the set threshold, the smoothness verification of the trajectory corresponding to the current target is successful; otherwise, the smoothness verification of the trajectory corresponding to the current target fails.

[0016] Furthermore, step S6 also includes: updating the position information of the target whose trajectory smoothness verification was successful in the current frame image to the trajectory data corresponding to the target, updating the Kalman filter based on the updated trajectory data corresponding to the target, and using the updated Kalman filter to predict the trajectory points of each target in the next frame image.

[0017] Compared with the prior art, the present invention can achieve the following beneficial effects: This invention creates a multi-target data association and cataloging method that combines the Deep Sort algorithm with the association characteristics of temporal multi-target data. By introducing a CNN model, it extracts target appearance features for nearest neighbor matching during real-time target tracking and verifies trajectory smoothness. This solves the target tracking problem under occlusion and the problem of frequent target ID jumps, ensuring stable IDs during multi-target tracking. After obtaining the two-dimensional trajectories of multiple targets at a single measurement station, the intersection calculation of the two-dimensional trajectories of two or more measurement stations can be performed to obtain the three-dimensional spatial trajectory. Attached Figure Description

[0018] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A flowchart illustrating the multi-target data association and cataloging method described in the embodiments of the present invention; Figure 2 This is a schematic diagram illustrating trajectory smoothness verification as described in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0020] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0021] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0022] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0023] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] like Figure 1 As shown, this invention provides a multi-target cataloging and tracking method based on data association, specifically including the following steps: S1: Obtain the target trajectory set. The optical measurement device tracks multiple targets in real time with the main target as the field of view center and obtains the k-th frame image. k starts from 3. Based on the deep learning target detection algorithm, target detection is performed on the k-th frame image to obtain the position information of each target. S2: Based on the trajectory of each target in the (k-1)th frame image, predict the trajectory points of each target in the kth frame image. Based on the position information of each target in the previous (k-1)th frames image, determine whether the trajectory of each target in the kth frame image is a definite trajectory. Execute step S3 based on the definite trajectory and execute step S4 based on the uncertain trajectory. S3: Perform concatenated matching on the position information of each target in the k-th frame image and the trajectory point prediction results of each target in the k-th frame image. Execute step S6 based on each target that is successfully matched and the trajectory corresponding to each target. Execute step S4 based on each target that fails to be matched. S4: Perform IOU matching on each target and each trajectory that failed to match, and perform IOU matching on each target and the uncertain trajectory that failed to match. If either of them matches successfully, then execute step S6 based on each target that successfully matches and the trajectory corresponding to each target. Otherwise, update the number of failed matches for each target and trajectory that failed to match, and execute step S5. The position information of each target that failed to match is used as the first trajectory point of each new target and added to the target trajectory set. S5: Determine if there are any targets and trajectories whose number of failed matches exceeds the threshold. If so, delete the targets and trajectories whose number of failed matches exceeds the threshold and update the target trajectory set. Otherwise, proceed to step S6. S6: Construct an evaluation function, use the evaluation function to verify the trajectory smoothness of each target corresponding to the cascaded matching or the target corresponding to the IOU matching, update the position information of each target that has successfully verified the trajectory smoothness to the trajectory data corresponding to each target, and update the Kalman filter based on the updated trajectory data corresponding to each target, and add each target that fails the trajectory smoothness verification as the first trajectory point of each new target to the target trajectory set. S7: Determine whether the target tracking task has ended. If not, output the multi-target data association cataloging result based on the target trajectory set corresponding to the k-th frame image, and replace the k-th frame target image with the (k+1)-th frame target image to execute steps S1-S6 until the target tracking task ends. If yes, output the multi-target data association cataloging result based on the target trajectory set corresponding to the k-th frame image.

[0025] It should be noted that when the optical measurement equipment tracks the main target, there are multiple targets within the field of view. This invention needs to catalog and track all targets. First, a target recognition algorithm framework based on deep learning is used to detect and recognize the targets in the input optical measurement image and obtain the information of all detected target bounding boxes. Next, a Kalman filter is used to predict the trajectory of the tracked target. Then, based on the Hungarian algorithm, cascade matching and IOU matching are performed between the predicted state and the measurement state of the current frame. The trajectory data is then verified a second time based on the matching results to ensure the correctness and reliability of the real-time catalog data. Finally, the parameters of the Kalman filter are updated based on the verification results.

[0026] In some embodiments, the method for obtaining the target trajectory set in step S1 is as follows: The optical measurement equipment tracks multiple targets in real time with the main target as the center of the field of view, obtains images arranged in the shooting sequence, performs target detection on the first frame image based on a deep learning target detection algorithm, obtains the position information of each target, obtains the first trajectory point of each target based on the position information of each target, generates the corresponding target number, and constructs an initial target trajectory set. Obtain the second frame image, perform target detection on the second frame image based on the deep learning target detection algorithm, and predict the trajectory points of each target in the second frame image based on the initial target trajectory set. Match the position information of each target in the second frame image with the trajectory point prediction results of each target in the second frame image. Add the position information of each successfully matched target to the trajectory data of the corresponding target as the second trajectory point of each target. Use the position information of each unmatched target as the first trajectory point of each new target. Create a new target number and add it to the current target trajectory set. Obtain the target trajectory set obtained in step S1, and update the number of unmatched targets and trajectories.

[0027] In some embodiments, in step S1, the deep learning object detection algorithm is the YOLO algorithm or the SSD algorithm, and the object detection result is represented in the form of an object detection box.

[0028] It should be noted that, for target images acquired sequentially by the optical measurement equipment, deep learning target recognition algorithm frameworks such as YOLO and SSD are used to detect all targets within the target images and output the target information. and categories, where and Indicates the center coordinates of the bounding box. and These represent the aspect ratio and height of the bounding box, respectively. Target detection bounding boxes for each target For each target, the trajectory of each target is recorded separately.

[0029] Furthermore, in the target trajectory set formed by the centroid coordinates of each target in the first frame target image, each target is first cataloged. When updating the target trajectory set, for successfully matched targets, new trajectory points are superimposed on the ID number of the initial catalog. For new targets, the corresponding ID number is obtained according to the existing ID number sorting rules. If a cataloged target is to be deleted, the target at the corresponding cataloging position is deleted, and the catalog ID number is retained to avoid ID number confusion.

[0030] In some embodiments, in step S2, a Kalman filter is used to predict the trajectory points of each target in the k-th frame image, wherein the trajectory numbered i in the k-th frame image is... The trajectory point prediction results are as follows: ; in, For the trajectory numbered i in the k-th frame of the image. The trajectory point prediction results For the trajectory numbered i in the (k-1)th frame of the image The prediction results For the trajectory numbered i in the (k-1)th frame of the image The target predicted velocity change, This represents the time interval between two adjacent frames.

[0031] It should be noted that the Kalman filter is based on a constant velocity motion and linear observation model. It uses information from the previous frame to predict the position information of each target in the current frame and obtain the target prediction value.

[0032] In some embodiments, in step S2, the specific method for determining whether the trajectory corresponding to each target is a determined trajectory based on the target trajectory set of the previous k-1 frames is as follows: for the trajectory numbered i in the current frame image... Statistical analysis of the trajectory with ID i The location information corresponding to the first k-1 frames of the image is recorded at least 3 times. If the number of records is not less than 3 times, it is determined as a definite trajectory; otherwise, it is determined as an uncertain trajectory.

[0033] In some embodiments, in step S3, a traversal approach is used to obtain the position information of the j-th target in the k-th frame image using the following formula. and the trajectory numbered i in the k-th frame image Perform cascading matching: ; in, As a weighting factor, The Mahalanobis distance metric. This is a cosine distance metric. This is the cascaded matching value.

[0034] Furthermore, the formula for calculating the Mahalanobis distance metric is as follows:

[0035] in, For the k-th frame of the image The detection box corresponding to each target For the trajectory numbered i in the k-th frame image The trajectory point prediction results Let be the covariance matrix between the location information corresponding to trajectory number i in the k-th frame and the average location information corresponding to trajectory number i in the previous k-1 frames.

[0036] Furthermore, the formula for calculating the cosine distance metric is as follows: ; in, For the detection box corresponding to the j-th target in the k-th frame image The descriptive vector of the extracted appearance features. This represents the set of descriptive vectors representing the appearance features of each target successfully associated with trajectory number i. Belongs to set The appearance feature description vector.

[0037] It should be noted that the predicted bounding box of the target image in the current frame is based on the Hungarian algorithm. Detection bounding box of the target image in the current frame Cascaded matching is performed, combining apparent features and motion features to improve the accuracy of data association. Regarding motion features, this invention uses Mahalanobis distance to match the predicted values ​​of the Kalman filter with the actual observed values ​​of the target. This takes into account the uncertainty of state estimation and uses the quantile of the 95% confidence interval calculated using the chi-square distribution as the threshold for sudden entry / exit Mahalanobis distance, i.e., 9.4877, to exclude impossible associations.

[0038] Mahalanobis distance can be a suitable correlation metric when target motion is relatively regular and uncertainty is low. However, in real-world image spaces, Kalman filtering provides a rather coarse estimate of target motion characteristics, offering only a rough estimate of target position. Therefore, a cosine distance metric is introduced on top of Mahalanobis distance to correlate targets, extracting the appearance features of the tracked target and measuring the differences in dimensions between different tracked targets. The correlation is measured by calculating the cosine distance between the appearance feature vector detected in the current frame and the appearance feature vectors of previously tracked targets. This represents a set of descriptive vectors representing the appearance features of each target successfully associated with trajectory number i. Typically, each tracked target corresponds to one set. The set retains the feature vector descriptions from the most recently associated frames for each target. When the next frame successfully associates with the corresponding target, the feature vectors in this set are updated, retaining the most recent and removing outdated appearance feature vectors. This is the result of the cosine distance metric.

[0039] When associating targets, the two metrics mentioned above can complement each other; therefore, a linear weighted average of Mahalanobis distance and cosine distance is used as the final metric. ; If the cascaded matching value When the correlation is within the set threshold, it can be considered a correct association, and it can be passed... The value adjusts the degree of influence of the two measurement methods on the final measurement result.

[0040] In some embodiments, the method used in step S3 to match uncertain targets and each target that failed to match is as follows: The first cost matrix (1-IOU) is formed by negatively taking the IOU value between the target detection and prediction results of uncertain targets as the matching cost. This first cost matrix is ​​then input into a Hungarian algorithm to obtain the optimal matching relationship, thus determining whether the current uncertain target has been successfully matched. Similarly, the second cost matrix (1-IOU) is formed by negatively taking the IOU value between the target detection and prediction results of each failed match as the matching cost. This second cost matrix is ​​then input into a Hungarian algorithm to obtain the optimal matching relationship, again determining whether the current failed match has been successfully matched. Furthermore, the method for matching the target detection results and target prediction results of the second frame target image is the same as the method used for matching uncertain targets and each failed match target separately.

[0041] The Hungarian algorithm optimizes the matching scheme step by step by subtracting the minimum value from rows / columns and finding independent zeros, thereby adjusting the matrix and taking the position of the independent zero as the optimal matching relationship between the corresponding detection and the trajectory.

[0042] It should be noted that the purpose of this matching is to select the correctly associated trajectory for trajectory smoothness verification; otherwise, IOU matching is performed. The IOU cost matrix is ​​calculated (preferably, cost matrix = 1 - IOU), and then the Hungarian algorithm is used for matching. If the trajectory matches successfully, the trajectory smoothness verification part is performed. If a detection box mismatch occurs during matching, the failed target is treated as a new target. Sometimes the tracked target may disappear, failing to match multiple times in subsequent tracking. When the number of matches exceeds a set threshold `max_age` (the threshold is set according to requirements), the corresponding target will be deleted.

[0043] In some embodiments, in step S6, the evaluation function The calculation formula is: ; in, and As a weighting factor, For the trajectory numbered i in the (k-2)th frame image The center coordinates, For the trajectory numbered i in the (k-1)th frame of the image The center coordinates, For the k-th frame of the image The center coordinates of each target; If the evaluation function value is less than the set threshold, the smoothness verification of the trajectory corresponding to the current target is successful; otherwise, the smoothness verification of the trajectory corresponding to the current target fails.

[0044] It should be noted that the specific content of the trajectory smoothness verification section is as follows: To ensure the accuracy and reliability of real-time cataloging trajectory data, secondary verification of the real-time cataloging trajectory data is required, such as... Figure 2 As shown, the basis for verifying the two-dimensional trajectory projection data is that image points P1, P2, and P3 will not undergo abrupt changes in the X and Y directions in a short period of time, and the trajectory formed by targets belonging to the same trajectory has good smoothness, and their travel direction and distance will not change abruptly between consecutive frames.

[0045] Preferably, the selection of the trajectory smoothness evaluation function is as follows: For conventional target types tracked by optical measurement equipment, generally speaking, their speed and direction of movement do not change suddenly or drastically. Therefore, it can be said that the trajectory is smooth, and the trajectories of multiple targets are path-dependent. Especially when the shooting time interval is short or the number of trajectory sampling points is very large, its smoothness is even more satisfied.

[0046] Multi-target trajectory recognition, or the problem of corresponding target points at different times, is based on trajectory smoothness. This means that adjacent sampling points on the same trajectory have similar movement speeds and directions. Based on the smoothness of the trajectory, an evaluation function for trajectory recognition is selected, with the principle being that the closer the movement speeds and directions are, the smaller the value.

[0047] The first term of the evaluation function corresponds to the change in motion direction, and its subtrahend is the direction cosine of the angle between the lines connecting the three adjacent target points. Clearly, the smaller the change in the target's motion direction, the smaller the value of the first term; if the target's motion direction is the same, its value is zero. The second term of the evaluation function corresponds to the change in motion velocity, and its subtrahend is the change in distance between the three adjacent target points. Clearly, the smaller the change in the target's motion velocity, the smaller the value of the second term; if the target's motion velocity remains constant, meaning the position of the middle target point is the geometric mean of the distances between the two preceding and following target points, its value is zero. This is how the evaluation function is chosen. Clearly, the selection principle of the evaluation function is satisfied: the velocity and direction of motion of the target points on the same trajectory are most similar and change the least. The value is the smallest.

[0048] If the evaluation function If the value is less than the set threshold (set according to requirements), it indicates that the smoothness verification of the target trajectory corresponding to the target image in the current frame is successful, and Kalman filtering can be used to update the parameters. Otherwise, it indicates that the smoothness verification of the target trajectory corresponding to the target image is unsuccessful, and the unsuccessful target needs to be taken as a new target and the new target trajectory needs to be recorded.

[0049] In some embodiments, step S6 further includes: updating the position information of the target whose trajectory smoothness verification is successful in the current frame image to the trajectory data corresponding to the target, updating the Kalman filter based on the updated trajectory data corresponding to the target, and using the updated Kalman filter to predict the trajectory points of each target in the next frame image.

[0050] It should be noted that the parameters of the Kalman filter are updated after both data association and trajectory smoothness verification are successful. After updating the Kalman filter parameters, trajectory prediction of the target in the next frame of the target image is then performed.

[0051] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0052] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-target cataloging and tracking method based on data association, characterized in that: Specifically, the steps include the following: S1: Obtain the target trajectory set. The optical measurement device tracks multiple targets in real time with the main target as the field of view center and obtains the k-th frame image. k starts from 3. Based on the deep learning target detection algorithm, target detection is performed on the k-th frame image to obtain the position information of each target. S2: Based on the trajectory of each target in the (k-1)th frame image, predict the trajectory points of each target in the kth frame image. Based on the position information of each target in the previous (k-1)th frames image, determine whether the trajectory of each target in the kth frame image is a definite trajectory. Execute step S3 based on the definite trajectory and execute step S4 based on the uncertain trajectory. S3: Perform concatenated matching on the position information of each target in the k-th frame image and the trajectory point prediction results of each target in the k-th frame image. Execute step S6 based on each target that is successfully matched and the trajectory corresponding to each target. Execute step S4 based on each target that fails to be matched. S4: Perform IOU matching on each target and each trajectory that failed to match, and perform IOU matching on each target and the uncertain trajectory that failed to match. If either of them matches successfully, then execute step S6 based on each target that successfully matches and the trajectory corresponding to each target. Otherwise, update the number of failed matches for each target and trajectory that failed to match, and execute step S5. The position information of each target that failed to match is used as the first trajectory point of each new target and added to the target trajectory set. S5: Determine if there are any targets and trajectories whose number of failed matches exceeds the threshold. If so, delete the targets and trajectories whose number of failed matches exceeds the threshold and update the target trajectory set. Otherwise, proceed to step S6. S6: Construct an evaluation function, use the evaluation function to verify the trajectory smoothness of each target corresponding to the cascaded matching or the target corresponding to the IOU matching, update the position information of each target that has successfully verified the trajectory smoothness to the trajectory data corresponding to each target, and update the Kalman filter based on the updated trajectory data corresponding to each target, and add each target that fails the trajectory smoothness verification as the first trajectory point of each new target to the target trajectory set. S7: Determine whether the target tracking task has ended. If not, output the multi-target data association cataloging result based on the target trajectory set corresponding to the k-th frame image, and replace the k-th frame target image with the (k+1)-th frame target image to execute steps S1-S6 until the target tracking task ends. If yes, output the multi-target data association cataloging result based on the target trajectory set corresponding to the k-th frame image.

2. The multi-target cataloging and tracking method based on data association according to claim 1, characterized in that: In step S1, the method for obtaining the target trajectory set is as follows: The optical measurement equipment tracks multiple targets in real time with the main target as the center of the field of view, obtains images arranged in the shooting sequence, performs target detection on the first frame image based on a deep learning target detection algorithm, obtains the position information of each target, obtains the first trajectory point of each target based on the position information of each target, generates the corresponding target number, and constructs an initial target trajectory set. Obtain the second frame image, perform target detection on the second frame image based on the deep learning target detection algorithm, and predict the trajectory points of each target in the second frame image based on the initial target trajectory set. Match the position information of each target in the second frame image with the trajectory point prediction results of each target in the second frame image. Add the position information of each successfully matched target to the trajectory data of the corresponding target as the second trajectory point of each target. Use the position information of each unmatched target as the first trajectory point of each new target. Create a new target number and add it to the current target trajectory set. Obtain the target trajectory set obtained in step S1, and update the number of unmatched targets and trajectories.

3. The multi-target cataloging and tracking method based on data association according to claim 1, characterized in that: In step S1, the deep learning object detection algorithm is either the YOLO algorithm or the SSD algorithm.

4. The multi-target cataloging and tracking method based on data association according to claim 1, characterized in that: In step S2, a Kalman filter is used to predict the trajectory points of each target in the k-th frame image, where the trajectory numbered i in the k-th frame image is... The trajectory point prediction results are as follows: ； in, For the trajectory numbered i in the k-th frame of the image. The trajectory point prediction results For the trajectory numbered i in the (k-1)th frame of the image The prediction results For the trajectory numbered i in the (k-1)th frame of the image The target predicted velocity change, This represents the time interval between two adjacent frames.

5. The multi-target cataloging and tracking method based on data association according to claim 1, characterized in that: In step S2, the specific method for determining whether the trajectory corresponding to each target is a definite trajectory based on the target trajectory set of the previous k-1 frames is as follows: For the trajectory numbered i in the current frame image... Statistical analysis of the trajectory numbered i The location information corresponding to the first k-1 frames of the image is recorded at least 3 times. If the number of records is not less than 3 times, it is determined as a definite trajectory; otherwise, it is determined as an uncertain trajectory.

6. The multi-target cataloging and tracking method based on data association according to claim 2, characterized in that: In step S3, the position information of the j-th target in the k-th frame image is obtained by traversal using the following formula. and the trajectory numbered i in the k-th frame image Perform cascading matching: ； in, As a weighting factor, The Mahalanobis distance metric. This is a cosine distance metric. This is the cascaded matching value.

7. The multi-target cataloging and tracking method based on data association according to claim 6, characterized in that: The formula for calculating the Mahalanobis distance metric is: in, For the k-th frame of the image The detection box corresponding to each target For the trajectory numbered i in the k-th frame image The trajectory point prediction results Let be the covariance matrix between the location information corresponding to trajectory number i in the k-th frame and the average location information corresponding to trajectory number i in the previous k-1 frames.

8. The multi-target cataloging and tracking method based on data association according to claim 6, characterized in that: The formula for calculating the cosine distance metric is: ； in, For the detection box corresponding to the j-th target in the k-th frame image The descriptive vector of the extracted appearance features. This represents the set of descriptive vectors representing the appearance features of each target successfully associated with trajectory number i. Belongs to set The appearance feature description vector.

9. The multi-target cataloging and tracking method based on data association according to claim 1, characterized in that: In step S6, the evaluation function The calculation formula is: ； in, and As a weighting factor, For the trajectory numbered i in the (k-2)th frame image The center coordinates, For the trajectory numbered i in the (k-1)th frame of the image The center coordinates, For the k-th frame of the image The center coordinates of each target; If the evaluation function value is less than the set threshold, the smoothness verification of the trajectory corresponding to the current target is successful; otherwise, the smoothness verification of the trajectory corresponding to the current target fails.

10. The multi-target cataloging and tracking method based on data association according to claim 1, characterized in that: Step S6 further includes: updating the position information of the target whose trajectory smoothness verification was successful in the current frame image to the trajectory data corresponding to the target, updating the Kalman filter based on the updated trajectory data corresponding to the target, and using the updated Kalman filter to predict the trajectory points of each target in the next frame image.

Images