Airport multi-target tracking and matching method and system

Abstract

The present invention discloses a method and system for tracking and matching multiple targets at an airport, and the method comprises: S1, constructing a similar target confusion relationship matrix model and a target detector result database; S2, using a target detector to detect and obtain an object label frame set of the current frame and dividing it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow; S3, matching the objects in the high-score object label frame set with the objects in the to-be-matched object label set Spool and classifying and dividing them after matching; S4, verifying and evaluating the objects through a similar target confusion relationship matrix model and correcting the object labels. The present invention performs multiple object matching and correction processing, improves the efficiency and accuracy of object tracking and matching, significantly improves the stability and accuracy of multi-target tracking, and provides technical support for airport monitoring, track tracing, hazard source identification, and security node detection.

Description

Technical Field

[0001] The present invention relates to the field of airport multi-target tracking, and in particular to an airport multi-target tracking and matching method and system. Background Art

[0002] Different areas of the airport have different monitoring and tracking requirements. In the terminal hall, the monitoring and tracking is mainly for passengers, staff, luggage or other equipment. In the apron area, it mainly involves various types of support vehicles (such as tank trucks, supply trucks, trailers, filling trucks, de-icing trucks, etc.), support personnel, and equipment for implementing support measures. In other areas, there are also other monitoring and tracking (such as perimeter security monitoring). Especially in the apron area, because multiple targets frequently appear in the monitoring frame and need to be tracked, target detectors are currently used. It is difficult for target detectors to maintain consistency in the judgment and matching of front and rear targets (the large size of the support vehicle will make it difficult to identify the target), resulting in the loss of tracking targets; and because the target vehicle or pedestrian will be frequently blocked by the wing or fuselage, it will frequently cause tracking target interruption, label jump (the target detector will mistakenly generate a new target object and create a new label after tracking the target disappears, resulting in the loss of the same target tracking and more noise. Later, manual data playback is required to check and correct the label), missed detection and false detection, etc. The existing target detector cannot solve the above technical problems of airport aircraft support. Target detectors suffer from misidentification and tracking loss, and existing target detectors cannot effectively use historical tracking information to correct and amend the recognition results. This leads to a series of problems and challenges in actual business scenarios such as airport monitoring, track tracing, hazard source identification, and security node detection. Summary of the invention

[0003] The purpose of the present invention is to solve the technical problems pointed out by the background technology, and to provide an airport multi-target tracking and matching method and system, which constructs a target detector result database to store the target detector results of subsequent object recognition so as to obtain all historical objects, and a similar target confusion relationship matrix model is stored as a similar target confusion relationship matrix according to the object matrix, and multiple object matching and correction processing are performed, which greatly reduces the probability of mismatching and improves the efficiency and accuracy of object tracking and matching.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A method for tracking and matching multiple targets at an airport, the method comprising:

[0006] S1. Target detectors are deployed in the airport study area and the detection results of the target detectors are obtained. Based on the detection results of the target detectors, a similar target confusion relationship matrix model and a target detector result database are constructed. The target detector result database includes an object label set Strk in a tracking state, a label set Slost in a lost state, and a current frame object label set. The object label set Strk in a tracking state includes an activated object label set and an inactivated object label set. The current frame object label set includes an activated object label set Tact in the current frame, a re-found object label set Trfd in the current frame, and a lost object label set Tlost in the current frame. The activated object labels in the object label set Strk and the lost label set Slost in the object label set are merged into a to-be-matched object label set Spool, and the inactivated object labels in the object label set Strk are recorded as an unconfirmed object label set Sucf.

[0007] S2, using the target detector to detect and obtain the object label frame set of the current frame and divide it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow;

[0008] S3, matching the objects in the high-scoring object label frame set with the objects in the to-be-matched object label set Spool, and classifying them after matching according to the following method:

[0009] The objects matched in the high-scoring object label frame set and in the activated state are stored in the object label set Tact; the objects matched in the high-scoring object label frame set and in the lost state are stored in the object label set Trfd; the remaining unmatched object labels in the high-scoring object label frame set are stored as the object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as the object label set Srest;

[0010] S4. Object verification and evaluation are performed through a similar target confusion relationship matrix model, and object label correction is performed.

[0011] In order to better implement the present invention, the present invention also includes the following method:

[0012] S5, perform object matching on the objects in the low-scoring object label frame set Dlow and the remaining unmatched objects in the to-be-matched object label set Spool; store the objects in the low-scoring object label frame set that are matched in an activated state into the object label set Tact; store the objects in the low-scoring object label frame set that are matched in a lost state into the object label set Trfd; store the remaining unmatched object labels in the to-be-matched object label set Spool into the object label set Srest;

[0013] S6. Object verification and evaluation are performed through a similar target confusion relationship matrix model, and object label correction is performed.

[0014] Preferably, the present invention also includes the following method:

[0015] S7, performing object matching on the objects in the object tag set Dhrest and the remaining unmatched objects in the unconfirmed object tag set Sucf; storing the matched objects in the object tag set Dhrest into the object tag set Tact;

[0016] S8. Store the unmatched objects in the object tag set Strk in the tracking state into the tag set Slost in the lost state; set new tags for the remaining objects in the object tag set Dhrest and store them in the object tag set Strk in the tracking state; store the object tag set Tact activated in the current frame into the object tag set Strk in the tracking state and perform object tracking matching for the next frame.

[0017] Preferably, the present invention also includes the following method:

[0018] S9. Use the similar target confusion relationship matrix model to identify the same object in time series and connect the trajectory points of the same object in time to form the trajectory data of the object.

[0019] Preferably, the similar target confusion relationship matrix model stores all historical objects in the target detector result database, and all historical objects are identified, extracted and added based on the historical frame data of the target detector; the similar target confusion relationship matrix model constructs a similar target confusion relationship matrix according to the features in the historical frame data of the target detector during identification and extraction, and the similar target confusion relationship matrix is ​​stored in a matrix according to the object. An object includes the object's real feature data and the associated object confusion feature data, the object's real feature data indicates that the object is real and contains features, and the object confusion feature data indicates features that are confused with the real object and need to be excluded as real objects, and the similar target confusion relationship matrix model inputs the target detector's historical frame data for model training.

[0020] Preferably, the similar target confusion relationship matrix model is stored in association with the real object, the confusion object, and the confusion probability, and the confusion probability expression is as follows: , It represents the probability that the object’s true label is i and is mistakenly identified as label j. Indicates the number of times the object's true label is i and is mistakenly identified as label j. Indicates the total number of times the object’s true label is i.

[0021] Preferably, in step S4, the similar target confusion relationship matrix model verifies whether the object matching in step S3 is correct. If it is verified to be the same object and the label is the same, no processing is performed; if it is verified to be the same object but the label is different, the label of the object is corrected by using the label initially set for the object or the statistical object historical label information and the label with the most historical label information; if it is verified to be different objects, the object is withdrawn from the object matching operation in step S3 and stored in the unconfirmed object label set Sucf at the same time;

[0022] In step S6, the similar target confusion relationship matrix model verifies whether the object matching in step S5 is correct. If it is verified to be the same object and the label is the same, no processing is performed; if it is verified to be the same object but the label is different, the label of the object is corrected using the label initially set for the object or the statistical object historical label information and the label with the most historical label information; if it is verified to be a different object, the object is withdrawn from the object matching operation in step S5 and stored in the unconfirmed object label set Sucf at the same time.

[0023] Preferably, in step S3, the object matching uses the detection range box to calculate the distance cost for matching judgment: ,in Represents the detection range box of object m in the object label set Spool to be matched, represents the detection range box of object n in the set of high-score object label boxes, Represents the distance cost between object m in the to-be-matched object label set Spool and object n in the high-score object label box set;

[0024] Set the distance cost threshold, Object m and object n whose distance cost threshold is not less than the distance cost threshold are matched accordingly.

[0025] An airport multi-target tracking and matching system comprises a similar target confusion relationship matrix model, a target detector result database, an object matching classification module, an object verification evaluation module and a target detector deployed in an airport research area, wherein the target detector result database comprises an object label set Strk in a tracking state, a label set Slost in a lost state and a current frame object label set, the object label set Strk in a tracking state comprises an activated object label set and an inactivated object label set, the current frame object label set comprises an activated object label set Tact in the current frame, a re-found object label set Trfd in the current frame and a lost object label set Tlost in the current frame; the object label set The activated object tags in Strk and the tag set Slost in the lost state are merged into the to-be-matched object tag set Spool, and the inactivated object tags in the object tag set Strk are recorded as the unconfirmed object tag set Sucf; a similar target confusion relationship matrix model is constructed based on the target detector detection results of the target detector result database; the object matching classification module collects the object label frame set of the current frame obtained by the target detector detection and divides it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow, and matches the objects in the high-score object label frame set with the objects in the to-be-matched object label set Spool and classifies them after matching according to the following method:

[0026] The objects matched in the high-scoring object label frame set and in the activated state are stored in the object label set Tact; the objects matched in the high-scoring object label frame set and in the lost state are stored in the object label set Trfd; the remaining unmatched object labels in the high-scoring object label frame set are stored as the object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as the object label set Srest;

[0027] The object verification and evaluation module uses a similar target confusion relationship matrix model to perform object verification and evaluation and to correct object labels.

[0028] In order to better realize the airport multi-target tracking and matching system of the present invention, the present invention also includes a trajectory data sorting module, the similar target confusion relationship matrix model stores all historical objects in the target detector result database, and all historical objects are identified, extracted and added based on the historical frame data of the target detector; the similar target confusion relationship matrix model constructs a similar target confusion relationship matrix according to features in the historical frame data of the target detector during identification and extraction, and the similar target confusion relationship matrix is ​​stored in a matrix according to the object, and a certain object includes the object's real feature data and the associated object confusion feature data, the object's real feature data indicates that the object is real and contains features, and the object confusion feature data indicates features that are confused with the real object and need to be excluded as the real object, and the similar target confusion relationship matrix model inputs the target detector historical frame data for model training;

[0029] The object matching classification module is also used to match the objects in the low-scoring object label frame set Dlow with the remaining unmatched objects in the to-be-matched object label set Spool; store the objects in the low-scoring object label frame set that are matched in an activated state into the object label set Tact; store the objects in the low-scoring object label frame set that are matched in a lost state into the object label set Trfd; store the remaining unmatched object labels in the to-be-matched object label set Spool into the object label set Srest;

[0030] The object matching and classification module is also used to match the objects in the object tag set Dhrest with the remaining unmatched objects in the unconfirmed object tag set Sucf; store the matched objects in the object tag set Dhrest into the object tag set Tact;

[0031] The object matching and classification module is also used to store the unmatched objects in the object tag set Strk in the tracking state into the tag set Slost in the lost state; set new tags for the remaining objects in the object tag set Dhrest and store them in the object tag set Strk in the tracking state; store the object tag set Tact activated in the current frame into the object tag set Strk in the tracking state and perform object tracking and matching for the next frame;

[0032] The trajectory data sorting module uses a similar target confusion relationship matrix model to identify the same object in time series and connects the trajectory points of the same object in time to form the trajectory data of the object.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] (1) The present invention constructs a target detector result database to store the target detector results of subsequent object recognition in order to obtain all historical objects. The similar target confusion relationship matrix model is stored in a matrix of similar target confusion relationship matrices according to the objects, and multiple object matching and correction processes are performed, which greatly reduces the probability of mismatching, improves the efficiency and accuracy of object tracking and matching, and significantly improves the stability and accuracy of multi-target tracking in airport monitoring scenarios, providing technical support for airport monitoring, track tracing, hazard source identification, security node detection and other businesses.

[0035] (2) The present invention can continuously perform object matching and verification processing on the object label frame set of the current frame, can unify the labels of the same object, can accurately obtain the object detection frame data of all frames of the same object, and is convenient for accurately obtaining the trajectory data of the same object. When the target object is occluded, the accurate target object trajectory can be obtained or the complete trajectory can be obtained by splicing and matching under the premise of achieving high-precision object target matching.

[0036] (3) The similar target confusion relationship matrix model of the present invention performs object matching verification in each step. The similar target confusion relationship matrix is ​​also derived from historical objects or sample data, which can solve the problem of long-term occlusion causing the tracking object to be interrupted and judged as a new object. The similar target confusion relationship matrix model is closely involved in each step of the present invention, realizing object verification and correction, and can also facilitate subsequent rapid splicing of trajectories. It has the advantages of high speed and high accuracy. The present invention can be expanded to the field of multi-target tracking and matching of existing monitoring systems, and has important significance in the field of monitoring and security. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a schematic diagram of the method principle of Example 1;

[0038] Figure 2 This is a schematic diagram of the method principle of Example 2;

[0039] Figure 3 This is an example of the comparison result of the ByteTrack algorithm object recognition captured at the 32nd frame;

[0040] Figure 4 This is a comparison result diagram of the intercepted portion of the object recognition method of the present invention at the 32nd frame;

[0041] Figure 5 This is an example of the comparison result of the ByteTrack algorithm object recognition captured at frame 264;

[0042] Figure 6 This is a comparison result diagram of a portion captured at the 264th frame of the object recognition method of the present invention;

[0043] Figure 7 It is a principle structure block diagram of the airport multi-target tracking and matching system of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with embodiments: Embodiment 1

[0045] like Figure 1 As shown, a method for tracking and matching multiple targets at an airport includes:

[0046] S1. Target detectors are deployed in the airport study area and the detection results of the target detectors are obtained. Based on the detection results of the target detectors, a similar target confusion relationship matrix model and a target detector result database are constructed. The target detector result database includes an object label set Strk in a tracking state, a label set Slost in a lost state, and a current frame object label set. The object label set Strk in a tracking state includes an activated object label set and an inactivated object label set. The current frame object label set includes an activated object label set Tact in the current frame, a re-found object label set Trfd in the current frame, and a lost object label set Tlost in the current frame. The activated object labels in the object label set Strk and the lost label set Slost in the object label set are merged into a to-be-matched object label set Spool, and the inactivated object labels in the object label set Strk are recorded as an unconfirmed object label set Sucf.

[0047] S2, using the target detector to detect and obtain the object label frame set of the current frame and divide it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow;

[0048] S3, matching the objects in the high-scoring object label frame set with the objects in the to-be-matched object label set Spool, and classifying them after matching according to the following method:

[0049] Store the objects in the high-score object label frame set that are matched and in the activated state into the object label set Tact;

[0050] The objects in the high-score object label frame set that match the lost state are stored in the object label set Trfd;

[0051] The remaining unmatched object labels in the high-score object label frame set are stored as the object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as the object label set Srest;

[0052] S4. Verify and evaluate the object and correct the object label through the similar target confusion relationship matrix model. The similar target confusion relationship matrix model verifies whether the object matching in step S3 is correct (the similar target confusion relationship matrix model verifies whether the objects are the same target and whether the object matching is correct through the similar target confusion relationship matrix). If it is verified to be the same object and the label is the same, no processing is done; if it is verified to be the same object but the label is different, the label of the object is corrected with the label set at the beginning of the object or the historical label information of the object is counted and the label with the most historical label information is used; if it is verified to be different objects, the object is withdrawn from the object matching operation in step S3 and stored in the unconfirmed object label set Sucf at the same time. Embodiment 2

[0053] like Figure 2 As shown, a method for tracking and matching multiple targets at an airport includes:

[0054] S1. Target detectors are deployed in the airport study area and the detection results of the target detectors are obtained. Based on the detection results of the target detectors, a similar target confusion relationship matrix model and a target detector result database are constructed. The target detector detection results mainly rely on the method of the present invention to detect objects in historical frame data and construct corresponding sample object data. The similar target confusion relationship matrix model stores all historical objects in the target detector result database. All historical objects are identified, extracted and added based on the historical frame data of the target detector (of course, some corresponding confusion objects can also be created as samples based on historical real objects. Historical real objects have different forms at different angles and positions); Similar target confusion relationship matrix model Based on the historical frame data of the target detector, a similar target confusion relationship matrix is ​​constructed according to the features during identification and extraction. The similar target confusion relationship matrix is ​​stored in a matrix according to the object. An object includes the real feature data of the object and the associated object confusion feature data. The real feature data of the object indicates that the object is real and contains features. The object confusion feature data indicates features that are confused with the real object and need to be excluded as real objects. Often a real object is constructed in groups. For example, a real object group has multiple real objects of the same object (at different angles or different positions). A real object group corresponds to multiple confused objects and performs model training to identify their distinguishing features, thereby forming a similar target confusion relationship matrix. The similar target confusion relationship matrix model inputs the historical frame data of the target detector for model training.

[0055] In some embodiments, the similar target confusion relationship matrix model is stored in association with the real object, the confusion object, and the confusion probability. The confusion probability expression is as follows: , It represents the probability that the object’s true label is i and is mistakenly identified as label j. Indicates the number of times the object's true label is i and is mistakenly identified as label j. Indicates the total number of times the object's true label is i; multiple confused objects corresponding to the real object group of the similar target confusion relationship matrix are also calculated and counted according to the above confusion probability. The object's true label is i, which means that the real object in the real object group is mistakenly identified as the confused object corresponding to label j. For objects with high confusion probability, sample data is constructed to facilitate accurate model training.

[0056] The target detector result database is grouped according to the following method: the target detector result database includes the object label set Strk in the tracking state, the label set Slost in the lost state and the current frame object label set. The object label set Strk in the tracking state includes the activated and inactivated object label sets. The current frame object label set includes the activated object label set Tact of the current frame, the re-found object label set Trfd of the current frame and the lost object label set Tlost of the current frame; the activated object labels in the object label set Strk and the label set Slost in the lost state are merged into the object label set Spool to be matched, and the inactivated object labels in the object label set Strk are recorded as the unconfirmed object label set Sucf.

[0057] S2. Use the target detector to detect and obtain the object label frame set of the current frame and divide it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow. After the target detector of the present invention obtains the object label frame set of the current frame, each object label frame frames the object and records the label, confidence and object range frame of the object, sets the high-score threshold high_thresh and the low-score threshold low_thresh based on the confidence, adds the object detection frame with a confidence greater than the high-score threshold high_thresh to the high-score object label frame set Dhigh, adds the object detection frame with a confidence between the high-score threshold high_thresh and the low-score threshold low_thresh to the low-score object label frame set Dlow, and excludes the object detection frame below the low-score threshold low_thresh (the object detection frame with too low confidence is likely to be misdetected, and exclusion processing is beneficial to improving processing efficiency).

[0058] S3, matching the objects in the high-scoring object label frame set with the objects in the to-be-matched object label set Spool, and classifying them after matching according to the following method:

[0059] The objects matched in the high-scoring object label frame set and in the activated state are stored in the object label set Tact; the objects matched in the high-scoring object label frame set and in the lost state are stored in the object label set Trfd; the remaining unmatched object labels in the high-scoring object label frame set are stored as the object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as the object label set Srest.

[0060] In some embodiments, object matching uses a detection range box to calculate the distance cost for matching judgment: ,in Represents the detection range box of object m in the object label set Spool to be matched, represents the detection range box of object n in the set of high-score object label boxes, Represents the distance cost between object m in the object label set Spool to be matched and object n in the high-score object label box set; sets the distance cost threshold, Object m is not less than the distance cost threshold and is matched with object n. The object matching of the present invention uses a Kalman filter. If the object in the high-scoring object label frame set is the same object as the object in the to-be-matched object label set Spool, since the object in the high-scoring object label frame set is the current frame, that is, the new position, the objects in the to-be-matched object label set Spool are mostly the positions of the objects in the previous frame. The present invention uses the Hungarian algorithm to perform one-to-one association matching between all objects in the to-be-matched object label set Spool and all objects in the high-scoring object label frame set (ensuring that each object and detection frame are matched once).

[0061] S4. Verify and evaluate the object and correct the object label through the similar target confusion relationship matrix model. In some embodiments, the similar target confusion relationship matrix model verifies whether the object matching in step S3 is correct (mainly performs feature recognition of mutually matching objects, that is, determines whether the object matching is in the same real object group and does not fall into its corresponding confused object). If it is verified to be the same object and the label is the same, no processing is performed; if it is verified to be the same object but the label is different, the label of the object is corrected with the label set at the beginning of the object or the historical label information of the object is counted and the label with the most historical label information is used (after correction, the object is kept consistent to prevent the target object from jumping); if it is verified to be a different object, the object is withdrawn from the object matching operation in step S3 and stored in the unconfirmed object label set Sucf at the same time.

[0062] In some embodiments, since step S3 is a matching operation for objects in the high-scoring object label frame set, the object matching accuracy is relatively high, and a method can be used to perform faster verification to improve verification efficiency:

[0063] The similar target confusion relationship matrix model is used to quickly screen the confusion probability in the similar target confusion relationship matrix. , where chaos_thresh represents the confusion probability threshold of the confusion relationship matrix, It represents the probability of confusion between the object qa in the high-score object label box set and the object pa in the to-be-matched object label set Spool.

[0064] S5. Match the objects in the low-scoring object label frame set Dlow with the remaining unmatched objects in the to-be-matched object label set Spool; store the objects in the low-scoring object label frame set that are matched in an activated state into the object label set Tact; store the objects in the low-scoring object label frame set that are matched in a lost state into the object label set Trfd; store the remaining unmatched object labels in the to-be-matched object label set Spool into the object label set Srest.

[0065] In some embodiments, object matching uses a detection range box to calculate the distance cost for matching judgment: ,in Represents the detection range box of object f in the object label set Spool to be matched, Represents the detection range box of object g in the low-score object label box set Dlow, Represents the distance cost between object f in the object label set Spool to be matched and object g in the low-score object label box set; sets the distance cost threshold, Object f that is not less than the distance cost threshold is matched with object g.

[0066] S6. Verify and evaluate the object and correct the object label through the similar target confusion relationship matrix model. In some embodiments, the similar target confusion relationship matrix model verifies whether the object matching in step S5 is correct. If it is verified to be the same object and the label is the same, no processing is performed; if it is verified to be the same object but the label is different, the label of the object is corrected with the label initially set for the object or the historical label information of the object is counted and the label with the most historical label information; if it is verified to be a different object, the object is withdrawn from the object matching operation in step S5 and stored in the unconfirmed object label set Sucf at the same time.

[0067] S7. Perform object matching on the objects in the object tag set Dhrest and the remaining unmatched objects in the unconfirmed object tag set Sucf; store the matched objects in the object tag set Dhrest into the object tag set Tact.

[0068] In some embodiments, object matching uses a detection range box to calculate the distance cost for matching judgment: ,in Represents the detection range box of object b in the object label set Dhrest, represents the detection range box of object d in the unconfirmed object label set Sucf, Indicates the distance cost between object d in the unconfirmed object label set Sucf and object b in the object label set Dhrest; sets the distance cost threshold, Object b and object d whose distance cost threshold is not less than each other are matched accordingly.

[0069] S8. Store the unmatched objects in the object tag set Strk in the tracking state into the tag set Slost in the lost state; set new tags for the remaining objects in the object tag set Dhrest and store them in the object tag set Strk in the tracking state; store the object tag set Tact activated in the current frame into the object tag set Strk in the tracking state and perform object tracking matching for the next frame.

[0070] S9. Use the similar target confusion relationship matrix model to identify the same object in time series and connect the trajectory points of the same object in time to form the trajectory data of the object. Taking the occlusion of a target object as an example, the traditional method will generate many tracking labels. If the trajectory of the object is to be formed, multiple discontinuous and different objects will be formed (due to different labels, the traditional method will think that there are multiple target objects, but in fact they are the same target object), which will form multiple trajectories. The present invention constructs a similar target confusion relationship matrix model and a target detector result database to continuously perform object matching and verification processing on the object label frame set of the current frame, which can unify the label of the same object, and can accurately obtain the object detection frame data of all frames of the same object, so as to accurately obtain the trajectory data of the same object. For the occlusion of the target object, the same object has an old trajectory S i and a new trajectory S j , the two are judged by Euclidean distance in the bipartite graph and the splicing matching result of the old trajectory and the new trajectory is obtained by the Hungarian algorithm. The present invention can effectively solve the data clutter situation such as the target object being blocked for a long time and the subsequent tracking label jumping.

[0071] This embodiment uses the ByteTrack algorithm commonly used by existing target detectors to perform a test comparison on target object recognition on a general road. Figure 3 , the figure shows that at frame 32, the target detector made a recognition error and the bench ( Figure 4 The blue box with the target label bench) is misidentified as a suitcase ( Figure 3 The green box of the target label is suitcase. Figure 3 , Figure 4Only the label information of the bench as the target is displayed, and the label information of other people or objects is not displayed), and the ByteTrack algorithm of the target detector does not process this; the present invention performs object matching, object and label correction, and multiple verification and correction based on the similar target confusion relationship matrix model, so that no recognition error will occur in the same 32nd frame, and accurate recognition is always performed. Figure 5 ,Test comparison of tracking results after long-term occlusion,After a long period of occlusion, at frame 264, the bench target reappears, but the ByteTrack algorithm of the existing target detector assigns it a new tracking label ( Figure 5 The blue box with the tracking tag ID 33). The bench target of the method of the present invention can keep the original tracking tag (see Figure 6 The yellow box with the tracking tag ID being 15).

[0072] The present invention is compared with two relatively advanced target tracking methods, SORT and ByteTrack, and four indicators, MOTA, IDF1, HOTA and IDs, are used to evaluate the method. Among them, MOTA is a general indicator for evaluating the accuracy of multi-target tracking systems, IDF1 is an indicator for evaluating the identity preservation ability in target tracking, HOTA is an indicator for more comprehensive evaluation of tracking performance, and IDs is the total number of tracking tags. Table 1 shows the comparison results of the present invention and the two tracking methods, SORT and ByteTrack, in the standard data set MOT20. As can be seen from Table 1, the present invention is particularly outstanding in the measurement indicators of tracking tag switching, which is far ahead of the other two algorithms.

[0073] Table 1: Comparison of the present invention with SORT and ByteTrack tracking methods

[0074]

[0075] like Figure 7As shown, an airport multi-target tracking and matching system includes a similar target confusion relationship matrix model, a target detector result database, an object matching classification module, an object verification evaluation module and a target detector deployed in the airport research area, the target detector result database includes an object label set Strk in a tracking state, a label set Slost in a lost state and a current frame object label set, the object label set Strk in the tracking state includes an activated object label set and an inactivated object label set, the current frame object label set includes an activated object label set Tact in the current frame, a re-found object label set Trfd in the current frame and a lost object label set Tlost in the current frame; the object label set The activated object tags in Strk and the tag set Slost in the lost state are merged into the to-be-matched object tag set Spool, and the inactivated object tags in the object tag set Strk are recorded as the unconfirmed object tag set Sucf; a similar target confusion relationship matrix model is constructed based on the target detector detection results of the target detector result database; the object matching classification module collects the object label frame set of the current frame obtained by the target detector detection and divides it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow, and matches the objects in the high-score object label frame set with the objects in the to-be-matched object label set Spool and classifies them after matching according to the following method:

[0076] The objects matched in the high-scoring object label frame set and in the activated state are stored in the object label set Tact; the objects matched in the high-scoring object label frame set and in the lost state are stored in the object label set Trfd; the remaining unmatched object labels in the high-scoring object label frame set are stored as the object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as the object label set Srest;

[0077] The object verification and evaluation module uses a similar target confusion relationship matrix model to perform object verification and evaluation and to correct object labels.

[0078] The airport multi-target tracking and matching system of the present invention also includes a trajectory data sorting module, a similar target confusion relationship matrix model stores all historical objects in a target detector result database, and all historical objects are identified, extracted and added based on the historical frame data of the target detector; the similar target confusion relationship matrix model constructs a similar target confusion relationship matrix according to features in the historical frame data of the target detector during identification and extraction, and the similar target confusion relationship matrix is ​​stored in a matrix according to objects, and a certain object includes object real feature data and associated object confusion feature data, the object real feature data indicates that the object is real and contains features, and the object confusion feature data indicates features that are confused with the real object and need to be excluded as the real object, and the similar target confusion relationship matrix model inputs the historical frame data of the target detector for model training;

[0079] The object matching classification module is also used to match the objects in the low-scoring object label frame set Dlow with the remaining unmatched objects in the to-be-matched object label set Spool; store the objects in the low-scoring object label frame set that are matched in an activated state into the object label set Tact; store the objects in the low-scoring object label frame set that are matched in a lost state into the object label set Trfd; store the remaining unmatched object labels in the to-be-matched object label set Spool into the object label set Srest;

[0080] The object matching and classification module is also used to match the objects in the object tag set Dhrest with the remaining unmatched objects in the unconfirmed object tag set Sucf; store the matched objects in the object tag set Dhrest into the object tag set Tact;

[0081] The object matching classification module is also used to store the unmatched objects in the object tag set Strk in the tracking state into the tag set Slost in the lost state; set new tags for the remaining objects in the object tag set Dhrest and store them in the object tag set Strk in the tracking state; store the object tag set Tact activated in the current frame into the object tag set Strk in the tracking state and perform object tracking matching for the next frame;

[0082] The trajectory data sorting module uses the similar target confusion relationship matrix model to identify the same object in time series and connects the trajectory points of the same object in time to form the trajectory data of the object.

[0083] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for tracking and matching multiple targets at an airport, characterized in that: The methods include: S1. Target detectors are deployed in the airport study area and the detection results of target detectors are obtained. A similar target confusion relationship matrix model and a target detector result database are constructed based on the detection results of the target detector. The similar target confusion relationship matrix model is based on the historical frame data of the target detector. The similar target confusion relationship matrix is ​​constructed according to the features during identification and extraction. The similar target confusion relationship matrix is ​​stored in a matrix according to the object. An object includes the real feature data of the object and the associated confused feature data of the object. The real feature data of the object indicates that the object is real and contains features. The confused feature data of the object indicates features that are confused with the real object and need to be excluded as the real object. The similar target confusion relationship matrix model inputs the historical frame data of the target detector for model training. The detector result database includes an object tag set Strk in a tracking state, a tag set Slost in a lost state, and a current frame object tag set. The object tag set Strk in a tracking state includes an activated and an inactivated object tag set. The current frame object tag set includes an activated object tag set Tact in the current frame, a re-found object tag set Trfd in the current frame, and a lost object tag set Tlost in the current frame. The activated object tags in the object tag set Strk and the tag set Slost in a lost state are merged into a to-be-matched object tag set Spool, and the inactivated object tags in the object tag set Strk are recorded as an unconfirmed object tag set Sucf. S2, using the target detector to detect and obtain the object label frame set of the current frame and divide it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow; S3, matching the objects in the high-scoring object label frame set with the objects in the to-be-matched object label set Spool, and classifying them after matching according to the following method: Store the objects in the high-score object label frame set that are matched and in the activated state into the object label set Tact; The objects in the high-score object label frame set that match the lost state are stored in the object label set Trfd; The remaining unmatched object labels in the high-score object label frame set are stored as an object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as an object label set Srest; S4, object verification and evaluation and object label correction through similar target confusion relationship matrix model; S5, performing object matching on objects in the low-score object label box set Dlow; S6, object verification and evaluation and object label correction through similar target confusion relationship matrix model; S7, performing object matching and storage processing on the objects in the object tag set Dhrest and the remaining unmatched objects in the unconfirmed object tag set Sucf; S8. Use the similar target confusion relationship matrix model to identify the same object in time series and connect the trajectory points of the same object in time to form the trajectory data of the object.

2. The airport multi-target tracking and matching method according to claim 1, characterized in that: S5 also includes the following methods: Match the objects in the low-score object label frame set Dlow with the remaining unmatched objects in the to-be-matched object label set Spool; store the matched objects in the low-score object label frame set in the active state into the object label set Tact; The objects in the low-score object label frame set that match the lost state are stored in the object label set Trfd; The remaining unmatched object tags in the to-be-matched object tag set Spool are stored in the object tag set Srest.

3. The airport multi-target tracking and matching method according to claim 2 is characterized in that: S7 also includes the following methods: Store the matching objects in the object tag set Dhrest into the object tag set Tact; The unmatched objects in the object tag set Strk in the tracking state are stored in the tag set Slost in the lost state; new tags are set for the remaining objects in the object tag set Dhrest and stored in the object tag set Strk in the tracking state; the object tag set Tact activated in the current frame is stored in the object tag set Strk in the tracking state and the next frame object tracking matching is performed.

4. The airport multi-target tracking and matching method according to claim 1, characterized in that: The similar target confusion relationship matrix model stores all historical objects in the target detector result database, and all historical objects are identified, extracted and added based on the historical frame data of the target detector.

5. The airport multi-target tracking and matching method according to claim 1, characterized in that: The similar target confusion relationship matrix model is stored in association with the real object, the confusion object, and the confusion probability. The confusion probability expression is as follows: , It represents the probability that the object’s true label is i and is mistakenly identified as label j. Indicates the number of times the object's true label is i and is mistakenly identified as label j. Indicates the total number of times the object’s true label is i.

6. The airport multi-target tracking and matching method according to claim 2 is characterized in that: In step S4, the similar target confusion relationship matrix model verifies whether the object matching in step S3 is correct. If it is verified to be the same object and the label is the same, no processing is performed; if it is verified to be the same object but the label is different, the label of the object is corrected by using the label initially set for the object or the statistical object historical label information and the label with the most historical label information; if it is verified to be different objects, the object matching operation in step S3 is withdrawn and stored in the unconfirmed object label set Sucf at the same time; In step S6, the similar target confusion relationship matrix model verifies whether the object matching in step S5 is correct. If it is verified to be the same object and the label is the same, no processing is performed; if it is verified to be the same object but the label is different, the label of the object is corrected using the label initially set for the object or the statistical object historical label information and the label with the most historical label information; if it is verified to be a different object, the object is withdrawn from the object matching operation in step S5 and stored in the unconfirmed object label set Sucf at the same time.

7. An airport multi-target tracking and matching system implementing the airport multi-target tracking and matching method according to claim 1, characterized in that: It includes a similar target confusion relationship matrix model, a target detector result database, an object matching classification module, an object verification and evaluation module, and a target detector deployed in the airport research area. The target detector result database includes an object label set Strk in a tracking state, a label set Slost in a lost state, and a current frame object label set. The object label set Strk in a tracking state includes activated and inactivated object label sets. The current frame object label set includes an activated object label set Tact in the current frame, a re-found object label set Trfd in the current frame, and a lost object label set Tlost in the current frame. The activated object labels in the object label set Strk and the lost label set Slost in the object label set Strk are merged into a to-be-matched object label set Spool, and the inactivated object labels in the object label set Strk are recorded as an unconfirmed object label set Sucf. The similar target confusion relationship matrix model is constructed based on the target detector detection results of the target detector result database; the object matching classification module collects the object label frame set of the current frame obtained by the target detector detection and divides it into a high-score object label frame set Dhigh and a low-score object label frame set Dlow, and matches the objects in the high-score object label frame set with the objects in the to-be-matched object label set Spool and classifies them after matching according to the following method: The objects in the high-score object label frame set that match the active state are stored in the object label set Tact; the objects in the high-score object label frame set that match the lost state are stored in the object label set Trfd; The remaining unmatched object labels in the high-score object label frame set are stored as an object label set Dhrest, and the remaining unmatched object labels in the to-be-matched object label set Spool are stored as an object label set Srest; The object verification and evaluation module uses a similar target confusion relationship matrix model to perform object verification and evaluation and to correct object labels.

8. The airport multi-target tracking and matching system according to claim 7, characterized in that: It also includes a trajectory data sorting module, wherein the similar target confusion relationship matrix model stores all historical objects in the target detector result database, and all historical objects are identified, extracted and added based on the historical frame data of the target detector; The similar target confusion relationship matrix model is based on the target detector's historical frame data. The similar target confusion relationship matrix is ​​constructed according to the features during identification and extraction. The similar target confusion relationship matrix is ​​stored in a matrix according to the object. An object includes the object's real feature data and the associated object confusion feature data. The object's real feature data indicates that the object is real and contains features. The object confusion feature data indicates features that are confused with the real object and need to be excluded as the real object. The similar target confusion relationship matrix model inputs the target detector's historical frame data for model training. The object matching and classification module is also used to match the objects in the low-scoring object label frame set Dlow with the remaining unmatched objects in the to-be-matched object label set Spool; and store the objects in the low-scoring object label frame set that are matched and in an activated state into the object label set Tact; The objects in the low-score object label frame set that are in a lost state are stored in the object label set Trfd; the remaining unmatched object labels in the to-be-matched object label set Spool are stored in the object label set Srest; The object matching and classification module is also used to match the objects in the object tag set Dhrest with the remaining unmatched objects in the unconfirmed object tag set Sucf; store the matched objects in the object tag set Dhrest into the object tag set Tact; The object matching and classification module is also used to store the unmatched objects in the object tag set Strk in the tracking state into the tag set Slost in the lost state; set new tags for the remaining objects in the object tag set Dhrest and store them in the object tag set Strk in the tracking state; store the object tag set Tact activated in the current frame into the object tag set Strk in the tracking state and perform object tracking and matching for the next frame; The trajectory data sorting module uses a similar target confusion relationship matrix model to identify the same object according to the time series and connects the trajectory points of the same object according to time association to form the trajectory data of the object.

Images