A traffic scene-oriented multi-target tracking method, device, equipment and storage medium

Abstract

The application discloses a traffic scene-oriented multi-target tracking method and device, equipment and storage medium, including: based on the related node of the candidate target of the current frame in the video sequence, a high-order hypergraph is constructed, so as to obtain the hyperedge intermediate representation vector of each hyperedge in the high-order hypergraph, according to the disturbance perception aggregation technology, the high-order feature representation of the target node is obtained, finally the tracking trajectory of the target is obtained through the linear distribution algorithm. The application realizes efficient fusion and comprehensive judgment of multi-source information at the hyperedge level by uniformly integrating candidate targets, scene local regions and disturbance factors into the tracking model, so as to more fully utilize the context information such as road structure, lane distribution and flow direction characteristics, effectively alleviate the false detection and misassociation in the scenes such as dense targets, complex interaction and long duration of occlusion, and significantly improve the accuracy and stability of the trajectory level result.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent transportation technology, and in particular relates to a multi-target tracking method, device, equipment and storage medium for traffic scenarios. Background Technology

[0002] Multi-target tracking technology detects and correlates targets such as vehicles and pedestrians across frames in video sequences to obtain continuous trajectories of these targets over time. It is a key foundational technology for applications such as road monitoring, intelligent transportation, and autonomous driving. In urban intersection electronic police systems, traffic flow, queue length, and dwell time need to be statistically analyzed based on target trajectories. In autonomous driving scenarios, continuous tracking of vehicles and pedestrians ahead and in the surrounding area is required to support path planning and collision warnings. In intelligent traffic management systems, trajectory analysis is also needed to detect abnormal behaviors such as driving against traffic, running red lights, and illegal parking. These applications place high demands on the real-time performance, stability, and robustness of multi-target tracking.

[0003] Traditional multi-object tracking methods often employ a "detection-association" pipeline structure, where a target detection network first generates candidate targets in each frame, and then performs cross-frame matching based on appearance features and motion information. With the development of deep learning, especially the Transformer architecture, end-to-end multi-object tracking methods have been extensively studied, further improving detection and tracking accuracy on standard datasets.

[0004] However, most of the above methods are evaluated under relatively regular datasets and ideal imaging conditions. In real traffic scenarios, there are a variety of visual disturbances: dense traffic of vehicles and pedestrians during peak hours leads to long-term occlusion and frequent interactions; backlighting in the morning and evening, nighttime lighting, and rain and fog cause drastic changes in image brightness and contrast; high-speed movement and camera shake bring obvious blurring and compression noise. As a result, detection networks are prone to missed detections, false alarms, and bounding box jitter. Existing methods generally have the following shortcomings: First, they lack a unified modeling mechanism for multiple types of disturbances at the image and trajectory levels during the training phase, making it difficult to systematically simulate the combined disturbances such as occlusion, blurring, lighting changes, and missed detections, trajectory interruptions, and position jitter at the detection level; Second, in the association phase, they are mostly based on pairwise similarity between targets, and have not yet incorporated targets, context, and disturbance factors into a unified structure for high-order inference; Third, the memory module usually only relies on confidence or fixed rules in its update strategy, lacking explicit modeling of disturbance intensity and observation quality. Once the memory features are contaminated by low-quality observations, it can easily lead to long-term trajectory drift or identity errors.

[0005] To address the aforementioned challenges, how to provide a solution to significantly improve correlation efficiency and result quality in complex traffic scenarios such as dense traffic, multi-lane intersections, and mixed pedestrian and vehicle traffic is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the aforementioned technical problems, the purpose of this application is to provide a multi-target tracking method, apparatus, device, and storage medium for traffic scenarios. By uniformly modeling multiple types of perturbations in the image layer and trajectory layer during the training phase, a high-order structure that can explicitly represent the relationship between the target, context, and perturbation is introduced, and a perturbation-aware memory module is combined to improve the robustness and stability of multi-target tracking in complex traffic environments.

[0007] To achieve the above objectives, this application provides a multi-target tracking method for traffic scenarios.

[0008] The above-mentioned objective of this application is achieved through the following technical solution:

[0009] A multi-target tracking method for traffic scenarios includes:

[0010] Acquire traffic scene video data and perform preprocessing to obtain a preprocessed video sequence;

[0011] Obtain a pre-trained multi-target tracking network, input the video sequence into the multi-target tracking network, and obtain the target tracking result;

[0012] Specifically, the multi-target tracking network is as follows:

[0013] Within a preset time window, the observation features of the candidate targets in the current frame of the video sequence are encoded and defined as target nodes, the scene local regions of the candidate targets are encoded and defined as context nodes, and the perturbation intensity information of the candidate targets is encoded and defined as perturbation nodes. The observation features of the candidate targets are fused with long-term memory features selected from the memory feature library and used as input features for the target nodes, thereby constructing a high-order hypergraph of the candidate targets.

[0014] Based on the constructed high-order hypergraph, feature representation is performed on each hyperedge to obtain the hyperedge intermediate representation vector of each hyperedge in the high-order hypergraph. Perturbation-aware aggregation is performed on the intermediate representation vectors of all hyperedges adjacent to the target node to obtain the high-order feature representation of the target node.

[0015] The higher-order feature representation is input into the association scoring function to obtain the association cost matrix between the target node and the historical trajectory, and then input into the linear allocation algorithm to update the candidate movement trajectory, and finally output the movement tracking trajectory of the candidate target in the video sequence.

[0016] Preferably, the observed features of the candidate target are fused with long-term memory features selected from the memory feature database, and this fusion serves as the input feature of the target node, including:

[0017] A long-term memory feature library is constructed based on the maintained historical trajectories. Candidate historical trajectories are selected from the long-term memory feature library according to the similarity of the observed features of the candidate targets and the consistency of the predicted position distance. The long-term memory features with the highest similarity of historical movement trajectories and the highest overlap of predicted positions are fused with the observed features of the candidate targets and used as the target-side input data of the hyperedge representation network.

[0018] Preferably, it further includes:

[0019] The candidate target's movement trajectory features are matched with historical movement trajectory features in the long-term memory feature library, and the long-term memory feature library is updated based on confidence level, position consistency, and disturbance intensity indication.

[0020] Preferably, the disturbance intensity information of the candidate target is encoded and defined as a disturbance node, specifically as follows:

[0021] The perturbation node includes image layer perturbation intensity information encoding and trajectory layer perturbation intensity information encoding.

[0022] Preferably, obtaining a pre-trained multi-target tracking network includes:

[0023] Obtain the video sequence for training;

[0024] Occlusion perturbation, blur perturbation, and photometric perturbation are injected into each frame of the video sequence to obtain a perturbation image sequence containing multiple visual degradation modes, which is then input into the multi-target tracking network for training.

[0025] Preferably, acquiring a pre-trained multi-target tracking network further includes:

[0026] In each frame of the video sequence, detection loss perturbation, position jitter perturbation, and trajectory discontinuity perturbation are injected into the target's movement trajectory to generate perturbation trajectory data containing multiple temporal degradation modes, which is then input into the multi-target tracking network for training.

[0027] Preferably, obtaining a pre-trained multi-target tracking network includes:

[0028] In the multi-target tracking network, a target detection loss function and a target trajectory association loss function are established, and a weighted combination is performed to obtain the total loss function.

[0029] The parameters in the multi-target tracking network are iteratively updated using a preset optimization algorithm;

[0030] When the total loss function converges or a preset condition is met, the optimization stops, and the trained multi-object tracking network is obtained.

[0031] A multi-target tracking device for traffic scenarios includes:

[0032] The preprocessing module is used to acquire traffic scene video data and perform preprocessing to obtain a preprocessed video sequence.

[0033] The trajectory acquisition module is used to acquire a pre-trained multi-target tracking network, input the video sequence into the multi-target tracking network, and obtain the target tracking result;

[0034] Specifically, the multi-target tracking network is as follows:

[0035] Within a preset time window, the observation features of the candidate targets in the current frame of the video sequence are encoded and defined as target nodes, the scene local regions of the candidate targets are encoded and defined as context nodes, and the perturbation intensity information of the candidate targets is encoded and defined as perturbation nodes. The observation features of the candidate targets are fused with long-term memory features selected from the memory feature library and used as input features for the target nodes, thereby constructing a high-order hypergraph of the candidate targets.

[0036] Based on the constructed high-order hypergraph, feature representation is performed on each hyperedge to obtain the hyperedge intermediate representation vector of each hyperedge in the high-order hypergraph. Perturbation-aware aggregation is performed on the intermediate representation vectors of all hyperedges adjacent to the target node to obtain the high-order feature representation of the target node.

[0037] The higher-order feature representation is input into the association scoring function to obtain the association cost matrix between the target node and the historical trajectory, and then input into the linear allocation algorithm to update the candidate movement trajectory, and finally output the movement tracking trajectory of the candidate target in the video sequence.

[0038] A multi-target tracking device for traffic scenarios includes:

[0039] Memory is used to store computer programs that run on the processor.

[0040] A processor is used to implement the multi-target tracking method for traffic scenarios when executing the computer program.

[0041] A multi-target tracking storage medium for traffic scenarios includes:

[0042] The multi-target tracking storage medium stores a computer program, which, when executed by a processor, implements the multi-target tracking method for traffic scenarios.

[0043] This application decomposes the complex spatiotemporal correlation problem into multiple sub-tasks, such as perturbation enhancement, hypergraph modeling, memory interaction, and linear assignment. It integrates local evidence through hyperedge representation and perturbation aggregation, and then completes trajectory-level matching and updating using the correlation cost matrix and linear assignment algorithm. This achieves efficient fusion and comprehensive judgment of multi-source information at the hyperedge level, making fuller use of contextual information such as road structure, lane distribution, and flow direction features. It effectively alleviates false detection and false association in scenarios with dense targets, complex interactions, and long duration of occlusion, significantly improving the accuracy and stability of trajectory-level results. It effectively reduces trajectory fragmentation and identity switching phenomena in scenarios with occlusion and dense targets, and significantly improves correlation efficiency and result quality in complex traffic scenarios such as dense vehicles, multi-lane weaving, and mixed pedestrian and vehicle traffic.

[0044] During the training phase of the multi-target tracking network, various perturbations such as occlusion, blurring, illumination changes, detection loss, position jitter, and trajectory discontinuity are considered simultaneously. The perturbations of the image layer and trajectory layer are modeled in a unified manner, and the perturbations are gradually introduced from weak to strong through a progressive perturbation scheduling strategy, which significantly improves the robustness of multi-target tracking in complex traffic scenarios. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a multi-target tracking method for traffic scenarios in an embodiment of this application;

[0047] Figure 2 This is a structural diagram of a multi-target tracking device for traffic scenarios according to an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] Furthermore, the technical features in the various embodiments or individual embodiments provided in this application can be arbitrarily combined with each other to form a feasible technical solution. Such combination is not constrained by the order of steps and / or the structural composition mode, but must be based on the ability of a person skilled in the art to implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0050] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described below are merely illustrative. For example, the division of units and modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or modules can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical or other forms.

[0051] In addition, each functional unit in the various embodiments of this application can be integrated into a single processor, or each unit can be a separate device, or two or more units can be integrated into a single device; each functional unit in the various embodiments of this application can be implemented in hardware or in the form of hardware plus software functional units.

[0052] Those skilled in the art will understand that all or part of the steps of the following method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the following method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0053] It should be understood that the use of terms such as "system," "device," "unit," and / or "module" in this application is merely one method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0054] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "a plurality of" or "several" means two or more, unless otherwise explicitly specified.

[0055] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which this application can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size should still fall within the scope of the technical content disclosed in this application, provided that they do not affect the effects and purposes that this application can produce.

[0056] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0057] It should also be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the article or device that includes the aforementioned element.

[0058] The implementation method of this application is written in a progressive manner.

[0059] A multi-target tracking method for traffic scenarios includes:

[0060] S1. Acquire traffic scene video data and perform preprocessing to obtain a preprocessed video sequence;

[0061] Specifically, raw traffic scene video data is collected through fixed road cameras, PTZ cameras, or vehicle-mounted camera equipment. The video data includes multiple targets such as motor vehicles, non-motor vehicles, and pedestrians.

[0062] The video data is preprocessed by converting data of different encoding formats in a unified manner, scaling videos of different resolutions in a unified manner, resampling the frame rate, and geometrically correcting fisheye lenses or wide-angle lenses, and finally obtaining the preprocessed video sequence.

[0063] S2. Obtain a pre-trained multi-target tracking network, input the video sequence into the multi-target tracking network, and obtain the target tracking results;

[0064] The multi-target tracking network specifically includes:

[0065] Within a preset time window, the observation features of candidate targets in the current frame of the video sequence are encoded and defined as target nodes, the scene local regions of candidate targets are encoded and defined as context nodes, and the perturbation intensity information of candidate targets is encoded and defined as perturbation nodes. The observation features of the candidate targets are fused with long-term memory features selected from the memory feature library and used as input features for target nodes, thereby constructing a high-order hypergraph of candidate targets.

[0066] Specifically, context nodes can be divided according to road topology, lane direction and traffic rules to ensure that the hypergraph structure can explicitly reflect the scene semantics;

[0067] The disturbance node includes the type, intensity, and historical statistical information encoding of the disturbance, and may also include information such as timestamps or the frequency of disturbance occurrence, which can be used to characterize the evolution of the disturbance in the time dimension;

[0068] Build target node Context nodes Disturbance nodes Fuse with long-term memory features of historical trajectories that have the highest similarity to candidate targets in the long-term memory feature library. This forms a high-order hypergraph structure of target-context-perturbation, calculated using the following formula:

[0069] ;

[0070] in,[ || ] indicates vector concatenation. , , , , For learnable weight matrix, This is a bias term.

[0071] Based on the construction of a high-order hypergraph, feature representation is performed on each hyperedge to obtain the hyperedge intermediate representation vector of each hyperedge in the high-order hypergraph. Perturbation-aware aggregation is performed on the intermediate representation vectors of all hyperedges adjacent to the target node to obtain the high-order feature representation of the target node.

[0072] Specifically, the formula for calculating the intermediate representation vector of each hyperedge is as follows:

[0073] ;

[0074] in, It is a non-linear activation function;

[0075] For the target node Context nodes With disturbance nodes The intermediate representation vector of the hyperedge corresponding to the formed hyperedge.

[0076] By communicating with the target node Connected context-perturbation combined index set (i.e., all with) Connected hyperedges Combination sets), and sets Any superedge encountered during the inner traversal (i.e., corresponding to any context node index) index of perturbation nodes 'Calculate the hyperedge intermediate representation vector obtained from the calculation to obtain the set' The unnormalized attention scores of all hyperedges, after being normalized by softmax, are calculated using the following formula:

[0077] ;

[0078] in,

[0079] This is a learnable attention vector;

[0080] Representing vectors The transpose, and The output is used to calculate the inner product, thereby obtaining the unnormalized attention score of the hyperedge.

[0081] Finally, the high-order feature representation of the target is obtained. :

[0082] .

[0083] The higher-order feature representation is input into the association scoring function to obtain the association cost matrix between the target node and the historical trajectory. This matrix is ​​then input into the linear assignment algorithm to update the candidate motion trajectories, and finally outputs the motion tracking trajectory of the candidate target in the video sequence.

[0084] Specifically, the correlation scoring function includes linear mapping and nonlinear activation;

[0085] The association cost matrix between the target node and the historical trajectory is as follows:

[0086] ;

[0087] in, Represented as the weight vector of the learning process The transpose of is used with the feature vector. Perform inner product to obtain scalar score;

[0088] This is a bias term used to shift the linear mapping result, thereby improving the expressive power of the scoring function.

[0089] The correlation cost matrix is ​​input into a linear allocation algorithm (such as the Hungarian algorithm) to solve for the optimal matching relationship, thereby realizing the creation, continuation, termination and reconnection of candidate target movement trajectories.

[0090] In some embodiments, the observed features of candidate targets are fused with long-term memory features selected from a memory feature database, and the resulting fused features are used as input features for the target node, including:

[0091] A long-term memory feature library is constructed based on the maintained historical trajectories. Candidate historical trajectories are selected from the long-term memory feature library based on the similarity of the observed features of the candidate targets and the consistency of the predicted position distance. The long-term memory features with the highest similarity of the historical movement trajectory and the highest overlap of the predicted position are fused with the observed features of the candidate targets and used as the target-side input data of the hyperedge representation network.

[0092] Specifically, long-term memory features are stored in the long-term memory bank according to historical trajectory instances, and the long-term memory bank is constructed based on the trajectory maintained by each tracking result;

[0093] When performing target-trajectory association calculation, for candidate targets in the current frame, candidate historical trajectories are first screened / retrieved in the memory bank based on appearance feature similarity and predicted position overlap (or distance consistency). Then, the long-term memory features corresponding to the candidate trajectories are read and fused with the current observation features as the target-side input of the hyperedge representation network.

[0094] In some embodiments, the observed features of candidate targets are fused with historical trajectories in a created memory feature library, and the resulting fusion is used as the input features for the target node, including:

[0095] The candidate target's movement trajectory features are matched with historical movement trajectory features in the long-term memory feature library, and the long-term memory feature library is updated based on confidence level, position consistency, and disturbance intensity indication.

[0096] In some embodiments, the matching of the candidate target's movement trajectory features with historical movement trajectories stored in a long-term memory feature database is also considered, such as confidence level. Location consistency and the current disturbance intensity indication Calculate the memory write gate coefficient:

[0097] ;

[0098] in, This represents the transpose of the weight vector. This is a bias term.

[0099] And update long-term memory features:

[0100] ;

[0101] in, The trajectory features observed for candidate targets in the current frame.

[0102] When the current observation quality is high and the disturbance intensity is low, the gating coefficient The gating coefficient tends to be larger, thus encouraging high-quality observations to be written into memory; when there is severe occlusion, missed detection, or strong interference, the gating coefficient is automatically reduced to suppress the contamination of long-term memory by abnormal observations.

[0103] The observation quality can be characterized by the target detection confidence and / or the consistency between the predicted position and the detected position. The perturbation intensity is estimated by the input degradation degree from the output of the composite perturbation module. The determination of "low observation quality" and "large perturbation intensity" can be achieved by using preset thresholds.

[0104] The preset threshold can be set through validation set search or engineering experience, or by using a continuous gating function to adaptively form an equivalent threshold boundary through learnable parameters during the training phase, thereby achieving the optimal effect of the set threshold.

[0105] By introducing perturbation sensing, the long-term memory of the trajectory is subjected to quality perception and dynamic update control. It can automatically assess the reliability of observations based on detection confidence, spatial matching degree, and current perturbation intensity, suppress the writing of low-quality observation features into memory, and strengthen the weight of high-confidence, low-perturbation observations in memory. This effectively avoids the problem of "more and more errors being recorded and errors accumulating over time" in traditional memory modules. In cases of long-term target occlusion, intermittent loss, or severe background interference, reliable re-association and trajectory continuation can be achieved by relying on clean and stable long-term memory, significantly improving the robustness and reliability of the system in long-term tracking tasks.

[0106] In other embodiments, the perturbation intensity information of the candidate target is encoded and defined as a perturbation node. Specifically, the perturbation node includes image layer perturbation intensity information encoding and trajectory layer perturbation intensity information encoding.

[0107] By classifying the disturbed nodes, we can take into account special situations in different environments and improve the robustness of multi-target tracking in complex traffic scenarios.

[0108] In some embodiments, obtaining a pre-trained multi-object tracking network includes:

[0109] Obtain the video sequence for training;

[0110] Occlusion perturbation, blur perturbation, and luminance perturbation are injected into each frame of the video sequence to obtain a perturbation image sequence containing multiple visual degradation modes, which is then input into a multi-object tracking network for training.

[0111] Specifically, when training a multi-target tracking network, the perturbation nodes of candidate targets can be adjusted for training purposes. For the processed [target name]... Frame Image Occlusion disturbances can be achieved by superimposing several occlusion masks on the image. The shape, size, and position of the occlusion mask can be generated based on the target detection box or by using a random strategy to simulate situations such as mutual occlusion between vehicles, mutual occlusion between pedestrians and vehicles, and occlusion by road poles and billboards.

[0112] The formula for calculating occlusion disturbance is:

[0113] ;

[0114] in, Indicates the first A masking code;

[0115] The parameters of the occlusion perturbation operator;

[0116] This indicates element-wise multiplication.

[0117] The fuzzy perturbation can be achieved through convolution, and the calculation formula is as follows:

[0118] ;

[0119] in, Represents Gaussian blur or motion blur operator along random directions;

[0120] For the parameters of the fuzzy perturbation operator;

[0121] This represents the corresponding convolution kernel parameters, used to simulate blurring caused by camera shake, high-speed motion, and compressed noise.

[0122] The formula for calculating photometric perturbation is:

[0123] ;

[0124] in: , This represents the brightness and contrast adjustment factors for random sampling;

[0125] This represents additive noise, used to simulate light degradation such as day-night cycles, backlighting, streetlight flickering, and rain / fog weather.

[0126] Image data is generated by combining occlusion perturbation, blur perturbation, and photometric perturbation. The calculation formula is:

[0127] ;

[0128] in, , , These are the intensity weights for occlusion perturbation, blur perturbation, and photometric perturbation, respectively.

[0129] In some embodiments, the encoding of trajectory layer perturbation intensity information includes:

[0130] In the video sequence, detection loss perturbation, position jitter perturbation and trajectory discontinuity perturbation are injected into the movement trajectory of the candidate target to generate perturbation trajectory data containing multiple temporal degradation modes.

[0131] Specifically, the trajectory of the candidate target within the time window is represented as follows:

[0132] ;

[0133] ;

[0134] in, Represents the target bounding box at time t;

[0135] These are the horizontal and vertical coordinates of the bounding box in the image coordinate system, respectively;

[0136] These represent the width and height of the bounding box, respectively.

[0137] After injecting disturbances into the movement trajectory, the calculation formula is as follows:

[0138] ;

[0139] in, This indicates that there is no detection box at this time step;

[0140] A flag indicating whether to retain this time step;

[0141] This represents the amount of positional jitter.

[0142] When a single time step When =0, it can simulate the detection of missed detections or short-term occlusions; when set within a continuous time interval... When =0, it can simulate situations where there is long-term occlusion or the target temporarily leaves the field of view; when the position jitter is set... When the size is large, it can simulate phenomena such as detection frame jitter and unstable positioning.

[0143] Furthermore, a combined composite perturbation sample can be obtained by combining the image layer perturbation intensity information encoding and the trajectory layer perturbation intensity information encoding, as shown in the formula:

[0144] ;

[0145] in, Indicates the first Original image of the frame The image after applying image layer perturbation;

[0146] Indicates the first The result of applying trajectory layer perturbation to the frame target bounding box or trajectory state is given by the following formula:

[0147] ;

[0148] in, For the first Number of frame targets;

[0149] For the first Frame number The perturbated bounding box of each target;

[0150] These are the horizontal and vertical position coordinates of the bounding box in the image coordinate system after applying trajectory layer perturbation;

[0151] These represent the bounding box's width and height in the image coordinate system after applying the trajectory layer perturbation.

[0152] By unifying the modeling of image layer perturbations and trajectory layer perturbations, and by gradually introducing perturbations from weak to strong through a progressive perturbation scheduling strategy for training, the robustness of the multi-target tracking network in complex traffic scenarios is significantly improved.

[0153] In some embodiments, obtaining a pre-trained multi-object tracking network includes:

[0154] In a multi-target tracking network, a target detection loss function and a target trajectory association loss function are established, and a weighted combination is performed to obtain the total loss function.

[0155] The parameters in the multi-target tracking network are iteratively updated using a pre-defined optimization algorithm;

[0156] The optimization stops when the total loss function converges or a preset condition is met, resulting in the trained multi-object tracking network.

[0157] Specifically, based on the object detection task and trajectory association task, an overall training is constructed, and the total loss function is obtained by weighting and combining the various losses. The calculation formula is as follows:

[0158] ;

[0159] in, For target detection loss, For trajectory association loss, , These are the corresponding weighting coefficients.

[0160] Meanwhile, the weights of the loss term can be adjusted according to different datasets and application scenarios to achieve a balance between detection accuracy, correlation stability and perturbation robustness.

[0161] In some embodiments, a gradual perturbation scheduling strategy is employed to control the perturbation probability.

[0162] Specifically, let the total number of training rounds be... The current round number is Perturbation probability It can be represented as:

[0163] ;

[0164] in, Let be the initial perturbation probability. This represents the maximum perturbation probability.

[0165] As the training progresses, the range of the blur kernel is gradually expanded, the occlusion area is increased, or the length of the trajectory discontinuity is increased, so that the model learns the basic pattern in the early stage and gradually adapts to complex perturbation combinations in the later stage.

[0166] The perturbation scheduling strategy can also be adaptively adjusted based on the performance of the validation set to achieve a balance between model convergence speed and perturbation diversity.

[0167] like Figure 2 As shown, a multi-target tracking device for traffic scenarios includes:

[0168] The preprocessing module is used to acquire traffic scene video data and perform preprocessing to obtain a preprocessed video sequence.

[0169] The trajectory acquisition module is used to acquire a pre-trained multi-target tracking network. The video sequence is input into the multi-target tracking network to obtain the target tracking result.

[0170] The multi-target tracking network specifically includes:

[0171] Within a preset time window, the observation features of candidate targets in the current frame of the video sequence are encoded and defined as target nodes, the scene local regions of candidate targets are encoded and defined as context nodes, and the perturbation intensity information of candidate targets is encoded and defined as perturbation nodes. The observation features of the candidate targets are fused with long-term memory features selected from the memory feature library and used as input features for target nodes, thereby constructing a high-order hypergraph of candidate targets.

[0172] Based on the construction of a high-order hypergraph, feature representation is performed on each hyperedge to obtain the hyperedge intermediate representation vector of each hyperedge in the high-order hypergraph. Perturbation-aware aggregation is performed on the intermediate representation vectors of all hyperedges adjacent to the target node to obtain the high-order feature representation of the target node.

[0173] The higher-order feature representation is input into the association scoring function to obtain the association cost matrix between the target node and the historical trajectory. This matrix is ​​then input into the linear assignment algorithm to update the candidate movement trajectories, and finally outputs the movement tracking trajectory of the candidate target in the video sequence.

[0174] A multi-target tracking device for traffic scenarios includes:

[0175] Memory is used to store computer programs that run on the processor.

[0176] A processor used to execute computer programs to implement multi-target tracking methods for traffic scenarios.

[0177] A multi-target tracking storage medium for traffic scenarios includes:

[0178] The multi-target tracking storage medium stores a computer program, which, when executed by a processor, implements a multi-target tracking method for traffic scenarios.

[0179] This application establishes a high-order hypergraph structure by uniformly representing candidate targets, local scene regions, and perturbation factors in the same graph space. Local evidence integration is completed through hyperedge representation and perturbation aggregation, and trajectory-level matching and updating are completed by association cost matrix and linear allocation algorithm. In complex traffic scenarios such as dense vehicles, multi-lane weaving, and mixed pedestrian and vehicle traffic, it can significantly improve association efficiency and tracking trajectory result quality.

[0180] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0181] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0182] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A traffic scene oriented multi-target tracking method, characterized in that, The method comprises the following steps: acquiring traffic scene video data and pre-processing to obtain a pre-processed video sequence; acquiring a pre-trained multi-target tracking network, inputting the video sequence into the multi-target tracking network to obtain a target tracking result; wherein the multi-target tracking network is specifically: encoding the observation feature of a candidate target in the current frame of the video sequence within a preset time window, defining it as a target node, encoding the local region of the candidate target in the scene, defining it as a context node, and encoding the disturbance intensity information of the candidate target, defining it as a disturbance node, and then fusing the observation feature of the candidate target with long-term memory features selected from a memory feature library as input features of the target node, thereby constructing a high-order hypergraph of the candidate target; according to the constructed high-order hypergraph, performing feature representation on each hyperedge to obtain a hyperedge intermediate representation vector of each hyperedge in the high-order hypergraph, and performing disturbance-aware aggregation on all hyperedge intermediate representation vectors adjacent to the target node to obtain a high-order feature representation of the target node; inputting the high-order feature representation into an association scoring function to obtain an association cost matrix between the target node and historical trajectories, and inputting it into a linear assignment algorithm to update the candidate moving trajectory, and finally outputting a moving tracking trajectory of the candidate target in the video sequence; acquiring a pre-trained multi-target tracking network, comprising: acquiring a video sequence for training; injecting occlusion disturbance, blur disturbance and luminosity disturbance into each frame image in the video sequence to obtain a disturbance image sequence containing multiple visual degradation modes; injecting detection loss disturbance, position jitter disturbance and trajectory discontinuity disturbance on the moving trajectory of the target in each frame image in the video sequence to generate disturbance trajectory data containing multiple time dimension degradation modes, and inputting it into the multi-target tracking network for training.

2. The traffic-scene-oriented multi-target tracking method according to claim 1, characterized in that, fusing the observation feature of the candidate target with long-term memory features selected from a memory feature library as input features of the target node, comprising: constructing a long-term memory feature library based on the maintained historical trajectories, and selecting candidate historical trajectories from the long-term memory feature library according to the similarity of the candidate target observation feature and the consistency of the predicted position distance, selecting the long-term memory feature with the highest historical moving trajectory similarity and predicted position overlap degree with the candidate target observation feature to fuse as the target side input data of the hyperedge representation network.

3. The traffic-scene-oriented multi-target tracking method according to claim 2, characterized in that, Further comprising: matching the moving trajectory feature of the candidate target with the historical moving trajectory feature matched in the long-term memory feature library, and confirming whether to update the long-term memory feature library according to the confidence, position consistency and disturbance intensity indicator.

4. The traffic-scene oriented multi-target tracking method according to claim 1, characterized in that, encoding the disturbance intensity information of the candidate target as a disturbance node, specifically: the disturbance node comprises image layer disturbance intensity information encoding and trajectory layer disturbance intensity information encoding.

5. The traffic-scene-oriented multi-target tracking method according to claim 4, characterized in that, acquiring a pre-trained multi-target tracking network, comprising: establishing a target detection loss function and a target trajectory association loss function in the multi-target tracking network, and performing weighted combination to obtain a total loss function; The parameters in the multi-target tracking network are iteratively updated by a preset optimization algorithm; The optimization is stopped when the total loss function converges or a preset condition is met, and a trained multi-target tracking network is obtained.

6. A traffic scene oriented multi-target tracking apparatus characterized by comprising: It comprises: a preprocessing module for obtaining traffic scene video data and preprocessing to obtain preprocessed video sequences; a trajectory acquisition module for acquiring a pre-trained multi-target tracking network, inputting the video sequences into the multi-target tracking network, and obtaining target tracking results; The multi-target tracking network specifically comprises: Within a preset time window, the observation feature of the candidate target in the current frame of the video sequence is encoded, defined as a target node, the local region of the candidate target in the scene is encoded, defined as a context node, and the disturbance intensity information of the candidate target is encoded, defined as a disturbance node. After the observation feature of the candidate target is fused with the long-term memory feature selected from the memory feature library, it is used as the input feature of the target node, thereby constructing a high-order hypergraph of the candidate target; According to the constructed high-order hypergraph, the feature representation of each hyperedge is obtained, and the high-order feature representation of the target node is obtained by aggregating all hyperedge intermediate representation vectors adjacent to the target node; The high-order feature representation is input into an association scoring function to obtain an association cost matrix between the target node and the historical trajectory, and is input into a linear assignment algorithm to update the candidate moving trajectory, and finally outputs the moving tracking trajectory of the candidate target in the video sequence. The pre-trained multi-target tracking network comprises: acquiring a video sequence for training; In each frame of the video sequence, inject occlusion disturbance, blur disturbance and luminosity disturbance to obtain a disturbance image sequence containing multiple visual degradation modes; inject detection loss disturbance, position jitter disturbance and trajectory interruption disturbance on the moving trajectory of the target in each frame of the video sequence to generate disturbance trajectory data containing multiple time dimension degradation modes, and input into the multi-target tracking network for training.

7. A traffic scene oriented multi-target tracking device, characterized by It comprises: a memory for storing computer programs running on the processor; a processor for executing the computer programs to implement the multi-target tracking method according to any one of claims 1 to 5.

8. A traffic scene oriented multi-target tracking storage medium, characterized by, It comprises: The multi-target tracking storage medium stores computer programs, and the computer programs are executed by the processor to implement the multi-target tracking method according to any one of claims 1 to 5.

Images