A multi-target tracking method and system based on Kalman filtering and end-to-end Transformer

Abstract

This invention relates to the field of data processing technology and discloses a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer. The method involves detecting reference vehicles within a reference frame, initializing a Kalman filter and an end-to-end Transformer model, detecting tracking vehicles within a tracking frame, using the Kalman filter to predict the position of the vehicle in the next tracking frame, and then using the end-to-end Transformer model to extract tracking vehicles that meet matching conditions within the next tracking frame based on the predicted position information. This yields the actual position information of each tracking vehicle, generates the actual driving trajectory, and finally, based on the distance between the actual and predicted position information, and considering the phased changes in road traffic flow, intelligently allocates the updating of the Kalman filter and end-to-end Transformer model and the computational load, improving the tracking efficiency and response speed of multi-target road vehicle tracking, reducing the difficulty of vehicle identification and tracking, and lowering the tracking trajectory error rate.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer. Background Technology

[0002] Multi-target vehicle tracking is of great significance and has wide application value in intelligent transportation systems. Multi-target tracking technology can provide real-time traffic flow and vehicle trajectory information, which helps traffic management departments optimize traffic signal control, improve road traffic efficiency, and alleviate traffic congestion.

[0003] Currently, multi-target tracking of road vehicles typically employs continuous frame image recognition and vehicle matching. However, considering the need to identify and compare each object in the frame image, there are limitations in tracking efficiency and response speed when road traffic is heavy. Furthermore, as traffic volume increases, the likelihood of similar vehicles appearing in the same frame image increases, leading to increased difficulty in vehicle recognition and a high error rate in tracking trajectories.

[0004] Therefore, improving the tracking efficiency and response speed of multi-target tracking of road vehicles, reducing the difficulty of vehicle identification and tracking, and lowering the tracking trajectory error rate are technical problems that urgently need to be solved. Summary of the Invention

[0005] This invention provides a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer, aiming to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above objectives, this invention provides a multi-target tracking method based on Kalman filtering and end-to-end Transformer, comprising the following steps:

[0007] Acquire a target analysis video set; wherein, the target analysis video set includes target analysis videos of road vehicles traveling, collected by several cameras set up at different sections of the target road;

[0008] Detect reference vehicles in each reference frame of each target analysis video, generate tracking initialization parameters based on the detection results of reference vehicles in each reference frame, and initialize the Kalman filter and end-to-end Transformer model using the tracking initialization parameters;

[0009] The tracking vehicles in each tracking frame of each target analysis video are detected, and the position of the tracking vehicles in the next tracking frame is predicted using a Kalman filter to obtain the predicted position information of each tracking vehicle in the next tracking frame. Based on the predicted position information, the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame are extracted using an end-to-end Transformer model to generate the actual position information of each tracking vehicle.

[0010] Based on the actual and predicted location information of each tracked vehicle in each optimization and adjustment cycle, the Kalman filter and end-to-end Transformer model for the next optimization and adjustment cycle are updated, and a data transmission mapping relationship between each target analysis video and the corresponding data processing server is established.

[0011] Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0012] Optionally, the steps for obtaining the target analysis video set include:

[0013] Access the road monitoring database; wherein, the road monitoring database stores several pieces of road monitoring information, each piece of road monitoring information including collected monitoring videos of road vehicles and the road coverage segment of each monitoring video;

[0014] Extract the start and end points of the target road for multi-target tracking, and match the monitoring videos of several road coverage segments corresponding to the target road in the road monitoring database;

[0015] The monitoring videos of the aforementioned road coverage segments are used as the target analysis video set, and each target analysis video in the target analysis video set is assigned a corresponding sequence number according to the segment order of each road coverage segment from the starting position to the ending position in the target road.

[0016] Optionally, the step of detecting reference vehicles in each reference frame of each target analysis video, and generating tracking initialization parameters based on the detection results of reference vehicles in each reference frame, specifically includes:

[0017] The target detection algorithm is used to detect and identify the reference vehicles in each reference frame, and the initial state information combination of each reference vehicle and the number of reference vehicles in each reference frame are extracted.

[0018] Based on the state variables in the initial state information combination of each reference vehicle, a state vector containing each state variable is constructed, and the elements on the diagonal of the diagonal matrix are configured as the covariance matrix of the variances of each state variable; wherein, the initial state information combination contains N kinds of state information.

[0019] Based on the number of reference vehicles in each reference frame, an initial matching feature combination is determined from a preset reference vehicle number value and matching feature combination comparison table. The number of matching features in the initial matching feature combination is positively correlated with the number of reference vehicles in each reference frame.

[0020] Optionally, the steps of initializing the Kalman filter and the end-to-end Transformer model using tracking initialization parameters include:

[0021] Based on the state vector and covariance matrix of each reference vehicle, an initial Kalman filter is constructed for each reference vehicle.

[0022] Based on the initial matching feature combination, the end-to-end Transformer model corresponding to the initial matching feature combination is matched in the pre-trained end-to-end Transformer model database to obtain the initial end-to-end Transformer model of the target analysis video.

[0023] Optionally, the steps of detecting the tracking vehicle in each tracking frame of the target analysis video, predicting the position of the tracking vehicle in the next tracking frame using a Kalman filter, and obtaining the predicted position information of each tracking vehicle in the next tracking frame specifically include:

[0024] A target detection algorithm is used to detect and identify the tracking vehicles in each tracking frame of the target analysis video, and extract the state information of each tracking vehicle in each tracking frame by combining the initial state information of each tracking vehicle.

[0025] The state information of each tracked vehicle in each tracking frame is input into the initial Kalman filter of each reference vehicle constructed accordingly, and the position of the tracked vehicle in each tracking frame in the next tracking frame is predicted to obtain the predicted position information of each tracked vehicle in the next tracking frame.

[0026] Optionally, based on the predicted location information, the step of extracting the tracking vehicles in the next tracking frame that satisfy the matching conditions with the tracking vehicles in each tracking frame using an end-to-end Transformer model, and generating the actual location information of each tracking vehicle, specifically includes:

[0027] Based on the predicted position information of each tracked vehicle in the next tracking frame, the predicted position information is used as the identification starting position, and similarity matching is performed on each tracked vehicle in the next tracking frame in ascending order of distance between each vehicle and the identification starting position.

[0028] When the first similarity score is detected to be no lower than the preset similarity threshold, the matching process is stopped and the location of the currently matched vehicle is used as the actual location information of each tracked vehicle.

[0029] Optionally, based on the actual and predicted position information of each tracked vehicle within each optimization adjustment period, the Kalman filter and end-to-end Transformer model for the next optimization adjustment period are updated. This includes:

[0030] The proportion of the distance between the actual location information and the predicted location information of each tracked vehicle in each optimization and adjustment cycle that exceeds a preset distance threshold is calculated, and it is determined whether the proportion is within the error proportion range.

[0031] If so, the actual position information and predicted position information for the next optimization adjustment cycle are determined using the Kalman filter and end-to-end Transformer model of the current optimization adjustment cycle.

[0032] If not, determine whether the ratio is higher than the error ratio range. If yes, perform the first optimization adjustment action of the Kalman filter and the end-to-end Transformer model. If no, perform the second optimization adjustment action of the Kalman filter and the end-to-end Transformer model.

[0033] The first optimization adjustment action includes: adding a target number of optimized state variables to the state information combination corresponding to the Kalman filter in the previous optimization adjustment period to obtain an updated state information combination, and using the state vectors corresponding to the state variables in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the first matching feature combination in the previous optimization adjustment period with an end-to-end Transformer model with a second matching feature combination in the end-to-end Transformer model library; wherein the second matching feature combination is configured to have more matching features than the first matching feature combination.

[0034] The second optimization adjustment action includes: reducing the number of recently added target optimization state variables in the state information combination corresponding to the Kalman filter in the previous optimization adjustment period to obtain an updated state information combination, and constructing an optimized Kalman filter using the state vectors corresponding to the state variables in the updated state information combination; and replacing the end-to-end Transformer model corresponding to the second matching feature combination in the previous optimization adjustment period with an end-to-end Transformer model with a first matching feature combination in the end-to-end Transformer model library; wherein the first matching feature combination is configured to have fewer matching features than the second matching feature combination.

[0035] Optionally, the step of establishing a data transmission mapping relationship between each target analysis video and its corresponding data processing server includes:

[0036] After updating the Kalman filter and end-to-end Transformer model for the next optimization adjustment period, obtain the number of optimized state variables of the Kalman filter and the number of matching features of the end-to-end Transformer model in the next optimization adjustment period.

[0037] According to the preset conversion rules, the number of optimization state variables and the number of matching features of each target analysis video in the next optimization adjustment cycle are converted into computational load quantization values ​​and accumulated to obtain the computational load task of each target analysis video;

[0038] Based on the computational load quantization value of the computational load task of each target analysis video and the computational idle quantization value of each data processing server in the next optimization and adjustment cycle, each target analysis video is assigned to the corresponding data processing server with the objective of minimizing the variance of the total computational load quantization value of the computational load tasks assigned to each data processing server.

[0039] Based on the allocation relationship between the target analysis video and the corresponding data processing server, establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server.

[0040] Optionally, the steps include obtaining the actual position information of each tracked vehicle in each tracking frame of each target analysis video, and generating the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information. Specifically, this includes:

[0041] Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the regional driving trajectory of each tracked vehicle in each target analysis video according to the timestamp order of each tracking frame corresponding to each actual position information;

[0042] The system summarizes the regional driving trajectories of each target analysis video in the target analysis video set, removes duplicate trajectories, and obtains the actual driving trajectory of each tracked vehicle in the target analysis video set.

[0043] Furthermore, to achieve the above objectives, the present invention also provides a multi-target tracking system based on Kalman filtering and end-to-end Transformer, comprising:

[0044] The acquisition module is used to acquire a target analysis video set; wherein, the target analysis video set includes target analysis videos of road vehicles traveling, collected by several cameras set up at different sections of the target road;

[0045] The first detection module is used to detect reference vehicles in each reference frame of each target analysis video, generate tracking initialization parameters based on the detection results of reference vehicles in each reference frame, and initialize the Kalman filter and end-to-end Transformer model using the tracking initialization parameters.

[0046] The second detection module is used to detect the tracking vehicles in each tracking frame of each target analysis video, use a Kalman filter to predict the position of the tracking vehicles in the next tracking frame, and obtain the predicted position information of each tracking vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame, and generate the actual position information of each tracking vehicle.

[0047] The update module is used to update the Kalman filter and end-to-end Transformer model for the next optimization adjustment period based on the actual and predicted position information of each tracked vehicle in each optimization adjustment period, and to establish the data transmission mapping relationship between each target analysis video and the corresponding data processing server.

[0048] The generation module is used to obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0049] The beneficial effects of this invention are as follows: It proposes a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer. By detecting reference vehicles within reference frames in the video analysis for each target, the Kalman filter and end-to-end Transformer model are initialized based on the detection results of the reference vehicles. Then, by detecting tracking vehicles within each tracking frame in the video analysis for each target, the Kalman filter is used to predict the position in the next tracking frame based on the detection results. Finally, based on the predicted position information, the end-to-end Transformer model is used to extract tracking vehicles in the next tracking frame that satisfy the matching conditions with the tracking vehicles in each tracking frame, thus generating... Based on the actual location information of each tracked vehicle in the analysis video of each target, the actual driving trajectory is generated. Finally, based on the predicted and actual location information in each optimization and adjustment cycle, and considering the characteristics of the phased changes in road traffic flow and the impact of traffic flow changes on the accuracy of vehicle prediction location and the difficulty of vehicle recognition, the Kalman filter and end-to-end Transformer model update and the intelligent allocation of computational load tasks for the next optimization and adjustment cycle are realized. This improves the tracking efficiency and response speed of multi-target tracking of road vehicles, reduces the difficulty of vehicle recognition and tracking, and lowers the tracking trajectory error rate. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the multi-target tracking method based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of the structure of a multi-target tracking system based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention.

[0052] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] This invention provides a multi-target tracking method based on Kalman filtering and end-to-end Transformer, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the multi-target tracking method based on Kalman filtering and end-to-end Transformer in an embodiment of the present invention.

[0055] In this embodiment, the multi-target tracking method based on Kalman filtering and end-to-end Transformer includes the following steps:

[0056] S1: Obtain the target analysis video set; wherein, the target analysis video set includes target analysis videos of road vehicles traveling, collected by several cameras set up at different sections of the target road;

[0057] S2: Detect reference vehicles in each reference frame of each target analysis video, generate tracking initialization parameters based on the detection results of reference vehicles in each reference frame, and initialize the Kalman filter and end-to-end Transformer model using the tracking initialization parameters;

[0058] S3: Detect the tracking vehicles in each tracking frame of each target analysis video, use a Kalman filter to predict the position of the tracking vehicles in the next tracking frame, and obtain the predicted position information of each tracking vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame, and generate the actual position information of each tracking vehicle.

[0059] S4: Based on the actual and predicted location information of each tracked vehicle in each optimization and adjustment cycle, update the Kalman filter and end-to-end Transformer model for the next optimization and adjustment cycle, and establish the data transmission mapping relationship between each target analysis video and the corresponding data processing server.

[0060] S5: Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0061] It should be noted that currently, multi-target tracking of road vehicles usually adopts the method of continuous frame image recognition and vehicle matching. However, considering that each object in the frame image needs to be identified and compared, there are limitations in tracking efficiency and response speed when the traffic volume is high. Furthermore, as the traffic volume increases, the possibility of similar vehicles appearing in the same frame image increases, which leads to increased difficulty in vehicle recognition and high tracking trajectory error rate.

[0062] To address the aforementioned issues, this embodiment detects reference vehicles within reference frames in each target analysis video. Based on the detection results of these reference vehicles, it initializes a Kalman filter and an end-to-end Transformer model. Then, it detects tracking vehicles within each tracking frame in each target analysis video. Based on the detection results, it uses a Kalman filter to predict the position of the vehicle in the next tracking frame. Then, based on the predicted position information, it uses an end-to-end Transformer model to extract tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame, generating the actual position information for each tracking vehicle. Based on the actual position information of each tracking vehicle in each target analysis video, it generates the actual driving trajectory. Finally, based on the predicted and actual position information within each optimization adjustment cycle, and considering the distance between the actual and predicted position information, it takes into account the characteristics of phased changes in road traffic flow and the impact of traffic flow changes on the accuracy of vehicle prediction and the difficulty of vehicle recognition. This enables intelligent allocation of the Kalman filter and end-to-end Transformer model updates and computational load tasks for the next optimization adjustment cycle, improving the tracking efficiency and response speed of multi-target road vehicle tracking, reducing the difficulty of vehicle recognition and tracking, and lowering the tracking trajectory error rate.

[0063] In a preferred embodiment, the step of obtaining the target analysis video set specifically includes:

[0064] S11: Access the road monitoring database; wherein, the road monitoring database stores several pieces of road monitoring information, each piece of road monitoring information including collected monitoring videos of road vehicles and the road coverage segment of each monitoring video;

[0065] S12: Extract the start and end positions of the target road for multi-target tracking, and match the monitoring videos of several road coverage segments corresponding to the target road in the road monitoring database;

[0066] S13: Take the monitoring videos of the several road coverage segments as the target analysis video set, and assign a corresponding sequence number to each target analysis video in the target analysis video set according to the segment order of each road coverage segment from the starting position to the ending position in the target road.

[0067] In this embodiment, the target analysis video set is obtained by accessing the road monitoring database, extracting the start and end points of the target road, finding several monitoring videos covering the target road in the road monitoring database, and then sequentially numbering each monitoring video according to the segment order from the start to the end point, thus obtaining the target analysis video set. This facilitates obtaining the overall tracking trajectory of the vehicle on the target road by summarizing multiple trajectories after tracking each target analysis video separately.

[0068] In a preferred embodiment, the step of detecting reference vehicles in each reference frame of each target analysis video, and generating tracking initialization parameters based on the detection results of reference vehicles in each reference frame, specifically includes:

[0069] S21: Use a target detection algorithm to detect and identify reference vehicles in each reference frame, and extract the initial state information combination of each reference vehicle and the reference vehicle count value of each reference frame.

[0070] S22: Based on the state variables in the initial state information combination of each reference vehicle, construct a state vector containing each state variable and a covariance matrix whose diagonal elements are configured as the variances of each state variable; wherein, the initial state information combination contains N kinds of state information;

[0071] S23: Based on the reference vehicle quantity value of each reference frame, determine the initial matching feature combination in the matching feature set for vehicle association recognition by the end-to-end Transformer model in a preset reference vehicle quantity value and matching feature combination comparison table; wherein, the number of matching features in the initial matching feature combination is positively correlated with the reference vehicle quantity value of each reference frame.

[0072] Based on this, the steps for initializing the Kalman filter and the end-to-end Transformer model using tracking initialization parameters include:

[0073] S24: Construct the initial Kalman filter for each reference vehicle based on the state vector and covariance matrix of each reference vehicle;

[0074] S25: Based on the initial matching feature combination, match the end-to-end Transformer model corresponding to the initial matching feature combination in the pre-trained end-to-end Transformer model database to obtain the initial end-to-end Transformer model of the target analysis video.

[0075] In this embodiment, a target detection algorithm is first used to detect and identify reference vehicles in the reference frames (the first N image frames in each detection cycle are reference frames, and the remaining image frames are tracking frames). Then, a state vector and covariance matrix are constructed based on the initial state information group of the reference vehicles (the state variables in the initial state information group include vehicle position, vehicle speed in the X and Y directions, etc.). Subsequently, a Kalman filter is constructed. The initial matching feature combination is determined based on the number of reference vehicles (the matching features in the initial matching feature combination include vehicle size, color distribution, window features, wheel features, roof features, etc.). The end-to-end Transformer model corresponding to the initial matching feature combination is matched in a pre-trained end-to-end Transformer model database. In this way, the Kalman filter and end-to-end Transformer model used to perform initial vehicle tracking are obtained.

[0076] In a preferred embodiment, the steps of detecting the tracked vehicle in each tracking frame of each target analysis video, predicting the position of the tracked vehicle in the next tracking frame using a Kalman filter, and obtaining the predicted position information of each tracked vehicle in the next tracking frame specifically include:

[0077] S31: Use a target detection algorithm to detect and identify the tracking vehicles in each tracking frame of the target analysis video, and extract the state information of each tracking vehicle in each tracking frame by combining the initial state information of each tracking vehicle.

[0078] S32: Input the state information of each tracked vehicle in each tracking frame into the initial Kalman filter of each reference vehicle constructed accordingly, predict the position of the tracked vehicle in the next tracking frame in each tracking frame, and obtain the predicted position information of each tracked vehicle in the next tracking frame.

[0079] Based on this, and using the predicted location information, the step of extracting the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame using an end-to-end Transformer model, and generating the actual location information of each tracking vehicle, specifically includes:

[0080] S33: Based on the predicted position information of each tracked vehicle in the next tracking frame, taking the predicted position information as the identification starting position, perform similarity matching with each tracked vehicle in the next tracking frame in ascending order of distance between each vehicle and the identification starting position;

[0081] S34: When the first similarity is detected to be no less than the preset similarity threshold, stop the matching process and use the location of the currently matched vehicle as the actual location information of each tracked vehicle.

[0082] In this embodiment, after obtaining the Kalman filter and the end-to-end Transformer model, the Kalman filter is first used to predict the position of the tracked vehicle in the next tracking frame, obtaining predicted position information. Then, based on the predicted position information, the end-to-end Transformer model is used to perform vehicle similarity matching in the next tracking frame in ascending order of distance between each vehicle and the predicted position information, until the first vehicle with a similarity not lower than a preset similarity threshold is detected, which is then taken as the actual position of the tracked vehicle in the next tracking frame. Therefore, by starting from the predicted position information and performing similarity matching in order of distance, it achieves higher recognition and tracking efficiency compared to the traditional method of performing similarity matching for each vehicle across the entire image frame.

[0083] In a preferred embodiment, the step of updating the Kalman filter and end-to-end Transformer model for the next optimization adjustment period based on the actual and predicted position information of each tracked vehicle within each optimization adjustment period specifically includes:

[0084] S41: Calculate the proportion of the distance between the actual location information and the predicted location information of each tracked vehicle exceeding a preset distance threshold within each optimization and adjustment cycle, and determine whether the proportion is within the error proportion range.

[0085] S42: If so, use the Kalman filter and end-to-end Transformer model of the current optimization adjustment period to determine the actual position information and predicted position information for the next optimization adjustment period;

[0086] S43: If not, determine whether the ratio is higher than the error ratio range. If yes, execute the first optimization adjustment action of the Kalman filter and the end-to-end Transformer model. If no, execute the second optimization adjustment action of the Kalman filter and the end-to-end Transformer model.

[0087] The first optimization adjustment action includes: adding a target number of optimized state variables to the state information combination corresponding to the Kalman filter in the previous optimization adjustment period to obtain an updated state information combination, and using the state vectors corresponding to the state variables in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the first matching feature combination in the previous optimization adjustment period with an end-to-end Transformer model with a second matching feature combination in the end-to-end Transformer model library; wherein the second matching feature combination is configured to have more matching features than the first matching feature combination.

[0088] The second optimization adjustment action includes: reducing the number of recently added target optimization state variables in the state information combination corresponding to the Kalman filter in the previous optimization adjustment period to obtain an updated state information combination, and constructing an optimized Kalman filter using the state vectors corresponding to the state variables in the updated state information combination; and replacing the end-to-end Transformer model corresponding to the second matching feature combination in the previous optimization adjustment period with an end-to-end Transformer model with a first matching feature combination in the end-to-end Transformer model library; wherein the first matching feature combination is configured to have fewer matching features than the second matching feature combination.

[0089] Based on this, the steps for establishing the data transmission mapping relationship between each target analysis video and the corresponding data processing server include:

[0090] S44: After updating the Kalman filter and end-to-end Transformer model for the next optimization adjustment period, obtain the number of optimized state variables of the Kalman filter and the number of matching features of the end-to-end Transformer model in the next optimization adjustment period.

[0091] S45: According to the preset conversion rules, the number of optimization state variables and the number of matching features of each target analysis video in the next optimization adjustment cycle are converted into computational load quantization values ​​and accumulated to obtain the computational load task of each target analysis video;

[0092] S46: Based on the computational load quantization value of the computational load task of each target analysis video and the computational idle quantization value of each data processing server in the next optimization adjustment cycle, and with the objective of minimizing the variance of the total computational load quantization value of the computational load tasks assigned to each data processing server, each target analysis video is assigned to the corresponding data processing server.

[0093] S47: Based on the allocation relationship between the target analysis video and the corresponding data processing server, establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server.

[0094] In this embodiment, based on the predicted and actual location information within each optimization adjustment cycle, and considering the distance between the actual and predicted location information, the characteristics of the phased changes in road traffic flow and the impact of traffic flow changes on the accuracy of vehicle prediction location and the difficulty of vehicle recognition are taken into account. This enables intelligent allocation of Kalman filter and end-to-end Transformer model updates and computational load tasks for the next optimization adjustment cycle, thereby improving the tracking efficiency and response speed of multi-target road vehicle tracking, reducing the difficulty of vehicle recognition and tracking, and lowering the tracking trajectory error rate.

[0095] In a preferred embodiment, the steps of obtaining the actual position information of each tracked vehicle in each tracking frame of each target analysis video, and generating the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information, specifically include:

[0096] S51: Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the regional driving trajectory of each tracked vehicle in each target analysis video according to the order of the timestamps of the tracking frames corresponding to each actual position information.

[0097] S52: Summarize the regional driving trajectory of each target analysis video in the target analysis video set, delete duplicate trajectories, and obtain the actual driving trajectory of each tracked vehicle in the target analysis video set.

[0098] In this embodiment, a Kalman filter is used to predict the position in the next tracking frame. Then, based on the predicted position information, an end-to-end Transformer model is used to extract the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame, generating the actual position information of each tracking vehicle. Based on the actual position information of each tracking vehicle in each target analysis video, vehicle tracking is performed on each target analysis video, and the overall tracking trajectory of the vehicle on the target road is obtained by summarizing multiple trajectories, generating the actual driving trajectory of each tracking vehicle.

[0099] Reference Figure 2 , Figure 2 This is a schematic diagram of the structure of a multi-target tracking system based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention.

[0100] like Figure 2 As shown, the multi-target tracking system based on Kalman filtering and end-to-end Transformer proposed in this embodiment of the invention includes:

[0101] The acquisition module 10 is used to acquire a target analysis video set; wherein, the target analysis video set includes target analysis videos of road vehicles traveling, collected by several cameras set up in different sections of the target road;

[0102] The first detection module 20 is used to detect reference vehicles in each reference frame of each target analysis video, generate tracking initialization parameters based on the detection results of reference vehicles in each reference frame, and initialize the Kalman filter and end-to-end Transformer model using the tracking initialization parameters.

[0103] The second detection module 30 is used to detect the tracking vehicles in each tracking frame of each target analysis video, use a Kalman filter to predict the position of the tracking vehicles in each tracking frame in the next tracking frame, and obtain the predicted position information of each tracking vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame, and generate the actual position information of each tracking vehicle.

[0104] The update module 40 is used to update the Kalman filter and end-to-end Transformer model for the next optimization adjustment period based on the actual and predicted position information of each tracked vehicle in each optimization adjustment period, and to establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server.

[0105] The generation module 50 is used to obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0106] Other embodiments or specific implementations of the multi-target tracking system based on Kalman filtering and end-to-end Transformer of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0107] It is understood that in the description of this specification, references to terms such as "one embodiment," "another embodiment," "other embodiments," or "first embodiment to Nth embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0108] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0109] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A multi-target tracking method based on Kalman filtering and end-to-end Transformer, characterized in that, Includes the following steps: Acquire a target analysis video set; wherein, the target analysis video set includes target analysis videos of road vehicles traveling, collected by several cameras set up at different sections of the target road; Detect reference vehicles in each reference frame of each target analysis video, generate tracking initialization parameters based on the detection results of reference vehicles in each reference frame, and initialize the Kalman filter and end-to-end Transformer model using the tracking initialization parameters; The tracking vehicles in each tracking frame of each target analysis video are detected, and the position of the tracking vehicles in the next tracking frame is predicted using a Kalman filter to obtain the predicted position information of each tracking vehicle in the next tracking frame. Based on the predicted position information, the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame are extracted using an end-to-end Transformer model to generate the actual position information of each tracking vehicle. Based on the actual and predicted location information of each tracked vehicle in each optimization and adjustment cycle, the Kalman filter and end-to-end Transformer model for the next optimization and adjustment cycle are updated, and a data transmission mapping relationship between each target analysis video and the corresponding data processing server is established. Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

2. The Kalman filter and end-to-end Transformer based multi-target tracking method of claim 1, wherein, The steps to obtain the target analysis video set include: Access the road monitoring database; wherein, the road monitoring database stores several pieces of road monitoring information, each piece of road monitoring information including collected monitoring videos of road vehicles and the road coverage segment of each monitoring video; Extract the start and end points of the target road for multi-target tracking, and match the monitoring videos of several road coverage segments corresponding to the target road in the road monitoring database; The monitoring videos of the aforementioned road coverage segments are used as the target analysis video set, and each target analysis video in the target analysis video set is assigned a corresponding sequence number according to the segment order of each road coverage segment from the starting position to the ending position in the target road.

3. The Kalman filter and end-to-end Transformer based multi-target tracking method of claim 1, wherein, The steps for detecting reference vehicles in each reference frame of each target analysis video, and generating tracking initialization parameters based on the detection results of reference vehicles in each reference frame, specifically include: The target detection algorithm is used to detect and identify the reference vehicles in each reference frame, and the initial state information combination of each reference vehicle and the number of reference vehicles in each reference frame are extracted. Based on the state variables in the initial state information combination of each reference vehicle, a state vector containing each state variable is constructed, and the elements on the diagonal of the diagonal matrix are configured as the covariance matrix of the variances of each state variable; wherein, the initial state information combination contains N kinds of state information. Based on the number of reference vehicles in each reference frame, an initial matching feature combination is determined from a preset reference vehicle number value and matching feature combination comparison table. The number of matching features in the initial matching feature combination is positively correlated with the number of reference vehicles in each reference frame.

4. The multi-target tracking method based on Kalman filtering and end-to-end Transformer as described in claim 3, characterized in that, The steps for initializing the Kalman filter and end-to-end Transformer model using tracking initialization parameters specifically include: Based on the state vector and covariance matrix of each reference vehicle, an initial Kalman filter is constructed for each reference vehicle. Based on the initial matching feature combination, the end-to-end Transformer model corresponding to the initial matching feature combination is matched in the pre-trained end-to-end Transformer model database to obtain the initial end-to-end Transformer model of the target analysis video.

5. The Kalman filter and end-to-end Transformer based multi-target tracking method of claim 4, wherein, The steps of detecting tracked vehicles in each tracking frame of the video for each target analysis, predicting the position of the tracked vehicle in the next tracking frame using a Kalman filter, and obtaining the predicted position information of each tracked vehicle in the next tracking frame specifically include: A target detection algorithm is used to detect and identify the tracking vehicles in each tracking frame of the target analysis video, and extract the state information of each tracking vehicle in each tracking frame by combining the initial state information of each tracking vehicle. The state information of each tracked vehicle in each tracking frame is input into the initial Kalman filter of each reference vehicle constructed accordingly, and the position of the tracked vehicle in each tracking frame in the next tracking frame is predicted to obtain the predicted position information of each tracked vehicle in the next tracking frame.

6. The Kalman filter and end-to-end Transformer based multi-target tracking method of claim 5, wherein, Based on the predicted location information, the step of extracting the tracking vehicles in the next tracking frame that satisfy the matching conditions with the tracking vehicles in each tracking frame using an end-to-end Transformer model, and generating the actual location information of each tracking vehicle, specifically includes: Based on the predicted position information of each tracked vehicle in the next tracking frame, the predicted position information is used as the identification starting position, and similarity matching is performed on each tracked vehicle in the next tracking frame in ascending order of distance between each vehicle and the identification starting position. When the first similarity score is detected to be no lower than the preset similarity threshold, the matching process is stopped and the location of the currently matched vehicle is used as the actual location information of each tracked vehicle.

7. The multi-target tracking method based on Kalman filtering and end-to-end Transformer as described in claim 6, characterized in that, The steps for updating the Kalman filter and end-to-end Transformer model for the next optimization period, based on the actual and predicted position information of each tracked vehicle within each optimization period, specifically include: The proportion of the distance between the actual location information and the predicted location information of each tracked vehicle in each optimization and adjustment cycle that exceeds a preset distance threshold is calculated, and it is determined whether the proportion is within the error proportion range. If so, the actual position information and predicted position information for the next optimization adjustment cycle are determined using the Kalman filter and end-to-end Transformer model of the current optimization adjustment cycle. If not, determine whether the ratio is higher than the error ratio range. If yes, perform the first optimization adjustment action of the Kalman filter and the end-to-end Transformer model. If no, perform the second optimization adjustment action of the Kalman filter and the end-to-end Transformer model. The first optimization adjustment action includes: adding a target number of optimized state variables to the state information combination corresponding to the Kalman filter in the previous optimization adjustment period to obtain an updated state information combination, and using the state vectors corresponding to the state variables in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the first matching feature combination in the previous optimization adjustment period with an end-to-end Transformer model with a second matching feature combination in the end-to-end Transformer model library; wherein the second matching feature combination is configured to have more matching features than the first matching feature combination. The second optimization adjustment action includes: reducing the number of recently added target optimization state variables in the state information combination corresponding to the Kalman filter in the previous optimization adjustment period to obtain an updated state information combination, and constructing an optimized Kalman filter using the state vectors corresponding to the state variables in the updated state information combination; and replacing the end-to-end Transformer model corresponding to the second matching feature combination in the previous optimization adjustment period with an end-to-end Transformer model with a first matching feature combination in the end-to-end Transformer model library; wherein the first matching feature combination is configured to have fewer matching features than the second matching feature combination.

8. The Kalman filter and end-to-end Transformer based multi-target tracking method of claim 7, wherein, The steps for establishing a data transmission mapping relationship between each target analysis video and its corresponding data processing server include: After updating the Kalman filter and end-to-end Transformer model for the next optimization adjustment period, obtain the number of optimized state variables of the Kalman filter and the number of matching features of the end-to-end Transformer model in the next optimization adjustment period. According to the preset conversion rules, the number of optimization state variables and the number of matching features of each target analysis video in the next optimization adjustment cycle are converted into computational load quantization values ​​and accumulated to obtain the computational load task of each target analysis video. Based on the computational load quantization value of the computational load task of each target analysis video and the computational idle quantization value of each data processing server in the next optimization and adjustment cycle, each target analysis video is assigned to the corresponding data processing server with the objective of minimizing the variance of the total computational load quantization value of the computational load tasks assigned to each data processing server. Based on the allocation relationship between the target analysis video and the corresponding data processing server, establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server.

9. The Kalman filter and end-to-end Transformer based multi-target tracking method of claim 1, wherein, The steps for obtaining the actual position information of each tracked vehicle in each tracking frame of each target analysis video, and generating the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information, specifically include: Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the regional driving trajectory of each tracked vehicle in each target analysis video according to the timestamp order of each tracking frame corresponding to each actual position information; The system summarizes the regional driving trajectories of each target analysis video in the target analysis video set, removes duplicate trajectories, and obtains the actual driving trajectory of each tracked vehicle in the target analysis video set. 10.A multi-target tracking system based on Kalman filtering and end-to-end Transformer, characterized in that, include: The acquisition module is used to acquire a target analysis video set; wherein, the target analysis video set includes target analysis videos of road vehicles traveling, collected by several cameras set up at different sections of the target road; The first detection module is used to detect reference vehicles in each reference frame of each target analysis video, generate tracking initialization parameters based on the detection results of reference vehicles in each reference frame, and initialize the Kalman filter and end-to-end Transformer model using the tracking initialization parameters. The second detection module is used to detect the tracking vehicles in each tracking frame of each target analysis video, and use a Kalman filter to predict the position of the tracking vehicles in the next tracking frame to obtain the predicted position information of each tracking vehicle in the next tracking frame. Based on the predicted position information, an end-to-end Transformer model is used to extract the tracking vehicles in the next tracking frame that meet the matching conditions with the tracking vehicles in each tracking frame, and generate the actual position information of each tracking vehicle. The update module is used to update the Kalman filter and end-to-end Transformer model for the next optimization adjustment period based on the actual and predicted position information of each tracked vehicle in each optimization adjustment period, and to establish the data transmission mapping relationship between each target analysis video and the corresponding data processing server. The generation module is used to obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

Images