Multi-target tracking method and system based on Kalman filtering and end-to-end Transform

Abstract

The invention relates to the technical field of data processing, and discloses a multi-target tracking method and system based on Kalman filtering and an end-to-end Transformer, and the method comprises the steps: detecting a reference vehicle in a reference frame, initializing a Kalman filter and an end-to-end Transformer model, and then detecting a tracking vehicle in a tracking frame, thereby achieving the multi-target tracking. The method comprises the following steps: performing position prediction in a next tracking frame by using a Kalman filter, then extracting a tracking vehicle in the next tracking frame, which meets a matching condition with the tracking vehicle, by using an end-to-end Transformer model based on predicted position information, obtaining actual position information of each tracking vehicle, generating an actual driving track, and transmitting the actual driving track to the next tracking frame. And finally, according to the distance value between the actual position information and the predicted position information, considering the characteristic of staged change of the traffic flow of the road, realizing updating of a Kalman filter and an end-to-end Transform model and intelligent distribution of operation load tasks, improving the tracking efficiency and response speed of multi-target tracking of the road vehicle, and improving the tracking efficiency and response speed of the multi-target tracking of the road vehicle. The vehicle identification and tracking difficulty is reduced; and the tracking trajectory error rate is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer. Background Art

[0002] Road vehicle multi-target tracking is of great significance and has wide application value in intelligent transportation systems. Multi-target tracking technology can provide real-time traffic flow and vehicle trajectory information, which helps traffic management departments optimize traffic signal control, improve road traffic efficiency, and alleviate traffic congestion.

[0003] At present, the method of continuous frame image recognition and vehicle matching is usually adopted for multi-target tracking of road vehicles. However, considering the need to identify and compare each object in the frame image, there are limitations on tracking efficiency and response speed when the road traffic is heavy. In addition, as the traffic volume increases, the possibility of similar vehicles appearing in the same frame image becomes greater, which will increase the difficulty of vehicle identification and the high error rate of tracking trajectory.

[0004] Therefore, how to improve the tracking efficiency and response speed of multi-target tracking of road vehicles, reduce the difficulty of vehicle identification and tracking, and reduce the error rate of tracking trajectories is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer, aiming to solve at least one of the above technical problems.

[0006] To achieve the above object, the present invention provides a multi-target tracking method based on Kalman filtering and end-to-end Transformer, comprising the following steps: Acquire a target analysis video set; wherein the target analysis video set includes target analysis videos of road vehicles traveling collected by a plurality of cameras arranged at different sections on the target road; Detecting a reference vehicle in each reference frame in each target analysis video, generating tracking initialization parameters based on the detection results of the reference vehicle in each reference frame, and initializing a Kalman filter and an end-to-end Transformer model using the tracking initialization parameters; Detect the tracked vehicles in each tracking frame in each target analysis video, predict the position of the tracked vehicles in each tracking frame in the next tracking frame using a Kalman filter, and obtain the predicted position information of each tracked vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracked vehicles in the next tracking frame that meet the matching conditions with the tracked vehicles in each tracking frame, and generate the actual position information of each tracked vehicle; According to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment cycle, the Kalman filter and end-to-end Transformer model of the next optimization adjustment cycle are updated, and the data transmission mapping relationship between each target analysis video and the corresponding data processing server is established; The actual position information of each tracked vehicle in each target analysis video in each tracking frame is obtained, and the actual driving trajectory of each tracked vehicle in the target analysis video set is generated based on the timestamp of the tracking frame corresponding to each actual position information.

[0007] Optionally, the step of obtaining the target analysis video set includes: Accessing a road monitoring database; wherein the road monitoring database stores a plurality of road monitoring information, each road monitoring information including a collected monitoring video of a vehicle traveling on the road and a road coverage section of each monitoring video; Extracting the starting point position and the end point position in the target road for performing multi-target tracking, and matching the monitoring videos of several road coverage sections corresponding to the target road in the road monitoring database; The surveillance videos of the several road coverage sections are used as a target analysis video set, and a corresponding sequence number is assigned to each target analysis video in the target analysis video set according to the sequence of each road coverage section from the starting position to the ending position in the target road.

[0008] Optionally, detecting a reference vehicle in each reference frame in each target analysis video, and generating a tracking initialization parameter step based on the detection result of the reference vehicle in each reference frame specifically includes: The target detection algorithm is used to detect and identify the reference vehicles in each reference frame, and the initial state information combination of each reference vehicle and the number of reference vehicles in each reference frame are extracted; Based on the state variables in the initial state information combination of each reference vehicle, construct a state vector containing each state variable and a covariance matrix in which the elements on the diagonal of the diagonal matrix are configured as the variances of each state variable; wherein the initial state information combination contains N kinds of state information; Based on the reference vehicle quantity value of each reference frame, an initial matching feature combination in a matching feature set for vehicle association identification by an end-to-end Transformer model is determined in a preset comparison table of the reference vehicle quantity value and the matching feature combination; wherein the number of matching features in the initial matching feature combination is positively correlated with the reference vehicle quantity value of each reference frame.

[0009] Optionally, use the tracking initialization parameters to initialize the Kalman filter and the end-to-end Transformer model steps, including: Constructing an initial Kalman filter for each reference vehicle based on the constructed state vector and covariance matrix of each reference vehicle; According to the initial matching feature combination, the end-to-end Transformer model corresponding to the initial matching feature combination is matched in a pre-trained end-to-end Transformer model database to obtain the initial end-to-end Transformer model of the target analysis video.

[0010] Optionally, detecting a tracking vehicle in each tracking frame in each target analysis video, predicting the position of the tracking vehicle in each tracking frame in the next tracking frame using a Kalman filter, and obtaining the predicted position information of each tracking vehicle in the next tracking frame specifically includes: The target detection algorithm is used to detect and identify the tracking vehicle in each tracking frame in the target analysis video, and the state information of the initial state information combination of each tracking vehicle in each tracking frame is extracted; The state information of each tracked vehicle in each tracking frame is input into the initial Kalman filter of each reference vehicle constructed accordingly, and the position of the tracked vehicle in each tracking frame in the next tracking frame is predicted to obtain the predicted position information of each tracked vehicle in the next tracking frame.

[0011] Optionally, based on the predicted position information, using an end-to-end Transformer model to extract a tracked vehicle in a next tracking frame that meets a matching condition with a tracked vehicle in each tracking frame, and generating the actual position information of each tracked vehicle, specifically includes: Based on the predicted position information of each tracked vehicle in the next tracking frame, the predicted position information is used as the identification starting position, and similarity matching is performed with the tracked vehicles in order of the distance between each vehicle and the identification starting position in the next tracking frame from small to large; When it is detected that the first similarity is not lower than the preset similarity threshold, the matching process is stopped and the position of the currently successfully matched vehicle is used as the actual position information of each tracked vehicle.

[0012] Optionally, according to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment period, the Kalman filter and the end-to-end Transformer model steps of the next optimization adjustment period are updated, specifically including: Counting the ratio of the distance between the actual position information and the predicted position information of each tracked vehicle in each optimization adjustment cycle that exceeds a preset distance threshold, and determining whether the ratio is within an error ratio range; If yes, the actual position information and predicted position information of the next optimization adjustment cycle are determined by using the Kalman filter and the end-to-end Transformer model of the current optimization adjustment cycle; If not, determining whether the ratio is higher than the error ratio range, and if so, performing a first optimization adjustment action of the Kalman filter and the end-to-end Transformer model, and if not, performing a second optimization adjustment action of the Kalman filter and the end-to-end Transformer model; The first optimization and adjustment action includes: adding a target number of optimized state variables to the state information combination corresponding to the Kalman filter in the previous optimization and adjustment cycle to obtain an updated state information combination, and using the state vector corresponding to the state variable in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the first matching feature combination in the previous optimization and adjustment cycle with an end-to-end Transformer model with a second matching feature combination in the end-to-end Transformer model library; wherein the second matching feature combination is configured to have a greater number of matching features than the first matching feature combination; Among them, the second optimization and adjustment action includes: reducing the most recently added target number of optimized state variables in the state information combination corresponding to the Kalman filter in the previous optimization and adjustment cycle to obtain an updated state information combination, and using the state vectors corresponding to the state variables in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the second matching feature combination in the previous optimization and adjustment cycle with an end-to-end Transformer model with a first matching feature combination in the end-to-end Transformer model library; wherein the first matching feature combination is configured to have fewer matching features than the second matching feature combination.

[0013] Optionally, the step of establishing a data transmission mapping relationship between each target analysis video and a corresponding data processing server specifically includes: After updating the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle, obtaining the number of optimized state variables of the Kalman filter and the number of matching features of the end-to-end Transformer model in the next optimization adjustment cycle; According to the preset conversion rules, the number of the optimization state variables and the number of matching features of each target analysis video in the next optimization adjustment cycle are converted into a calculation load quantization value and accumulated to obtain the calculation load task of each target analysis video; According to the computational load quantization value of the computational load task of each target analysis video and the computational idle quantization value of each data processing server in the next optimization adjustment cycle, based on the computational idle quantization value and the computational load quantization value, with the goal of minimizing the variance of the total computational load quantization value of the computational load task assigned to each data processing server, each target analysis video is assigned to a corresponding data processing server; According to the allocation relationship between the target analysis video and the corresponding data processing server, a data transmission mapping relationship between each target analysis video and the corresponding data processing server is established.

[0014] Optionally, the steps of obtaining the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generating the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information, specifically include: Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the regional driving trajectory of each tracked vehicle in each target analysis video according to the time stamp sequence of each tracking frame corresponding to each actual position information; The regional driving trajectory of each target analysis video in the target analysis video set is summarized, and the duplicate trajectories are deleted to obtain the actual driving trajectory of each tracked vehicle in the target analysis video set.

[0015] In addition, in order to achieve the above object, the present invention also provides a multi-target tracking system based on Kalman filtering and end-to-end Transformer, comprising: An acquisition module is used to acquire a target analysis video set; wherein the target analysis video set includes target analysis videos of road vehicles traveling collected by a plurality of cameras arranged at different sections on the target road; A first detection module is used to detect a reference vehicle in each reference frame in each target analysis video, generate tracking initialization parameters based on the detection results of the reference vehicle in each reference frame, and use the tracking initialization parameters to initialize the Kalman filter and the end-to-end Transformer model; The second detection module is used to detect the tracked vehicles in each tracking frame in each target analysis video, predict the position of the tracked vehicles in each tracking frame in the next tracking frame by using the Kalman filter, and obtain the predicted position information of each tracked vehicle in the next tracking frame; based on the predicted position information, the end-to-end Transformer model is used to extract the tracked vehicles in the next tracking frame that meet the matching conditions with the tracked vehicles in each tracking frame, and generate the actual position information of each tracked vehicle; An update module is used to update the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle according to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment cycle, and to establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server; The generation module is used to obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0016] The beneficial effects of the present invention are as follows: a multi-target tracking method and system based on Kalman filtering and end-to-end Transformer are proposed, by detecting a reference vehicle in a reference frame in each target analysis video, initializing the Kalman filter and the end-to-end Transformer model according to the detection result of the reference vehicle, and then detecting the tracking vehicle in each tracking frame in each target analysis video, according to the detection result of the tracking vehicle, using the Kalman filter to predict the position in the next tracking frame, and then based on the predicted position information, using the end-to-end Transformer model to extract the tracking vehicle in the next tracking frame that meets the matching condition with the tracking vehicle in each tracking frame, and generating The actual position information of each tracked vehicle is used to generate the actual driving trajectory based on the actual position information of each tracked vehicle in each target analysis video. Finally, according to the predicted position information and the actual position information in each optimization adjustment cycle, the distance value between the actual position information and the predicted position information is used, and the characteristics of the phased changes in road traffic flow and the impact of traffic flow changes on the accuracy of vehicle predicted position and vehicle identification difficulty are considered. The Kalman filter and end-to-end Transformer model of the next optimization adjustment cycle are updated, and the computing load tasks are intelligently allocated, thereby improving the tracking efficiency and response speed of multi-target tracking of road vehicles, reducing the difficulty of vehicle identification and tracking, and reducing the error rate of tracking trajectories. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a multi-target tracking method based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention; Figure 2It is a structural schematic diagram of a multi-target tracking system based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention.

[0018] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] The embodiment of the present invention provides a multi-target tracking method based on Kalman filtering and end-to-end Transformer, referring to Figure 1 , Figure 1 The figure is a flow chart of a multi-target tracking method based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention.

[0021] In this embodiment, the multi-target tracking method based on Kalman filtering and end-to-end Transformer includes the following steps: S1: Obtain a target analysis video set; wherein the target analysis video set includes target analysis videos of road vehicles traveling collected by a number of cameras set at different sections on the target road; S2: Detect a reference vehicle in each reference frame in each target analysis video, generate tracking initialization parameters based on the detection results of the reference vehicle in each reference frame, and use the tracking initialization parameters to initialize the Kalman filter and the end-to-end Transformer model; S3: Detect the tracking vehicle in each tracking frame in each target analysis video, predict the position of the tracking vehicle in each tracking frame in the next tracking frame using a Kalman filter, and obtain the predicted position information of each tracking vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracking vehicle in the next tracking frame that meets the matching condition with the tracking vehicle in each tracking frame, and generate the actual position information of each tracking vehicle; S4: According to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment cycle, the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle are updated, and a data transmission mapping relationship between each target analysis video and the corresponding data processing server is established; S5: Obtain the actual position information of each tracked vehicle in each target analysis video in each tracking frame, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0022] It should be noted that, at present, the method of continuous frame image recognition and vehicle matching is usually adopted for multi-target tracking of road vehicles. However, considering the need to identify and compare each object in the frame image, there are limitations on tracking efficiency and response speed when the road traffic volume is heavy. Moreover, as the traffic volume increases, the possibility of similar vehicles appearing in the same frame image becomes greater, which will increase the difficulty of vehicle identification and the high error rate of tracking trajectory.

[0023] In order to solve the above problems, this embodiment detects the reference vehicle in the reference frame in each target analysis video, initializes the Kalman filter and the end-to-end Transformer model according to the detection result of the reference vehicle, then detects the tracking vehicle in each tracking frame in each target analysis video, and uses the Kalman filter to predict the position in the next tracking frame according to the detection result of the tracking vehicle, and then uses the end-to-end Transformer model to extract the tracking vehicle in the next tracking frame that meets the matching condition with the tracking vehicle in each tracking frame based on the predicted position information, and generates the actual position information of each tracking vehicle. Based on the actual position information of each tracking vehicle in each target analysis video, the actual driving trajectory is generated, and finally, according to the predicted position information and the actual position information in each optimization adjustment period, according to the distance value between the actual position information and the predicted position information, the characteristics of the staged change of road traffic flow and the influence of the change of traffic flow on the accuracy of the predicted position of the vehicle and the difficulty of vehicle identification are considered, the Kalman filter and the end-to-end Transformer model of the next optimization adjustment period are updated, and the intelligent allocation of the computing load task is realized, so as to improve the tracking efficiency and response speed of multi-target tracking of road vehicles, reduce the difficulty of vehicle identification and tracking, and reduce the error rate of tracking trajectory.

[0024] In a preferred embodiment, the step of obtaining a target analysis video set specifically includes: S11: accessing a road monitoring database; wherein the road monitoring database stores a plurality of road monitoring information, each of which includes a collected monitoring video of a vehicle traveling on the road and a road coverage section of each monitoring video; S12: extracting the starting point position and the end point position in the target road for performing multi-target tracking, and matching the monitoring videos of several road coverage sections corresponding to the target road in the road monitoring database; S13: taking the surveillance videos of the plurality of road coverage sections as a target analysis video set, and assigning a corresponding sequence number to each target analysis video in the target analysis video set according to the sequence of each road coverage section from a starting position to an end position in the target road.

[0025] In this embodiment, the target analysis video set is obtained by accessing a road monitoring database, extracting the starting position and the end position in the target road, finding several monitoring videos covering the target road in the road monitoring database, and then sequentially numbering each monitoring video according to the sequence of sections from the starting position to the end position, and finally obtaining the target analysis video set. In this way, it is convenient to obtain the overall tracking trajectory of the vehicle on the target road by summarizing multiple trajectories after subsequent vehicle tracking for each target analysis video.

[0026] In a preferred embodiment, detecting a reference vehicle in each reference frame in each target analysis video, and generating a tracking initialization parameter step based on the detection result of the reference vehicle in each reference frame specifically includes: S21: Detect and identify the reference vehicles in each reference frame using a target detection algorithm, and extract the initial state information combination of each reference vehicle and the number of reference vehicles in each reference frame; S22: constructing a state vector including each state variable and a covariance matrix in which the elements on the diagonal of the diagonal matrix are configured as variances of each state variable based on the state variables in the initial state information combination of each reference vehicle; wherein the initial state information combination includes N kinds of state information; S23: Based on the reference vehicle quantity value of each reference frame, determine the initial matching feature combination in the matching feature set for vehicle association identification by the end-to-end Transformer model in a preset comparison table of the reference vehicle quantity value and the matching feature combination; wherein the number of matching features in the initial matching feature combination is positively correlated with the reference vehicle quantity value of each reference frame.

[0027] On this basis, the Kalman filter and end-to-end Transformer model steps are initialized using tracking initialization parameters, including: S24: constructing an initial Kalman filter for each reference vehicle according to the constructed state vector and covariance matrix of each reference vehicle; S25: According to the initial matching feature combination, the end-to-end Transformer model corresponding to the initial matching feature combination is matched in a pre-trained end-to-end Transformer model database to obtain the initial end-to-end Transformer model of the target analysis video.

[0028] In this embodiment, a target detection algorithm is first used to detect and identify a reference vehicle in a reference frame (the first N image frames in each detection cycle are reference frames, and the remaining image frames are tracking frames), and then a state vector and a covariance matrix are constructed based on an initial state information group of the reference vehicle (the state variables in the initial state information group include the vehicle position, the vehicle speed in the X and Y directions, etc.), and then a Kalman filter is constructed. An initial matching feature combination is determined based on the reference vehicle quantity value of the reference vehicle (the matching features in the initial matching feature combination include vehicle size, color distribution, window features, wheel features, roof features, etc.), and an end-to-end Transformer model corresponding to the initial matching feature combination is matched in a pre-trained end-to-end Transformer model database, thereby obtaining a Kalman filter and an end-to-end Transformer model for performing initial vehicle tracking.

[0029] In a preferred embodiment, the steps of detecting a tracked vehicle in each tracking frame in each target analysis video, predicting the position of the tracked vehicle in each tracking frame in the next tracking frame using a Kalman filter, and obtaining the predicted position information of each tracked vehicle in the next tracking frame specifically include: S31: using a target detection algorithm to detect and identify the tracking vehicle in each tracking frame in the target analysis video, and extracting state information of the initial state information combination of each tracking vehicle in each tracking frame; S32: Input the state information of each tracked vehicle in each tracking frame into the initial Kalman filter of each reference vehicle constructed accordingly, predict the position of the tracked vehicle in each tracking frame in the next tracking frame, and obtain the predicted position information of each tracked vehicle in the next tracking frame.

[0030] On this basis, based on the predicted position information, an end-to-end Transformer model is used to extract the tracked vehicles in the next tracking frame that meet the matching conditions with the tracked vehicles in each tracking frame, and the actual position information of each tracked vehicle is generated, which specifically includes: S33: Based on the predicted position information of each tracked vehicle in the next tracking frame, the predicted position information is used as the identification starting position, and similarity matching is performed with the tracked vehicles in order of the distance between each vehicle and the identification starting position in the next tracking frame from small to large; S34: When it is detected that the first similarity is not lower than the preset similarity threshold, the matching process is stopped and the position of the currently successfully matched vehicle is used as the actual position information of each tracked vehicle.

[0031] In this embodiment, after obtaining the Kalman filter and the end-to-end Transformer model, the Kalman filter is first used to predict the position of the tracked vehicle in the tracking frame in the next tracking frame to obtain the predicted position information. Then, based on the predicted position information, the end-to-end Transformer model is used to perform vehicle similarity matching in the order of the distance between each vehicle and the predicted position information in the next tracking frame from small to large, until the first vehicle whose similarity is not less than the preset similarity threshold is detected, and it is used as the actual position of the tracked vehicle in the next tracking frame. Thus, starting from the predicted position information and performing similarity matching in order of distance, it has higher recognition and tracking efficiency than the traditional similarity matching for each vehicle in the entire image frame.

[0032] In a preferred embodiment, the steps of updating the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle according to the actual position information and the predicted position information of each tracked vehicle in each optimization adjustment cycle specifically include: S41: Counting the ratio of the distance between the actual position information and the predicted position information of each tracked vehicle in each optimization adjustment period exceeding a preset distance threshold, and determining whether the ratio is within an error ratio range; S42: If yes, the actual position information and predicted position information of the next optimization adjustment cycle are determined by using the Kalman filter and the end-to-end Transformer model of the current optimization adjustment cycle; S43: If not, determine whether the ratio is higher than the error ratio range, if so, perform a first optimization adjustment action of the Kalman filter and the end-to-end Transformer model, if not, perform a second optimization adjustment action of the Kalman filter and the end-to-end Transformer model; The first optimization and adjustment action includes: adding a target number of optimized state variables to the state information combination corresponding to the Kalman filter in the previous optimization and adjustment cycle to obtain an updated state information combination, and using the state vector corresponding to the state variable in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the first matching feature combination in the previous optimization and adjustment cycle with an end-to-end Transformer model with a second matching feature combination in the end-to-end Transformer model library; wherein the second matching feature combination is configured to have a greater number of matching features than the first matching feature combination; Among them, the second optimization and adjustment action includes: reducing the most recently added target number of optimized state variables in the state information combination corresponding to the Kalman filter in the previous optimization and adjustment cycle to obtain an updated state information combination, and using the state vectors corresponding to the state variables in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the second matching feature combination in the previous optimization and adjustment cycle with an end-to-end Transformer model with a first matching feature combination in the end-to-end Transformer model library; wherein the first matching feature combination is configured to have fewer matching features than the second matching feature combination.

[0033] On this basis, the steps of establishing the data transmission mapping relationship between each target analysis video and the corresponding data processing server include: S44: after updating the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle, obtaining the number of optimized state variables of the Kalman filter and the number of matching features of the end-to-end Transformer model in the next optimization adjustment cycle; S45: According to a preset conversion rule, the number of the optimization state variables and the number of matching features of each target analysis video in the next optimization adjustment cycle are converted into a computational load quantization value and accumulated to obtain a computational load task for each target analysis video; S46: according to the computation load quantization value of the computation load task of each target analysis video and the computation idle quantization value of each data processing server in the next optimization adjustment cycle, based on the computation idle quantization value and the computation load quantization value, with the goal of minimizing the variance of the total computation load quantization value of the computation load task assigned to each data processing server, each target analysis video is assigned to a corresponding data processing server; S47: According to the allocation relationship between the target analysis video and the corresponding data processing server, a data transmission mapping relationship between each target analysis video and the corresponding data processing server is established.

[0034] In this embodiment, according to the predicted position information and the actual position information in each optimization and adjustment cycle, based on the distance value between the actual position information and the predicted position information, the characteristics of the phased changes in road traffic flow and the impact of traffic flow changes on the accuracy of vehicle predicted position and vehicle identification difficulty are considered, and the Kalman filter and end-to-end Transformer model of the next optimization and adjustment cycle are updated and the computing load tasks are intelligently allocated, thereby improving the tracking efficiency and response speed of road vehicle multi-target tracking, reducing the difficulty of vehicle identification and tracking, and reducing the tracking trajectory error rate.

[0035] In a preferred embodiment, the steps of obtaining the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generating the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information, specifically include: S51: Acquire the actual position information of each tracked vehicle in each target analysis video in each tracking frame, and generate the regional driving trajectory of each tracked vehicle in each target analysis video according to the time stamp sequence of each tracking frame corresponding to the actual position information; S52: Summarize the regional driving trajectory of each target analysis video in the target analysis video set, delete duplicate trajectories, and obtain the actual driving trajectory of each tracked vehicle in the target analysis video set.

[0036] In this embodiment, a Kalman filter is used to predict the position in the next tracking frame, and then based on the predicted position information, an end-to-end Transformer model is used to extract the tracked vehicles in the next tracking frame that meet the matching conditions with the tracked vehicles in each tracking frame, and the actual position information of each tracked vehicle is generated. Based on the actual position information of each tracked vehicle in each target analysis video, the vehicle is tracked for each target analysis video respectively, and then the overall tracking trajectory of the vehicle on the target road is obtained by summarizing multiple trajectories to generate the actual driving trajectory of each tracked vehicle.

[0037] Reference Figure 2 , Figure 2 It is a structural schematic diagram of a multi-target tracking system based on Kalman filtering and end-to-end Transformer according to an embodiment of the present invention.

[0038] like Figure 2 As shown, the multi-target tracking system based on Kalman filtering and end-to-end Transformer proposed in an embodiment of the present invention includes: The acquisition module 10 is used to acquire a target analysis video set; wherein the target analysis video set includes target analysis videos of road vehicles traveling collected by a plurality of cameras arranged at different sections on the target road; A first detection module 20 is used to detect a reference vehicle in each reference frame in each target analysis video, generate a tracking initialization parameter based on the detection result of the reference vehicle in each reference frame, and use the tracking initialization parameter to initialize the Kalman filter and the end-to-end Transformer model; The second detection module 30 is used to detect the tracked vehicle in each tracking frame in each target analysis video, predict the position of the tracked vehicle in each tracking frame in the next tracking frame by using a Kalman filter, and obtain the predicted position information of each tracked vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracked vehicle in the next tracking frame that meets the matching condition with the tracked vehicle in each tracking frame, and generate the actual position information of each tracked vehicle; An updating module 40 is used to update the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle according to the actual position information and the predicted position information of each tracked vehicle in each optimization adjustment cycle, and to establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server; The generation module 50 is used to obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

[0039] Other embodiments or specific implementations of the multi-target tracking system based on Kalman filtering and end-to-end Transformer of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0040] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0041] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0042] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-target tracking method based on Kalman filtering and end-to-end Transformer, characterized in that: The following steps are involved: Acquire a target analysis video set; wherein the target analysis video set includes target analysis videos of road vehicles traveling collected by a plurality of cameras arranged at different sections on the target road; Detecting a reference vehicle in each reference frame in each target analysis video, generating tracking initialization parameters based on the detection results of the reference vehicle in each reference frame, and initializing a Kalman filter and an end-to-end Transformer model using the tracking initialization parameters; Detect the tracked vehicles in each tracking frame in each target analysis video, predict the position of the tracked vehicles in each tracking frame in the next tracking frame using a Kalman filter, and obtain the predicted position information of each tracked vehicle in the next tracking frame; based on the predicted position information, use an end-to-end Transformer model to extract the tracked vehicles in the next tracking frame that meet the matching conditions with the tracked vehicles in each tracking frame, and generate the actual position information of each tracked vehicle; According to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment cycle, the Kalman filter and end-to-end Transformer model of the next optimization adjustment cycle are updated, and the data transmission mapping relationship between each target analysis video and the corresponding data processing server is established; The actual position information of each tracked vehicle in each target analysis video in each tracking frame is obtained, and the actual driving trajectory of each tracked vehicle in the target analysis video set is generated based on the timestamp of the tracking frame corresponding to each actual position information.

2. The multi-target tracking method based on Kalman filtering and end-to-end Transformer according to claim 1, characterized in that: The steps for obtaining the target analysis video set include: Accessing a road monitoring database; wherein the road monitoring database stores a plurality of road monitoring information, each road monitoring information including a collected monitoring video of a vehicle traveling on the road and a road coverage section of each monitoring video; Extracting the starting point position and the end point position in the target road for performing multi-target tracking, and matching the monitoring videos of several road coverage sections corresponding to the target road in the road monitoring database; The surveillance videos of the several road coverage sections are used as a target analysis video set, and a corresponding sequence number is assigned to each target analysis video in the target analysis video set according to the sequence of each road coverage section from the starting position to the ending position in the target road.

3. The multi-target tracking method based on Kalman filtering and end-to-end Transformer according to claim 1, characterized in that: The step of detecting a reference vehicle in each reference frame in each target analysis video and generating a tracking initialization parameter based on the detection result of the reference vehicle in each reference frame specifically includes: The target detection algorithm is used to detect and identify the reference vehicles in each reference frame, and the initial state information combination of each reference vehicle and the number of reference vehicles in each reference frame are extracted; Based on the state variables in the initial state information combination of each reference vehicle, construct a state vector containing each state variable and a covariance matrix in which the elements on the diagonal of the diagonal matrix are configured as the variances of each state variable; wherein the initial state information combination contains N kinds of state information; Based on the reference vehicle quantity value of each reference frame, an initial matching feature combination in a matching feature set for vehicle association identification by an end-to-end Transformer model is determined in a preset comparison table of the reference vehicle quantity value and the matching feature combination; wherein the number of matching features in the initial matching feature combination is positively correlated with the reference vehicle quantity value of each reference frame.

4. The multi-target tracking method based on Kalman filtering and end-to-end Transformer as claimed in claim 3, characterized in that: The steps to initialize the Kalman filter and end-to-end Transformer model using tracking initialization parameters include: Constructing an initial Kalman filter for each reference vehicle based on the constructed state vector and covariance matrix of each reference vehicle; According to the initial matching feature combination, the end-to-end Transformer model corresponding to the initial matching feature combination is matched in a pre-trained end-to-end Transformer model database to obtain the initial end-to-end Transformer model of the target analysis video.

5. The multi-target tracking method based on Kalman filtering and end-to-end Transformer as claimed in claim 4, characterized in that: The steps of detecting the tracking vehicle in each tracking frame in each target analysis video, predicting the position of the tracking vehicle in each tracking frame in the next tracking frame by using a Kalman filter, and obtaining the predicted position information of each tracking vehicle in the next tracking frame specifically include: The target detection algorithm is used to detect and identify the tracking vehicle in each tracking frame in the target analysis video, and the state information of the initial state information combination of each tracking vehicle in each tracking frame is extracted; The state information of each tracked vehicle in each tracking frame is input into the initial Kalman filter of each reference vehicle constructed accordingly, and the position of the tracked vehicle in each tracking frame in the next tracking frame is predicted to obtain the predicted position information of each tracked vehicle in the next tracking frame.

6. The multi-target tracking method based on Kalman filtering and end-to-end Transformer as claimed in claim 5, characterized in that: Based on the predicted position information, an end-to-end Transformer model is used to extract the tracked vehicles in the next tracking frame that meet the matching condition with the tracked vehicles in each tracking frame, and the actual position information of each tracked vehicle is generated, specifically comprising: Based on the predicted position information of each tracked vehicle in the next tracking frame, the predicted position information is used as the identification starting position, and similarity matching is performed with the tracked vehicles in order of the distance between each vehicle and the identification starting position in the next tracking frame from small to large; When it is detected that the first similarity is not lower than the preset similarity threshold, the matching process is stopped and the position of the currently successfully matched vehicle is used as the actual position information of each tracked vehicle.

7. The multi-target tracking method based on Kalman filtering and end-to-end Transformer according to claim 6, characterized in that: According to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment cycle, the Kalman filter and end-to-end Transformer model steps of the next optimization adjustment cycle are updated, including: Counting the ratio of the distance between the actual position information and the predicted position information of each tracked vehicle in each optimization adjustment cycle that exceeds a preset distance threshold, and determining whether the ratio is within an error ratio range; If yes, the actual position information and predicted position information of the next optimization adjustment cycle are determined by using the Kalman filter and the end-to-end Transformer model of the current optimization adjustment cycle; If not, determining whether the ratio is higher than the error ratio range, and if so, performing a first optimization adjustment action of the Kalman filter and the end-to-end Transformer model, and if not, performing a second optimization adjustment action of the Kalman filter and the end-to-end Transformer model; The first optimization and adjustment action includes: adding a target number of optimized state variables to the state information combination corresponding to the Kalman filter in the previous optimization and adjustment cycle to obtain an updated state information combination, and using the state vector corresponding to the state variable in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the first matching feature combination in the previous optimization and adjustment cycle with an end-to-end Transformer model with a second matching feature combination in the end-to-end Transformer model library; wherein the second matching feature combination is configured to have a greater number of matching features than the first matching feature combination; Among them, the second optimization and adjustment action includes: reducing the most recently added target number of optimized state variables in the state information combination corresponding to the Kalman filter in the previous optimization and adjustment cycle to obtain an updated state information combination, and using the state vectors corresponding to the state variables in the updated state information combination to construct an optimized Kalman filter; and replacing the end-to-end Transformer model corresponding to the second matching feature combination in the previous optimization and adjustment cycle with an end-to-end Transformer model with a first matching feature combination in the end-to-end Transformer model library; wherein the first matching feature combination is configured to have fewer matching features than the second matching feature combination.

8. The multi-target tracking method based on Kalman filtering and end-to-end Transformer as claimed in claim 7, characterized in that: The steps of establishing a data transmission mapping relationship between each target analysis video and the corresponding data processing server specifically include: After updating the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle, obtaining the number of optimized state variables of the Kalman filter and the number of matching features of the end-to-end Transformer model in the next optimization adjustment cycle; According to the preset conversion rules, the number of the optimization state variables and the number of matching features of each target analysis video in the next optimization adjustment cycle are converted into a calculation load quantization value and accumulated to obtain the calculation load task of each target analysis video; According to the computational load quantization value of the computational load task of each target analysis video and the computational idle quantization value of each data processing server in the next optimization adjustment cycle, based on the computational idle quantization value and the computational load quantization value, with the goal of minimizing the variance of the total computational load quantization value of the computational load task assigned to each data processing server, each target analysis video is assigned to a corresponding data processing server; According to the allocation relationship between the target analysis video and the corresponding data processing server, a data transmission mapping relationship between each target analysis video and the corresponding data processing server is established.

9. The multi-target tracking method based on Kalman filtering and end-to-end Transformer according to claim 1, characterized in that: The steps of obtaining the actual position information of each tracked vehicle in each tracking frame in each target analysis video and generating the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information include: Obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the regional driving trajectory of each tracked vehicle in each target analysis video according to the time stamp sequence of each tracking frame corresponding to each actual position information; The regional driving trajectory of each target analysis video in the target analysis video set is summarized, and the duplicate trajectories are deleted to obtain the actual driving trajectory of each tracked vehicle in the target analysis video set.

10. A multi-target tracking system based on Kalman filtering and end-to-end Transformer, characterized in that: include: An acquisition module is used to acquire a target analysis video set; wherein the target analysis video set includes target analysis videos of road vehicles traveling collected by a plurality of cameras arranged at different sections on the target road; A first detection module is used to detect a reference vehicle in each reference frame in each target analysis video, generate tracking initialization parameters based on the detection results of the reference vehicle in each reference frame, and use the tracking initialization parameters to initialize the Kalman filter and the end-to-end Transformer model; The second detection module is used to detect the tracked vehicles in each tracking frame in each target analysis video, predict the position of the tracked vehicles in each tracking frame in the next tracking frame by using the Kalman filter, and obtain the predicted position information of each tracked vehicle in the next tracking frame; based on the predicted position information, the end-to-end Transformer model is used to extract the tracked vehicles in the next tracking frame that meet the matching conditions with the tracked vehicles in each tracking frame, and generate the actual position information of each tracked vehicle; An update module is used to update the Kalman filter and the end-to-end Transformer model of the next optimization adjustment cycle according to the actual position information and predicted position information of each tracked vehicle in each optimization adjustment cycle, and to establish a data transmission mapping relationship between each target analysis video and the corresponding data processing server; The generation module is used to obtain the actual position information of each tracked vehicle in each tracking frame in each target analysis video, and generate the actual driving trajectory of each tracked vehicle in the target analysis video set based on the timestamp of the tracking frame corresponding to each actual position information.

Images