A traffic scene-based target tracking method and system

Abstract

The application discloses a traffic scene-based target tracking method and system, and relates to the field of intelligent traffic management.The method comprises the following steps: achieving target tracking by using a denoising learning process in a diffusion model, using visual memory information and search area information as conditional input, introducing a noise bounding box corresponding to the target to predict the real position of the target, and decomposing the denoising process into multiple denoising blocks to improve the prediction efficiency of the model and achieve real-time tracking, so that the denoising process is implemented once in each denoising block, and finally, accurate target position prediction is achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic management, and in particular to a target tracking method and system based on traffic scenarios. Background Technology

[0002] In recent years, high-position video technology has developed rapidly. By installing high-position video cameras on the roadside to collect data and using visual algorithms to analyze the collected data in a comprehensive manner, it is beneficial to achieve more refined and accurate management of roadside parking. In addition, it can provide traffic management departments with more evidence of vehicle violations and irregular driving, and enable more intelligent application and management of the entire traffic flow, thus playing a positive role in promoting various aspects of urban traffic management and driving safety.

[0003] Among them, target tracking algorithms have important and wide-ranging application value in traffic scenarios, covering multiple aspects such as traffic monitoring, intelligent transportation systems, traffic safety, data analysis, emergency response, smart city construction, and public transportation optimization. Through target tracking technology, the level of intelligence in traffic management can be improved, traffic safety can be enhanced, traffic resource allocation can be optimized, and strong support can be provided for the sustainable development of urban transportation. For example, in roadside high-position video surveillance scenarios, through target tracking technology, traffic management departments can monitor targets such as vehicles and pedestrians on the road in real time, promptly detect abnormal situations such as traffic congestion and accidents, collect traffic flow statistics, help analyze traffic conditions, optimize traffic light control strategies, and improve road traffic efficiency.

[0004] Currently, target tracking methods can be broadly categorized into two types: one is to directly predict the bounding box of the target based on image features; the other is to use a coordinate autoregressive framework. The image feature-based tracking method heavily relies on the matching result between the template and the search region, and cannot utilize location priors. The autoregressive method, on the other hand, must use the bounding boxes from the previous frame to train the model, which means it can only utilize existing bounding boxes in the training set, resulting in low target tracking efficiency. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a target tracking method and system based on traffic scenarios, which can solve the problem of low efficiency in existing target tracking methods.

[0006] To achieve the above objectives, the present invention provides a target tracking method based on traffic scenarios, the method comprising:

[0007] Obtain the visual memory information and search area information corresponding to the target based on the previous frame image and the current frame image;

[0008] The noise bounding box corresponding to the target is obtained based on the true bounding box and Gaussian noise.

[0009] The location information of the target is obtained by using a pre-set extended model based on the visual memory information and search area information corresponding to the target, as well as the noise box corresponding to the target.

[0010] Further, the step of obtaining the noise bounding box corresponding to the target based on the true bounding box and Gaussian noise includes:

[0011] According to the formula Obtain the noise bounding box corresponding to the target, whereby the noise bounding box is defined as X. I The true bounding box is defined as X0, and the Gaussian noise is ∈, ∈ ∈ N(0, I). Indicates noise scheduling parameters, α j =1-β j ,β j ∈(0,1) is the noise scheduling parameter, and T represents the time step.

[0012] Further, the step of obtaining the location information corresponding to the target by using a pre-set extended model based on the visual memory information and search region information corresponding to the target, as well as the noise box corresponding to the target, includes:

[0013] According to the formula Obtain the location information corresponding to the target, where L represents a total of L denoised blocks, j represents the j-th denoised block, ∈ j This represents the noise predicted for the j-th denoised block.

[0014] Furthermore, the step of obtaining the visual memory information and search region information corresponding to the target based on the previous frame image and the current frame image includes:

[0015] The previous frame image is cropped to obtain the visual memory information corresponding to the target, and the current frame image is cropped to obtain the search area information corresponding to the target.

[0016] Furthermore, the method also includes:

[0017] According to the formula and The preset extended model is updated, wherein, l cls Here, CE is the classification loss function, and CE is the cross-entropy loss function. The ID category represents the predicted output, L represents the true value, the predicted bounding box is denoted as P, and the labeled bounding box is denoted as G.

[0018] Furthermore, the present invention provides a target tracking system based on traffic scenarios, the system comprising:

[0019] The acquisition module is used to acquire visual memory information and search area information corresponding to the target based on the previous frame image and the current frame image;

[0020] The acquisition module is further configured to acquire the noise bounding box corresponding to the target based on the true bounding box and Gaussian noise corresponding to the target;

[0021] The prediction module is used to obtain the location information of the target based on the visual memory information and search region information corresponding to the target and the noise box corresponding to the target through a preset extended model.

[0022] Furthermore, the acquisition module is specifically used to obtain information according to the formula. Obtain the noise bounding box corresponding to the target, whereby the noise bounding box is defined as X. I The true bounding box is defined as X0, and the Gaussian noise is ∈, ∈ ∈ N(0, I). Indicates noise scheduling parameters, α j =1-β j ,β j ∈(0,1) is the noise scheduling parameter, and T represents the time step.

[0023] Furthermore, the prediction module is specifically used to predict according to the formula. Obtain the location information corresponding to the target, where L represents a total of L denoised blocks, j represents the j-th denoised block, ∈ j This represents the noise predicted for the j-th denoised block.

[0024] Furthermore, the acquisition module is specifically used to crop the previous frame image to obtain the visual memory information corresponding to the target, and to crop the current frame image to obtain the search region information corresponding to the target.

[0025] Furthermore, the acquisition module is also used to obtain information according to the formula. and The preset extended model is updated, wherein, l cls Here, CE is the classification loss function, and CE is the cross-entropy loss function. The ID category represents the predicted output, L represents the true value, the predicted bounding box is denoted as P, and the labeled bounding box is denoted as G.

[0026] This invention provides a target tracking method and system based on traffic scenarios. It achieves target tracking by utilizing the denoising learning process in a diffusion model. It uses visual memory information and search region information as conditional inputs, and introduces the noisy bounding box corresponding to the target to predict the true position of the target. In order to improve the model prediction efficiency and achieve real-time tracking, the denoising process is decomposed into multiple denoising blocks, and each denoising block performs a denoising process once, ultimately achieving accurate target position prediction. Attached Figure Description

[0027] Figure 1 This is a flowchart of a target tracking method based on a traffic scenario provided by the present invention;

[0028] Figure 2 This is a schematic diagram of a target tracking system based on a traffic scenario provided by the present invention. Detailed Implementation

[0029] The device structure and implementation of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0030] This invention provides a target tracking method based on traffic scenarios, such as... Figure 1 As shown, the specific steps include the following:

[0031] 101. Obtain the visual memory information and search area information corresponding to the target based on the previous frame image and the current frame image.

[0032] Specifically, the previous frame image is cropped to obtain the visual memory information corresponding to the target, and the current frame image is cropped to obtain the search region information corresponding to the target. The input includes the visual memory, the search region, and a noise box with added noise. The visual memory stores the matching template, which is cropped based on the previous frame image; the search region is cropped based on the current frame and includes areas where the target may exist.

[0033] For embodiments of the present invention, step 101 may further include: constructing a target tracking dataset for traffic scenes. Specifically, the target tracking dataset includes the target category, the target ID, and the target's two-dimensional bounding box information. The present invention constructs the dataset in two ways: one is by collecting data manually using a visual sensor; the other is by using open-source datasets, including but not limited to KITTI and MOT Challenge datasets. For collected data, video image data of traffic scenes is obtained from high-position video cameras by setting up sensor devices such as high-position video cameras, and target tracking and annotation are performed on target types such as vehicles, pedestrians, and non-motorized vehicles, including the category, ID, and two-dimensional bounding box of each target. For open-source data, the present invention mainly uses the KITTI dataset, which provides bounding boxes and ID annotation information for targets such as vehicles, pedestrians, and bicycles, covering traffic road scene data under various lighting, weather, and traffic conditions.

[0034] 102. Obtain the noise bounding box corresponding to the target based on the true bounding box and Gaussian noise.

[0035] Specifically, according to the formula Obtain the noise bounding box corresponding to the target, whereby the noise bounding box is defined as X. I The true bounding box is defined as X0, and the Gaussian noise is ∈, ∈ ∈ N(0, I). Indicates noise scheduling parameters, α j =1-β j ,β j ∈(0,1) is the noise scheduling parameter, and T represents the time step.

[0036] 103. Obtain the location information corresponding to the target by using a pre-set extended model based on the visual memory information and search area information corresponding to the target and the noise box corresponding to the target.

[0037] Specifically, according to the formula Obtain the location information corresponding to the target, where L represents a total of L denoised blocks, j represents the j-th denoised block, ∈ j This represents the noise predicted for the j-th denoised block.

[0038] For example, a noisy bounding box can be used as input, with visual memory and the search region as conditional inputs; specifically, Gaussian noise is added to the ground truth bounding box to obtain the noisy bounding box, defined as... Where * indicates the addition of noise; This represents the xy coordinates of the top-left corner of the bounding box. This represents the x and y coordinates of the bottom right corner of the bounding box;

[0039] Specifically, the noisy bounding boxes are mapped to a C-dimensional space using a word embedding model to obtain the noisy box embedding representation X. I ∈R 4xC C represents dimension; specifically, word embedding is a technique in Natural Language Processing (NLP) used to map words to a low-dimensional continuous vector space; then, an image embedding model is used to map the template and search region in visual memory to a high-dimensional space, resulting in a template embedding representation. and search embedding representation Where, N z This indicates the size of the template embedding representation. n represents the number of templates, H z W z Indicates the height and width of the template image; H s W s Indicates the height and width of the search image;

[0040] Furthermore, the input to the denoised block includes the noise box embedding representation X. I and search embedding representation The attention mechanism receives its input via a linear layer, including the query value of the noise box. Search key value k s Search value v s Then, a denoising attention mechanism is used for the first denoising step, specifically as follows: d represents the dimension value, which is used to avoid excessively large values ​​when calculating the dot product, which could lead to gradient instability. Then, a feedforward neural network layer (FFN) is used to enhance the denoising attention output X′. I Specifically, it is represented as: X′ I =Attention denoising (s,X I )+X I ,X″ I =X′ I +FFN(X′ I ), X′ I X″ represents the output after applying the denoising attention operation. I This represents the enhanced output using a feedforward neural network (FFN); finally, two linear layers are used for noise prediction to perform a second round of denoising, specifically expressed as: ∈=NoisePred(X″ I =Linear(ReLU(Linear(X″)) I In this context, Linear() represents a linear layer, ReLU represents an activation function, and NoisePred represents a noise prediction module, which consists of a linear layer and an activation function; ∈ represents the predicted noise; multiple denoising blocks can be set, defined as L blocks, and the denoised bounding box embedding representation X0 after L denoising blocks is represented as:

[0041] Where, L represents a total of L denoised blocks, j represents the j-th denoised block, ∈ j This represents the noise predicted for the j-th denoised block;

[0042] Furthermore, according to the formula and The preset extended model is updated, wherein, l cls Here, CE is the classification loss function, and CE is the cross-entropy loss function. The ID category represents the predicted output, L represents the true value, the predicted bounding box is denoted as P, and the labeled bounding box is denoted as G.

[0043] This invention provides a target tracking method based on traffic scenarios. The method achieves target tracking by utilizing the denoising learning process in a diffusion model. It uses visual memory information and search area information as conditional inputs, and introduces the noise bounding box corresponding to the target to predict the true position of the target. In order to improve the model prediction efficiency and achieve real-time tracking, the denoising process is decomposed into multiple denoising blocks, and each denoising block performs a denoising process once, ultimately achieving accurate target position prediction.

[0044] As Figure 1 The specific implementation of the method shown in this invention provides a target tracking system based on traffic scenarios, such as... Figure 2 As shown, the system includes: an acquisition module 21, used to acquire visual memory information and search area information corresponding to the target based on the previous frame image and the current frame image;

[0045] The acquisition module 21 is further configured to acquire the noise bounding box corresponding to the target based on the true bounding box corresponding to the target and the Gaussian noise;

[0046] Prediction module 22 is used to obtain the location information corresponding to the target by using a preset extended model based on the visual memory information and search area information corresponding to the target and the noise box corresponding to the target.

[0047] Furthermore, the acquisition module 21 is specifically used to obtain information according to the formula. Obtain the noise bounding box corresponding to the target, whereby the noise bounding box is defined as X. I The true bounding box is defined as X0, and the Gaussian noise is ∈, ∈ ∈ N(0, I). Indicates noise scheduling parameters, α j =1-β j ,β j ∈(0,1) is the noise scheduling parameter, and T represents the time step.

[0048] Furthermore, the prediction module 22 is specifically used to predict according to the formula Obtain the location information corresponding to the target, where L represents a total of L denoised blocks, j represents the j-th denoised block, ∈ j This represents the noise predicted for the j-th denoised block.

[0049] Furthermore, the acquisition module 21 is specifically used to crop the previous frame image to obtain the visual memory information corresponding to the target, and to crop the current frame image to obtain the search area information corresponding to the target.

[0050] Furthermore, the acquisition module 21 is also used to obtain information according to the formula. and The preset extended model is updated, wherein, l cls Here, CE is the classification loss function, and CE is the cross-entropy loss function. The ID category represents the predicted output, L represents the true value, the predicted bounding box is denoted as P, and the labeled bounding box is denoted as G.

[0051] This invention provides a target tracking system based on a traffic scenario. The system achieves target tracking by utilizing the denoising learning process in a diffusion model. It uses visual memory information and search area information as input conditions, and introduces the noise bounding box corresponding to the target to predict the target's true location. In order to improve the model's prediction efficiency and achieve real-time tracking, the denoising process is decomposed into multiple denoising blocks, and each denoising block performs a denoising process once, ultimately achieving accurate target location prediction.

[0052] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0053] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0054] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0055] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0056] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0057] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0058] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0059] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0060] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target tracking method based on traffic scenarios, characterized in that, The method includes: Obtain the visual memory information and search area information corresponding to the target based on the previous frame image and the current frame image; The noise bounding box corresponding to the target is obtained based on the true bounding box and Gaussian noise. The location information of the target is obtained by using a pre-set extended model based on the visual memory information and search area information corresponding to the target, as well as the noise box corresponding to the target. The step of obtaining the noise bounding box corresponding to the target based on the true bounding box and Gaussian noise includes: According to the formula = + Obtain the noise bounding box corresponding to the target. The noise bounding box is defined as follows: The true bounding box is defined as Gaussian noise is , (0,I), Indicates noise scheduling parameters, = , =1- , (0,1) represents the noise scheduling parameter, and T represents the time step. The step of obtaining the location information of the target by means of a pre-set extended model based on the visual memory information and search region information corresponding to the target and the noise box corresponding to the target includes: According to the formula = Obtain the location information corresponding to the target, where L represents the total number of denoised blocks, and j represents the j-th denoised block. This represents the noise predicted for the j-th denoised block.

2. The target tracking method based on a traffic scenario according to claim 1, characterized in that, The steps of obtaining the visual memory information and search region information corresponding to the target based on the previous frame image and the current frame image include: The previous frame image is cropped to obtain the visual memory information corresponding to the target, and the current frame image is cropped to obtain the search area information corresponding to the target.

3. The target tracking method based on a traffic scenario according to claim 1, characterized in that, The method further includes: According to the formula and The preset extended model is updated, wherein, Here, CE is the classification loss function, and CE is the cross-entropy loss function. The ID category represents the predicted output, L represents the true value, the predicted bounding box is denoted as P, and the labeled bounding box is denoted as G.

4. A target tracking system based on traffic scenarios, characterized in that, The system includes: The acquisition module is used to acquire visual memory information and search area information corresponding to the target based on the previous frame image and the current frame image; The acquisition module is further configured to acquire the noise bounding box corresponding to the target based on the true bounding box and Gaussian noise corresponding to the target; The prediction module is used to obtain the location information of the target based on the visual memory information and search region information corresponding to the target and the noise box corresponding to the target through a preset extended model; The acquisition module is specifically used to obtain information according to the formula. = + Obtain the noise bounding box corresponding to the target. The noise bounding box is defined as follows: The true bounding box is defined as Gaussian noise is , (0,I), Indicates noise scheduling parameters, = , =1- , (0,1) represents the noise scheduling parameter, and T represents the time step. The prediction module is specifically used to predict based on the formula. = Obtain the location information corresponding to the target, where L represents the total number of denoised blocks, and j represents the j-th denoised block. This represents the noise predicted for the j-th denoised block.

5. A target tracking system based on a traffic scenario according to claim 4, characterized in that, The acquisition module is further configured to crop the previous frame image to obtain visual memory information corresponding to the target, and to crop the current frame image to obtain search region information corresponding to the target.

6. A target tracking system based on a traffic scenario according to claim 4, characterized in that, The acquisition module is also used to obtain information according to the formula. and The preset extended model is updated, wherein, Here, CE is the classification loss function, and CE is the cross-entropy loss function. The ID category represents the predicted output, L represents the true value, the predicted bounding box is denoted as P, and the labeled bounding box is denoted as G.

Images