Training of tracking models and methods and devices for tracking traffic objects

Abstract

This disclosure provides a method, apparatus, electronic device, and readable storage medium for training a tracking model and tracking traffic objects, relating to artificial intelligence technologies such as image processing, intelligent transportation, and deep learning. The method for training the tracking model includes: acquiring an image dataset; training a neural network model using the image dataset to obtain a detection model; acquiring a video dataset; and training the detection model using the video dataset to obtain a tracking model. The method for tracking traffic objects includes: extracting image frames to be detected from the video to be detected; inputting the image frames to be detected into the tracking model to obtain the position detection results, category detection results, and identification information of each current target in the image frames to be detected, output by the tracking model; and obtaining the tracking results of each current target in the image frames to be detected based on the position detection results, category detection results, and identification information of each current target.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the fields of artificial intelligence technology such as image processing, intelligent transportation, and deep learning. It provides a method, apparatus, electronic device, and readable storage medium for training a tracking model and tracking traffic objects. Background Technology

[0002] In road traffic scenarios, there are usually multiple targets of various types (e.g., motor vehicles, people, non-motor vehicles, etc.), with a large number of targets and a complex background.

[0003] However, existing tracking methods suffer from problems such as inability to track multiple categories and multiple targets, complex tracking logic, and low tracking efficiency. Summary of the Invention

[0004] According to a first aspect of this disclosure, a method for training a tracking model is provided, comprising: acquiring an image dataset, the image dataset including multiple first images and location and category annotations of each target in the multiple first images; training a neural network model using the multiple first images and the location and category annotations of each target in the multiple first images to obtain a detection model; acquiring a video dataset, the video dataset including multiple videos and location, category, and label annotations of each target in each second image of the multiple videos; training the detection model using each second image of the multiple videos and the location, category, and label annotations of each target in each second image to obtain a tracking model.

[0005] According to a second aspect of this disclosure, a method for tracking traffic objects is provided, comprising: extracting image frames to be detected from a video to be detected; inputting the image frames to be detected into a tracking model to obtain position detection results, category detection results, and identification information of each current target in the image frames to be detected output by the tracking model; and obtaining tracking results of each current target in the image frames to be detected based on the position detection results, category detection results, and identification information of each current target.

[0006] According to a third aspect of this disclosure, a training apparatus for a tracking model is provided, comprising: a first acquisition unit for acquiring an image dataset, the image dataset including multiple first images and location and category annotations of each target in the multiple first images; a first training unit for training a neural network model using the multiple first images and the location and category annotations of each target in the multiple first images to obtain a detection model; a second acquisition unit for acquiring a video dataset, the video dataset including multiple videos and location, category, and label annotations of each target in each second image of the multiple videos; and a second training unit for training the detection model using each second image of the multiple videos and the location, category, and label annotations of each target in each second image to obtain a tracking model.

[0007] According to a fourth aspect of this disclosure, a traffic object tracking device is provided, comprising: an extraction unit for extracting image frames to be detected from a video to be detected; a processing unit for inputting the image frames to be detected into a tracking model to obtain position detection results, category detection results, and identification information of each current target in the image frames to be detected, output by the tracking model; and a tracking unit for obtaining tracking results of each current target in the image frames to be detected based on the position detection results, category detection results, and identification information of each current target.

[0008] According to a fifth aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0009] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0010] According to a seventh aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0011] As can be seen from the above technical solutions, this embodiment obtains the tracking model through a two-stage training method. In addition to detecting the position and category of each target in the image, the tracking model can also identify the identification information of each target in the image. Furthermore, by using image datasets and video datasets to train the tracking model, the training cost of the tracking model can be reduced and the training efficiency of the tracking model can be improved.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0014] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0015] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0016] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0017] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0018] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0019] Figure 6 This is a block diagram of an electronic device used to implement the tracking model training and traffic object tracking method of the embodiments of this disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0021] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the training method of the tracking model in this embodiment specifically includes the following steps:

[0022] S101. Obtain an image dataset, wherein the image dataset includes multiple first images and the location and category annotation results of each target in the multiple first images;

[0023] S102. Train a neural network model using multiple first images and the location and category annotations of each target in the multiple first images to obtain a detection model;

[0024] S103. Obtain a video dataset, which includes multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image of the multiple videos;

[0025] S104. Train the detection model using the second images of multiple videos, the location annotation results, category annotation results, and identifier annotation results of each target in each second image, to obtain the tracking model.

[0026] The training method of the tracking model in this embodiment first uses the acquired image dataset to train a neural network model to obtain a detection model. The obtained detection model has the ability to detect the position and category of each target in the image. Then, the acquired video dataset is used to train the detection model to obtain a tracking model. The obtained tracking model can not only detect the position and category of each target in the image, but also recognize the identification information of each target in the image, thereby completing the tracking of the target. Furthermore, the method of training the tracking model using image datasets and video datasets can reduce the training cost of the tracking model and improve the training efficiency of the tracking model.

[0027] In this embodiment, the image dataset obtained by executing S101 is a dataset used for training the detection model. In the first image contained in the image dataset, the location annotation result of each target is the bounding box corresponding to each target, and the category annotation result of each target is the category to which each target belongs, such as car, pedestrian, bicycle, etc.

[0028] In this embodiment, after executing S101 to obtain an image dataset containing multiple first images and the location and category annotation results of each target in the multiple first images, S102 is executed to train a neural network model using the obtained multiple first images and the location and category annotation results of each target in the multiple first images to obtain a detection model.

[0029] In this embodiment, the neural network model trained by executing S102 can be an anchor-free detector, thereby avoiding the ambiguity problem of different anchor points corresponding to the same feature vector; in this embodiment, any type of detector can also be used as a neural network model for training.

[0030] Specifically, in this embodiment, when performing S102, the acquired multiple first images and the location and category annotation results of each target in the multiple first images are used to train the neural network model to obtain the detection model, the optional implementation method is as follows: input the multiple first images into the neural network model respectively to obtain the location prediction result and category prediction result output by the neural network model for each target in each first image; adjust the parameters of the neural network model according to the loss function value calculated based on the location prediction result, location annotation result, category prediction result and category annotation result of each target in each first image, until the neural network model converges to obtain the detection model.

[0031] In addition, when performing S102, this embodiment can also combine a multi-scale training strategy to train the detection model, that is, use first images of different sizes to train the neural network model, thereby improving the robustness of the trained detection model.

[0032] In this embodiment, after training the detection model in S102, S103 is executed to obtain the video dataset. The obtained video dataset includes multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image of the multiple videos.

[0033] In this embodiment, the video dataset obtained by executing S103 is a dataset used for training the tracking model. Each video in the video dataset consists of multiple second images. In the second images contained in the video dataset, the location annotation result of each target is the bounding box corresponding to each target, the category annotation result of each target is the category to which each target belongs, and the identification annotation result of each target is the identification information (e.g., target ID) corresponding to each target. It can be understood that the same target in the same video has a unique identification annotation result.

[0034] In this embodiment, after executing S103 to obtain a video dataset containing multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image of the multiple videos, S104 is executed to train a detection model using each second image of the multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image to obtain a tracking model.

[0035] Since video datasets are expensive to acquire, this embodiment employs a two-stage training strategy to obtain the tracking model. This reduces the requirements for video datasets and maximizes the use of more readily available image datasets. Even with a small amount of video dataset, the tracking model can be trained, thereby reducing the training cost of the tracking model.

[0036] Specifically, in this embodiment, when training the detection model using the second images of multiple videos and the location annotation, category annotation, and label annotation results of each target in each second image in S104 to obtain the tracking model, the optional implementation method is as follows: input multiple second images into the detection model respectively to obtain the location prediction result, category prediction result, and label prediction result output by the detection model for each target in each second image. In this embodiment, the multiple second images input in one training process correspond to the same video; adjust the parameters of the detection model based on the loss function value calculated according to the location prediction result, location annotation result, category prediction result, category annotation result, label prediction result, and label annotation result of each target in each second image until the detection model converges to obtain the tracking model.

[0037] In this embodiment, when executing S104, the loss function value can be calculated based on a preset calculation method. The preset calculation method can be the calculation method of Additive Angular Margin Loss.

[0038] In other words, this embodiment performs joint learning of detection and tracking tasks when training the tracking model, so that the detection can combine the features of the tracking, and the tracking can combine the features of the detection, thereby improving the performance of the trained tracking model.

[0039] Figure 2 This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2 As shown, the traffic object tracking method in this embodiment specifically includes the following steps:

[0040] S201. Extract the image frames to be detected from the video to be detected;

[0041] S202. Input the image frame to be detected into the tracking model to obtain the position detection result, category detection result and identification information of each current target in the image frame to be detected output by the tracking model;

[0042] S203. Based on the position detection results, category detection results and identification information of each current target, the tracking results of each current target in the image frame to be detected are obtained.

[0043] The traffic object tracking method in this embodiment extracts the image frames to be detected from the video to be detected, and then inputs the extracted image frames to be detected into a pre-trained tracking model. The model outputs the position detection results, category detection results, and identification information of each current target in the image frame to be detected. Then, based on the detection results and identification information of each current target, the tracking results of each current target in the image frame to be detected are obtained. This method achieves the purpose of tracking multiple targets (vehicles, pedestrians, cyclists, and other traffic objects) of different categories in the image, which simplifies the tracking logic and improves the tracking efficiency.

[0044] In this embodiment, when executing S201, the video to be detected is first acquired, and then the image frames to be detected are extracted from the acquired video to be detected. In this embodiment, when acquiring the video to be detected in S201, the video captured in real time can be used as the video to be detected, or the video that has been captured can be used as the video to be detected.

[0045] In this embodiment, after executing S201 to acquire the image frame to be detected, S202 is executed to input the acquired image frame to be detected into the tracking model, and the position detection result, category detection result and identification information of each current target in the image frame to be detected are obtained from the tracking model output.

[0046] In this embodiment, the current target's position detection result obtained by executing S202 is the bounding box corresponding to the current target, the current target's category detection result is the category to which the current target belongs, and the current target's identification information is the target identifier of the current target, such as the target ID.

[0047] In this embodiment, after executing S202 to obtain the detection results and identification information of each current target, S203 is executed to obtain the tracking results of each current target in the image frame to be detected based on the detection results and identification information of each current target.

[0048] In this embodiment, when S203 is executed, the position detection results, category detection results and identification information of each current target can be displayed in the image frame to be detected, so that the display results of each current target can be used as the tracking results of each target in the image frame to be detected.

[0049] To further improve the accuracy of target tracking, in this embodiment, when executing S203 to obtain the tracking results of each current target in the image frame to be detected based on the obtained position detection results, category detection results, and identification information of each current target, the optional implementation method can be as follows: generate a current tracking pool based on the position detection results, category detection results, and identification information of each current target in the image frame to be detected, that is, the tracking pool will record different targets and the detection results of each target; generate a historical tracking pool based on the position detection results, category detection results, and identification information of each historical target in the previous image frame of the image frame to be detected; and obtain the tracking results of each current target in the image frame to be detected based on the current tracking pool and the historical tracking pool.

[0050] In this embodiment, when executing S203 to obtain the tracking results of each current target in the image frame to be detected based on the current tracking pool and the historical tracking pool, the optional implementation method can be as follows: matching each current target in the current tracking pool with each historical target in the historical tracking pool; determining two targets that meet the preset conditions based on the matching degree, the preset conditions being that the matching degree between the two targets exceeds a preset threshold; and when it is determined that the two targets have the same identification information, using the position detection result, category detection result, and identification information of the current target in the image frame to be detected as the tracking result of the current target.

[0051] In other words, this embodiment can also combine the detection results of previous image frames to correct the tracking results of each current target in the current image frame, avoid tracking errors, and make the obtained tracking results more accurate.

[0052] In this embodiment, after determining that two targets have the same identification information, S203 may also include the following: obtaining the target features of the two targets and fusing them, wherein the target features are extracted from the targets by the tracking model and used to obtain the target's identification information; using the fusion result as the target feature of the current target in the image frame to be detected, and using the fused target features for the next target matching.

[0053] In this embodiment, when generating the tracking pool based on the position detection results, category detection results, and identification information of each current target in S203, the tracking pool can be generated after processing the detection results using the Kalman filter algorithm and then based on the processing results of the detection results of each current target and the identification information.

[0054] In this embodiment, when performing S203 to match the targets in the current tracking pool with the targets in the historical tracking pool, the Hungarian algorithm can be used to match a current target in the current tracking pool with a historical target in the historical tracking pool, and then determine whether the current target and the historical target are the same target based on the matching result.

[0055] In this embodiment, when performing S203 to match each current target in the current tracking pool with each current target in the historical tracking pool, the following may also be included: obtaining the position detection results and target features of the current target and historical targets; and performing matching between the current target and historical targets based on the obtained position detection results and target features. In other words, this embodiment uses two types of information to match targets, which can improve the accuracy of target matching.

[0056] The traffic object tracking method in this embodiment can manage parking spaces based on the tracking results of each current target in the obtained image. Since it can track multiple targets of different categories, it can improve the accuracy of parking space management.

[0057] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure. Figure 3 The diagram illustrates the traffic object tracking process in this embodiment. In addition to annotating the bounding boxes and categories of different types of targets in the image frame, it also annotates the identification information of each target, such as "10 person", "9 person", "2 car", "6 car", etc.

[0058] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4 As shown, the training device 400 for the tracking model in this embodiment includes:

[0059] The first acquisition unit 401 is used to acquire an image dataset, wherein the image dataset includes multiple first images and the location and category annotation results of each target in the multiple first images;

[0060] The first training unit 402 is used to train a neural network model using multiple first images and the location and category annotation results of each target in the multiple first images to obtain a detection model;

[0061] The second acquisition unit 403 is used to acquire a video dataset, which includes multiple videos and the location annotation results, category annotation results and identifier annotation results of each target in each second image of the multiple videos;

[0062] The second training unit 404 is used to train the detection model using each second image from multiple videos, the location annotation results, category annotation results, and identifier annotation results of each target in each second image, to obtain a tracking model.

[0063] The image dataset acquired by the first acquisition unit 401 is a dataset used for training the detection model. In the first image contained in the image dataset, the location annotation result of each target is the bounding box corresponding to each target, and the category annotation result of each target is the category to which each target belongs, such as car, pedestrian, bicycle, etc.

[0064] In this embodiment, after the first acquisition unit 401 acquires an image dataset containing multiple first images and the location and category annotation results of each target in the multiple first images, the first training unit 402 uses the acquired multiple first images and the location and category annotation results of each target in the multiple first images to train a neural network model and obtain a detection model.

[0065] The neural network model trained by the first training unit 402 can be an anchor-free detector, thereby avoiding the ambiguity problem of different anchor points corresponding to the same feature vector; in this embodiment, any type of detector can also be used as a neural network model for training.

[0066] Specifically, when the first training unit 402 trains the neural network model using the acquired multiple first images and the location and category annotation results of each target in the multiple first images to obtain the detection model, the optional implementation method is as follows: input the multiple first images into the neural network model respectively to obtain the location prediction result and category prediction result output by the neural network model for each target in each first image; adjust the parameters of the neural network model according to the loss function value calculated based on the location prediction result, location annotation result, category prediction result and category annotation result of each target in each first image until the neural network model converges to obtain the detection model.

[0067] In addition, the first training unit 402 can also combine a multi-scale training strategy to train the detection model, that is, use first images of different sizes to train the neural network model, thereby improving the robustness of the trained detection model.

[0068] In this embodiment, after the detection model is trained by the first training unit 402, the second acquisition unit 403 acquires the video dataset. The acquired video dataset includes multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image of the multiple videos.

[0069] The video dataset acquired by the second acquisition unit 403 is a dataset used for training the tracking model. Each video in the video dataset consists of multiple second images. In the second images contained in the video dataset, the location annotation result of each target is the bounding box corresponding to each target, the category annotation result of each target is the category to which each target belongs, and the identification annotation result of each target is the identification information (e.g., target ID) corresponding to each target. It can be understood that the same target in the same video has a unique identification annotation result.

[0070] In this embodiment, after the second acquisition unit 403 acquires a video dataset containing multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image of the multiple videos, the second training unit 404 uses each second image of the multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image to train a detection model and obtain a tracking model.

[0071] Specifically, when the second training unit 404 trains the detection model using the second images of multiple videos and the location annotation, category annotation, and label annotation results of each target in each second image to obtain the tracking model, the optional implementation method is as follows: input multiple second images into the detection model respectively to obtain the location prediction, category prediction, and label prediction results output by the detection model for each target in each second image; adjust the parameters of the detection model based on the loss function value calculated by the location prediction, location annotation, category prediction, category annotation, label prediction, and label annotation results of each target in each second image until the detection model converges to obtain the tracking model.

[0072] The second training unit 404 can calculate the loss function value based on a preset calculation method, which can be the calculation method of Additive Angular Margin Loss.

[0073] In other words, when training the tracking model, the second training unit 404 performs joint learning of the detection and tracking tasks, so that the detection can combine the features of the tracking, and the tracking can combine the features of the detection, thereby improving the performance of the trained tracking model.

[0074] Figure 5 This is a schematic diagram according to the fifth embodiment of this disclosure. (See diagram below.) Figure 5 As shown, the traffic object tracking device 500 of this embodiment includes:

[0075] Extraction unit 501 is used to extract the image frames to be detected from the video to be detected;

[0076] Processing unit 502 is used to input the image frame to be detected into the tracking model to obtain the position detection result, category detection result and identification information of each current target in the image frame to be detected output by the tracking model;

[0077] The tracking unit 503 is used to obtain the tracking results of each current target in the image frame to be detected based on the position detection results, category detection results and identification information of each current target.

[0078] The extraction unit 501 can first acquire the video to be detected, and then extract the image frames to be detected from the acquired video to be detected. When acquiring the video to be detected, the extraction unit 501 can use the video captured in real time as the video to be detected, or it can use the video that has been captured.

[0079] In this embodiment, after the extraction unit 501 acquires the image frame to be detected, the processing unit 502 inputs the acquired image frame to be detected into the tracking model to obtain the position detection result, category detection result and identification information of each current target in the image frame to be detected output by the tracking model.

[0080] The current target's position detection result obtained by the processing unit 502 is the bounding box corresponding to the current target, the current target's category detection result is the category to which the current target belongs, and the current target's identification information is the target identifier of the current target, such as the target ID.

[0081] In this embodiment, after the processing unit 502 obtains the detection results and identification information of each current target, the tracking unit 503 obtains the tracking results of each current target in the image frame to be detected based on the detection results and identification information of each current target.

[0082] The tracking unit 503 can display the position detection results, category detection results and identification information of each current target in the image frame to be detected, thereby using the display results of each current target as the tracking results of each target in the image frame to be detected.

[0083] To further improve the accuracy of target tracking, when the tracking unit 503 obtains the tracking results of each current target in the image frame to be detected based on the obtained position detection results, category detection results, and identification information of each current target, the optional implementation method can be as follows: generate a current tracking pool based on the position detection results, category detection results, and identification information of each current target in the image frame to be detected, that is, the tracking pool will record different targets and the detection results of each target; generate a historical tracking pool based on the position detection results, category detection results, and identification information of each historical target in the previous image frame of the image frame to be detected; and obtain the tracking results of each current target in the image frame to be detected based on the current tracking pool and the historical tracking pool.

[0084] When the tracking unit 503 obtains the tracking results of each current target in the image frame to be detected based on the current tracking pool and the historical tracking pool, the optional implementation method can be as follows: matching each current target in the current tracking pool with each historical target in the historical tracking pool; determining two targets that meet the preset conditions based on the matching degree; and taking the position detection result, category detection result and identification information of the current target in the image frame to be detected as the tracking result of the current target when it is determined that the two targets have the same identification information.

[0085] In other words, the tracking unit 503 can also combine the detection results of the previous image frames to correct the tracking results of each current target in the current image frame, so as to avoid tracking errors and make the obtained tracking results more accurate.

[0086] After determining that two targets have the same identification information, the tracking unit 503 may also include the following: acquiring the target features of the two targets and fusing them; and using the fusion result as the target feature of the current target in the image frame to be detected.

[0087] When generating a tracking pool based on the position detection results, category detection results, and identification information of each current target, the tracking unit 503 can process the detection results using the Kalman filter algorithm, and then generate the tracking pool based on the processing results of the detection results of each current target and the identification information.

[0088] When matching targets in the current tracking pool with targets in the historical tracking pool, the tracking unit 503 can use the Hungarian algorithm to match a current target in the current tracking pool with a historical target in the historical tracking pool, and then determine whether the current target and the historical target are the same target based on the matching result.

[0089] When matching each current target in the current tracking pool with each current target in the historical tracking pool, the tracking unit 503 may also include the following: acquiring the position detection results and target features of the current and historical targets; and performing matching between the current and historical targets based on the acquired position detection results and target features. In other words, the tracking unit 503 uses two types of information to match targets, which improves the accuracy of target matching.

[0090] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0091] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0092] like Figure 6 The diagram shown is a block diagram of an electronic device for training a tracking model and a traffic object tracking method according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0093] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0094] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0095] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the training of a tracking model and traffic object tracking methods. For example, in some embodiments, the training of a tracking model and traffic object tracking methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608.

[0096] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the tracking model training and traffic object tracking method described above may be performed. Alternatively, in other embodiments, computing unit 601 may be configured to perform the tracking model training and traffic object tracking method by any other suitable means (e.g., by means of firmware).

[0097] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other training and traffic object tracking device for a programmable tracking model, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0102] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the management difficulties and weak business scalability inherent in traditional physical hosts and VPS (Virtual Private Server) services. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0103] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for tracking traffic objects, comprising: Extract the image frames to be detected from the video to be detected; The image frame to be detected is input into the tracking model to obtain the position detection result, category detection result and identification information of each current target in the image frame to be detected output by the tracking model; Based on the location detection results, category detection results, and identification information of each current target, the tracking results of each current target in the image frame to be detected are obtained; The tracking model is obtained by training the detection model using an image dataset and then training the detection model using a video dataset. The step of obtaining the tracking results of each current target in the image frame to be detected based on the position detection results, category detection results, and identification information of each current target includes: The current tracking pool is generated based on the position detection results, category detection results, and identification information of each current target in the image frame to be detected; A historical tracking pool is generated based on the position detection results, category detection results, and identification information of each historical target in the previous image frame of the image frame to be detected. Based on the current tracking pool and the historical tracking pool, the tracking results of each current target in the image frame to be detected are obtained.

2. The method according to claim 1, wherein training the tracking model comprises: Obtain the image dataset, which includes multiple first images and the location and category annotations of each target in the multiple first images; The detection model is obtained by training a neural network model using multiple first images and the location and category annotations of each target in the multiple first images; Obtain the video dataset, which includes multiple videos and the location annotation, category annotation, and identifier annotation results of each target in each second image of the multiple videos; The detection model is trained using the second images of multiple videos, the location annotations, category annotations, and identifier annotations of each target in each second image, to obtain the tracking model.

3. The method according to claim 2, wherein, The step of training a neural network model using multiple first images and the location and category annotations of each target in the multiple first images to obtain the detection model includes: Multiple first images are input into the neural network model to obtain the location prediction result and category prediction result of each target in each first image output by the neural network model; Based on the loss function values ​​calculated from the location prediction results, location labeling results, category prediction results, and category labeling results of each target in each first image, the parameters of the neural network model are adjusted until the neural network model converges, thus obtaining the detection model.

4. The method according to claim 2, wherein, The detection model is trained using second images from multiple videos, the location annotations, category annotations, and identifier annotations of each target in each second image, resulting in the tracking model, which includes: Multiple second images are input into the detection model to obtain the location prediction result, category prediction result, and label prediction result of the detection model for each target in each second image; Based on the loss function values ​​calculated from the location prediction results, location annotation results, category prediction results, category annotation results, label prediction results, and label annotation results of each target in each second image, the parameters of the detection model are adjusted until the detection model converges, thus obtaining the tracking model.

5. The method according to claim 1, wherein, The step of obtaining the tracking results of each current target in the image frame to be detected based on the current tracking pool and the historical tracking pool includes: Match each current target in the current tracking pool with each historical target in the historical tracking pool; Based on the matching degree, two targets that meet the preset conditions are identified; If it is determined that the two targets have the same identification information, the position detection result, category detection result and identification information of the current target in the image frame to be detected are used as the tracking result of the current target.

6. The method according to claim 5, wherein, The step of matching each current target in the current tracking pool with each historical target in the historical tracking pool includes: Obtain the location detection results and target features of the current target and historical targets; Based on the location detection results and target features, a match is made between the current target and historical targets.

7. The method according to claim 5, further comprising: After determining that the two targets have the same identification information, the target features of the two targets are obtained and fused. The fusion result is used as the target feature of the current target in the image frame to be detected.

8. A traffic object tracking device, comprising: The extraction unit is used to extract the image frames to be detected from the video to be detected; The processing unit is used to input the image frame to be detected into the tracking model and obtain the position detection result, category detection result and identification information of each current target in the image frame to be detected output by the tracking model; The tracking unit is used to obtain the tracking results of each current target in the image frame to be detected based on the position detection results, category detection results and identification information of each current target; The tracking model is obtained by training the detection model using an image dataset and then training the detection model using a video dataset. When the tracking unit obtains the tracking results of each current target in the image frame to be detected based on the position detection results, category detection results, and identification information of each current target, it specifically performs the following: The current tracking pool is generated based on the position detection results, category detection results, and identification information of each current target in the image frame to be detected; A historical tracking pool is generated based on the position detection results, category detection results, and identification information of each historical target in the previous image frame of the image frame to be detected. Based on the current tracking pool and the historical tracking pool, the tracking results of each current target in the image frame to be detected are obtained.

9. The apparatus according to claim 8, further comprising a training device for the tracking model: The first acquisition unit is used to acquire the image dataset, which includes multiple first images and the location and category annotation results of each target in the multiple first images; The first training unit is used to train a neural network model using multiple first images, the location annotation results and category annotation results of each target in the multiple first images, to obtain the detection model; The second acquisition unit is used to acquire the video dataset, which includes multiple videos and the location annotation results, category annotation results, and identifier annotation results of each target in each second image of the multiple videos; The second training unit is used to train the detection model using each second image from multiple videos, the location annotation results, category annotation results, and identifier annotation results of each target in each second image, so as to obtain the tracking model.

10. The apparatus according to claim 9, wherein, When the first training unit trains the neural network model using multiple first images and the location and category annotations of each target in the multiple first images to obtain the detection model, it specifically performs the following: Multiple first images are input into the neural network model to obtain the location prediction result and category prediction result of each target in each first image output by the neural network model; Based on the loss function values ​​calculated from the location prediction results, location labeling results, category prediction results, and category labeling results of each target in each first image, the parameters of the neural network model are adjusted until the neural network model converges, thus obtaining the detection model.

11. The apparatus according to claim 9, wherein, When the second training unit trains the detection model using second images from multiple videos, the location annotations, category annotations, and identifier annotations of each target in each second image, and obtains the tracking model, it specifically executes the following: Multiple second images are input into the detection model to obtain the location prediction result, category prediction result, and label prediction result of the detection model for each target in each second image; Based on the loss function values ​​calculated from the location prediction results, location annotation results, category prediction results, category annotation results, label prediction results, and label annotation results of each target in each second image, the parameters of the detection model are adjusted until the detection model converges, thus obtaining the tracking model.

12. The apparatus according to claim 8, wherein, When the tracking unit obtains the tracking results of each current target in the image frame to be detected based on the current tracking pool and the historical tracking pool, it specifically performs the following: Match each current target in the current tracking pool with each historical target in the historical tracking pool; Based on the matching degree, two targets that meet the preset conditions are identified; If it is determined that the two targets have the same identification information, the position detection result, category detection result and identification information of the current target in the image frame to be detected are used as the tracking result of the current target.

13. The apparatus according to claim 12, wherein, When the tracking unit matches each current target in the current tracking pool with each historical target in the historical tracking pool, it specifically performs the following: Obtain the location detection results and target features of the current target and historical targets; Based on the location detection results and target features, a match is made between the current target and historical targets.

14. The apparatus of claim 12, wherein the tracking unit is further configured to perform, After determining that the two targets have the same identification information, the target features of the two targets are obtained and fused. The fusion result is used as the target feature of the current target in the image frame to be detected.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images