Vehicle tracking method, system and device in complex road scene and medium

Abstract

The invention discloses a vehicle tracking method, system and device in a complex road scene and a medium. The method comprises the following steps: acquiring a current road image; the current road image is input into the trained vehicle tracking network, vehicle tracking is carried out according to the result output by the vehicle tracking network, and the method, the system, the equipment and the medium are not prone to the problems of false detection, missing detection and tracking trajectory interruption of the target vehicle.

Description

Technical Field

[0001] The present invention belongs to the field of multi-target tracking, and relates to a vehicle tracking method, system, equipment and medium in a complex road scene. Background Art

[0002] Car travel is the main way for people to travel, which brings with it various problems such as the increase in the number of cars, vehicle management and traffic safety. For example, in the complex road traffic environment, the number of illegal behaviors such as vehicle driving violations, license plate duplication, vehicle theft, traffic accidents and hit-and-run has increased, making it increasingly difficult to manually identify vehicles and follow up on cases in a timely manner. A large number of cameras are installed in modern public transportation systems. The video monitoring and capture of the cameras can record a large amount of vehicle driving information at all times. It is imperative to monitor vehicles through target detection and tracking technology, which can effectively control traffic conditions and reduce driving time and accident rates.

[0003] In this context, vehicle tracking algorithms not only need to have efficient target detection capabilities to accurately identify each vehicle in dense traffic. To achieve this goal, researchers continue to explore and integrate a variety of advanced technologies, such as deep learning, computer vision, and optimization algorithms, to build a more robust and accurate vehicle tracking system. Redmon et al. proposed that YOLO divides the image into N*N sub-regions, and each sub-region predicts and outputs the probability, category, and location of a target, which greatly improves the detection speed and reduces the number of parameters and calculations. Although the YOLO series has achieved remarkable results in target detection, there are still some problems, such as anchor box prior design, anchor box overlap, and a large number of redundant anchor boxes. Convolutional neural networks (CNNs) can automatically learn and extract target features in video frames, and then perform target state prediction and data association, which greatly improves accuracy and robustness. Compared with single target tracking, multi-target tracking needs to consider the characteristics and motion information of multiple targets at the same time, and distinguish and match targets. Therefore, multi-target tracking requires more efficient and accurate algorithms to achieve.

[0004] In real road scenes, the surrounding environment is complex and changeable. There are factors such as large variations in target scales, similar contours of target types, and easy occlusion between targets. These factors make it difficult for target detection algorithms to detect targets, resulting in problems such as false detection and missed detection of target vehicles and interruption of tracking trajectories. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a vehicle tracking method, system, device and medium in a complex road scene, which is not prone to the problems of false detection, missed detection and interruption of tracking trajectory of target vehicles.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] In one aspect, the present invention provides a vehicle tracking method in a complex road scene, comprising:

[0008] Get the current road image;

[0009] The current road image is input into the trained vehicle tracking network, and the vehicle is tracked according to the result output by the vehicle tracking network.

[0010] Furthermore, before inputting the current road image into the trained vehicle tracking network, the process further includes:

[0011] Constructing a vehicle tracking network, wherein the vehicle tracking network includes an input layer, a detection network, a dynamic convolutional network, an attention mechanism layer, and an output layer;

[0012] The vehicle tracking network is trained to obtain a trained vehicle tracking network.

[0013] Furthermore, the detection network is improved based on the YOLOv8 recognition network and is called YOLOv8-LID.

[0014] Furthermore, the calculations in the self-attention mechanism layer are represented as square matrix calculations.

[0015] In a second aspect of the present invention, the present invention provides a vehicle tracking system in a complex road scene, comprising:

[0016] An acquisition module is used to acquire the current road image;

[0017] The tracking module is used to input the current road image into the trained vehicle tracking network and perform vehicle tracking according to the result output by the vehicle tracking network.

[0018] Furthermore, it also includes:

[0019] A construction module, used to construct a vehicle tracking network, wherein the vehicle tracking network includes an input layer, a detection network, a dynamic convolutional network, an attention mechanism layer and an output layer;

[0020] The training module is used to train the vehicle tracking network to obtain a trained vehicle tracking network.

[0021] Furthermore, the detection network is improved based on the YOLOv8 recognition network and is called YOLOv8-LID.

[0022] Furthermore, the calculations in the self-attention mechanism layer are represented as square matrix calculations.

[0023] In a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the vehicle tracking method in the complex road scene when executing the computer program.

[0024] In a fourth aspect of the present invention, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle tracking method in the complex road scene are implemented.

[0025] The present invention has the following beneficial effects:

[0026] During specific operation, the vehicle tracking method, system, device and medium in a complex road scene described in the present invention input the current road image into a trained vehicle tracking network, perform vehicle tracking based on the output result of the vehicle tracking network, and utilize the vehicle tracking network to achieve vehicle tracking, thereby solving the problems of false detection, missed detection and interruption of tracking trajectory of the target vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0028] Figure 1 is a flow chart of the method of the present invention;

[0029] Figure 2 The structure diagram of the vehicle tracking network in the present invention;

[0030] Figure 3 This is the architecture diagram of the Improve-SPPF structure;

[0031] Figure 4 This is the effect diagram of the simulation experiment. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0034] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0036] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0037] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0040] Embodiment 1

[0041] It should be noted that the present invention is based on the YOLOv8 detection algorithm and the BoT-SORT tracking algorithm, introduces a lightweight attention mechanism (Light-attention), enhances feature representation, and enables the model to better pay attention to each target in the image; uses the spatial pyramid algorithm (Improve-SPPF) of the global receptive field to enhance the model's ability to capture global information, so as to enhance its detection accuracy for objects of different sizes; introduces a dynamic convolutional network (DCNV4) to improve the model's adaptability to deformation, thereby improving the model's detection ability in complex scenes, and achieving accurate and continuous tracking of multiple vehicles in complex road scenes. In addition, a vehicle data set in complex road scenes is established, and training and verification are performed on the data set.

[0042] refer to Figures 1 to 3 Specifically, the vehicle tracking method in a complex road scene of the present invention includes:

[0043] 1) Build a vehicle tracking network;

[0044] The vehicle tracking network includes an input layer, a detection network, a dynamic convolutional network, an attention mechanism layer and an output layer.

[0045] The detection network is built based on the YOLOv8 algorithm. The improved network structure is:

[0046] The backbone feature extraction network of the YOLOv8 algorithm uses a series of convolution and deconvolution layers, and uses residual connections and bottleneck structures to reduce the size of the network and improve performance. This part uses the C2f module as the basic building block. Compared with the C3 module, the C2f module has fewer parameters and better feature extraction capabilities.

[0047] In addition, in response to the phenomenon that the scale of vehicle targets varies greatly, the spatial pyramid algorithm (Improve-SPPF) with global receptive field is used to more effectively fuse the local receptive field and the global receptive field, enhance the model's ability to capture global information, and more accurately locate and identify targets of different sizes.

[0048] Spatial Pyramid Pooling (SPP) divides the input feature map into grids of different sizes, performs pooling operations in each grid, and finally concatenates all pooling results as feature representation, thereby achieving effective processing of images of different scales. Fast Spatial Pyramid Pooling (SPPF) reduces the number of parameters through a serial structure while maintaining the multi-scale feature aggregation capability of SPP. Large Separable Kernel Attention Mechanism (LSKA) uses large and separable convolution kernels and spatial dilated convolutions to capture the extensive contextual information of the image, generate an attention map, and weight the original features through the attention map, thereby enhancing the network's attention to important features and improving the performance of the model. In the Improve-SPPF structure, the LSKA module is added after all maximum pooling layer operations are completed and before the second convolution layer.

[0049] The YOLOv8 recognition network consists of two parts: the neck network Neck and the head network Detect_head. The Neck part is responsible for multi-scale feature fusion, which enhances the feature representation capability by fusing feature maps from different stages of the Backbone. The Head part is responsible for the final target detection and classification tasks.

[0050] In addition, to improve the accuracy of the detection stage, a more efficient attention mechanism, Light-attention, is proposed by referring to the self-attention mechanism in the transformer architecture. It is added after the last convolution layer of the backbone to improve the global modeling ability of the model and enhance the feature representation, so that the model can better pay attention to each target in the image.

[0051] The self-attention mechanism layer is a type of attention mechanism, but unlike the traditional attention mechanism, the self-attention mechanism does not rely on external information, but only learns feature representation through information within the sequence.

[0052] The calculation in the original self-attention mechanism is represented as a square matrix calculation to reduce the amount of calculation in the model's self-attention process, reduce the calculation overhead, and improve the model training speed. First, the input features are subjected to sine and cosine operations to complete the position encoding, and three sets of vectors, query, key, and value, are generated based on the position encoding and input features, as shown in formulas (1), (2), and (3):

[0053] query=x+position#(1)

[0054] key=x+position#(2)

[0055] value=x#(3)

[0056] Among them, x represents the original feature and position represents the position encoding.

[0057] Calculate all input query vectors, key vectors, and value vectors, and the generated results are recorded as query matrix Q, key matrix K, and value matrix V.

[0058] Since the position encoding can be regarded as a sine-cosine operation on the input feature map, Q, K, V can be directly expressed as the weighted result of the input feature X, that is:

[0059] Q=XW Q #(4)

[0060] K=XW K #(5)

[0061] V=XW V #(6)

[0062] Among them, X represents the input feature map, and W Q ,W K ,W V They represent weight vectors respectively.

[0063] Original attention formula:

[0064]

[0065] Where T is the total number of keys, d k is the dimension of the key vector, which is used to scale the dot product attention to prevent numerical instability during the calculation process; the softmax function is used to convert the original attention score into a probability distribution, that is, the attention weight.

[0066] So we can convert the original self-attention formula into It is expressed as:

[0067]

[0068] in,

[0069]

[0070] That is, the original self-attention mechanism formula can be simplified as:

[0071] attention=softmax(XWX T )V#(10)

[0072] Modeling the matrix operations in the original self-attention mechanism as square matrix operations greatly reduces the number of parameters in the calculation process of the self-attention mechanism and reduces the computational overhead in the model training process.

[0073] In addition, the present invention introduces a dynamic convolutional network (DCNv4) to improve the model's adaptability to deformation, thereby enhancing the model's detection capability in complex scenarios.

[0074] 2) training the vehicle tracking network to obtain a trained vehicle tracking network;

[0075] Specifically, 21) create a vehicle tracking dataset in complex road scenes, and classify vehicles into three categories: car, bus, and truck according to different target types;

[0076] 22) Building a backbone network in a multi-target tracking network to extract and learn features of the targets in the vehicle data set in step 21);

[0077] 23) Building a detection network in the multi-target tracking network for further detection and identification of the features extracted in step 22);

[0078] 24) Building a tracking model (dynamic convolutional network) in a multi-target tracking network to track the targets detected in step 23);

[0079] 25) Using the vehicle data set to train the vehicle tracking network to obtain a trained vehicle tracking network.

[0080] 3) Obtain the current road image;

[0081] 4) Inputting the current road image into the trained vehicle tracking network, and performing vehicle tracking according to the result output by the vehicle tracking network.

[0082] Simulation experiment

[0083] In this experiment, the collected images are derived from real road surveillance videos. The collected surveillance videos are extracted into frames, and each frame is preprocessed, such as resizing and normalization, to meet the input requirements of the detection model. LabelImg is used to annotate vehicles and divide them into three categories: car, bus, and truck. The annotated frame images are divided into three categories: train (training set), val (validation set), and test (test set) in a ratio of 8:1:1. The samples in the train file are used to train the model, the samples in the val file are used to verify the model, and the samples in the test file are used to test the model.

[0084] During the training process, the training epoch is 300, and the confidence threshold is set to 0.5, which is the network's confidence in the detection frame. When performing vehicle tracking tasks, the vehicle tracking algorithm has multiple detections of the same vehicle. The NMS algorithm is used to make the algorithm only get one detection frame for each vehicle. When the training is completed, an optimal weight file is obtained, which saves the important parameters for vehicle tracking.

[0085] The test analysis indicators are: IDF1, IDP, IDR and MOTA.

[0086] The present invention was tested using Python 3.8 on the Ubuntu operating platform. The specific software and hardware environment is shown in Table 1:

[0087] Table 1

[0088]

[0089] Table 2

[0090]

[0091] Table 2 shows the experimental results. Figure 4 , indicating that the detection accuracy of the present invention is high.

[0092] Embodiment 2

[0093] The vehicle tracking system in a complex road scene of the present invention comprises:

[0094] An acquisition module is used to acquire the current road image;

[0095] The tracking module is used to input the current road image into the trained vehicle tracking network and perform vehicle tracking according to the result output by the vehicle tracking network.

[0096] As an embodiment of the present invention, it also includes:

[0097] A construction module, used to construct a vehicle tracking network, wherein the vehicle tracking network includes an input layer, a detection network, a dynamic convolutional network, an attention mechanism layer and an output layer;

[0098] The training module is used to train the vehicle tracking network to obtain a trained vehicle tracking network.

[0099] As an implementation mode of the present invention, the detection network is improved based on the YOLOv8 recognition network, and is called YOLOv8-LID.

[0100] As an implementation mode of the present invention, the calculation in the self-attention mechanism layer is represented as square matrix calculation.

[0101] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0102] Embodiment 3

[0103] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the vehicle tracking method under the complex road scene are implemented. The memory may include a memory, such as a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk memory, etc. The processor, the network interface, and the memory are interconnected through an internal bus. The internal bus may be an industrial standard architecture bus, a peripheral component interconnection standard bus, an extended industrial standard structure bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs. Specifically, the program may include a program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0104] Embodiment 4

[0105] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle tracking method in a complex road scene are implemented. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1A step that specifies a function in one or more boxes.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A vehicle tracking method in a complex road scene, characterized in that: include: Get the current road image; The current road image is input into the trained vehicle tracking network, and the vehicle is tracked according to the result output by the vehicle tracking network.

2. The vehicle tracking method in complex road scenes according to claim 1, characterized in that: Before inputting the current road image into the trained vehicle tracking network, the following steps are also included: Constructing a vehicle tracking network, wherein the vehicle tracking network includes an input layer, a detection network, a dynamic convolutional network, an attention mechanism layer, and an output layer; The vehicle tracking network is trained to obtain a trained vehicle tracking network.

3. The vehicle tracking method in complex road scenes according to claim 2, characterized in that: The detection network is improved based on the YOLOv8 recognition network and is called YOLOv8-LID.

4. The vehicle tracking method in complex road scenes according to claim 2, characterized in that: The calculations in the self-attention mechanism layer are represented as square matrix calculations.

5. A vehicle tracking system in a complex road scene, characterized in that: include: An acquisition module is used to acquire the current road image; The tracking module is used to input the current road image into the trained vehicle tracking network and perform vehicle tracking according to the result output by the vehicle tracking network.

6. The vehicle tracking system in complex road scenes according to claim 5, characterized in that: Also includes: A construction module, used to construct a vehicle tracking network, wherein the vehicle tracking network includes an input layer, a detection network, a dynamic convolutional network, an attention mechanism layer and an output layer; The training module is used to train the vehicle tracking network to obtain a trained vehicle tracking network.

7. The vehicle tracking system in complex road scenes according to claim 6, characterized in that: The detection network is improved based on the YOLOv8 recognition network and is called YOLOv8-LID.

8. The vehicle tracking system in complex road scenes according to claim 6, characterized in that: The calculations in the self-attention mechanism layer are represented as square matrix calculations.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the vehicle tracking method in a complex road scene as described in any one of claims 1-4 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the vehicle tracking method in a complex road scene as claimed in any one of claims 1 to 4 are implemented.

Images