Cross-camera multi-vehicle tracking method combining weak cues and boundary expansion clustering

By combining weak cues and boundary expansion clustering, a similarity matrix is ​​constructed using the bounding box area and appearance features of vehicle target detection boxes. This solves the problems of low trajectory feature quality and poor clustering performance in multi-vehicle tracking systems across cameras in highway scenarios, and achieves more accurate vehicle trajectory matching and global trajectory generation.

CN118570259BActive Publication Date: 2024-11-05HANGZHOU PIXEL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411035233.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-05
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

In highway scenarios, existing technologies face problems such as low trajectory feature quality and poor trajectory clustering performance in multi-vehicle tracking systems with multiple cameras. Especially in high-speed environments and dimly lit tunnels, vehicle features are difficult to match accurately.

Method used

We employ a weak cue and boundary expansion clustering approach, combining the bounding box area of ​​the vehicle target detection box as a weak cue and the appearance feature as a strong cue to construct a similarity matrix. Then, we perform trajectory clustering using the boundary expansion clustering algorithm, taking into account camera order and scene characteristics.

Benefits of technology

It improves the quality of trajectory features and clustering performance, ensuring accurate matching of vehicle trajectories and generation of global trajectories in high-speed and tunnel environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118570259B_ABST
    Figure CN118570259B_ABST
Patent Text Reader

Abstract

The present invention provides a cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering. The scheme uses the vehicle appearance features of each vehicle target detection frame as a strong clue and the boundary frame area of ​​each vehicle target detection frame as a weak clue, and combines the weak clues and the strong clues to obtain the similarity matrix of multi-vehicle tracking trajectories of different cameras; the similarity matrix is ​​clustered by using boundary extension clustering according to the shooting order of the cameras to obtain the vehicle tracking trajectory of each vehicle, so as to provide high-quality trajectory features and high-performance trajectory clustering.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cross-camera multi-vehicle tracking, and in particular to a high-speed cross-camera multi-vehicle tracking method that uses weak cues to enhance vehicle representation and combines strong appearance cues to enhance the similarity matrix, and considers scene characteristics and camera order to associate and reorder tracks using a boundary extension clustering method. Background Art

[0002] Multi-target multi-camera tracking (MTMCT) system is an important component of intelligent transportation system, which aims to match vehicles across multiple cameras and generate global trajectories. The MTMCT process includes several key steps. First, vehicle detection is performed to extract the vehicle from the camera frame. Second, vehicle re-identification (ReID) is used to extract the ReID features of the detected vehicles. Subsequently, trajectories are generated within each camera based on the vehicle detection and ReID features. Finally, the vehicle trajectories from different cameras are clustered to obtain the global trajectory and generate the final result. Trajectory clustering is a key step in generating global trajectories and plays a vital role in achieving the superior performance of the MTMCT system. Although some excellent works have been proposed, applying the MTMCT system to highway scenarios still faces the following challenges:

[0003] 1) Low-quality trajectory features: In the MTMCT system, the trajectory is represented as a collection of ReID features that capture the movement of a vehicle in a specific time period. However, the quality of trajectory features may be disturbed due to factors such as the high-speed movement of vehicles in high-speed environments and dim lighting conditions in tunnels;

[0004] 2) Low-performance trajectory clustering: When clustering vehicle trajectories, clustering algorithms are often used to match trajectories from different cameras. However, when a vehicle passes through different cameras, its observed trajectory may be affected by multiple factors such as camera viewing angle and traffic conditions;

[0005] In summary, the current cross-camera multi-vehicle tracking technology used in urban streets is difficult to be directly applied to the cross-camera multi-vehicle tracking scenario on highways. Summary of the invention

[0006] This scheme provides a cross-camera multi-vehicle tracking method that combines weak cues and boundary extension clustering. This method combines the enhanced similarity matrix of strong cues and weak cues, and considers the scene characteristics and camera order of the boundary extension clustering method to associate the trajectories of cross-matching tracked multi-vehicle trajectories to achieve multi-camera multi-vehicle tracking on highways.

[0007] To achieve the above invention objectives, in a first aspect, the present invention provides a cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering, comprising the following steps:

[0008] Get tracking video streams from multiple cameras on the highway;

[0009] Input the video frame of the tracking video of each camera into the pre-trained vehicle target detection module to obtain the vehicle target detection frame, input the vehicle target detection frame image corresponding to the vehicle target detection frame into the pre-trained feature extraction module to output the ReID feature and the bounding box area of ​​the vehicle target frame, fuse the ReID feature to obtain the vehicle appearance feature, input the vehicle appearance feature and the corresponding vehicle target detection frame into the BoT-SORT target tracking module to output the multi-vehicle tracking trajectory of each camera, take the vehicle appearance feature of each vehicle target detection frame as the strong clue, take the bounding box area of ​​each vehicle target detection frame as the weak clue, and combine the weak clues and the strong clues to obtain the similarity matrix of the multi-vehicle tracking trajectories of different cameras;

[0010] The similarity matrix is ​​clustered by using boundary extension clustering according to the shooting order of the cameras to obtain the vehicle tracking trajectory of each vehicle.

[0011] Secondly, this solution provides a cross-camera multi-vehicle tracking system that implements the same technical content by combining weak clues and boundary extension clustering.

[0012] On the third aspect, the present solution provides an electronic device that implements the same technical content.

[0013] In a fourth aspect, the present solution provides a readable storage medium that implements the same technical solution.

[0014] Compared with the prior art, this scheme has the following characteristics and beneficial effects: 1. This scheme adopts the area size of the vehicle target detection box boundary box as a weak clue and the vehicle appearance features as a strong clue, and integrates the strong clues and weak clues to form a new feature representation of the vehicle at time T, and constructs a new similarity matrix to represent the similarity between calculated trajectories. In this way, the problem that the quality of trajectory features may be disturbed due to factors such as the high-speed movement of vehicles in high-speed environments and the dim lighting conditions in tunnels is solved, thereby improving the quality of the track; 2. This scheme adopts the boundary extension clustering method to generate new trajectories, comprehensively considering the scene characteristics and camera sequence, and improving the clustering quality by clustering adjacent cameras in pairs and then clustering them with the trajectories near them. At the same time, the boundary extension clustering method also considers the influence of the boundary of the scene section on the clustering, thereby improving the quality of trajectory clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a baseline flowchart of the cross-camera multi-vehicle tracking method combining weak clues and boundary expansion clustering according to the present scheme;

[0016] Figure 2is a flowchart of a cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering according to the present solution;

[0017] Figure 3 is a flowchart of a boundary extension clustering (BEC) of a cross-camera multi-vehicle tracking method combining weak cues and boundary extension clustering according to the present scheme;

[0018] Figure 4 This is a pseudo code diagram of a cross-camera multi-vehicle tracking method combining weak cues and boundary expansion clustering;

[0019] Figure 5 This is a schematic diagram of the structure of the cross-camera multi-vehicle tracking system combining weak clues and boundary extension clustering in this scheme;

[0020] Figure 6 It is a schematic diagram of the hardware structure of the electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0022] Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0023] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0024] This scheme proposes a cross-camera multi-vehicle tracking method that combines weak clues and boundary extension clustering. The video frames taken by the camera on the highway are input into the YOLOV8 vehicle target detection model to output the vehicle target detection frame, and then the vehicle frame detection image corresponding to the vehicle target detection frame is input into the ReID model to extract four ReID features, and the four vehicle ReID features are fused to form a comprehensive and robust vehicle appearance feature; the vehicle target detection frame obtained based on the vehicle target detection model and the comprehensive and robust vehicle appearance feature representation are sent to the BoT-SORT target tracking module to obtain the multi-vehicle tracking trajectory under each camera; the similarity between trajectories is calculated based on the vehicle appearance features to form a similarity matrix. In addition, the bounding box size of the vehicle target detection frame should also be regarded as a weak clue. As an effective component of the similarity matrix, the area of ​​the bounding box is combined as a weak clue and the vehicle appearance feature as a strong clue to obtain a similarity matrix that integrates weak clues and strong clues; finally, the boundary extension clustering stage. Each step of clustering selects adjacent cameras for clustering. When the clustering of the trajectories of adjacent cameras is completed, the trajectories of adjacent clusters will be selected, and the clustering process will be repeated until the global trajectory is obtained.

[0025] Embodiment 1 A cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering;

[0026] Figure 1 The baseline flowchart of the proposed method for cross-camera multi-vehicle tracking combining weak cues and boundary extension clustering is provided. Figure 2 The logical flow chart of the cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering is provided. Figure 1 and Figure 2 It can be seen that the cross-camera multi-vehicle tracking method provided by this scheme combining weak clues and boundary extension clustering includes the following steps:

[0027] Get tracking video streams from multiple cameras on the highway;

[0028] Input the video frame of the tracking video of each camera into the pre-trained vehicle target detection module to obtain the vehicle target detection frame, input the vehicle target detection frame image corresponding to the vehicle target detection frame into the pre-trained feature extraction module to output the ReID feature and the bounding box area of ​​the vehicle target frame, fuse the ReID feature to obtain the vehicle appearance feature, input the vehicle appearance feature and the corresponding vehicle target detection frame into the BoT-SORT target tracking module to output the multi-vehicle tracking trajectory of each camera, take the vehicle appearance feature of each vehicle target detection frame as the strong clue, take the bounding box area of ​​each vehicle target detection frame as the weak clue, and combine the weak clues and the strong clues to obtain the similarity matrix of the multi-vehicle tracking trajectories of different cameras;

[0029] The similarity matrix is ​​clustered by using boundary extension clustering according to the shooting order of the cameras to obtain the vehicle tracking trajectory of each vehicle.

[0030] About obtaining the vehicle target detection frame:

[0031] The vehicle target detection module selected in this solution is the YOLOV8 model. The video frames of the tracking video of each camera are input into the pre-trained YOLOV8 model to output the vehicle target detection frame. Specifically, the video frame is input into the YOLOV8 model to obtain a multi-scale feature map through multi-scale extraction mechanism analysis. The multi-scale feature map is adaptively fused to obtain fusion features. The fusion features are input into the classification network and the bounding box prediction network to output the vehicle target detection frame, where the target detection frame information includes the vehicle type, the target frame confidence, and the vehicle target detection frame image corresponding to the current vehicle target detection frame.

[0032] Specifically, the video frames of the tracking video of each camera are uniformly cropped according to a predetermined size and then input into a pre-trained vehicle target detection module, wherein the vehicle target detection module is trained with video frames marked with vehicle target detection frames.

[0033] It should be noted that the target frame information of the vehicle target detection frame output by the vehicle target detection module includes the vehicle category (car, truck, bus) and the confidence level. The corresponding vehicle target detection frame image can be disassembled according to the vehicle target detection frame.

[0034] About feature extraction module:

[0035] The feature extraction module of this solution can perform feature extraction on each vehicle target detection frame image to output ReID features and the bounding box area of ​​the vehicle target frame, wherein the ReID feature is output by inputting the vehicle target detection frame image into the ReID model, and the bounding box area of ​​the vehicle target frame is calculated based on the area of ​​the vehicle target detection frame.

[0036] The ReID model of this scheme includes four ReID networks with different backbones. The vehicle target detection frame image is input into the ReID networks with different backbones, and the corresponding ReID features are obtained through feature extraction. The different ReID features are fused to obtain the final robust vehicle appearance features.

[0037] Taking into account the large number of cameras on the highway, which may easily lead to an excessive number of video frames, this scheme uses vehicle target detection frame images to be batch-transferred into different backbone ReID networks to extract the ReID features understood by each network to capture the multi-perspective appearance representation of the vehicle, thereby enriching the diversity of ReID features. Subsequently, this scheme fuses multiple ReID features to form a comprehensive representation of the vehicle's appearance features. The vehicle appearance features have stronger robustness, can more comprehensively describe the vehicle's appearance features, and fuse the ReID features of different focus points for overall representation.

[0038] In a specific embodiment of the present solution, the ReID model includes four ReID networks with different backbones, namely, IBN-densenet169, IBN-ResNet101, Resnet50, and IBN-Se-ResNet101. Correspondingly, the four ReID networks are fused to obtain the vehicle appearance features.

[0039] It should be noted that the vehicle appearance features are indeed features that can directly represent the difference between the appearance of a vehicle and that of other vehicles. However, for video frames on highways, when a vehicle passes through a tunnel section or other dimly lit or changing lighting scenes, the vehicle's appearance features may be seriously affected; at this time, the area of ​​the vehicle's vehicle target detection frame will not change, so the size of the bounding box of the vehicle target detection frame can be used as a weak clue to make up for the similarity comparison of the vehicle's appearance features. In other words, the applicant team further discovered that when defining a single-camera vehicle tracking trajectory, the single-camera vehicle tracking trajectory is not only composed of vehicle appearance features, but also includes the size of the bounding box of the vehicle target detection frame. In addition, since the vehicle appearance features are extracted based on the detected vehicle target detection frame, the use of the bounding box size to supplement the vehicle appearance features to construct a subsequent similarity matrix can affect the final trajectory aggregation result based on the similarity matrix.

[0040] Acquisition of multi-vehicle tracking trajectories with a single camera:

[0041] Based on the vehicle target detection frame and vehicle appearance features obtained in the first two stages, this solution inputs the vehicle target detection frame and vehicle appearance features into the BoT-SORT target tracking module to obtain the multi-vehicle tracking trajectory of a single camera. The BoT-SORT target tracking module selected in this solution combines the Transformer architecture and existing target detection technology to achieve efficient real-time multi-target tracking. The BoT-SORT target tracking module assigns a unique ID to each vehicle based on the vehicle target detection frame and vehicle appearance features, uses a Kalman filter or a similar prediction mechanism to predict the vehicle's position in the next frame, matches the detection results in the current frame with the predicted trajectory, and updates the trajectory to obtain the multi-vehicle tracking trajectory of multiple vehicles under a single camera.

[0042] Correspondingly, the BoT-SORT target tracking module assigns a unique identification ID to the vehicle based on the vehicle appearance features and the corresponding vehicle target detection frame, and tracks the vehicle unique identification ID based on the vehicle appearance features and the vehicle target detection frame to form a multi-vehicle tracking trajectory of a single camera.

[0043] Specifically, the vehicle appearance features of the same vehicle are associated to form a multi-vehicle tracking trajectory of a single camera:

[0044] ;

[0045] in Indicates the tracking trajectory associated with the unique identification ID of the corresponding vehicle. represents the time frame, represents the vehicle target detection box, Indicates the appearance characteristics of the vehicle.

[0046] About obtaining the similarity matrix:

[0047] This application proposal calculates the similarity between trajectories based on appearance features to form a similarity matrix. In addition, the bounding box size of the vehicle target detection box is regarded as a weak clue, and the vehicle appearance features are used as strong clues to calculate the similarity between multiple vehicle tracking trajectories across cameras, and obtain a similarity matrix that fuses weak clues and strong clues, thereby affecting the performance of the system.

[0048] In the step of "combining weak clues and strong clues to obtain a similarity matrix of multi-vehicle tracking trajectories of different cameras", weak clues and strong clues are fused to obtain a fused feature representation, trajectory features of multi-vehicle tracking trajectories of different cameras are obtained based on the fused feature representation, and the similarity between different trajectory features is obtained to obtain a similarity matrix.

[0049] Specifically, the weak cues and strong cues are fused to obtain the fused feature representation:

[0050] ;

[0051] in is the fusion feature representation, It is the feature representation before fusion, corresponding to the strong clue; Indicates the bounding box size of the vehicle target detection box, corresponding to the weak clue, Represents the fusion factor.

[0052] Find the similarity between different trajectory features to obtain the similarity matrix:

[0053] ;

[0054] ;

[0055] in is the i-th multi-vehicle tracking trajectory, is the j-th multi-vehicle tracking trajectory, is the m-th multi-vehicle tracking trajectory, is the trajectory feature corresponding to the i-th multi-vehicle tracking trajectory, is the trajectory feature of the j-th multi-vehicle tracking trajectory, represents the similarity between the i-th multi-vehicle tracking trajectory and the j-th multi-vehicle tracking trajectory.

[0056] About obtaining vehicle tracking trajectory:

[0057] This solution uses boundary extension clustering to perform boundary clustering based on the similarity matrix to obtain vehicle tracking trajectories across cameras. Each step of boundary extension clustering selects adjacent cameras for clustering. When the clustering of the trajectories of adjacent cameras is completed, boundary extension clustering will select the trajectories of adjacent clusters and repeat the clustering process until the global vehicle tracking trajectory is obtained. In other words, the logic of boundary extension clustering is: first cluster the trajectories of adjacent cameras and gradually expand them to the global trajectory, and then concentrate on processing the trajectories close to the boundary.

[0058] Different from traditional clustering methods, boundary expansion clustering takes into account the shooting order of cameras and clusters them according to their sequential arrangement. First, in the entire clustering process of boundary expansion clustering, each clustering step selects adjacent cameras for clustering. Once the clustering of the trajectories of adjacent cameras is completed, the trajectories clustered with their vicinity are selected, and then the clustering process is repeated until the global trajectory is obtained. Second, for cameras close to the boundary (i.e., the starting point and the end point), boundary expansion clustering aims to cluster the trajectories of these cameras in a concentrated manner. This is because the motion pattern of the same vehicle is more consistent and stable when entering and leaving the road segment than when it is in the middle of the road segment. For example, in a highway scenario, a vehicle in the middle of the road segment may encounter disturbances such as sudden exits from the middle intersection, unexpected deceleration or delays, and the simultaneous presence of highly similar vehicles.

[0059] In the step of "using boundary extension clustering to cluster the trajectories of the similarity matrix according to the shooting order of the cameras to obtain the vehicle tracking trajectory of each vehicle", the cameras are numbered according to the shooting order from front to back to obtain a camera sequence, the cameras in the camera sequence are grouped in sequence according to the boundary grouping rule to obtain multiple clustering groups, and the multiple vehicle tracking trajectories in all clustering groups are clustered and merged in sequence according to the similarity matrix to obtain the global vehicle tracking trajectory.

[0060] As mentioned above, the boundary extension clustering algorithm provided by this solution combines the shooting order of the cameras and the correlation of the inherent features of the scene to perform trajectory clustering, where the shooting order of the cameras is reflected in the boundary clustering algorithm, which selects cameras and adjacent cameras for hierarchical clustering, and the inherent features of the scene are reflected in the boundary clustering algorithm, which is grouping based on boundary grouping rules.

[0061] Specifically, in the step of "grouping the cameras in the camera sequence in sequence according to the boundary grouping rule to obtain multiple clustering groups", the cameras in the camera sequence are grouped in sequence according to the camera sequence, the starting camera number, the ending camera number and the boundary length, and the cameras within the boundary length in the camera sequence are taken as the starting point with the starting camera number as the starting point to form the same clustering group, and the cameras within the boundary length in the camera sequence are taken as the ending point to form the same clustering group, and the remaining cameras in the camera sequence and their adjacent cameras are taken as the same clustering group.

[0062] In this solution, the start camera number refers to the number of the first start camera that appears in the scene section, the end camera number refers to the head number of the last end camera that appears in the scene section, and the boundary length depends on the position of the boundary camera close to the start camera or the end camera in the scene section. Specifically, for the start camera, the boundary length is the distance between the boundary camera of the first intersection or traffic light that appears in the scene section and the start camera; for the end camera, the boundary length is the distance between the boundary camera of the last intersection or traffic light that appears in the scene section and the end camera.

[0063] For example, the camera sequence is set to camseq, the starting camera number of the starting camera is startId, the ending camera number of the ending camera is endId, the boundary length is L, k represents the camera ID of the first traffic light or the first intersection in the section, and the boundary length L corresponding to the starting camera is set to k-startId; y represents the camera ID of the last traffic light or the last intersection in the section, and the boundary length L corresponding to the ending camera is set to y-endId.

[0064] In the step of "clustering and merging multiple vehicle tracking trajectories in all cluster groups in sequence according to the similarity matrix to obtain a global vehicle tracking trajectory", multiple vehicle tracking trajectories with high similarity in the cluster group are merged according to the similarity matrix to obtain a local merged trajectory, and then the local merged trajectories are clustered in sequence to obtain a global vehicle tracking trajectory. In other words, this solution also obtains a global vehicle tracking trajectory by merging multiple vehicle tracking trajectories from different cameras according to the similarity matrix.

[0065] Embodiment 2 A cross-camera multi-vehicle tracking system combining weak clues and boundary extension clustering,

[0066] As shown in Figure 5, this scheme provides a cross-camera multi-vehicle tracking system that combines weak clues and boundary extension clustering, including:

[0067] A video stream acquisition module, used to acquire tracking video streams of multiple cameras on the highway;

[0068] A vehicle target detection frame acquisition module is used to input the video frame of the tracking video of each camera into the pre-trained vehicle target detection module to obtain a vehicle target detection frame;

[0069] The vehicle appearance feature acquisition module is used to input the vehicle target detection frame image corresponding to the vehicle target detection frame into the pre-trained feature extraction module to output the ReID feature and the bounding box area of ​​the vehicle target frame, and fuse the ReID feature to obtain the vehicle appearance feature;

[0070] The multi-vehicle tracking trajectory acquisition module is used to input the vehicle appearance features and the corresponding vehicle target detection frame into the BoT-SORT target tracking module to output the multi-vehicle tracking trajectory of each camera;

[0071] A similarity matrix acquisition module is used to obtain the similarity matrix of multiple vehicle tracking trajectories of different cameras by combining the weak clues and the strong clues, using the vehicle appearance features of each vehicle target detection frame as strong clues and the bounding box area of ​​each vehicle target detection frame as weak clues;

[0072] The tracking module is used to cluster the trajectories of the similarity matrix using boundary extension clustering according to the shooting order of the cameras to obtain the vehicle tracking trajectory of each vehicle.

[0073] The technical contents in the second embodiment that are the same as those in the first embodiment will not be described in detail here.

[0074] Embodiment 3

[0075] This embodiment also provides an electronic device, referring to FIG6 , comprising a memory 304 and a processor 302 , wherein the memory 304 stores a computer program, and the processor 302 is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the cross-camera multi-vehicle tracking method or system combining weak clues and boundary extension clustering.

[0076] Specifically, the processor 302 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0077] The memory 304 may include a large capacity memory 304 for data or instructions. By way of example and not limitation, the memory 304 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. The memory 304 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 302.

[0078] The processor 302 reads and executes the computer program instructions stored in the memory 304 to implement any one of the cross-camera multi-vehicle tracking methods combining weak clues and boundary extension clustering in the above embodiments.

[0079] Optionally, the electronic device may further include a transmission device 306 and an input / output device 308 , wherein the transmission device 306 is connected to the processor 302 , and the input / output device 308 is connected to the processor 302 .

[0080] The transmission device 306 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 306 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0081] The input and output device 308 is used to input or output information. In this embodiment, the input information may be a tracking video stream, etc., and the output information may be a cross-camera vehicle tracking trajectory, etc.

[0082] Optionally, in this embodiment, the processor 302 may be configured to perform the following steps through a computer program:

[0083] Get tracking video streams from multiple cameras on the highway;

[0084] Input the video frame of the tracking video of each camera into the pre-trained vehicle target detection module to obtain the vehicle target detection frame, input the vehicle target detection frame image corresponding to the vehicle target detection frame into the pre-trained feature extraction module to output the ReID feature and the bounding box area of ​​the vehicle target frame, fuse the ReID feature to obtain the vehicle appearance feature, input the vehicle appearance feature and the corresponding vehicle target detection frame into the BoT-SORT target tracking module to output the multi-vehicle tracking trajectory of each camera, take the vehicle appearance feature of each vehicle target detection frame as the strong clue, take the bounding box area of ​​each vehicle target detection frame as the weak clue, and combine the weak clues and the strong clues to obtain the similarity matrix of the multi-vehicle tracking trajectories of different cameras;

[0085] The similarity matrix is ​​clustered by using boundary extension clustering according to the shooting order of the cameras to obtain the vehicle tracking trajectory of each vehicle.

[0086] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0087] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0088] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.

[0089] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A cross-camera multi-vehicle tracking method combining weak cues and boundary extension clustering, characterized in that: The following steps are involved: Get tracking video streams from multiple cameras on the highway; Input the video frame of the tracking video of each camera into the pre-trained vehicle target detection module to obtain the vehicle target detection frame, input the vehicle target detection frame image corresponding to the vehicle target detection frame into the pre-trained feature extraction module to output the ReID feature and the bounding box area of ​​the vehicle target frame, fuse the ReID feature to obtain the vehicle appearance feature, input the vehicle appearance feature and the corresponding vehicle target detection frame into the BoT-SORT target tracking module to output the multi-vehicle tracking trajectory of each camera, take the vehicle appearance feature of each vehicle target detection frame as the strong clue, take the bounding box area of ​​each vehicle target detection frame as the weak clue, and combine the weak clues and the strong clues to obtain the similarity matrix of the multi-vehicle tracking trajectories of different cameras; The cameras are numbered according to the shooting order from front to back to obtain a camera sequence. The cameras in the camera sequence are grouped in sequence according to the camera sequence, the starting camera number, the ending camera number and the boundary length. The cameras within the boundary length in the camera sequence are taken as the starting point with the starting camera number as the same cluster group, and the cameras within the boundary length in the camera sequence are taken as the end point with the ending camera number as the same cluster group. The remaining cameras in the camera sequence and their adjacent cameras are taken as the same cluster group. The multi-vehicle tracking trajectories in all cluster groups are clustered and merged in sequence according to the similarity matrix to obtain the global vehicle tracking trajectory, where the starting camera number refers to the number of the first starting camera that appears in the scene section, the ending camera number refers to the number of the last ending camera that appears in the scene section, and the boundary length depends on the position of the boundary camera close to the starting camera or the ending camera in the scene section.

2. The cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering according to claim 1, characterized in that: The ReID feature is output by inputting the vehicle target detection box image into the ReID model, and the bounding box area of ​​the vehicle target box is calculated based on the area of ​​the vehicle target detection box.

3. The cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering according to claim 1, characterized in that: The BoT-SORT target tracking module assigns a unique ID to the vehicle based on the vehicle's appearance features and the corresponding vehicle target detection frame, and tracks the vehicle's unique ID based on the vehicle's appearance features and the vehicle target detection frame to form a single-camera multi-vehicle tracking trajectory.

4. The cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering according to claim 1, characterized in that: The weak cues and the strong cues are fused to obtain a fused feature representation. Based on the fused feature representation, the trajectory features of multiple vehicle tracking trajectories from different cameras are obtained, and the similarities between different trajectory features are calculated to obtain a similarity matrix.

5. The cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering according to claim 1, characterized in that: The multi-vehicle tracking trajectories with high similarity in the cluster group are merged according to the similarity matrix to obtain the local merged trajectories, and then the local merged trajectories are clustered in sequence to obtain the global vehicle tracking trajectory.

6. A cross-camera multi-vehicle tracking system combining weak clues and boundary extension clustering, characterized in that: include: A video stream acquisition module, used to acquire tracking video streams of multiple cameras on the highway; A vehicle target detection frame acquisition module is used to input the video frame of the tracking video of each camera into the pre-trained vehicle target detection module to obtain a vehicle target detection frame; The vehicle appearance feature acquisition module is used to input the vehicle target detection frame image corresponding to the vehicle target detection frame into the pre-trained feature extraction module to output the ReID feature and the bounding box area of ​​the vehicle target frame, and fuse the ReID feature to obtain the vehicle appearance feature; The multi-vehicle tracking trajectory acquisition module is used to input the vehicle appearance features and the corresponding vehicle target detection frame into the BoT-SORT target tracking module to output the multi-vehicle tracking trajectory of each camera; A similarity matrix acquisition module is used to obtain the similarity matrix of multiple vehicle tracking trajectories of different cameras by combining the weak clues and the strong clues, using the vehicle appearance features of each vehicle target detection frame as strong clues and the bounding box area of ​​each vehicle target detection frame as weak clues; A tracking module is used to number the cameras according to the shooting order of the cameras from front to back to obtain a camera sequence, group the cameras in the camera sequence in sequence according to the camera sequence, the starting camera number, the ending camera number and the boundary length, take the starting camera number as the starting point and take the cameras within the boundary length in the camera sequence as the same cluster group, take the ending camera number as the end point and take the cameras within the boundary length in the camera sequence as the same cluster group, take the remaining cameras in the camera sequence and their adjacent cameras as the same cluster group, and cluster and merge the multi-vehicle tracking trajectories in all cluster groups in sequence according to the similarity matrix to obtain a global vehicle tracking trajectory, wherein the starting camera number refers to the number of the first starting camera that appears in the scene section, the ending camera number refers to the head number of the last ending camera that appears in the scene section, and the boundary length depends on the position of the boundary camera close to the starting camera or the ending camera in the scene section.

7. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the cross-camera multi-vehicle tracking method combining weak clues and boundary extension clustering as described in any one of claims 1 to 5.