Method, device, electronic device, and storage medium for cross-camera vehicle tracking

Abstract

The present application relates to the field of intelligent transportation technology and discloses a method for cross-camera vehicle tracking, comprising: receiving vehicle videos captured by multiple single cameras; analyzing the vehicle videos captured by each single camera to obtain vehicle trajectories of multiple single cameras; and inputting the vehicle trajectories of multiple single cameras into a pre-trained road segmentation model to obtain cross-camera vehicle trajectories. The present application adopts technical means applied to the field of deep learning technology. In this process, the search space is reduced through the road segmentation model, and global operations are used in the feature extraction process without image dimensionality reduction, thereby reducing the loss of details and the lack of feature association, and improving the accuracy of cross-camera vehicle tracking. The present application also discloses a cross-camera vehicle tracking device, electronic device, and storage medium.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, for example, to a method, device, electronic device, and storage medium for cross-camera vehicle tracking. Background Art

[0002] For the transportation industry, the essence of new infrastructure is to transform traditional transportation infrastructure construction using digital and intelligent technologies, and to make intelligent transportation intelligent through digital information technologies such as the Internet and the Internet of Things. With the development of the economy, the deep integration of emerging technologies such as artificial intelligence and big data with the transportation industry has brought new development opportunities to the industry. In traffic scenarios, vehicle trajectories can provide almost all the dynamic information of moving vehicles. Analyzing vehicle trajectories in monitoring scenarios can grasp dynamic road traffic information. Cross-camera correlation of target trajectories in multiple cameras can break the isolation of target information between individual cameras and obtain the overall road operation status in a large-scale video monitoring area, which helps road traffic managers conduct traffic analysis, prediction, and control.

[0003] To achieve cross-camera correlation of target trajectories in multiple cameras, a cross-camera road space fusion and vehicle target detection and tracking method has been proposed. This method uses camera calibration technology to calibrate the cross-camera scene, leverages common areas in the cross-camera scene to achieve road space fusion, and employs deep learning methods to detect vehicle targets and complete cross-camera vehicle tracking. Specifically, it includes the following steps: Step 1: Input traffic scene background images p1 and p2 from Scene 1 and Scene 2, and video frame image sequence groups s1 and s2; Step 2: Construct a coordinate system and model to complete camera calibration; Step 3: Set control points to identify the road area; Step 4: Generate a road space fusion image Mp with spatial information; Step 5: Detect targets across cameras; and Step 6: Track targets across cameras.

[0004] During the implementation of the embodiments of the present disclosure, it was found that at least the following problems exist in the related art:

[0005] In related technologies, cross-camera vehicle trajectory detection requires constructing a coordinate system to generate a fused road image with spatial information. This fused image generation process typically requires gradually reducing the spatial resolution of the feature map, decoding the encoded information, and then increasing the feature map resolution back to the original input resolution, either all at once or gradually. This resolution change inevitably results in information loss, making cross-camera vehicle tracking accuracy unreliable. Summary of the Invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] Embodiments of the present disclosure provide a method, apparatus, electronic device, and storage medium for cross-camera vehicle tracking to improve the accuracy of cross-camera vehicle tracking.

[0008] In some embodiments, the method for cross-camera vehicle tracking includes:

[0009] Receive vehicle videos on the same main road captured by multiple single cameras;

[0010] Analyze the vehicle video captured by each single camera to obtain the vehicle trajectories of multiple single cameras;

[0011] The vehicle trajectories of multiple single cameras are input into the pre-trained road segmentation model to obtain the cross-camera vehicle trajectories.

[0012] Optionally, multiple single-camera vehicle trajectories are fed into a pre-trained road segmentation model to obtain cross-camera vehicle trajectories, including:

[0013] Input the vehicle trajectories from multiple single cameras into the pre-trained road segmentation model to obtain the road segmentation results;

[0014] Perform single-camera vehicle trajectory filtering based on the road segmentation results to obtain filtered vehicle trajectories;

[0015] According to the filtered vehicle trajectories, the vehicle trajectories across cameras are obtained.

[0016] Optionally, performing single-camera vehicle trajectory filtering based on the road segmentation result to obtain filtered vehicle trajectory includes:

[0017] Perform intersection trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles traveling on the main road; and / or,

[0018] Perform time threshold trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles that meet the time threshold;

[0019] The trajectory of the vehicle after trajectory filtering is used as the filtered vehicle trajectory.

[0020] Optionally, a vehicle trajectory across cameras is obtained based on the filtered vehicle trajectory, including:

[0021] Calculate the similarity between filtered vehicle trajectories;

[0022] Clustering is performed based on the calculation results to obtain vehicle trajectories across cameras.

[0023] Optionally, the vehicle video captured by each single camera is analyzed to obtain the vehicle trajectories of multiple single cameras, including:

[0024] Input the collected vehicle videos into the pre-trained local vehicle detection model to obtain the target detection box of each vehicle;

[0025] Input the target detection box of each vehicle into the pre-trained local vehicle identification model to obtain the identification features of each vehicle;

[0026] According to the target detection frame and identification features, the vehicle trajectory of a single camera that matches the target detection frame is obtained.

[0027] Optionally, the training process of the local vehicle detection model includes:

[0028] Create a local dataset;

[0029] The vehicle target detection model is adjusted using the local dataset to obtain a local vehicle detection model.

[0030] Optionally, obtaining a vehicle trajectory of a single camera that matches the target detection frame according to the target detection frame and the identification features includes:

[0031] Using the target detection frame and identification features, the degree of motion matching and performance matching is determined;

[0032] The priority is assigned to the determined matching degree through cascade matching to obtain the vehicle trajectory of a single camera that matches the target detection frame.

[0033] In some embodiments, the apparatus for cross-camera vehicle tracking includes:

[0034] A processor and a memory storing program instructions, wherein the processor is configured to execute the above-mentioned vehicle tracking method across cameras when running the program instructions.

[0035] In some embodiments, the electronic device includes:

[0036] As described above for a vehicle tracking device across cameras.

[0037] In some embodiments, the storage medium includes:

[0038] Program instructions are stored, and when the program instructions are executed, the method for vehicle tracking across cameras as described above is executed.

[0039] The method, device, electronic device, and storage medium for cross-camera vehicle tracking provided by the embodiments of the present disclosure can achieve the following technical effects:

[0040] This application uses deep learning techniques to analyze vehicle videos captured by multiple single cameras to obtain vehicle trajectories from multiple cameras. This analysis then uses a road segmentation model to determine the road segmentation results and correlate the trajectories. This process reduces the search space through the road segmentation model, and employs global operations during feature extraction, eliminating the need for image dimensionality reduction. This minimizes the loss of detail and lack of feature correlation, improving the accuracy of cross-camera vehicle tracking.

[0041] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] One or more embodiments are exemplarily described by corresponding drawings. These exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation. In addition,

[0043] Figure 1 is a schematic diagram of a vehicle tracking system environment across cameras provided by an embodiment of the present disclosure;

[0044] Figure 2 is a schematic diagram of a vehicle tracking method across cameras provided by an embodiment of the present disclosure;

[0045] Figure 3 is a schematic diagram of another vehicle tracking method across cameras provided by an embodiment of the present disclosure;

[0046] Figure 4 is another schematic diagram of a vehicle tracking system environment across cameras according to an embodiment of the present disclosure;

[0047] Figure 5 is a schematic diagram of another vehicle tracking method across cameras provided by an embodiment of the present disclosure;

[0048] Figure 6 is a schematic diagram of adjusting a vehicle target detection model provided by an embodiment of the present disclosure;

[0049] Figure 7 Schematic diagram of a vehicle tracking device across cameras provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0051] In the description and claims of the embodiments of the present disclosure, as well as in the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar items and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate to describe the embodiments of the present disclosure herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions.

[0052] Unless otherwise stated, the term "plurality" means two or more.

[0053] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0054] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0055] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0056] CBS stands for Conv, BN, and Silu, where Conv stands for convolution, BN (Batch Normalization) is an algorithm frequently used in deep networks to accelerate neural network training, convergence speed, and stability. It is an indispensable part of current deep networks, and Silu (Sigmoid Weighted Liner Unit) is an activation function.

[0057] Combine Figure 1 As shown in FIG, a schematic diagram of a vehicle tracking system environment for a cross-camera vehicle is provided in an embodiment of the present disclosure. In the figure, positions 01, 02, 03, 04, and 05 are all equipped with surveillance cameras, and the vehicle video image signal is collected by each surveillance camera. In actual application, the distribution of surveillance cameras can be as follows: Figure 1As shown, vehicle video images are collected along the main road, which can also be adjusted as needed, where the main road refers to the road consisting of position 01, position 02, position 03, position 04, and position 05. It should be understood that the distribution and number of surveillance cameras include but are not limited to Figure 1 The method shown can be set according to actual working conditions, as long as it can be used to reflect the vehicle video image signals collected by multiple cameras, and the corresponding road sections represented by the corresponding main roads also change randomly.

[0058] Combine Figure 2 As shown, an embodiment of the present disclosure provides a vehicle tracking method for crossing cameras, comprising:

[0059] S21, the cross-camera vehicle tracking system receives vehicle videos on the same main road captured by multiple single cameras.

[0060] In the embodiment of the present disclosure, receiving vehicle videos captured by multiple single cameras may be performed by multiple Figure 1 The vehicle video captured by the surveillance camera shown is a real surveillance video in a road traffic scene.

[0061] S22, the cross-camera vehicle tracking system analyzes the vehicle video captured by each single camera to obtain vehicle trajectories of multiple single cameras.

[0062] In the disclosed embodiment, analyzing the vehicle video captured by each single camera includes decoding the vehicle video using the ffmpeg library and extracting key frames from the video image. The single-camera vehicle trajectory corresponding to each surveillance camera is then obtained by manually annotating the image and combining it with a pre-trained detection model. Decoding the captured vehicle video and extracting key frames from the video image can be performed by inputting them into the pre-trained detection model at fixed intervals. This fixed interval can be one frame every two seconds, one frame every five seconds, or other intervals, which are not specifically limited in this application.

[0063] S23, the cross-camera vehicle tracking system inputs the vehicle trajectories of multiple single cameras into the pre-trained road segmentation model to obtain cross-camera vehicle trajectories.

[0064] In the embodiment of the present disclosure, the vehicle trajectory of a single camera is input into a pre-trained road segmentation model, the road segmentation result is determined, and the trajectory matching is completed by calculation, thereby obtaining the vehicle trajectory across cameras.

[0065] The cross-camera vehicle tracking method provided by the disclosed embodiments analyzes vehicle videos captured by multiple single cameras to obtain multiple single-camera vehicle trajectories. A road segmentation model is then used to determine the road segmentation results, thereby correlating the trajectories. This process reduces the search space using the road segmentation model, and global operations are used during feature extraction, eliminating the need for image dimensionality reduction. This minimizes the loss of detail and lack of feature correlation, thereby improving the accuracy of cross-camera vehicle tracking.

[0066] Combine Figure 3 As shown, an embodiment of the present disclosure provides a method for tracking a vehicle across cameras, comprising:

[0067] S31, receiving vehicle videos on the same main road captured by multiple single cameras.

[0068] S32: Analyze the vehicle video captured by each single camera to obtain vehicle trajectories of multiple single cameras.

[0069] S33, inputting the vehicle trajectories of the multiple single cameras into a pre-trained road segmentation model to obtain a road segmentation result.

[0070] In the embodiment of the present disclosure, the road segmentation result obtained by the pre-trained road segmentation model refers to the determination of whether the input image belongs to a road by the pre-trained road segmentation model. Wherein, determining whether the input image belongs to a road refers to performing semantic segmentation judgment on the input image by the road segmentation model to determine whether the input image is a main road. In practical applications, such as Figure 1 The road shown including position 01 to position 05 is a main road, and the remaining sections are secondary roads.

[0071] Optionally, the training process of the road segmentation model includes:

[0072] Create a local road segmentation dataset.

[0073] Pre-trained road segmentation models on open source datasets.

[0074] The trained road segmentation model is fine-tuned on the local road segmentation dataset.

[0075] In an embodiment of the present disclosure, the process of establishing a local road segmentation dataset includes receiving road scene videos captured by multiple single cameras, extracting video keyframes, and annotating roads in the keyframes to obtain the local road segmentation dataset.

[0076] In practical applications, the extraction of video key frames can be performed using the ffmpeg library or other tools for recording and converting videos. This application does not specifically limit this, and the specific method can be selected according to the actual working conditions. The extraction of video key frames using tools for recording and converting videos is a relatively mature technology, and this application will not elaborate on this.

[0077] In practical applications, the road annotation form of the keyframe is a polygon detection box that tightly contains the road. The road is segmented by manually annotating the polygon detection box, providing a data basis for the subsequent determination of the road segmentation result.

[0078] In the embodiment of the present disclosure, pre-training the road segmentation model in the open source dataset means training the road segmentation model by using the open source dataset with a large amount of data, thereby ensuring the effect of the model training and the accuracy of the model results.

[0079] In practical applications, open source datasets can be the open source CamVid and Cityscape datasets, and the road segmentation model can use the HRNet road segmentation model or a VGG16-based road segmentation model. This allows the road segmentation model to be pre-trained using a large open source dataset. It should be understood that the choice of open source dataset and road segmentation model can be determined based on actual operating conditions, and this application does not impose specific limitations on this.

[0080] In this disclosed embodiment, the trained road segmentation model is adjusted on a local road segmentation dataset to obtain a road segmentation model more suitable for the local scene. This adjustment process specifically involves adjusting the output categories of the pre-trained road segmentation model to determine whether the road section corresponding to the vehicle trajectories input by multiple single cameras is a main road.

[0081] In practical applications, the adjustment of the output category of the pre-trained road segmentation model can be to adjust the output category of the original HRNet model to 2, that is, to output whether each pixel of the input image is a main road.

[0082] S34, filtering the single-camera vehicle trajectory according to the road segmentation result to obtain a filtered vehicle trajectory.

[0083] Optionally, performing single-camera vehicle trajectory filtering based on the road segmentation result to obtain filtered vehicle trajectory includes:

[0084] Perform intersection trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles traveling on the main road; and / or,

[0085] Perform time threshold trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles that meet the time threshold;

[0086] The trajectory of the vehicle after trajectory filtering is used as the filtered vehicle trajectory.

[0087] In the disclosed embodiment, single-camera vehicle trajectory filtering is performed based on the road segmentation results to obtain filtered vehicle trajectories. This includes: performing intersection trajectory filtering based on the road segmentation results to obtain trajectories of vehicles traveling on the main road; and using the trajectory of the vehicle after trajectory filtering as the filtered vehicle trajectory. The intersection trajectory filtering is used to eliminate vehicle trajectories that have left the main road.

[0088] In practical applications, such as Figure 4 Figure 2 shows a schematic diagram of a cross-camera vehicle tracking system environment according to an embodiment of the present disclosure. Based on the road segmentation results, the sections corresponding to a1 and a3 are determined to be primary roads, while the sections corresponding to a2 and a4 are determined to be secondary roads. If the vehicle's route or direction changes, such as when it moves from a1 or a3 to a2 or a4, the intersection trajectory filtering method deletes the corresponding track from a1 to a2 or a4, and the track of the vehicle after the trajectory filtering is used as the filtered vehicle track.

[0089] In an embodiment of the present disclosure, single-camera vehicle trajectory filtering is performed based on the road segmentation results to obtain filtered vehicle trajectories. This includes performing time threshold trajectory filtering based on the road segmentation results to obtain vehicle trajectories that meet the time threshold. Time threshold trajectory filtering is used to eliminate trajectories with excessively short time lengths. In an embodiment of the present disclosure, time threshold trajectory filtering eliminates corresponding trajectories if the vehicle trajectory does not meet the time threshold. The time threshold can be a threshold for measuring the time length of the trajectory or a threshold for measuring the number of trajectory frames. When video image key frames are captured at fixed time intervals and input into a pre-trained detection model, the time threshold is expressed as a threshold for measuring the number of trajectory frames. That is, the trajectory time length is determined by combining the number of video frames with the fixed time interval to determine whether the vehicle trajectory meets the time threshold. The vehicle trajectory after trajectory filtering is then used as the filtered vehicle trajectory. It should be understood that the time threshold can be 6 minutes, 8 minutes, or other time length units, or 180 frames, 200 frames, or other frame units, and this application does not specifically limit this.

[0090] In the disclosed embodiment, single-camera vehicle trajectory filtering is performed based on the road segmentation result to obtain filtered vehicle trajectory, including:

[0091] Perform intersection trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles traveling on the main road; and,

[0092] Perform time threshold trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles that meet the time threshold;

[0093] The trajectory of the vehicle after trajectory filtering is used as the filtered vehicle trajectory.

[0094] In practical applications, the filtering of single-camera vehicle trajectories can be performed by intersection trajectories filtering and time threshold trajectories filtering. The execution method is the same as the above embodiment, and this application will not elaborate on this.

[0095] In this way, the single-camera vehicle trajectory is filtered by intersection trajectory filtering and / or time threshold trajectory filtering based on the road segmentation results to further eliminate vehicle trajectories, thereby reducing the search space through the road segmentation model, improving trajectory tracking efficiency, and improving the accuracy of cross-camera vehicle tracking.

[0096] S35 , obtaining a vehicle trajectory across cameras based on the filtered vehicle trajectory.

[0097] Optionally, a vehicle trajectory across cameras is obtained based on the filtered vehicle trajectory, including:

[0098] Calculate the similarity between filtered vehicle trajectories;

[0099] Clustering is performed based on the calculation results to obtain vehicle trajectories across cameras.

[0100] Optionally, calculating the similarity between the filtered vehicle trajectories includes determining a feature of each filtered vehicle trajectory based on all vehicle identification features in the filtered vehicle trajectories, and calculating the similarity between the filtered vehicle trajectories based on the feature of each filtered vehicle trajectory.

[0101] In the embodiment of the present disclosure, determining the characteristics of each filtered vehicle trajectory based on all vehicle identification features in the filtered vehicle trajectory means that the filtered vehicle trajectory already contains the identification features of all vehicles, and averaging the identification features of all vehicles in the corresponding vehicle trajectory as the characteristic of the corresponding vehicle trajectory.

[0102] In the embodiment of the present disclosure, based on the characteristics of each filtered vehicle trajectory, the similarity between the filtered vehicle trajectories can be calculated according to the following formula:

[0103]

[0104] Among them, T1 and T2 are different vehicle trajectories, f(T1) represents the characteristics of vehicle trajectory T1, f(T2) represents the characteristics of vehicle trajectory T2, cos(T 1, T2) represents the similarity between vehicle trajectory T1 and vehicle trajectory T2.

[0105] Optionally, clustering is performed based on the calculation results to obtain vehicle trajectories across cameras, including:

[0106] According to the similarity calculation results, the similarity matrix between each trajectory is obtained;

[0107] The similarity matrix between each track is clustered to obtain the vehicle track across cameras.

[0108] In the embodiment of the present disclosure, the similarity matrix between each track can be expressed as follows:

[0109]

[0110] Where M represents the similarity matrix of n vehicle trajectories, and n is a positive integer.

[0111] In the embodiment of the present disclosure, clustering the similarity matrices between the trajectories to obtain the cross-camera vehicle trajectory means treating each trajectory as a cluster, and merging the two clusters when the distance between the two clusters is less than the first clustering threshold, until all clusters are merged to obtain a first fusion result. And based on the first fusion result, when the distance between the two clusters is less than the second clustering threshold, the two clusters are merged until all clusters are merged to obtain a second fusion result, that is, the cross-camera vehicle trajectory. Among them, the first clustering threshold is less than the second clustering threshold. The first clustering threshold can be 0.4, 0.5, 0.6 or other values, and the second clustering threshold can be 0.7, 0.8, 0.9 or other values.

[0112] In practical applications, trajectory fusion based on the first clustering threshold refers to preliminary trajectory fusion, which can be performed using hierarchical clustering, k-means clustering, or other clustering methods to merge target trajectories with high similarity, thus obtaining the first fusion result. Trajectory fusion based on the second clustering threshold refers to cross-camera trajectory fusion, which can be performed using hierarchical clustering, k-means clustering, or other clustering methods to fuse multiple single-camera trajectories after preliminary fusion into a multi-camera trajectory, thus obtaining the second fusion result.

[0113] By calculating the similarity of each vehicle's trajectory and clustering it based on the calculated results, the single-camera vehicle trajectory is fused into a cross-camera vehicle trajectory, improving the accuracy of cross-camera vehicle tracking. Furthermore, because the extraction of vehicle identification features is a global operation, the trajectory features derived from these features also have the properties of a global operation, meaning that no image dimensionality reduction is required, thus minimizing the loss of detail and the lack of feature association, further improving the accuracy of cross-camera vehicle tracking.

[0114] Combine Figure 5 As shown, an embodiment of the present disclosure provides a method for tracking a vehicle across cameras, comprising:

[0115] S51: Receive vehicle videos on the same main road captured by multiple single cameras.

[0116] S52: Input the video stream into the pre-trained local vehicle detection model to obtain the target detection frame of each vehicle.

[0117] In the embodiment of the present disclosure, inputting a video stream into a pre-trained local vehicle detection model refers to inputting a captured vehicle video into the pre-trained local vehicle detection model.

[0118] Optionally, the training process of the local vehicle detection model includes:

[0119] Create a local dataset;

[0120] The vehicle target detection model is adjusted using the local dataset to obtain a local vehicle detection model.

[0121] In the disclosed embodiment, the process of establishing a local dataset includes receiving road vehicle videos captured by multiple single cameras, extracting video key frames, and annotating images in the key frames to generate tracking trajectory annotations.

[0122] In practical applications, the extraction of video key frames can be performed using the ffmpeg library or other tools for recording and converting videos. This application does not specifically limit this, and the specific method can be selected according to the actual working conditions. The extraction of video key frames using tools for recording and converting videos is a relatively mature technology, and this application will not elaborate on this.

[0123] In practical applications, tracking trajectory annotation takes the form of a target detection box, which can be a rectangular box. The annotation includes the current frame ID, the vehicle target ID, the coordinates of the upper left corner of the target detection box, the detection box width, and the detection box height. Tracking trajectory annotation provides the data foundation for subsequently determining the vehicle's target detection box, its identification features, and deriving the cross-camera vehicle trajectory from a single-camera vehicle trajectory.

[0124] Optionally, the annotation content also includes a flag.

[0125] In practical applications, the flag bit has a range of two values, which can be 0 or 1, or a or b, or other types. The flag bit value is used to indicate whether the target is a valid target. When the flag bit value is 0 or 1, 0 can indicate that the corresponding target is blurry or too small or other reasons make the target an invalid target, and 1 can indicate that the corresponding target is a valid target. It should be understood that the flag bit value can be set according to the actual working conditions, and this application does not make specific restrictions on this, as long as it can be used to reflect whether the target indicated by the flag is a valid target. In this way, by marking the flag bit, road vehicle videos captured by multiple single cameras can be screened and invalid targets can be eliminated, thereby reducing the subsequent search space and improving the accuracy of cross-camera vehicle tracking.

[0126] In the disclosed embodiments, a local vehicle detection model is adjusted using a local dataset to obtain a local vehicle detection model, including a pre-trained vehicle detection model on an open source dataset. The trained vehicle detection model is adjusted on the local dataset to obtain a local vehicle detection model.

[0127] In practical applications, the open source dataset can be an open source object detection dataset, such as the MS-COCO object detection dataset, or other open source object detection datasets. The vehicle object detection model can be a vehicle object detection model based on the YOLOX algorithm, or other vehicle object detection models. It should be understood that the selection of open source object detection datasets and vehicle object detection models can be based on actual working conditions, and this application does not impose specific restrictions on this.

[0128] In practical applications, combined with Figure 6 As shown in FIG, a schematic diagram of adjusting a vehicle target detection model provided by an embodiment of the present disclosure is provided, which exemplifies the process of using a local data set to adjust a vehicle target detection model based on the YOLOX algorithm to obtain a local vehicle detection model. The adjustment process involves modifying the original YOLOX network head part, and the modified structure is as follows Figure 6 As shown in the figure, the convolution kernel of the Cls branch is modified to a 1*1 convolution kernel. After being activated by the activation function, Conv outputs the probability that the target is a vehicle. The Obj branch outputs the probability that the target is a foreground, and the Loc branch outputs the target detection box.

[0129] In practical applications, the probability of the target output by the Cls branch being a vehicle can be used to filter the target detection frame of each vehicle by setting a probability threshold. The probability threshold can be 0.4, 0.5, or other values, and can be set according to actual working conditions. The probability of the foreground output by the Obj branch refers to the probability that the output result is not a target detection frame of a vehicle, that is, the probability of interference factors. The target detection frame output by the Loc branch refers to the target detection frame of each vehicle, where the output target detection frame includes the detection frame, the coordinates of the upper left corner of the detection frame, the detection frame width, and the detection frame height.

[0130] By feeding the video stream into a pre-trained local vehicle detection model and adjusting it based on the local dataset, we can generate a target detection frame for each vehicle. During this adjustment process, we can filter the target detection frames based on their probability and threshold, reducing data interference and improving cross-camera vehicle tracking accuracy.

[0131] S53: Input the target detection frame of each vehicle into the pre-trained local vehicle identification model to obtain the identification features of each vehicle.

[0132] In the embodiment of the present disclosure, the training process of the local vehicle identification model includes pre-training the vehicle identification model in an open source dataset and adjusting the trained vehicle identification model on the local dataset to obtain the local vehicle identification model.

[0133] In practical applications, the open-source dataset can be an open-source vehicle re-identification dataset, such as the VeRi dataset, or other datasets. The vehicle identification model can be a vehicle re-identification model based on transReID, or other vehicle re-identification models. It should be understood that the selection of open-source datasets and vehicle re-identification models can be based on actual operating conditions and is not specifically limited in this application.

[0134] In practical applications, vehicle re-identification models are used to extract identifying features of vehicles, which is crucial for achieving accurate object association. The transReID-based vehicle re-identification model takes a fixed image patch (i.e., the object detection bounding box of each vehicle) as input and outputs global features of the image. Since this process is a global operation, image dimensionality reduction is not required, minimizing the loss of detail and feature association, thereby improving the accuracy of cross-camera vehicle tracking.

[0135] S54 , obtaining a vehicle trajectory of a single camera that matches the target detection frame according to the target detection frame and the identification features.

[0136] Optionally, obtaining a vehicle trajectory of a single camera that matches the target detection frame according to the target detection frame and the identification features includes:

[0137] Using the target detection frame and identification features, the degree of motion matching and performance matching is determined;

[0138] The priority is assigned to the determined matching degree through cascade matching to obtain the vehicle trajectory of a single camera that matches the target detection frame.

[0139] In the embodiment of the present disclosure, using the target detection frame and the identification features to determine the degree of motion matching and the degree of performance matching means using the Mahalanobis distance to reflect the degree of motion matching and using the minimum cosine distance to reflect the degree of performance matching based on the target detection frame and the identification features.

[0140] In practical applications, the matching degree between the targets in the previous and next frames is calculated by combining the Mahalanobis distance and the minimum cosine distance between the most recent k deep feature sets of the target that are successfully tracked and the feature vector of the detection result. Among them, the Mahalanobis distance is used for variance normalization, the Mahalanobis distance is used as a distance measurement method, and the cosine distance is used as a similarity measurement method. The Mahalanobis distance distinguishes based on position, and the cosine distance is used for direction. Using the cosine distance, the differences between dimensions of different individuals can be measured, and the differences between dimensions in each individual can be compensated by the Mahalanobis distance, so that a relatively comprehensive difference measurement can be achieved as a whole. This optimizes the measurement method and better completes the matching. It should be understood that the use of the Mahalanobis distance to reflect the degree of motion matching and the use of the minimum cosine distance to reflect the degree of performance matching are relatively mature existing technologies, and this application will not elaborate on this.

[0141] In the disclosed embodiments, cascade matching is used to assign priorities to the determined matching levels. Obtaining a single-camera vehicle trajectory that matches the target detection frame involves performing matching at different priorities. In cascade matching, the target that matched first in the previous frame is given the highest priority, while targets that failed to match successfully in several frames are gradually given lower priorities until they are gradually abandoned.

[0142] In practical applications, obtaining a single-camera vehicle trajectory that matches the target detection frame based on the target detection frame and identification features can be done by inputting the target detection frame and identification features into a single-camera vehicle tracking model based on a Deep-Sort target tracking algorithm or a Sort target tracking algorithm to obtain the single-camera vehicle trajectory that matches the target detection frame. The single-camera vehicle trajectory includes the vehicle ID, frame ID, the vehicle's target detection frame, and the vehicle's identification features.

[0143] S55 , inputting the vehicle trajectories of multiple single cameras into a pre-trained road segmentation model to obtain a cross-camera vehicle trajectory.

[0144] In this way, by analyzing vehicle videos captured by multiple single cameras, the vehicle trajectories of multiple single cameras are obtained, and global operations are used in the feature extraction process without the need for image dimensionality reduction, thereby reducing the loss of details and the lack of feature association, and improving the accuracy of cross-camera vehicle tracking.

[0145] Combine Figure 7As shown, an embodiment of the present disclosure provides a device for cross-camera vehicle tracking, including a processor 700 and a memory 701. Optionally, the device may further include a communication interface 702 and a bus 703. The processor 700, the communication interface 702, and the memory 701 may communicate with each other via the bus 703. The communication interface 702 may be used for information transmission. The processor 700 may call the logic instructions in the memory 701 to execute the cross-camera vehicle tracking method of the above embodiment.

[0146] In addition, the logic instructions in the memory 701 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0147] Memory 701, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. Processor 700 executes the program instructions / modules stored in memory 701 to perform functional applications and data processing, thereby implementing the cross-camera vehicle tracking method in the above-described embodiments.

[0148] The memory 701 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory 701 may include high-speed random access memory and non-volatile memory.

[0149] An embodiment of the present disclosure provides an electronic device, comprising the above-mentioned vehicle tracking device for crossing cameras.

[0150] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned vehicle tracking method for cross-cameras.

[0151] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to perform the above-mentioned vehicle tracking method across cameras.

[0152] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0153] The technical solution of the embodiments of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code, or a transient storage medium.

[0154] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to also include plural forms. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, an element defined by the sentence "comprising a..." does not exclude the presence of other identical elements in the process, method or device that includes the element. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments can be referenced to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0155] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0156] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0157] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for tracking a vehicle across cameras, characterized in that include: Receive vehicle videos on the same main road captured by multiple single cameras; Input the collected vehicle videos into the pre-trained local vehicle detection model to obtain the target detection box of each vehicle; Input the target detection box of each vehicle into the pre-trained local vehicle identification model to obtain the identification features of each vehicle; Using the target detection frame and identification features, the degree of motion matching and performance matching is determined; By assigning priorities to the determined matching degrees through cascade matching, the vehicle trajectory of a single camera that matches the target detection frame is obtained; Input the vehicle trajectories from multiple single cameras into the pre-trained road segmentation model to obtain the road segmentation results; Perform single-camera vehicle trajectory filtering based on the road segmentation results to obtain filtered vehicle trajectories; According to the filtered vehicle trajectory, the vehicle trajectory across cameras is obtained; The single-camera vehicle trajectory filtering is performed according to the road segmentation result to obtain the filtered vehicle trajectory, including: According to the road segmentation results, intersection trajectories are filtered to obtain the trajectories of vehicles traveling on the main road; and / or, Perform time threshold trajectory filtering based on the road segmentation results to obtain the trajectory of vehicles that meet the time threshold; The trajectory of the vehicle after trajectory filtering is used as the filtered vehicle trajectory.

2. The method according to claim 1, characterized in that The step of obtaining a vehicle trajectory across cameras based on the filtered vehicle trajectory includes: Calculate the similarity between filtered vehicle trajectories; Clustering is performed based on the calculation results to obtain vehicle trajectories across cameras.

3. The method according to claim 2, characterized in that The calculating of the similarity between the filtered vehicle trajectories includes: determining a feature of each filtered vehicle trajectory based on all vehicle identification features in the filtered vehicle trajectory; Based on the features of each filtered vehicle trajectory, the similarity between the filtered vehicle trajectories is calculated.

4. The method according to claim 2, characterized in that The clustering is performed based on the calculation results to obtain the vehicle trajectory across cameras, including: According to the similarity calculation results, the similarity matrix between each trajectory is obtained; The similarity matrix between each track is clustered to obtain the vehicle track across cameras.

5. The method according to claim 1, wherein The training process of the local vehicle detection model includes: Create a local dataset; The vehicle target detection model is adjusted using the local dataset to obtain a local vehicle detection model.

6. The method according to claim 5, characterized in that The process of building a local dataset includes: Receive road vehicle videos captured by multiple single cameras; Extract video key frames; Annotate the images in the keyframes to generate tracking trajectory annotations.

7. A vehicle tracking device for use across cameras, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for cross-camera vehicle tracking according to any one of claims 1 to 6 when running the program instructions.

8. An electronic device, characterized in that: The invention comprises a vehicle tracking device for crossing cameras as claimed in claim 7.

9. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the processor executes the vehicle tracking method across cameras according to any one of claims 1 to 6.

Images