Cross-lens tracking identification method and device for vehicle, equipment and storage medium

Abstract

The invention discloses a vehicle cross-lens tracking identification method and device, equipment and a storage medium. Comprising the following steps: tracking and identifying a vehicle in a video frame shot by a current camera based on a pre-trained target detection model; when the vehicle passes through a preset overlapping line in the cameras, matching the vehicle identified by the current camera and the vehicle identified by the adjacent previous camera in pairs by adopting a Hungary algorithm to obtain a vehicle matching result; matching results of the same vehicle in all the cameras are connected in series, and a cross-lens vehicle movement track is constructed. Based on a deep learning model, a Hungary algorithm and other methods, effective identification and accurate tracking of the same vehicle among different monitoring cameras are realized, the automation and precision of the system are greatly improved without adding hardware facilities, and powerful support is provided for traffic management.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a method, device, equipment and storage medium for cross-lens tracking and identification of a vehicle. Background Art

[0002] In the field of intelligent traffic monitoring, with the acceleration of urbanization and the continuous growth of traffic flow, higher requirements are placed on the real-time monitoring and management of traffic flow. Traditional monitoring systems often rely on a single camera and simple video analysis technology. These systems have obvious limitations when dealing with vehicle tracking in a multi-camera environment, such as the inability to achieve continuous tracking across cameras, poor adaptability to complex environments, and performance degradation in special environments such as tunnels. In addition, due to hardware limitations and cost considerations, adding additional hardware facilities is not always feasible.

[0003] In recent years, the development of deep learning technology has brought revolutionary changes to the field of traffic monitoring. In particular, object detection models are widely used in intelligent traffic monitoring systems to detect and identify objects in traffic videos, such as vehicles and pedestrians, in real time due to their high efficiency and accuracy. However, a single object detection model often has difficulty in handling cross-camera vehicle tracking problems, especially when vehicles frequently switch camera fields of view.

[0004] Therefore, existing technologies still face challenges in how to achieve seamless tracking across cameras, how to improve the automation and accuracy of the system, and how to make full use of existing monitoring resources without adding additional hardware facilities. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, device and storage medium for tracking and identifying a vehicle across cameras, so as to at least solve the technical problem in the related art that it is difficult to implement tracking and identifying a vehicle across cameras.

[0006] According to one aspect of an embodiment of the present application, a method for cross-shot tracking and identification of a vehicle is provided, comprising:

[0007] Track and identify vehicles in the video frame captured by the current camera based on the pre-trained object detection model;

[0008] When the vehicle crosses an overlap line preset in the camera, a Hungarian algorithm is used to perform pairwise matching of the vehicle identified by the current camera and the vehicle identified by the adjacent previous camera to obtain a vehicle matching result;

[0009] The matching results of the same vehicle in all cameras are connected in series to construct the vehicle motion trajectory across lenses.

[0010] According to another aspect of an embodiment of the present application, a cross-lens tracking and identification device for a vehicle is provided, comprising:

[0011] A tracking and recognition module is used to track and recognize vehicles in the video frame captured by the current camera based on a pre-trained target detection model;

[0012] A vehicle matching module, used for matching the vehicle identified by the current camera with the vehicle identified by the adjacent previous camera using the Hungarian algorithm when the vehicle crosses the overlap line preset in the camera, to obtain a vehicle matching result;

[0013] The full trajectory construction module is used to connect the matching results of the same vehicle in all cameras in series to construct the vehicle motion trajectory across lenses.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the cross-shot tracking and identification method of the vehicle through the computer program.

[0015] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned cross-shot tracking and identification method of the vehicle when running.

[0016] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0017] The vehicle cross-lens tracking and recognition method of the present application combines the target recognition capability of the deep learning target detection model, recognizes the vehicle features in the video frame, and matches the vehicles under adjacent cameras based on the Hungarian algorithm. The tracking accuracy is improved. The cross-lens tracking and recognition capability is further optimized by generating overlapping lines, and in practical applications, near-real-time cross-lens vehicle tracking can be achieved. Through the calculation of overlapping lines, the system can accurately identify the movement path of the vehicle from one camera to another, providing strong support for real-time tracking. This system can make full use of existing video surveillance resources without adding additional hardware facilities, reducing costs and improving resource utilization. In addition, the system reduces the need for manual intervention through automated target detection and vehicle tracking, improves the degree of automation of monitoring, and provides strong support for traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a flow chart of an optional cross-shot tracking and identification method of a vehicle according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a cross-lens tracking and identification method for a vehicle according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a method for calculating similarity based on a VGG model according to an embodiment of the present application;

[0022] Figure 4 This is a flow chart of a matching method based on the Hungarian algorithm according to an embodiment of the present application;

[0023] Figure 5 is a flow chart of overlapping line calculation according to an embodiment of the present application;

[0024] Figure 6 is a real-time operation flow chart of a system according to an embodiment of the present application;

[0025] Figure 7 is a vehicle state flow diagram according to an embodiment of the present application;

[0026] Figure 8 is a schematic diagram of an overlapping line according to an embodiment of the present application;

[0027] Fig. 9 This is a diagram showing the effect of operation of a solution according to an embodiment of the present application;

[0028] Fig.10 is a schematic diagram of a cross-lens tracking and identification device for a vehicle according to an embodiment of the present application;

[0029] Fig.11 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The following is combined with Figure 1-9 The vehicle cross-lens tracking and identification method of the embodiment of the present application is described in detail. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in this respect. On the contrary, the implementation of the present application can be applied to any applicable scenario. Figure 1 As shown, the method mainly includes the following steps:

[0033] S101 tracks and identifies vehicles in the video frame captured by the current camera based on a pre-trained target detection model.

[0034] Specifically, first, a target detection model is trained based on the YOLOv8 neural network; the video frame captured by the current camera is input into the target detection model to obtain the identified vehicle information.

[0035] YOLOv8 is a deep learning model and an advanced algorithm in the field of target detection. It supports image classification, object detection, and instance segmentation tasks. Obtain and annotate the dataset, and train the model based on the training interface provided by YOLOv8. This involves setting model parameters, such as the number of training rounds, the size of the input image, and the batch size, and running the training process on the specified device to obtain the target detection model for vehicle recognition.

[0036] The target detection model is the basic module of the entire system. It is responsible for identifying vehicle targets in each surveillance camera video and extracting the detection frame information of each vehicle, including the detection frame coordinates, vehicle location information, license plate number, appearance features, confidence, etc. Through the continuous monitoring of the target detection model, the system can record the trajectory and motion status of each vehicle in the video, providing the necessary initial data for subsequent cross-camera tracking.

[0037] Furthermore, the identified vehicle is tracked across frames based on a preset target tracking algorithm.

[0038] Among them, the preset target tracking algorithms include BoT-SORT, ByteTrack and other tracking algorithms. Based on the target tracking algorithm, the cross-frame tracking of the vehicle is realized.

[0039] S102 When a vehicle crosses an overlapping line preset in the camera, a Hungarian algorithm is used to perform pairwise matching of a vehicle identified by the current camera and a vehicle identified by an adjacent previous camera to obtain a vehicle matching result.

[0040] In one embodiment, an overlap line is set for each camera, and when a vehicle crosses the overlap line, it indicates that the vehicle has entered the field of view of the front camera from the field of view of the current camera. Then, the Hungarian algorithm is used to match the vehicle identified by the current camera with the vehicle identified by the adjacent previous camera.

[0041] Specifically, the vehicle image recognized by the current camera and the vehicle image recognized by the adjacent previous camera are input into the VGG model to extract vehicle features; the similarity between every two images is calculated based on the extracted vehicle features to obtain the similarity between every two vehicles; based on the similarity between multiple images corresponding to every two vehicles, the average similarity between every two vehicles is calculated.

[0042] In an optional implementation, a preset number of vehicle images with a high confidence level for each vehicle in the current camera are obtained according to a preset time window, such as 0.5 seconds; a preset number of vehicle images with a high confidence level for each vehicle in the adjacent previous camera are obtained; and the confidence level is the confidence level output by the target detection model. For example, 5 frames with the highest confidence level are selected for each vehicle in the current camera and the adjacent previous camera, and vehicle images are captured. A preset number of vehicle images with a high confidence level in the current camera and the adjacent previous camera are input into the VGG model. The specific number of images is not limited in this application.

[0043] Then, the images are compared one-to-one using the VGG model according to the confidence level from high to low. This includes image preprocessing, such as resizing and normalization. The image is resized to the input size required by the model (such as 224x224), and the pixel values ​​are normalized to [0,1].

[0044] Furthermore, the two preprocessed images are respectively input into the VGG model to extract their high-level features. Assuming that the feature vectors extracted from the model for images P1 and P2 are f1 and f2, respectively, then f1 = VGG(P1); f2 = VGG(P2).

[0045] Calculate the similarity of the extracted feature vectors. Commonly used similarity metrics include cosine similarity or Euclidean distance. Cosine similarity formula: Euclidean distance formula:

[0046] Based on the calculated similarity results, the similarity between the two images is determined. Generally, the closer the cosine similarity value is to 1, the more similar the images are; the smaller the Euclidean distance value is, the more similar the images are. This solution uses cosine similarity.

[0047] like Figure 3 The figure shows a similarity calculation process. It includes: selecting 10 frames with the highest confidence for each vehicle, defining the speed limit between adjacent monitoring, such as 5km / h-170km / h, traversing the vehicles of the rear camera, and calculating the first appearance time range of the front camera for each vehicle's first appearance time. Perform similarity calculation for the front camera vehicles within the range. Calculate the similarity of the 10 frames of the highest confidence images of the two vehicles using VGG, and take the average similarity as the similarity between the two vehicles.

[0048] In an exemplary scenario, the current camera identifies three vehicles, namely a, b, and c, and the front camera identifies three vehicles, namely A, B, and C. The similarity between a and A, B, and C, the similarity between b and A, B, and C, and the similarity between c and A, B, and C are calculated respectively. Because there are multiple pictures of vehicles a, b, and c, and there are also multiple pictures of vehicles A, B, and C, the average similarity between the two can be calculated based on the multiple pictures of a and the multiple pictures of A.

[0049] Furthermore, the average similarity between every two vehicles is input into the Hungarian algorithm to obtain the vehicle pairwise matching results.

[0050] Specifically, the present application adopts the Hungarian algorithm for vehicle matching. The Hungarian algorithm is a classic algorithm for solving the optimal matching problem of bipartite graphs. Here, it is suitable for pairwise matching between multiple vehicles. The steps are as follows:

[0051] Step a1: Represent the problem as a matrix, where each element represents the similarity between any two cars. If the matrix is ​​not a square matrix, it needs to be expanded to a square matrix by adding virtual rows or columns. For elements without values, a maximum value of 10 is assigned, so that the value is generally not adopted.

[0052] Step a2: For each row, subtract each element of the row from the minimum value in the row, which ensures that there is at least one zero in each row.

[0053] Step a3: Perform similar operations for each column, subtracting each element of each column from the minimum value in that column, which ensures that there is at least one zero in each column.

[0054] Step a4: Use as few straight lines (horizontal or vertical lines) as possible to cover all zeros in the matrix, and determine whether the number of covered zeros is equal to the dimension of the matrix. If the number of straight lines is equal to the dimension of the matrix, it means that the optimal match has been found. If the number of straight lines is less than the dimension of the matrix, proceed to the next step.

[0055] Step a5: Find the minimum value among the uncovered elements, then subtract the minimum value from the uncovered elements, add the minimum value to the elements that are covered by the intersection of the two straight lines, and keep the other elements unchanged.

[0056] Step a6: Repeat steps a4 and a5 until the number of lines used is equal to the dimension of the matrix.

[0057] Step a7: Determine the optimal allocation through zero elements. Each row and column corresponding to zero represents the matching result of the vehicle of the front camera to the vehicle of the rear camera.

[0058] like Figure 4 As shown in the figure, the matching algorithm process includes: obtaining the similarity comparison result of the VGG model and listing the vehicles involved in the comparison of two adjacent cameras. Define a matrix to record the similarity of the vehicles of the front and rear adjacent cameras. It can be understood that the matrix can be predefined; the similarity between two vehicles is regarded as the weight between two points. For two points that do not have similarity, add the maximum similarity of 10. Add virtual nodes to match the number of vehicles corresponding to the front and rear cameras to be the same. Run the Hungarian algorithm to obtain the global optimal solution.

[0059] In an exemplary scenario, the current camera recognizes three cars, namely a, b, and c, and the front camera recognizes three cars, namely A, B, and C. By matching them pairwise using the Hungarian algorithm, it can be found that a matches A, b matches B, and c matches C.

[0060] S103 connects the matching results of the same vehicle in all cameras in series to construct a vehicle motion trajectory across lenses.

[0061] After obtaining the matching information of the vehicle in different cameras, the trajectory can be connected in series based on the vehicle ID and the time information of the camera shooting, and finally the vehicle movement trajectory across the lenses can be constructed.

[0062] It can be understood that the system also includes configuration calculations before actually running.

[0063] In an optional embodiment, before tracking and identifying a vehicle in a video frame captured by a current camera based on a pre-trained target detection model, the method further includes:

[0064] Select videos shot by multiple continuous surveillance cameras within a preset time period; track and identify vehicles in the video frames of each camera based on the target detection model; obtain the time difference range between adjacent cameras based on the maximum and minimum vehicle speeds within the monitoring range and the distance between cameras; and obtain vehicle recognition results that meet the time difference range.

[0065] Specifically, video clips within the same time period are selected from continuous surveillance cameras to ensure that the time periods covered by these videos overlap, and it is usually recommended to be at least 10 minutes in order to capture sufficient vehicle crossing information.

[0066] Furthermore, advanced target detection models such as YOLOv8 are used to detect vehicles in the video of each camera, and tracking algorithms such as BoT-SORT and ByteTrack are used to achieve cross-frame tracking of the same vehicle.

[0067] Furthermore, according to the actual monitoring range and traffic conditions, the maximum and minimum speeds of the vehicle are set, such as 5km / h to 160km / h. According to the speed range and the distance between the cameras, the time difference range of the same vehicle appearing between adjacent surveillance cameras is calculated to provide a time reference for subsequent matching. For example, if the distance between the cameras is 150 meters and the speed range is 5km / h to 160km / h, then the time interval for the first appearance of the same vehicle between adjacent cameras should be 3.375s to 108s. Vehicles within this time interval are matched.

[0068] The vehicle recognition records that meet the time difference are selected, and the vehicles in the videos shot by adjacent cameras are matched pairwise based on the VGG model and the Hungarian algorithm to generate a cross-shot video of the same vehicle. For the specific similarity calculation method and the Hungarian matching algorithm, refer to step S102 and will not be described in detail here.

[0069] The cross-lens videos of the same vehicle are sent to manual review, which manually identifies the videos and selects the correct matches. The correct cross-lens videos are obtained.

[0070] Furthermore, the overlapping lines of adjacent cameras that match the correct cross-lens video are calculated, and the overlapping line pixel coordinate expression of each camera is saved. That is, the same vehicle should theoretically be recognized by the next camera at the same time after passing the overlapping line. For those without overlapping lines, a special expression formed by the pixel coordinates of the upper right corner of the video is assigned.

[0071] In an optional embodiment, the overlap line of adjacent cameras that match the correct cross-lens video is calculated, including: for each pair of adjacent cameras, recording the time when the vehicle first appears and the coordinates of the matching vehicle in the front camera; forming a cluster of multiple points based on all correctly matched vehicle coordinates, randomly selecting K initial centroids, and assigning data points to the nearest centroid cluster according to distance; repeatedly calculating the mean of the cluster as the new centroid, and redistributing the data points until the centroid is stable or the iteration limit is reached; finding a straight line that minimizes the vertical distance through the centroid, and translating the straight line to the lower left corner so that all points are located to the upper right of the straight line; determining and expressing the translated straight line, and recording the straight line expression as the overlap line of the front lens.

[0072] Specifically, the following detailed steps can be used to generate overlapping lines:

[0073] Step a: For every two adjacent cameras, record the first appearance time t_post of the vehicle in the rear camera;

[0074] Step b: in the video frame of the front camera, find the first frame that is greater than or equal to the first appearance time t_post, and record the pixel coordinates (x, y) of the lower left corner of the vehicle detection frame of the first frame;

[0075] Step c: Repeat steps a and b, record the lower left corner pixel coordinates of all correct vehicle matches, and form a cluster of multiple points;

[0076] Step d: Randomly select K initial centroids; these centroids are usually selected from data points;

[0077] Step e: Calculate the distance from each data point to all centroids and assign each data point to the cluster to which the nearest centroid belongs;

[0078] Step f: For each cluster, recalculate the mean of all data points in the cluster and use the mean as the new centroid; the centroid is the average value of all data points in the cluster in each dimension;

[0079] Step g: Repeat step e and step f until the centroid no longer changes or a predetermined number of iterations is reached;

[0080] Step h: draw a straight line through the centroid so that the vertical distance from all centroids to the straight line is the minimum;

[0081] Step i: Translate the straight line to the lower left corner so that all points are located at the upper right of the straight line, and use this straight line as the expression of the overlapping line.

[0082] like Figure 5The figure shows a flow chart for generating overlapping lines, including: obtaining the final matching data, and generating a single-vehicle cross-border video according to the final matching result. Manually browsing the video and recording the correct matching results. Corresponding the first appearance time of the rear vehicle to the detection frame of the front vehicle. Recording the lower left corner coordinates of the detection frame, recording all the paired lower left corner coordinates, and the program draws a straight line to draw all coordinate points to the upper right of the straight line. Recording the straight line expression as the overlapping line of the front lens.

[0083] like Figure 8 FIG. 1 is a schematic diagram of a generated overlap line. An overlap line can be seen in the shot. According to this step, the overlap line can be pre-configured. The cross-shot tracking recognition capability is further optimized by generating the overlap line.

[0084] In order to facilitate understanding of the method of the embodiment of the present application, Figure 2 Further description. Figure 2 As shown, the vehicle cross-shot tracking and identification method of the present application includes a configuration calculation phase and a normal operation phase.

[0085] In the configuration calculation stage, we first obtain continuous camera videos, run target detection and tracking to identify the vehicle in each frame, define the time range of adjacent cameras to splice vehicles, match the vehicles of adjacent cameras, generate matching videos and manually screen them, and form the overlapping line expression of adjacent cameras.

[0086] During normal operation, configure the overlapping lines of each camera, run target detection to track vehicles in real time, and segment cross-border vehicles through overlapping lines for matching.

[0087] like Figure 6 As shown in the figure, when the cross-lens recognition system is running, the target detection tracks the vehicle of each lens in real time, updates the matching status of the tracked vehicle every s seconds, matches the vehicle to be matched backward in the front lens with the vehicle to be matched forward in the rear lens, records the matching results of adjacent lenses, and updates the vehicle matching status.

[0088] like Figure 7 As shown, the matching status of the vehicle includes: tracking status, at this time the vehicle is being tracked and identified, and has not passed the overlap line. To-be-matched status, at this time the vehicle has crossed the overlap line. Matching failure status, the vehicle matching waiting time exceeds the threshold. Front matching failure: the vehicle successfully matches backwards, and the forward matching waiting time exceeds the threshold. To-be-matched front: the vehicle successfully matches backwards, and waits for forward matching. To-be-matched rear: the vehicle successfully matches forwards, and waits for backward matching. Rear matching failure: the vehicle successfully matches forwards, and the backward matching waiting time exceeds the threshold. Matched: the same vehicle in the front and rear cameras is successfully matched.

[0089] like Fig. 9As shown in FIG. 1 , in multiple consecutive camera images, the same vehicle can be tracked across lenses. The box in the figure indicates the vehicle to be tracked.

[0090] The solution disclosed in this application combines the target dynamic tracking capability of the deep learning target detection model and the image similarity matching capability of the VGG model, forms the optimal solution under known conditions through the Hungarian algorithm, and further optimizes the cross-lens tracking and recognition capability by generating overlapping lines. In practical applications, it can achieve near real-time cross-lens vehicle tracking. This system can make full use of existing video surveillance resources without adding additional hardware facilities, reducing costs and improving resource utilization. In addition, the system reduces the need for manual intervention through automated target detection and vehicle tracking, improves the degree of automation of monitoring, and provides strong support for traffic management.

[0091] According to another aspect of the embodiment of the present application, a vehicle cross-shot tracking and identification device for implementing the above-mentioned vehicle cross-shot tracking and identification method is also provided. Fig.10 As shown, the device comprises:

[0092] A tracking and identification module 1001 is used to track and identify vehicles in a video frame captured by a current camera based on a pre-trained target detection model;

[0093] The vehicle matching module 1002 is used to use the Hungarian algorithm to perform pairwise matching between the vehicle identified by the current camera and the vehicle identified by the adjacent previous camera when the vehicle crosses the overlap line preset in the camera, so as to obtain a vehicle matching result;

[0094] The full trajectory construction module 1003 is used to connect the matching results of the same vehicle in all cameras in series to construct the vehicle motion trajectory across lenses.

[0095] It should be noted that the cross-lens tracking and identification device for a vehicle provided in the above embodiment only uses the division of the above functional modules as an example when executing the cross-lens tracking and identification method for a vehicle. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the cross-lens tracking and identification device for a vehicle provided in the above embodiment and the cross-lens tracking and identification method for a vehicle are of the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0096] According to another aspect of an embodiment of the present application, an electronic device corresponding to the cross-shot tracking and identification method of a vehicle provided in the aforementioned embodiment is also provided to execute the above-mentioned cross-shot tracking and identification method of a vehicle.

[0097] Please refer to Fig.11, which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Fig.11 As shown, the electronic device includes: a processor 1100, a memory 1101, a bus 1102 and a communication interface 1103, and the processor 1100, the communication interface 1103 and the memory 1101 are connected via the bus 1102; the memory 1101 stores a computer program that can be run on the processor 1100, and when the processor 1100 runs the computer program, it executes the cross-shot tracking and identification method of the vehicle provided in any of the aforementioned embodiments of the present application.

[0098] The memory 1101 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 1103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0099] The bus 1102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 1101 is used to store programs, and the processor 1100 executes the programs after receiving the execution instructions. The vehicle cross-lens tracking and recognition method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 1100, or implemented by the processor 1100.

[0100] The processor 1100 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1100. The above processor 1100 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1101, and the processor 1100 reads the information in the memory 1101 and completes the steps of the above method in combination with its hardware.

[0101] The electronic device provided in the embodiment of the present application and the cross-lens tracking and identification method of a vehicle provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0102] According to another aspect of the embodiments of the present application, a computer-readable storage medium corresponding to the cross-shot tracking and identification method of a vehicle provided in the aforementioned embodiments is also provided, on which a computer program (i.e., a program product) is stored. When the computer program is executed by a processor, the cross-shot tracking and identification method of a vehicle provided in any of the aforementioned embodiments will be executed.

[0103] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0104] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the cross-lens tracking and identification method of a vehicle provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0105] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

Claims

1. A vehicle cross-lens tracking and recognition method, characterized in that: include: Track and identify vehicles in the video frame captured by the current camera based on the pre-trained object detection model; When the vehicle crosses an overlap line preset in the camera, a Hungarian algorithm is used to perform pairwise matching of the vehicle identified by the current camera and the vehicle identified by the adjacent previous camera to obtain a vehicle matching result; The matching results of the same vehicle in all cameras are connected in series to construct the vehicle motion trajectory across lenses.

2. The method according to claim 1, characterized in that The Hungarian algorithm is used to match the vehicle identified by the current camera with the vehicle identified by the previous camera in pairs, and the vehicle matching results are obtained, including: Input the vehicle image recognized by the current camera and the vehicle image recognized by the previous camera into the VGG model to extract vehicle features; The similarity between every two images is calculated based on the extracted vehicle features to obtain the similarity between every two vehicles; Based on the similarity between multiple pictures corresponding to each two cars, the average similarity between each two cars is calculated; The average similarity between every two vehicles is input into the Hungarian algorithm to obtain the vehicle pairwise matching results.

3. The method according to claim 2, characterized in that The vehicle image recognized by the current camera and the vehicle image recognized by the previous camera are input into the VGG model, including: Obtain a preset number of vehicle images with a high confidence level for each vehicle in the current camera; obtain a preset number of vehicle images with a high confidence level for each vehicle in the adjacent previous camera; the confidence level is the confidence level output by the target detection model; A preset number of vehicle images with high confidence from the current camera and the adjacent previous camera are input into the VGG model.

4. The method according to claim 1, characterized in that: Before tracking and identifying the vehicle in the video frame captured by the current camera based on the pre-trained target detection model, it also includes: Select videos shot by multiple continuous surveillance cameras within a preset time period; Based on the VGG model and the Hungarian algorithm, vehicles in videos shot by adjacent cameras are matched pairwise to generate cross-shot videos of the same vehicle. The cross-shot video of the same vehicle is sent to manual review to obtain a correctly matched cross-shot video; Calculate the overlapping lines of adjacent cameras that match the correct cross-lens video, and save the overlapping line pixel coordinate expression of each camera.

5. The method according to claim 4, characterized in that Before pairwise matching of vehicles in videos shot by adjacent cameras based on the VGG model and the Hungarian algorithm, the following steps are also included: Track and identify vehicles in each camera’s video frame based on the object detection model; Based on the maximum and minimum vehicle speeds within the monitoring range and the distance between cameras, the time difference range between adjacent cameras is obtained; The time range for selecting the monitored vehicle is determined based on the time difference range, and a vehicle identification result that meets the time range is obtained.

6. The method according to claim 4, characterized in that Calculating the overlapping lines of adjacent cameras of the correctly matched cross-lens video includes: For each pair of adjacent cameras, record the time when the vehicle first appears and the coordinates of the matching vehicle in the front camera; Form clusters of multiple points based on all correctly matched vehicle coordinates, randomly select K initial centroids, and assign data points to the nearest centroid cluster based on distance; Repeatedly calculate the cluster mean as the new centroid and redistribute the data points until the centroid is stable or the iteration limit is reached; Find the line that minimizes the vertical distance through the centroid, and translate the line to the lower left corner so that all points are located to the upper right of the line; The translated straight line is determined and expressed, and the straight line is used as an expression of the overlapping line.

7. The method according to claim 1, characterized in that The vehicle identification and matching process when the system is running includes: Track vehicles in the camera lens in real time through the object detection model; Match the vehicle to be matched backwards by the front camera with the vehicle to be matched forwards by the rear camera, and periodically update the matching status of the vehicles; The matching status includes one or more of a tracking status, a pending matching status, a pre-matching failure status, a post-matching failure status, and a matched status.

8. A vehicle cross-lens tracking and identification device, characterized in that: include: A tracking and recognition module is used to track and recognize vehicles in the video frame captured by the current camera based on a pre-trained target detection model; A vehicle matching module, used for matching the vehicle identified by the current camera with the vehicle identified by the adjacent previous camera using the Hungarian algorithm when the vehicle crosses the overlap line preset in the camera, to obtain a vehicle matching result; The full trajectory construction module is used to connect the matching results of the same vehicle in all cameras in series to construct the vehicle motion trajectory across lenses.

9. An electronic device, characterized in that: The invention comprises a processor and a memory storing program instructions, wherein the processor is configured to execute the cross-shot tracking and identification method of a vehicle as claimed in any one of claims 1 to 7 when executing the program instructions.

10. A computer-readable medium, characterized in that Computer-readable instructions are stored thereon, and the computer-readable instructions are executed by a processor to implement a cross-shot tracking and identification method for a vehicle as described in any one of claims 1 to 7.

Images