Vehicle Re-identification Method, Device and Equipment Incorporating Monitoring Network Topology Information

Abstract

The present invention provides a vehicle re-identification method, device and equipment that fuse monitoring network topology information. The vehicle re-identification method that fuses monitoring network topology information includes: obtaining an initial image data set and processing the initial image data set; using the processed initial image data set as the input of a preset convolutional neural network for vehicle re-identification to obtain a vehicle re-identification model; inputting a target vehicle image to be identified into the vehicle re-identification model to obtain a feature tensor of the target vehicle image to be identified, and determining the similarity between the template vehicle image to be identified and each image in the gallery set based on the feature tensor to form a preliminary result of vehicle re-identification; sorting the preliminary result of vehicle re-identification to obtain a final vehicle re-identification result. The present invention can directly perform vehicle re-identification on traffic monitoring videos, effectively improving the accuracy of vehicle re-identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a vehicle re-identification method, device and equipment that integrate monitoring network topology information. Background Art

[0002] With the prosperous development of economy and technology, on the one hand, the coverage rate of monitoring videos is getting higher and higher, and government administrators are gradually aware of the application prospects of image-based intelligent technologies in many aspects and scenarios such as public security, traffic management, crime fighting, criminal investigation evidence collection, etc., such as face recognition, abnormal event detection, etc. On the other hand, the number of civilian cars in possession is increasing, and vehicles have gradually become an indispensable part of current social life and an important participant in public scenes and traffic roads. Therefore, vehicle-related visual tasks emerge in an endless stream and have also received widespread attention from the academic and industrial circles. Vehicle Re-Identification refers to re-identifying the same vehicle that appears in non-overlapping views, that is, judging whether the car images collected from different monitoring cameras belong to the same vehicle, which has positive significance for fields such as traffic management and public security.

[0003] The research on vehicle re-identification mainly goes through three stages: In the 1990s, due to the imperfect construction of video monitoring and the immaturity of image processing technology, sensor-based methods were generally adopted. Due to hardware dependence, the application scenarios were limited to highways and the accuracy was also greatly defective; at the beginning of the 21st century, image processing technology became increasingly mature, and researchers began to use traditional manually designed features to identify and match vehicles. First, traditional image processing technology was used to obtain vehicle feature attributes such as color and corner information, and then the feature matching problem was transformed into a probability classification problem of judging whether two vehicle images from two different cameras came from the same vehicle, and a classifier was trained to achieve the vehicle re-identification goal. After 2012, the development of deep learning was in full swing, which promoted the vehicle re-identification task to return to the vision of the industry and research, and it is also the main means adopted by the current mainstream vehicle re-identification methods. Its main steps are to construct a deep learning network architecture, use a large amount of vehicle image data to perform supervised learning on vehicle features, map the vehicle image learning into the feature space, and calculate the metric similarity of the vehicle image in the feature space, believing that the more similar the vehicle images are, the more likely their identities are the same.

[0004] There are three main methods used in existing similar vehicle re-identification technologies. One is to combine global and local features, train the global feature network and the local feature network to obtain corresponding features, and then assign appropriate weights to the global features and local features to achieve a better feature mapping relationship, such as PGAN, etc.; one is multi-feature network splicing, that is, training multiple feature networks, and then splicing the trained feature networks together as the final feature vector, such as VehicleNet, etc.; another is to use multi-dimensional information, such as vehicle model, vehicle color, license plate number, timestamp and other information, to split the re-identification process into a multi-step matching process, which improves the accuracy of vehicle re-identification. The representative system is PROVID. These methods all have a unified feature: multiple neural networks need to be trained, and then reasonably assembled to complete the final re-identification result. The overall process requires a lot of manual docking operations, and it is difficult to form an automated application pipeline.

[0005] Although more and more researchers have noticed the challenges brought by the vehicle re-identification problem, the accuracy that the industry can achieve is only satisfactory. The main difficulties lie in two points. First, the differences between vehicle types are small, and different vehicles may only be distinguished by relying on smaller details; second, the imaging differences are large. Different images of the same vehicle may have large differences in angle, lighting, etc. Summary of the invention

[0006] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a vehicle re-identification method, device and equipment integrating monitoring network topology information, so as to improve the accuracy of vehicle re-identification.

[0007] To achieve the above-mentioned purpose and other related purposes, an embodiment of the present invention provides a vehicle re-identification method integrating monitoring network topology information, comprising: obtaining an initial image data set and processing the initial image data set; using the processed initial image data set as an input of a preset convolutional neural network for vehicle re-identification to obtain a vehicle re-identification model; inputting a target vehicle image to be identified into the vehicle re-identification model to obtain a feature tensor of the target vehicle image to be identified, and determining the similarity between the template vehicle image to be identified and each image in a gallery set based on the feature tensor to form a preliminary vehicle re-identification result; and sorting the preliminary vehicle re-identification results to obtain a final vehicle re-identification result.

[0008] In one embodiment of the present invention, the initial image data set includes: a training set and a test set; the test set includes a query set and a gallery set.

[0009] In one embodiment of the present invention, the processing of the initial image data set includes: performing any one or more combinations of horizontal flipping, random erasing, adding random patches, and color jittering on the initial image data.

[0010] In one embodiment of the present invention, the backbone network of the preset convolutional neural network is a deep residual network based on IBN-Net. The average pooling layer of the deep residual network based on IBN-Net is replaced with an adaptive average pooling layer. A batch normalization layer is added after the last fully connected layer to reduce the feature dimension, and then a fully connected layer is connected to output the final prediction result.

[0011] In one embodiment of the present invention, the preset convolutional neural network calculates the loss with a mixed loss function of triplet loss and cross-entropy loss; the preset convolutional neural network uses a stochastic gradient descent optimizer to calculate the gradient and iteratively update the parameters; a warm-up learning rate combined with cosine annealing is adopted to adjust the learning rate as the learning rate adjustment strategy; non-local blocks are respectively added to the tails of the last half of the residual blocks in the third and fourth layers of the deep residual network to capture long-range dependencies.

[0012] In one embodiment of the present invention, an implementation manner for determining the similarity between the template vehicle image to be recognized and each image in the gallery set based on the feature tensor includes: obtaining the feature tensors of each image in the gallery set; using the preset convolutional neural network to obtain the query feature tensor of the template vehicle image to be recognized; calculating the Euclidean distance between the query feature tensor of the template vehicle image to be recognized and the feature tensors of each image in the gallery set; determining the similarity based on the Euclidean distance, where the smaller the Euclidean distance, the higher the similarity.

[0013] In one embodiment of the present invention, an implementation manner for sorting the preliminary results of vehicle re-identification to obtain the final vehicle re-identification result includes: maintaining the monitoring network topology structure information in the form of an undirected graph; taking the template vehicle images to be recognized with a similarity above a preset value as candidate images and saving them in descending order; dividing the top k candidate images into a sorted list; determining the set of image IDs of all candidate images based on a confidence strategy; updating the sorted list based on whether the image IDs of each candidate image exist in a preset candidate image ID set; obtaining the final vehicle re-identification result according to the updated sorted list: the image ID with the highest similarity and meeting the similarity threshold is used as the final recognition result.

[0014] An embodiment of the present invention further provides a vehicle re-identification device that integrates monitoring network topology information, including: a vehicle detection and tracking module for acquiring an initial image dataset and processing the initial image dataset; a vehicle re-identification calculation module for using the processed initial image dataset as the input of a preset convolutional neural network for vehicle re-identification, obtaining a vehicle re-identification model, inputting a target vehicle image to be identified into the vehicle re-identification model, obtaining a feature tensor of the target vehicle image to be identified, and determining the similarity between the template vehicle image to be identified and each image in the gallery set based on the feature tensor, forming a preliminary result of vehicle re-identification, and sorting the preliminary result of vehicle re-identification to obtain a final vehicle re-identification result.

[0015] In an embodiment of the present invention, the backbone network of the preset convolutional neural network is a deep residual network based on IBN-Net, and the average pooling layer of the deep residual network based on IBN-Net is replaced with an adaptive average pooling layer. A batch normalization layer is added after the last fully connected layer to reduce the feature dimension, and then a fully connected layer is connected to output a final prediction result.

[0016] An embodiment of the present invention further provides an electronic device, including a processor and a memory, where the memory stores program instructions; the processor runs the program instructions to implement the vehicle re-identification method that integrates monitoring network topology information as described above.

[0017] As described above, the vehicle re-identification method, device, and equipment that integrate monitoring network topology information of the present invention have the following beneficial effects:

[0018] The present invention can directly perform vehicle re-identification on traffic monitoring videos, effectively improving the accuracy of vehicle re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It shows a schematic flowchart of the vehicle re-identification method that integrates monitoring network topology information of the present invention.

[0020] Figure 2 It shows a schematic principle structure diagram of the vehicle re-identification device that integrates monitoring network topology information of the present invention.

[0021] Figure 3 It shows a schematic operation principle diagram in the vehicle re-identification device that integrates monitoring network topology information of the present invention.

[0022] Figure 4 It shows a schematic structure diagram of an electronic device in an embodiment of the present application.

[0023] Description of Component Labels

[0024] 10 Electronic device

[0025] 1101 Processor

[0026] 1102 Memory

[0027] 100 Vehicle Re-identification Device Integrating Monitoring Network Topology Information

[0028] 110 Vehicle Detection and Tracking Module

[0029] 120 Vehicle Re-identification Calculation Module

[0030] Steps S100 - S400 Specific Embodiment

[0031] The following uses specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0032] The purpose of this embodiment is to provide a vehicle re-identification method, device and equipment integrating monitoring network topology information, for improving the accuracy of vehicle re-identification.

[0033] The following will elaborate in detail on the principle and implementation of the vehicle re-identification method, device and equipment integrating monitoring network topology information in this embodiment, so that those skilled in the art can understand the vehicle re-identification method, device and equipment integrating monitoring network topology information of the present invention without creative labor.

[0034] Example 1

[0035] As Figure 1 shown, this embodiment provides a vehicle re-identification method integrating monitoring network topology information, including:

[0036] Step S100: Obtain an initial image dataset and process the initial image dataset;

[0037] Step S200: Use the processed initial image dataset as the input of a vehicle re-identification preset convolutional neural network to obtain a vehicle re-identification model;

[0038] Step S300: Input the target vehicle image to be recognized into the vehicle re-identification model to obtain the feature tensor of the target vehicle image to be recognized, and determine the similarity between the template vehicle image to be recognized and each image in the gallery set based on the feature tensor to form a preliminary vehicle re-identification result;

[0039] Step S400: Sort the preliminary vehicle re-identification result to obtain the final vehicle re-identification result.

[0040] The steps S100 to S400 of the vehicle re-identification method integrating the monitoring network topology information in this embodiment are described in detail below.

[0041] Step S100: Obtain the initial image dataset and process the initial image dataset.

[0042] Obtain the initial image dataset and perform preprocessing operations such as partitioning and data augmentation on it.

[0043] Specifically, in this embodiment, the initial image dataset includes: a training set and a test set; the test set includes a query set and a gallery set.

[0044] In this embodiment, the processing of the initial image dataset includes: performing any one or a combination of horizontal flipping, random erasing, adding random patches, and color jittering on the initial image data.

[0045] That is, in this embodiment, after obtaining the dataset, the dataset is divided into a training set and a test set according to a reasonable ratio, and the test set is further divided into a query set and a gallery set; a horizontal flipping enhancement operation is performed on the initial training images, that is, the images are flipped left and right; a random erasing enhancement operation is performed on the initial training images, that is, the pixels in a randomly selected rectangular area are replaced with random values.

[0046] Specifically, in this embodiment, to prepare a dedicated dataset for the vehicle re-identification task, it is necessary to include a large number of pictures of vehicles taken by multiple traffic monitoring cameras, with complete annotation information, and the dataset is divided into a training set and a test set according to a suitable ratio, and the training set is further divided into a query set and a gallery set. In the experimental stage of the present invention, the open-source datasets VeRi and VeRi-Wild are mainly used for training.

[0047] Load the training set and perform data augmentation processing on the vehicle images in the training set. Specifically, the data augmentation processing includes:

[0048] 1) Horizontal flipping, that is, mirror-flipping the original image, saving it as a new image, and the annotation information of the new image obtained after horizontal flipping is the same as that of the original image.

[0049] 2) Random erasing, that is, randomly selecting the original image for erasing processing according to a certain probability. If the original image is selected for random erasing processing, a rectangular area is randomly selected in the original image according to the given area ratio range and aspect ratio, and all its pixel values are modified to random values.

[0050] 3) In addition, according to the training results, various data augmentation methods such as adding random patches and color jitter can be tried.

[0051] Step S200: Use the processed initial image dataset as the input of the preset convolutional neural network for vehicle re-identification to obtain a vehicle re-identification model.

[0052] In this embodiment, the backbone network of the preset convolutional neural network is a deep residual network based on IBN-Net. The average pooling layer of the deep residual network based on IBN-Net is replaced with an adaptive average pooling layer. A batch normalization layer is added after the last fully connected layer to reduce the feature dimension, and then a fully connected layer is connected to output the final prediction result.

[0053] Furthermore, in this embodiment, the preset convolutional neural network calculates the loss using a mixed loss function of triplet loss and cross-entropy loss; the preset convolutional neural network uses a Stochastic Gradient Descent (SGD) optimizer to calculate the gradient and iteratively update the parameters; the learning rate adjustment strategy is to use Warmup combined with Cosine Annealing LR; non-local blocks are added to the tails of the last half of the residual blocks in the third and fourth layers of the deep residual network to increase the receptive field, capture long-range dependencies, and simply implement the self-attention mechanism.

[0054] Among them, the partial batch normalization processing is replaced with Instance Batch Normalization (IBN). The specific replacement part is as Figure 1 shown. IBN unifies Instance Normalization (IN) and Batch Normalization (BN), that is, the input is evenly distributed to a BN module and an IN module, and then the respective outputs are connected as the overall output. Utilize the appearance invariance of IBN itself to enhance the generalization ability of the model and reduce the influence of different imaging qualities and backgrounds.

[0055] Replace the original average pooling layer in ResNet-50 with an adaptive average pooling layer to automatically adjust the stride and kernel size and accelerate the training process.

[0056] The entire model calculates the loss using a hybrid loss function that combines the cross-entropy loss function and the triplet loss function. That is, the cross-entropy loss function is used to calculate the classification results obtained by Softmax, and the triplet loss function is used to calculate the feature representation vectors. The loss values of the two are added together as the total loss value for training convergence.

[0057] The Stochastic Gradient Descent (SGD) optimizer is used, combined with the Cosine Annealing LR strategy for Warmup to adjust the learning rate, update the parameters, and achieve better training results.

[0058] Step S300: Input the target vehicle image to be recognized into the vehicle re-identification model to obtain the feature tensor of the target vehicle image to be recognized, and determine the similarity between the template vehicle image to be recognized and each image in the gallery set based on the feature tensor, forming a preliminary result of vehicle re-identification.

[0059] In this embodiment, one implementation of determining the similarity between the template vehicle image to be recognized and each image in the gallery set based on the feature tensor includes:

[0060] 1) Obtain the feature tensors of each image in the gallery set;

[0061] 2) Use a preset convolutional neural network to obtain the query feature tensor of the template vehicle image to be recognized;

[0062] 3) Calculate the Euclidean distance between the query feature tensor of the template vehicle image to be recognized and the feature tensors of each image in the gallery set;

[0063] 4) Determine the similarity based on the Euclidean distance. Among them, the smaller the Euclidean distance, the higher the similarity.

[0064] The feature values of the input image are obtained using the above network, and the Euclidean distance is calculated with the feature values of the gallery set images. The specific calculation formula is:

[0065]

[0066] Among them, f(a) and f(b) are the feature tensors of two vehicle images respectively, n is the feature dimension, and the smaller the calculated distance value d, the higher the similarity score.

[0067] Arrange the similarities between the input image and the images in the gallery set in descending order, and save the top n (n>4) recognition results as the preliminary recognition result candidate list.

[0068] Step S400: sorting the preliminary vehicle re-identification results to obtain final vehicle re-identification results.

[0069] In this embodiment, one implementation method of sorting the preliminary vehicle re-identification results to obtain the final vehicle re-identification results includes:

[0070] 1) Maintain monitoring network topology information in the form of an undirected graph;

[0071] 2) The template vehicle images to be identified with similarity above a preset value are taken as candidate images and saved in order from high to low;

[0072] 3) Divide the top k candidate images into a ranked list;

[0073] 4) Determine the image ID set of all candidate images based on the confidence strategy;

[0074] 5) updating the sorting list based on whether the image ID of each candidate image exists in the preset candidate image ID set;

[0075] 6) Obtain the final vehicle re-identification result according to the updated sorted list: the image ID with the highest similarity and meeting the similarity threshold is taken as the final recognition result.

[0076] Specifically, the process of obtaining vehicle re-identification results is as follows:

[0077] 1) Maintain monitoring network topology information in the form of an undirected graph;

[0078] 2) Save the candidate images with similarity scores above t in order from high to low;

[0079] 3) Divide the first k (k>2) candidate images into rank_list;

[0080] 4) Adopt a confident strategy, that is, assume that the first two results with the highest similarity scores are trustworthy, and set the trust base, that is, find all paths and nodes between rank_list[0] and rank_list[1], that is, the camera_id set of all candidate images, set as candidate_cids;

[0081] 5) Traverse rank_list from index 2 and maintain a variable drift_num with an initial value of 0, which indicates the number of candidate images whose camera_id does not belong to candidate_cids. For each candidate image: if the camera_id exists in candidate_cids, increase the similarity score of the current candidate image by e; if not, increase drift_num by 1;

[0082] 6) Sort according to the latest similarity scores, and determine whether drift_num exceeds the threshold. If it exceeds, reduce e, decrement trust_base by 1, and return to step 5; otherwise, output the latest rank_list as the result.

[0083] In this embodiment, after obtaining the vehicle re-identification result, the training parameters can be adjusted according to the vehicle re-identification result, and data augmentation means can be increased or decreased according to the training and testing results. After multiple optimizations, the final preset vehicle re-identification network model is obtained.

[0084] Example 2

[0085] As Figure 2 shown, a vehicle re-identification device 100 integrating monitoring network topology information in this embodiment, the vehicle re-identification device 100 integrating monitoring network topology information includes: a vehicle detection and tracking module 110 and a vehicle re-identification calculation module 120.

[0086] In this embodiment, the vehicle detection and tracking module 110 is used to obtain an initial image dataset and process the initial image dataset.

[0087] The user extracts the bounding box of the vehicle from the image frames of the video stream and crops it as the input image of the preset vehicle re-identification network.

[0088] Specifically, the vehicle detection and tracking module 110 trains the YOLOv5 network using the COCO open-source dataset to detect vehicles in the monitoring video stream and obtain vehicle bounding boxes.

[0089] In this embodiment, as Figure 3 shown, the vehicle re-identification device 100 integrating monitoring network topology information mainly uses YOLOv5 as the object detection tool, uses the preset vehicle re-identification network implemented above, and uses DeepSORT as the object tracking tool to implement a three-stage vehicle re-identification device for directly processing the monitoring video stream.

[0090] In this embodiment, the vehicle detection and tracking module 110 trains the YOLOv5 network using the open-source dataset COCO or directly downloads the open-source YOLOv5 weights, removes all other classes and only saves vehicles as the classification result, and constructs a vehicle object detector for YOLOv5. The vehicle detection and tracking module 110 converts the format of the open-source dataset VeRi into the dataset format supported by DeepSORT, and then uses it to train the DeepSORT deep appearance model. It can also directly download the pre-trained model based on the pedestrian dataset, but the accuracy of the latter will be affected, and a DeepSORT vehicle tracker is constructed.

[0091] Among them, the vehicle detection and tracking module 110 uses the DeepSORT algorithm to achieve stable tracking of vehicle identities in the video stream: performs non-maximum suppression screening on the vehicle bounding box images detected by YOLOv5 that meet the confidence threshold, inputs them into DeepSORT for prediction and matching. After cascade matching and IOU matching, the tracker and the feature set are updated simultaneously, and the vehicle ID is recorded. That is, the vehicle detection and tracking module 110 first uses YOLOv5 as the object detection network. After obtaining the vehicle detection result, it is input into the DeepSort algorithm module to achieve the tracking of the vehicle.

[0092] In addition, the other specific implemented technical features of the vehicle detection and tracking module 110 are basically the same as those in step S100 of the vehicle re-identification method for fusing monitoring network topology information in Embodiment 1, and the technical content that can be shared among the embodiments will not be repeated.

[0093] In this embodiment, the vehicle re-identification calculation module 120 is used to take the processed initial image dataset as the input of the preset convolutional neural network for vehicle re-identification, obtain the vehicle re-identification model, input the target vehicle image to be identified into the vehicle re-identification model, obtain the feature tensor of the target vehicle image to be identified, and determine the similarity between the template vehicle image to be identified and each image in the gallery set based on the feature tensor, form a preliminary result of vehicle re-identification, and sort the preliminary result of vehicle re-identification to obtain the final result of vehicle re-identification.

[0094] The vehicle re-identification calculation module 120 takes the vehicle bounding box obtained in the previous step as the vehicle image to be queried, inputs it together with the existing vehicle images into the preset vehicle re-identification network, obtains the previous re-identification relationship between the vehicle image to be queried and the existing vehicle images, and assigns an identity to the vehicle to be queried according to the result.

[0095] In this embodiment, the vehicle re-identification calculation module 120 inputs the vehicle bounding box image detected by YOLOv5 into the above-mentioned preset vehicle re-identification network to obtain the vehicle re-identification result, that is, the vehicle identity information. If the similarity threshold is met, the identity information is updated to the tracking ID list of DeepSORT; otherwise, a new vehicle identity is created.

[0096] The specific technical features implemented by the vehicle re-identification calculation module 120 are basically the same as those in steps S200 to S400 of the vehicle re-identification method for fusing monitoring network topology information in Embodiment 1. The technical content that can be shared among embodiments will not be repeated here.

[0097] In addition, the vehicle re-identification device 100 for fusing monitoring network topology information stores the results of identification, tracking, and identity information, and prints the information in the video output stream.

[0098] It should be noted that it should be understood that the division of each module of the above device is only a logical function division. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in hardware. For example, the vehicle detection and tracking module 110 can be a separately established processing element, or can be integrated in a certain chip of an electronic terminal. In addition, it can also be stored in the memory of the above terminal in the form of program code, and the function of the above tracking calculation module is called and executed by a certain processing element of the above terminal. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together or can be independently implemented. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit or software-form instructions in the processor element.

[0099] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain above module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0100] Example 3

[0101] As Figure 4 shown, this embodiment also provides an electronic device 10, and the electronic device 10 includes a processor 1101 and a memory 1102.

[0102] The electronic device 100 may be, for example, a fixed terminal, such as a server, a desktop computer, etc.; it may also be a mobile terminal, such as a notebook computer, a smart phone or a tablet computer, etc., or it may also be a vehicle-mounted terminal, etc.

[0103] The memory 1102 is connected to the processor 1101 through a system bus and completes mutual communication. The memory 1102 is used to store a computer program. The processor 1101 is coupled to the display 1003 and the memory 1002. The processor 1101 is used to run the computer program so that the electronic device 10 executes the vehicle re-identification method for fusing monitoring network topology information described in Embodiment 1. Embodiment 1 has described the vehicle re-identification method for fusing monitoring network topology information in detail, and will not be elaborated here.

[0104] The vehicle re-identification method for fusing monitoring network topology information can be applied to various types of electronic devices 10. In an exemplary embodiment, the electronic device 10 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, cameras or other electronic components for executing the above vehicle re-identification method for fusing monitoring network topology information.

[0105] The memory may include Random Access Memory (RAM), and may also include non-volatile memory, such as at least one disk memory.

[0106] The aforementioned processor 1101 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0107] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk, or optical disc, etc., all kinds of media that can store program codes.

[0108] In summary, the present invention can directly perform vehicle re-identification on traffic surveillance videos, effectively improving the accuracy of vehicle re-identification. Therefore, the present invention effectively overcomes various drawbacks in the prior art and has high industrial utilization value.

[0109] The above embodiments merely illustrate the principles and effects of the present invention, rather than limiting the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those of ordinary skill in the art within the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A vehicle re-identification method that integrates monitoring network topology information, characterized in that: Including: Obtain an initial image dataset and process the initial image dataset; Use the processed initial image dataset as the input of a pre-set convolutional neural network for vehicle re-identification to obtain a vehicle re-identification model; Input a target vehicle image to be identified into the vehicle re-identification model to obtain a feature tensor of the target vehicle image to be identified, and determine the similarity between the vehicle image to be identified and each image in the gallery set based on the feature tensor to form a preliminary result of vehicle re-identification; Sort the preliminary result of vehicle re-identification to obtain a final vehicle re-identification result; the implementation method of sorting the preliminary result of vehicle re-identification to obtain a final vehicle re-identification result includes: Maintain the monitoring network topology information in the form of an undirected graph; Use the vehicle images to be identified with a similarity above a preset value as candidate images and save them in descending order; Divide the top k candidate images into a sorted list; Determine the set of image IDs of all candidate images based on a confidence strategy: consider the top two results with the highest similarity scores to be believable, set a trust base trust_base, find all paths and their nodes between rank_list[0] and rank_list[1], that is, the set of image IDs of all candidate images, denoted as candidate_cids; Update the sorted list based on whether the image ID of each candidate image exists in a preset candidate image ID set, and obtain the final vehicle re-identification result according to the updated sorted list: the image ID with the highest similarity and meeting the similarity threshold is used as the final identification result; specifically, it includes: traversing the sorted list starting from the index, and maintaining a variable drift_num with an initial value of 0, where drift_num represents the number of candidate images whose image IDs do not belong to candidate_cids. For each candidate image: if the image ID of the candidate image exists in candidate_cids, increase the similarity score of the current candidate image by e; if not, increment drift_num by 1, sort according to the latest similarity score, and update the sorted list; determine whether drift_num exceeds the threshold. If it exceeds, decrease e, that is, decrease the trust base trust_base by 1, and return to update the sorted list; if not, output the latest sorted list as the final vehicle re-identification result.

2. The vehicle re-identification method for fusing monitored network topology information according to claim 1, characterized in that: The initial image dataset includes: a training set and a test set; the test set includes a query set and a gallery set.

3. The vehicle re-identification method for fusing monitored network topology information according to claim 1 or 2, characterized in that: The processing of the initial image dataset includes: performing any one or a combination of horizontal flipping, random erasing, adding random patches, and color jitter on the initial image data.

4. The vehicle re-identification method for fusing monitored network topology information according to claim 1, characterized in that: The backbone network of the pre-set convolutional neural network is a deep residual network based on IBN-Net, replace the average pooling layer of the deep residual network based on IBN-Net with an adaptive average pooling layer, add a batch normalization layer after the last fully connected layer to reduce the feature dimension, and then connect a fully connected layer to output the final prediction result.

5. The vehicle re-identification method for fusing monitored network topology information according to claim 4, characterized in that: The preset convolutional neural network calculates the loss using a hybrid loss function of triplet loss and cross - entropy loss; the preset convolutional neural network uses a stochastic gradient descent optimizer to calculate the gradient and iteratively update the parameters; a warm - up learning rate combined with cosine annealing is adopted to adjust the learning rate as the learning rate adjustment strategy; non - local blocks are added to the tails of the last half of the residual blocks in the third and fourth layers of the deep residual network respectively to capture long - range dependencies.

6. The vehicle re-identification method for fusing monitored network topology information according to claim 1, characterized in that: One implementation of determining the similarity between the vehicle image to be recognized and each image in the gallery set based on the feature tensor includes: Obtain the feature tensors of each image in the gallery set; Use the preset convolutional neural network to obtain the query feature tensor of the vehicle image to be recognized; Calculate the Euclidean distance between the query feature tensor of the vehicle image to be recognized and the feature tensors of each image in the gallery set; Determine the similarity based on the Euclidean distance, where the smaller the Euclidean distance, the higher the similarity.

7. A vehicle re-identification device integrating monitoring network topology information, characterized in that: Includes: A vehicle detection and tracking module for obtaining an initial image dataset and processing the initial image dataset; A vehicle re - identification calculation module for using the processed initial image dataset as the input of a vehicle re - identification preset convolutional neural network, obtaining a vehicle re - identification model, inputting the target vehicle image to be recognized into the vehicle re - identification model to obtain the feature tensor of the target vehicle image to be recognized, and determining the similarity between the vehicle image to be recognized and each image in the gallery set based on the feature tensor to form a preliminary vehicle re - identification result, and sorting the preliminary vehicle re - identification result to obtain the final vehicle re - identification result; the implementation of sorting the preliminary vehicle re - identification result to obtain the final vehicle re - identification result includes: Maintain the monitoring network topology structure information in the form of an undirected graph; Take the vehicle images to be recognized with similarity above a preset value as candidate images and save them in descending order; Divide the top k candidate images into a sorted list; Determine the set of image IDs of all candidate images based on a confidence strategy: consider the top two results with the highest similarity scores to be believable, set a trust base trust_base, find all paths and their nodes between rank_list[0] and rank_list[1], that is, the set of image IDs of all candidate images, denoted as candidate_cids; Update the sorted list based on whether the image ID of each candidate image exists in the preset candidate image ID set, and obtain the final vehicle re-identification result according to the updated sorted list: the image ID with the highest similarity and meeting the similarity threshold is used as the final recognition result; specifically including: traverse the sorted list starting from the index, and maintain a variable drift_num with an initial value of 0. drift_num represents the number of candidate images whose image IDs of their candidate images do not belong to candidate_cids. For each candidate image: if the image ID of the candidate image exists in candidate_cids, increase the similarity score of the current candidate image by e; if not, increment drift_num by 1, sort according to the latest similarity score, and update the sorted list; determine whether drift_num exceeds the threshold. If it exceeds, decrease e, that is, decrease the trust base by 1, and return to update the sorted list; if not, output the latest sorted list as the final vehicle re-identification result.

8. The vehicle re-identification device for fusing monitored network topology information according to claim 7, wherein: The backbone network of the preset convolutional neural network is a deep residual network based on IBN-Net. Replace the average pooling layer of the deep residual network based on IBN-Net with an adaptive average pooling layer. Add a batch normalization layer after the last fully connected layer to reduce the feature dimension, and then connect a fully connected layer to output the final prediction result.

9. An electronic device, characterized in that: It includes a processor and a memory, and the memory stores program instructions; the processor runs the program instructions to implement the vehicle re-identification method for fusing monitoring network topology information as described in any one of claims 1 to 8.

Images