Vehicle re-identification method, system and terminal

Abstract

The application discloses a vehicle re-identification method, system and terminal, the method comprises the following steps: constructing a recall model and a ranking model according to a first vehicle image training set and a second vehicle image training set, and constructing a vehicle image database based on actual business requirements; obtaining a target retrieval image, matching the target retrieval image with each vehicle image in the vehicle image database to obtain a candidate vehicle image; inputting the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image; and performing similarity calculation on the target feature vector and a candidate feature vector corresponding to the candidate vehicle image through the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image. The application constructs a two-stage recall model and a ranking model to construct a vehicle re-identification system, and collects difficult sample data in an application environment to optimize the vehicle re-identification system, thereby greatly improving the accuracy of vehicle re-identification.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving perception technology, and in particular to a vehicle re-identification method, system, terminal, and computer-readable storage medium. Background Art

[0002] Vehicle re-identification (ReID) is a subproblem of re-identification (ReID). Given a vehicle image, it involves finding the same vehicle in a database of vehicle images. It can also be considered a subproblem of image retrieval. In real-world traffic monitoring systems, vehicle re-identification can be used for target vehicle location, monitoring, and criminal investigation. With the rise of deep neural networks and the introduction of large datasets, improving the accuracy of vehicle re-identification has become a research hotspot in computer vision and autonomous driving in recent years.

[0003] Re-ID methods are generally categorized by their loss function, typically into metric learning and representation learning. Metric learning involves learning the similarity between two images through a network. Specifically, for vehicle re-ID, different images of the same vehicle are more similar than different images of different vehicles. The network's loss function minimizes the distance between images of the same vehicle (positive pairs) (for example, after the network extracts features and abstracts them into vector space, the distance between two vectors can be measured, such as the cosine value between the two vectors; smaller distances indicate closer vectors; similarly, smaller distances indicate more similar images) while maximizing the distance between images of different vehicles (negative pairs). Common metric learning loss methods include constrative loss, triplet loss, quadruplet loss, trihard loss, and margin mining loss (MSML). Representation learning, on the other hand, does not directly consider image similarity when training the network, instead treating the vehicle re-ID task as a classification problem. For vehicle re-identification, the model is trained using vehicle IDs as training labels. The number of pedestrian IDs in the training set equals the number of categories in the network. The feature layer is followed by a fully-connected classification layer, which is activated using a Softmax (or a modified version of Softmax) activation function to calculate the cross-entropy loss. During testing, the feature vectors of the penultimate layer are used to represent the vehicle image, and the fully-connected classification layer is discarded. Representational learning methods, which use classification methods, are trained using large-scale, pre-constructed vehicle ID data. Models trained using classification methods are characterized by high generalization and higher recall. Metric learning methods construct targeted pairs, triplets, or quadruplets and train the model using the corresponding metric learning loss function. This method improves the model's discrimination for specific samples by constructing targeted pairs, triplets, or quadruplets, facilitating optimization for corner cases (difficult samples) and improving precision.

[0004] However, the vehicle re-identification methods currently available in the art mostly focus on the research of the model structure used for vehicle re-identification, but do not consider the iterative optimization of the entire vehicle re-identification system from the perspective of vehicle re-identification system construction and in the application environment, resulting in low accuracy of the obtained vehicle re-identification results.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] The main purpose of this application is to provide a vehicle re-identification method, system, terminal and computer-readable storage medium, aiming to solve the problem in the prior art that the iterative optimizability of the entire vehicle re-identification system cannot be considered from the perspective of vehicle re-identification system construction and in the application environment, resulting in low accuracy of the vehicle re-identification results.

[0007] A first aspect of the present application provides a vehicle re-identification method, comprising the following steps: constructing a recall model and a ranking model based on a first vehicle image training set and a second vehicle image training set, and constructing a vehicle image database based on actual business needs, wherein the vehicle image database contains all vehicle images to be retrieved required in actual business; obtaining a target retrieval image, inputting the target retrieval image into the recall model, and performing feature vector matching on the target retrieval image and each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image; inputting the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image; and performing similarity calculation on the target feature vector and the candidate feature vector corresponding to the candidate vehicle image through the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image.

[0008] Based on the above technical means, the embodiment of the present application combines two vehicle re-identification models with a search architecture to obtain a first-stage recall model and a second-stage ranking model. This is used to construct a vehicle re-identification system. By sequentially inputting the target retrieval image into the recall model and the ranking model for screening, it is possible to quickly and accurately obtain a vehicle re-identification image of the target retrieval image. At the same time, by continuously providing feedback to the vehicle re-identification system through real-time collection of difficult corner case sample data in the application environment, it is possible to further iteratively optimize the model and improve the accuracy of vehicle re-identification.

[0009] Optionally, in one embodiment of the present application, the recall model and the sorting model are constructed based on the first vehicle image training set and the second vehicle image training set, specifically including: obtaining the first vehicle image training set and the second vehicle image training set, training the classification model based on the first vehicle image training set to obtain the recall model, wherein the first vehicle image training set includes an open source data set or a vehicle ID data set constructed based on its own business, and the second vehicle image training set includes the vehicle ID data set constructed based on its own business, the open source data set, and a historical difficult sample data set; training the twin network model based on the second vehicle image training set to obtain a sorting model, wherein the backbone network of the twin network model is loaded with the backbone network of the recall model.

[0010] Based on the above technical means, the embodiment of the present application can train the classification model used for representation learning in the vehicle re-identification model by setting different image data sets to obtain a recall model, and then use the recall model as a pre-trained model for the twin network model to train a ranking model, thereby constructing a recall model and a ranking model. In addition, the present application also introduces a historical difficult sample data set into the second image data set, greatly improving the accuracy of the ranking model in screening and retrieving target retrieval images.

[0011] Optionally, in one embodiment of the present application, the model backbone network of the classification model is one of an Arcface model, a SwinTransformer model, a ConvNeX model, a ResNet18 model or a MobileNet model.

[0012] According to the above-mentioned technical means, the embodiment of the present application sets the classification model for representation learning in the vehicle re-identification model. The model backbone network in the classification model can be selected according to the limiting factors in the actual production environment. For large-scale data, the SwinTransformer model and the ConvNeX model can be used, and for lightweight models, the ResNet18 model and the MobileNet model can be used. The corresponding classification model is selected according to different environmental restrictions, which can make the recall model construction more reasonable and accurate.

[0013] Optionally, in one embodiment of the present application, the constructing of a vehicle image database based on actual business needs specifically includes: obtaining a vehicle image data set to be detected collected based on actual business needs, and inputting the vehicle image data set to be detected into the recall model to obtain a feature vector corresponding to each vehicle image in the vehicle image data set to be detected; constructing a recall positive library based on the vehicle images in the vehicle image data set to be detected and the feature vectors corresponding to the vehicle images, and constructing a recall reverse library based on the recall positive library, wherein the recall positive library and the recall reverse library are combined into the vehicle image database.

[0014] According to the above-mentioned technical means, the embodiment of the present application constructs a recall positive library and a recall reverse library by inputting the vehicle image data set to be detected into the recall model to obtain the feature vector of each image in the vehicle image data set to be detected. The recall model can be deployed based on the recall positive library and the recall reverse library, and through the establishment of the recall positive library and the recall reverse library, the target retrieval images subsequently input into the recall model can be screened. Through the corresponding screening of the recall reverse library, it is ensured that the candidate vehicle images whose similarity reaches the preset threshold are selected for the target retrieval image.

[0015] Optionally, in one embodiment of the present application, the target retrieval image is obtained, the target retrieval image is input into the recall model, and the target retrieval image is matched with each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image, specifically including: obtaining a target retrieval image, and inputting the target retrieval image into the recall inverted library to obtain a recall feature vector corresponding to the target retrieval image; obtaining a feature vector in the recall inverted library, and obtaining a candidate vehicle image based on the feature vector in the recall inverted library and the recall feature vector.

[0016] According to the above technical means, the embodiment of the present application can obtain the recall feature vector corresponding to the target retrieval image by inputting the target retrieval image into the recall inverted library. By comparing the similarity between the recall feature vector and the feature vector in the recall inverted library, a preset number of candidate images corresponding to the recall feature vector whose similarity with the target retrieval image reaches a preset threshold can be obtained. Through the setting of the recall positive library and the recall inverted library, candidate vehicle images in the recall model whose similarity with the target retrieval image reaches a preset threshold can be screened out.

[0017] Optionally, in one embodiment of the present application, obtaining the candidate vehicle image based on the feature vector in the recalled inverted library and the recalled feature vector specifically includes: calculating the similarity between the recalled feature vector and the feature vector in the recalled inverted library, and screening out the candidate vehicle image corresponding to the feature vector in the recalled inverted library whose similarity is greater than a first preset threshold.

[0018] According to the above technical means, the embodiment of the present application inputs the target retrieval image into the recall inverted library, and can obtain candidate vehicle images whose similarity with the target retrieval image reaches a preset threshold from the recall inverted library. By establishing the recall inverted library, the candidate feature images corresponding to the target retrieval image in the recall inverted library can be obtained quickly and accurately.

[0019] Optionally, in one embodiment of the present application, inputting the target retrieval image into the sorting model to obtain the target feature vector corresponding to the target retrieval image also includes: dividing the sorting model into a sorting positive library feature extraction module and a similarity calculation module.

[0020] According to the above technical means, the embodiment of the present application can divide the sorting model into two parts by using a layer in the sorting model for generating feature vectors for vehicle images, namely a sorting positive library feature extraction module and a similarity calculation module, which can execute different functions accordingly and can perform further screening according to the target retrieval image.

[0021] Optionally, in one embodiment of the present application, the sorting model is divided into a sorting positive library feature extraction module and a similarity calculation module, specifically including: dividing the sorting model according to the feature vector generation layer in the sorting model, setting the part before the feature vector generation layer as the sorting positive library feature extraction module, wherein the sorting positive library feature extraction module is used to generate a corresponding feature vector for the vehicle image input into the sorting model; setting the part after the feature vector generation layer as the similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the feature vectors obtained after inputting the sorting model.

[0022] According to the above-mentioned technical means, the embodiment of the present application deploys the sorting model to divide the sorting model into a sorting positive library feature extraction module and a similarity calculation module, which can respectively perform different functions. The sorting positive library feature extraction module can be used to obtain the feature vector of the target retrieval image, and the similarity calculation module can be used to compare the feature vector corresponding to the target retrieval image with the feature vector corresponding to the candidate vehicle image output in the recall model for similarity, so as to accurately obtain the vehicle re-identification image corresponding to the target retrieval image.

[0023] Optionally, in one embodiment of the present application, the part after the feature vector generation layer is set as a similarity calculation module, and then further includes: obtaining two vehicle images in the vehicle image data set to be detected, and inputting the two vehicle images into the sorting positive library feature extraction module respectively to obtain two sorting feature vectors; calculating the cosine similarity between the two sorting feature vectors, and training the similarity calculation module according to the cosine similarity.

[0024] According to the above technical means, the embodiment of the present application can input the vehicle image dataset to be detected into the similarity calculation module in the sorting model, and train the similarity calculation module with images in the vehicle image dataset to be detected, which is conducive to ensuring the accuracy of further fine screening of the target retrieval image.

[0025] Optionally, in one embodiment of the present application, the similarity calculation between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image is performed by the ranking model to obtain the vehicle re-identification image corresponding to the target retrieval image, specifically including: performing similarity calculation between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image in the similarity calculation module of the ranking model to obtain multiple similarity scores; extracting a preset number of feature vectors from the candidate feature vectors from high to low according to the multiple similarity scores, and outputting a final vehicle image corresponding to the preset number of feature vectors in the candidate feature vectors, wherein the final vehicle image is the vehicle re-identification image corresponding to the target retrieval image.

[0026] According to the above technical means, the embodiment of the present application can input the target retrieval image into the first-stage recall model and the second-stage sorting model in sequence to realize the vehicle re-identification process of the target retrieval image; when the target retrieval image is input into the recall model for coarse screening, the target retrieval image can be screened according to the recall positive library and the recall reverse library in the recall model, and the candidate vehicle image is obtained by sorting and extracting according to the similarity between the feature vector corresponding to the target retrieval image and the feature vector corresponding to the image in the recall reverse library, thereby realizing coarse screening of the target retrieval image; further, the feature vector corresponding to the candidate vehicle image and the feature vector obtained by inputting the sorting positive library feature extraction module are similarly calculated in the similarity calculation module in the sorting model, thereby ensuring the accuracy of the target retrieval image, thereby obtaining the vehicle re-identification image of the target retrieval image.

[0027] Optionally, in one embodiment of the present application, the target feature vector and the candidate feature vector corresponding to the candidate vehicle image are calculated similarity through the sorting model to obtain the vehicle re-identification image corresponding to the target retrieval image, and then the method further includes: obtaining difficult sample data appearing in the vehicle re-identification image, and constructing bigram data for the difficult sample data; using the bigram data as the input of the sorting model, and fine-tuning the sorting model according to the bigram data.

[0028] According to the above technical means, the embodiment of the present application can collect Corner cases, i.e., difficult sample data, that appear during the vehicle re-identification process by inputting the target retrieval image into the vehicle re-identification system after constructing the recall model and the ranking model in the vehicle re-identification system, and construct a binary image suitable for the twin network model based on the difficult sample data; the binary image is input into the ranking model with the recall model as the pre-trained model for training and optimization, thereby realizing feedback optimization of the ranking model, which is conducive to gradually improving the data construction strategy and improving the accuracy of the ranking model.

[0029] Optionally, in one embodiment of the present application, the fine-tuning of the sorting model based on the binary data specifically includes: inputting the binary data into the sorting positive library feature extraction module respectively to obtain two positive library feature vectors; calculating the similarity between the two positive library feature vectors, and fine-tuning the similarity calculation module in the sorting model according to the similarity between the positive library feature vectors.

[0030] According to the above technical means, the embodiment of the present application can obtain a feature vector by inputting the collected and constructed binary data into the sorting model, and can optimize the feedback in the sorting model by calculating the similarity between the feature vectors, that is, optimizing the sorting positive library feature extraction module in the twin network model, which is conducive to the real-time update of the sorting positive library feature extraction module, so as to facilitate the screening of vehicle re-identification images that are more accurate with the target retrieval image.

[0031] A second aspect of the present application provides a vehicle re-identification system, which includes: a two-stage model training module, which is used to construct a recall model and a ranking model based on a first vehicle image training set and a second vehicle image training set, and to build a vehicle image database based on actual business needs, wherein the vehicle image database contains all the vehicle images to be retrieved required in actual business; a candidate vehicle image acquisition module, which is used to obtain a target retrieval image, input the target retrieval image into the recall model, and perform feature vector matching on the target retrieval image with each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image; a target feature vector generation module, which is used to input the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image; and a re-identification image acquisition module, which is used to perform similarity calculation on the target feature vector and the candidate feature vector corresponding to the candidate vehicle image through the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image.

[0032] Optionally, in one embodiment of the present application, the two-stage model training module includes: a recall model training unit, used to obtain a first vehicle image training set and a second vehicle image training set, and train a classification model according to the first vehicle image training set to obtain a recall model, wherein the first vehicle image training set includes an open source data set or a vehicle ID data set constructed based on its own business, and the second vehicle image training set includes the vehicle ID data set constructed based on its own business, the open source data set, and a historical difficult sample data set; a sorting model training unit, used to train a twin network model according to the second vehicle image training set to obtain a sorting model, wherein, The backbone network of the twin network model is loaded with the backbone network of the recall model; the first vehicle image data set vector generation unit is used to obtain the vehicle image data set to be detected collected based on actual business needs, and input the vehicle image data set to be detected into the recall model to obtain the feature vector corresponding to each vehicle image in the vehicle image data set to be detected; the vehicle image database generation unit is used to construct a recall positive library based on the vehicle images in the vehicle image data set to be detected and the feature vectors corresponding to the vehicle images, and to construct a recall reverse library based on the recall positive library, wherein the recall positive library and the recall reverse library are combined into the vehicle image database.

[0033] Optionally, in one embodiment of the present application, the recall model training unit includes: a classification model selection subunit, which is used to select the model backbone network of the classification model to be used as one of the Arcface model, SwinTransformer model, ConvNeX model, ResNet18 model or MobileNet model.

[0034] Optionally, in one embodiment of the present application, the candidate vehicle image acquisition module includes: a recall feature vector generation unit, used to acquire a target retrieval image, and input the target retrieval image into the recall inverted library to obtain a recall feature vector corresponding to the target retrieval image; a candidate vehicle image generation unit, used to acquire a feature vector in the recall inverted library, and obtain a candidate vehicle image based on the feature vector in the recall inverted library and the recall feature vector.

[0035] Optionally, in one embodiment of the present application, the recall feature vector generation unit includes: a similarity comparison subunit, used to calculate the similarity between the recall feature vector and the feature vector in the recall inverted library, and screen out candidate vehicle images corresponding to the feature vector in the recall inverted library whose similarity is greater than a first preset threshold; a recall feature vector generation subunit, used to.

[0036] Optionally, in one embodiment of the present application, the target feature vector generation module includes: a ranking model division unit, configured to divide the ranking model into a ranking positive library feature extraction module and a similarity calculation module.

[0037] Optionally, in one embodiment of the present application, the sorting model division unit includes: a sorting positive library feature extraction module setting subunit, used to divide the sorting model according to the feature vector generation layer in the sorting model, and set the part before the feature vector generation layer as the sorting positive library feature extraction module, wherein the sorting positive library feature extraction module is used to generate a corresponding feature vector for the vehicle image input into the sorting model; a similarity calculation module setting subunit, used to set the part after the feature vector generation layer as the similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the feature vectors obtained after inputting the sorting model.

[0038] Optionally, in one embodiment of the present application, the target feature vector generation module further includes: a third and a fourth feature vector generation unit, used to obtain two vehicle images in the vehicle image data set to be detected, and input the two vehicle images into the sorting positive library feature extraction module respectively to obtain two sorting feature vectors; a similarity calculation module training unit, used to calculate the cosine similarity between the two sorting feature vectors, and train the similarity calculation module according to the cosine similarity.

[0039] Optionally, in one embodiment of the present application, the re-identification image acquisition module includes: multiple similarity score generation units, used to perform similarity calculation on the target feature vector and the candidate feature vector corresponding to the candidate vehicle image in the similarity calculation module of the sorting model to obtain multiple similarity scores; a candidate vehicle image generation unit, used to extract a preset number of feature vectors from the candidate feature vectors from high to low according to the multiple similarity scores, and output a final vehicle image corresponding to the preset number of feature vectors in the candidate feature vectors, wherein the final vehicle image is the vehicle re-identification image corresponding to the target retrieval image.

[0040] Optionally, in one embodiment of the present application, the system of the embodiment of the present application also includes: a bigram data construction unit, used to obtain difficult sample data appearing in the vehicle re-identification image, and construct bigram data for the difficult sample data; a sorting model fine-tuning unit, used to use the bigram data as the input of the sorting model, and fine-tune the sorting model based on the bigram data.

[0041] Optionally, in one embodiment of the present application, the sorting model fine-tuning unit includes: a sixth feature vector generation subunit, used to input the two-tuple data into the sorting positive library feature extraction module respectively to obtain two positive library feature vectors; a sorting positive library feature extraction module fine-tuning subunit, used to calculate the similarity between the two positive library feature vectors, and fine-tune the similarity calculation module in the sorting model according to the similarity between the positive library feature vectors.

[0042] A third aspect of the present application provides a terminal, comprising: a memory, a processor, and a vehicle re-identification program stored in the memory and executable on the processor, wherein the vehicle re-identification program, when executed by the processor, implements the steps of the vehicle re-identification method as described in the above embodiment.

[0043] A fourth aspect of the present application provides a computer-readable storage medium storing a vehicle re-identification program. When the vehicle re-identification program is executed by a processor, the steps of the vehicle re-identification method described in the above embodiment are implemented.

[0044] Beneficial effects of this application:

[0045] (1) The embodiment of the present application combines two vehicle re-identification models with a search architecture to obtain a first-stage recall model and a second-stage ranking model, thereby constructing a vehicle re-identification system. By sequentially inputting the target retrieval image into the recall model and the ranking model for screening, the vehicle re-identification image of the target retrieval image can be quickly and accurately obtained. At the same time, by continuously providing feedback to the vehicle re-identification system through real-time collection of corner case difficult sample data in the application environment, the model can be further iterated and optimized to improve the accuracy of vehicle re-identification.

[0046] (2) In the embodiment of the present application, different image datasets are set to train the classification model used for representation learning in the vehicle re-identification model to obtain a recall model, and the recall model is used as a pre-trained model for the twin network model to train a ranking model, thereby constructing a recall model and a ranking model. In addition, the present application also introduces a historical difficult sample dataset into the second image dataset, which greatly improves the accuracy of the ranking model in screening and retrieving target retrieval images.

[0047] (3) In the embodiment of the present application, the feature vector of each image in the vehicle image dataset to be detected is obtained by inputting the vehicle image dataset to the recall model to construct a recall positive library and a recall reverse library. The recall model can be deployed based on the recall positive library and the recall reverse library. Moreover, by establishing the recall positive library and the recall reverse library, the target retrieval images subsequently input into the recall model can be screened. By correspondingly screening the recall reverse library, it is ensured that candidate vehicle images whose similarity reaches a preset threshold are selected for the target retrieval images.

[0048] (4) The embodiment of the present application can obtain a recall feature vector corresponding to the target retrieval image by inputting the target retrieval image into the recall inverted library. By comparing the recall feature vector with the feature vector in the recall inverted library for similarity, a preset number of candidate images corresponding to the recall feature vector whose similarity with the target retrieval image reaches a preset threshold can be obtained. By setting the recall positive library and the recall inverted library, candidate vehicle images whose similarity with the target retrieval image reaches a preset threshold in the recall model can be screened out.

[0049] (5) The embodiment of the present application inputs the target retrieval image into the recalled inverted library, and can obtain candidate vehicle images whose similarity with the target retrieval image reaches a preset threshold from the recalled inverted library. By establishing the recalled inverted library, the candidate feature images corresponding to the target retrieval image in the recalled inverted library can be quickly and accurately obtained.

[0050] (6) In the embodiment of the present application, the target retrieval image can be sequentially input into the recall model of the first stage and the sorting model of the second stage to realize the vehicle re-identification process of the target retrieval image; when the target retrieval image is input into the recall model for coarse screening, the target retrieval image can be screened according to the recall positive library and the recall reverse library in the recall model, and the candidate vehicle image can be sorted and extracted according to the similarity between the feature vector corresponding to the target retrieval image and the feature vector corresponding to the image in the recall reverse library, thereby realizing coarse screening of the target retrieval image; further, the feature vector corresponding to the candidate vehicle image and the feature vector obtained by inputting the feature extraction module of the sorting positive library are similarly calculated in the similarity calculation module in the sorting model, thereby ensuring the accuracy of the target retrieval image, thereby obtaining the vehicle re-identification image of the target retrieval image.

[0051] (7) According to the above technical means, the embodiment of the present application can collect Corner cases, i.e., difficult sample data, that appear during the vehicle re-identification process by inputting the target retrieval image into the vehicle re-identification system after constructing the recall model and the ranking model in the vehicle re-identification system, and construct a binary image applicable to the twin network model based on the difficult sample data; the binary image is input into the ranking model with the recall model as the pre-training model for training and optimization, thereby realizing feedback optimization of the ranking model, which is conducive to gradually improving the data construction strategy and improving the accuracy of the ranking model.

[0052] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 It is a flow chart of a preferred embodiment of the vehicle re-identification method of the present application;

[0055] Figure 2 Schematic diagram of the structure of the recall model of the preferred embodiment of the vehicle re-identification method of the present application;

[0056] Figure 3 Schematic diagram of the structure of the sorting model of the preferred embodiment of the vehicle re-identification system of the present application;

[0057] Figure 4 Schematic diagram of the structure of the combination of the recall model and the ranking model of the preferred embodiment of the vehicle re-identification system of the present application;

[0058] Figure 5 This is a schematic diagram of the iterative optimization steps of the sorting model for CornerCase difficult sample data collected in a production environment in a preferred embodiment of the vehicle re-identification system of the present application;

[0059] Figure 6 This is a schematic diagram of the process steps of a preferred embodiment of the vehicle re-identification system of the present application;

[0060] Figure 7 is a block diagram of a vehicle re-identification system according to an embodiment of the present application;

[0061] Figure 8 This is a schematic structural diagram of a preferred embodiment of the terminal of this application.

[0062] Among them, 10 is a vehicle re-identification system; 100 is a two-stage model training module, 200 is a candidate vehicle image acquisition module, 300 is a target feature vector generation module and 400 is a re-identification image acquisition module; 501 is a memory, 502 is a processor and 503 is a communication interface. DETAILED DESCRIPTION

[0063] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0064] Vehicle re-identification (ReID) is a subproblem of re-identification (ReID). Given a vehicle image, it involves finding the same vehicle in a database of vehicle images. It can also be considered a subproblem of image retrieval. In real-world traffic monitoring systems, vehicle re-identification can be used for target vehicle location, monitoring, and criminal investigation. With the rise of deep neural networks and the introduction of large datasets, improving the accuracy of vehicle re-identification has become a research hotspot in computer vision and autonomous driving in recent years.

[0065] Re-ID methods are generally categorized by their loss function, typically into metric learning and representation learning. Metric learning involves learning the similarity between two images through a network. Specifically, for vehicle re-ID, different images of the same vehicle are more similar than different images of different vehicles. The network's loss function minimizes the distance between images of the same vehicle (positive pairs) (for example, after the network extracts features and abstracts them into vector space, the distance between two vectors can be measured, such as the cosine value between the two vectors; smaller distances indicate closer vectors; similarly, smaller distances indicate more similar images) while maximizing the distance between images of different vehicles (negative pairs). Common metric learning loss methods include constrative loss, triplet loss, quadruplet loss, trihard loss, and margin mining loss (MSML). Representation learning, on the other hand, does not directly consider image similarity when training the network, instead treating the vehicle re-ID task as a classification problem. For vehicle re-identification, the model is trained using vehicle IDs as training labels. The number of pedestrian IDs in the training set equals the number of categories in the network. The feature layer is followed by a fully-connected classification layer, which is activated using a Softmax (or a modified version of Softmax) activation function to calculate the cross-entropy loss. During testing, the feature vectors of the penultimate layer are used to represent the vehicle image, and the fully-connected classification layer is discarded. Representational learning methods, which use classification methods, are trained using large-scale, pre-constructed vehicle ID data. Models trained using classification methods are characterized by high generalization and higher recall. Metric learning methods construct targeted pairs, triplets, or quadruplets and train the model using the corresponding metric learning loss function. This method improves the model's discrimination for specific samples by constructing targeted pairs, triplets, or quadruplets, facilitating optimization for corner cases (difficult samples) and improving precision.

[0066] The re-identification problem is also an image retrieval problem. Currently, the information retrieval or search architecture widely used in the industry is in the form of a funnel, with coarse filtering in the recall stage and fine filtering in the ranking stage. The recall model is more efficient, performing preliminary screening from a large candidate pool while maintaining the recall rate, ensuring that as few candidate targets as possible are missed. In the ranking stage, a more complex model can be used to further screen the initially filtered candidate pool, with this stage being more inclined towards the model's precision metric. After multiple stages of funnel-style filtering, the final retrieval or search target is obtained. In different search scenarios, depending on the specific requirements and constraints of the actual scenario, the system can adopt a single-stage architecture, that is, only a recall model or a ranking model. However, in large-scale search scenarios, the entire system is usually built using a funnel architecture.

[0067] Currently, the actual application of vehicle re-ID in production scenarios faces the following major challenges:

[0068] The number of IDs in open-source vehicle re-ID datasets is still relatively small. The Vehicle-1M dataset, currently the largest in terms of IDs, contains 936,051 images of 55,527 vehicles, also with only 55,527 IDs. Other open-source datasets are even smaller. Building their own large-scale vehicle re-ID datasets requires significant time, manpower, and financial resources.

[0069] When the constructed vehicle re-ID system is applied in a real-world production environment, the online production environment continuously reports corner cases (this application refers to corner cases as difficult sample data, referring to edge cases that arise during the vehicle re-ID process, and refers to extreme cases or situations that are difficult to solve). To address these corner cases and optimize them, in the case of a typically small dataset, in addition to model-side optimization, more immediate improvements lie in dataset optimization, such as constructing difficult samples to solve corner cases for training. (This involves constructing difficult sample data for the model based on specific corner cases. For example, in vehicle re-ID, different angles of the same vehicle can be significantly different, making it difficult to identify the vehicle as the same. In this case, it is necessary to construct sample pairs of images of the same vehicle from different angles, labeled as the same vehicle, and add them to the data. Only when the model sees these examples can it correctly identify the vehicle.) Typical difficult examples in vehicle re-ID data exhibit "high intra-class variance and low inter-class variance." Specifically, there is significant variation within the same class, such as the front and back of a car, while there is minimal variation between different vehicles, such as similar models with only subtle differences, making it very easy for the model to identify them as the same vehicle. When optimizing corner cases, metric learning methods offer advantages over representation learning methods. While constructing data for corner cases with classified ID data is challenging, constructing targeted bigrams or triplets is much easier. In summary, after a vehicle re-ID system is deployed in production, iterative optimization of corner cases based on continuous online feedback using metric learning methods is more agile and effective. However, the challenge with metric learning methods is that models trained on bigrams or triplets typically generalize poorly compared to models trained using representation learning methods (classification). Existing vehicle re-ID methods primarily focus on model structure, without considering the iterative optimizability of the entire vehicle re-ID system from the perspective of system construction or within its application environment.

[0070] To address the aforementioned challenges of iterative optimization of the vehicle re-ID system and the difficulty of large-scale online retrieval after its launch, this paper proposes a two-stage vehicle re-ID method using a search architecture (funnel-style, coarse screening + fine screening) and leveraging the advantages of both representation learning and metric learning methods for the re-ID model. The first stage is the recall model, which uses representation learning to train a classification model, leveraging its high generalization to ensure recall, and then performs coarse screening. The second stage is the ranking model, which uses a metric learning-based twin network model to construct targeted binary data based on corner cases. The recall model from the first stage is used as a pre-trained model for model fine-tuning, enabling rapid iterative optimization for corner case resolution in production environments.

[0071] The following describes a vehicle re-identification method, system, and terminal according to an embodiment of the present application with reference to the accompanying drawings. Addressing the problem that the related art techniques mentioned in the background art fail to consider the iterative optimizability of the entire vehicle re-identification system from the perspective of vehicle re-identification system construction and in the application environment, resulting in low accuracy of vehicle re-identification results, the present application provides a vehicle re-identification method. In this method, two vehicle re-identification models are combined with a search architecture to obtain a first-stage recall model and a second-stage ranking model. This is used to construct a vehicle re-identification system. By sequentially inputting a target retrieval image into the recall model and the ranking model for screening, a vehicle re-identification image of the target retrieval image can be quickly and accurately obtained. Furthermore, by continuously providing feedback to the vehicle re-identification system through real-time collection of corner case difficult sample data in the application environment, the model can be further iteratively optimized, thereby improving the accuracy of vehicle re-identification. This solves the problem that the related art techniques fail to consider the iterative optimizability of the entire vehicle re-identification system from the perspective of vehicle re-identification system construction and in the application environment, resulting in low accuracy of vehicle re-identification results.

[0072] Specifically, Figure 1 A flowchart of a vehicle re-identification method provided in an embodiment of the present application.

[0073] like Figure 1 As shown, the vehicle re-identification method includes the following steps:

[0074] In step S101, a recall model and a ranking model are constructed based on the first vehicle image training set and the second vehicle image training set, and a vehicle image database is constructed based on actual business needs, wherein the vehicle image database contains all vehicle images to be retrieved required in actual business.

[0075] It can be understood that the embodiment of the present application obtains a first vehicle image training set and a second vehicle image training set, trains a classification model based on the first vehicle image training set to obtain a recall model, wherein the first vehicle image training set includes an open source data set or a vehicle ID data set built based on its own business, and the second vehicle image training set includes the vehicle ID data set built based on its own business, the open source data set, and a historical difficult sample data set; the twin network model is trained based on the second vehicle image training set to obtain a sorting model, wherein the backbone network of the twin network model is loaded with the backbone network of the recall model.

[0076] The model backbone network of the classification model is one of the Arcface model, SwinTransformer model, ConvNeX model, ResNet18 model or MobileNet model.

[0077] The Arcface model is used as the model. The backbone network can be selected based on the constraints of the actual production environment (such as memory limitations and computing power limitations). Large models can use SwinTransformer, ConvNeXt, etc., and lightweight models can use ResNet18, MobileNet, etc. The embedding length of the last layer of the backbone network is set to 128 dimensions, that is, the embedding feature vector of the vehicle image obtained after training is 128 dimensions.

[0078] Furthermore, a vehicle image dataset to be detected is obtained based on actual business needs, and the vehicle image dataset to be detected is input into the recall model to obtain a feature vector corresponding to each vehicle image in the vehicle image dataset to be detected; a recall positive library is constructed based on the vehicle images in the vehicle image dataset to be detected and the feature vectors corresponding to the vehicle images, and a recall negative library is constructed based on the recall positive library, wherein the recall positive library and the recall negative library are combined into the vehicle image database.

[0079] Among them, the vehicle image dataset to be detected is a vehicle gallery to be retrieved constructed according to actual business needs, and should be specifically defined according to actual business conditions. It is not completely inclusive with the previous first vehicle image training set and the second vehicle image training set, and may contain some.

[0080] like Figure 2 As shown in the figure, the first stage recall model training: based on the vehicle ID dataset or open source dataset, the representation learning method is used, and the Arcface model is selected for training and optimization to obtain the first stage recall model. Figure 2The Input Image in the figure is an image from a vehicle ID dataset or an open source dataset, Backbone is the backbone network of the classification model, ReID 128-dim Embedding is a 128-dimensional feature vector, FC (Fully Connected) is the fully connected layer in the network structure, and Arcface is the first loss function.

[0081] The recall model is trained using a classification method. Without building its own large-scale vehicle ID dataset, it is possible to fully utilize the open source vehicle re-identification dataset to train a vehicle re-identification model with good generalization performance during the recall phase (the coarse screening phase to maintain the recall rate).

[0082] The second stage of sorting model training: A two-tuple dataset is constructed based on the self-built vehicle ID dataset, open source dataset, and difficult sample data built for Cornercase. Using the metric learning method, the twin network and CoSENT Loss (a loss function used to calculate distance and loss) are selected. The recall model of the first stage is used as a pre-trained model (a pre-trained model is a model trained on other general tasks, and the parameters are loaded. On this basis, the data of your own task is used for retraining. This process is usually called model fine-tuning) for fine-tuning training (loading the model of the first stage as the pre-trained model, and using the new data of this stage to train the model. Fine-tuning can be understood as continuing training based on the pre-trained model).

[0083] The sorting model is trained using metric learning, and can construct difficult binary sample data for corner cases for model optimization. The binary method is easier and more targeted to construct data, making model optimization agile and efficient, and also improving model accuracy during the sorting stage.

[0084] In this embodiment, different image datasets are used to train the classification model used for representation learning in the vehicle re-identification model to obtain a recall model. The recall model is then used as a pre-trained model for the twin network model to train a ranking model, thereby constructing a recall model and a ranking model. In addition, this application also introduces a historical difficult sample dataset into the second image dataset, greatly improving the accuracy of the ranking model in screening and retrieving target retrieval images.

[0085] When training the Arcface model, if you do not have your own large-scale vehicle ID data, you can use open source vehicle re-identification data, such as Vehicle-1M, for training. The recall model network structure diagram is as follows: Figure 2After training is complete, the last Softmax layer is removed (when deploying, only the feature vector is needed; the Softmax layer is not needed and is used to calculate the loss during training). The 128-dimensional Embedding feature vector is the feature vector of the vehicle image.

[0086] like Figure 3 As shown in the figure, when training the twin network, the recall model is used as the pre-training model for model fine-tuning. The forward process is: the input is a pair of images. After data preprocessing, the 128-dimensional embedding feature vectors of the two images are obtained by sharing the Backbone. The two 128-dimensional embedding feature vectors are calculated using the CoSENT Loss to calculate the similarity loss. Figure 3 The Share Weights in is the shared backbone network, and CoSENT is the second loss function.

[0087] Furthermore, a first loss function is used to normalize the recall model, and a second loss function is used to control the convergence of the ranking model. The first loss function is Arcface Loss, short for Additive Angular Margin Loss. Arcface Loss improves on Softmax by normalizing feature vectors and weights and adding angular margins. Angular margins have a more direct effect on angles than cosine margins, maximizing the classification boundary directly in angular space.

[0088] Furthermore, the model uses a twin network to share the backbone, and the loss function uses CoSENT (CosineSentence) Loss. The network structure of the sorting model is as follows Figure 3 The loss function is based on sentence embedding methods in Natural Language Processing (NLP). A sentence embedding can be understood as a vector representation of a sentence extracted through a network. CoSENT is an optimization method based on InferSent, a mainstream supervised sentence embedding solution proposed by Facebook. Its training process is closer to prediction, and it achieves faster convergence and better results.

[0089] In step S102 , a target retrieval image is acquired, the target retrieval image is input into a recall model, and feature vector matching is performed between the target retrieval image and each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image.

[0090] Recall model deployment: Offline (offline state, extract features from a large number of images. Since online processing takes a lot of time, an offline process is adopted). Use the recall model to build the recall positive ranking library and the recall inverted ranking library (used in image retrieval deployment. The recall positive ranking library is a mapping library that uses the image ID as the key and the image extracted features as the value; the recall inverted ranking library is a mapping table that uses the image features as the key and the image ID as the value). Build the sorting model's positive ranking library feature extraction module offline.

[0091] It can be understood that in this application, a target retrieval image is obtained, and the target retrieval image is input into the recall inverted library to obtain a recall feature vector corresponding to the target retrieval image; the feature vector in the recall inverted library is obtained, and a candidate vehicle image is obtained based on the feature vector in the recall inverted library and the recall feature vector.

[0092] Furthermore, the similarity between the recalled feature vector and the feature vectors in the recalled inverted database is calculated, and candidate vehicle images corresponding to feature vectors in the recalled inverted database whose similarity is greater than a first preset threshold are screened out.

[0093] Deploying the recall model involves running all images in the vehicle image library (the dataset of vehicles to be tested) through the recall model to obtain a 128-dimensional embedding feature vector. This vector is then used to construct the recall positive and recall negative libraries. Deploying the ranking model involves running images in the vehicle image library (the dataset of vehicles to be tested) through the ranking model to obtain a 128-dimensional embedding feature vector. This vector is then used to construct the feature extraction module and similarity calculation module for the positive ranking library. The similarity calculation module of the ranking model must be deployed separately.

[0094] In the embodiment of the present application, a recall positive library and a recall negative library are constructed by inputting a vehicle image dataset to be detected into a recall model to obtain a feature vector of each image in the vehicle image dataset to be detected. The recall model can be deployed based on the recall positive library and the recall negative library. Moreover, through the establishment of the recall positive library and the recall negative library, the target retrieval images subsequently input into the recall model can be screened. Through the corresponding screening of the recall negative library, it is ensured that candidate vehicle images whose similarity reaches a preset threshold are selected for the target retrieval images.

[0095] In step S103, the target retrieval image is input into the ranking model to obtain a target feature vector corresponding to the target retrieval image.

[0096] When deploying the ranking model, the model is split based on the 128-dimensional embedding layer. The layer before the 128-dimensional embedding is used for the forward process to construct the ranking feature extraction module. The forward process inputs an image and produces a 128-dimensional embedding feature vector. The subsequent process is used for similarity calculation and requires a separate deployment service. The forward process inputs two 128-dimensional embedding feature vectors and outputs the cosine similarity score between the two embedding feature vectors.

[0097] It can be understood that, in the present application, the ranking model is divided into a ranking positive library feature extraction module and a similarity calculation module.

[0098] Furthermore, the sorting model is divided according to the feature vector generation layer in the sorting model, and the part before the feature vector generation layer is set as a sorting positive library feature extraction module, wherein the sorting positive library feature extraction module is used to generate corresponding feature vectors for the vehicle images input into the sorting model; and the part after the feature vector generation layer is set as a similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the feature vectors obtained after inputting the sorting model.

[0099] Furthermore, two vehicle images in the vehicle image data set to be detected are obtained, and the two vehicle images are respectively input into the sorting positive library feature extraction module to obtain two sorting feature vectors; the cosine similarity between the two sorting feature vectors is calculated, and the similarity calculation module is trained according to the cosine similarity.

[0100] The embodiment of the present application deploys the sorting model to divide the sorting model into a sorting positive library feature extraction module and a similarity calculation module, which can perform different functions respectively. The sorting positive library feature extraction module can be used to obtain the feature vector of the target retrieval image, and the similarity calculation module can be used to compare the feature vector corresponding to the target retrieval image with the feature vector corresponding to the candidate vehicle image output in the recall model. In this way, the vehicle re-identification image corresponding to the target retrieval image can be accurately obtained.

[0101] In step S104, similarity is calculated between the target feature vector and the candidate feature vectors corresponding to the candidate vehicle images using the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image.

[0102] It can be understood that in the present application, the target feature vector and the candidate feature vector corresponding to the candidate vehicle image are subjected to similarity calculation in the similarity calculation module of the sorting model to obtain multiple similarity scores; a preset number of feature vectors in the candidate feature vectors are extracted from high to low according to the multiple similarity scores, and the final vehicle image corresponding to the preset number of feature vectors in the candidate feature vectors is output, wherein the final vehicle image is the vehicle re-identification image corresponding to the target retrieval image.

[0103] like Figure 4 As shown in the figure, the forward process of the vehicle re-identification system is that the retrieved vehicle image first passes through the recall model to obtain a 128-dimensional Embedding vector feature, and the recall model's recall inverted library is preliminarily screened according to the sorting number TOP N (calculating the similarity of the vectors, the higher the score, the more similar they are, and TOP N is sorted from high to low according to the score, taking the highest N and screening them out first); the target retrieval image passes through the sorting model to obtain a 128-dimensional Embedding vector feature, and is queried according to the sorting forward library feature extraction module, and the similarity calculation module is used to calculate the similarity between the feature vector of the retrieval image and the sorting vector feature of the candidate pool image of the TOPN that has passed the recall stage, and the final retrieval result TOPM of the vehicle with the same ID is obtained by adjusting the threshold.

[0104] The process of the vehicle re-identification system in the actual production environment is to input the retrieved vehicle image into the recall and sorting models respectively, and query the vehicle image with the same ID in the vehicle image library.

[0105] The forward process of the vehicle re-identification system is that the retrieved image first passes through the recall model and the inverted index (when deployed, the key in the inverted index is a vector and the value is the image ID. In vector retrieval, the vectors are sorted according to their similarity. The higher the score, the more similar they are. What we do here is to calculate the similarity and then select the top N) to screen out the top N (the threshold is used to sort the feature vector similarity from high to low. You can directly take the top N, or set a threshold and only select those greater than the threshold) candidate vehicle images with the same ID; the retrieved image passes through the sorting model, and the similarity is calculated with the candidate vehicle images screened in the recall stage. The threshold is used to determine the final vehicle image with the same ID as this retrieved image.

[0106] In an embodiment of the present application, the target retrieval image can be sequentially input into the recall model of the first stage and the sorting model of the second stage to realize the vehicle re-identification process of the target retrieval image; when the target retrieval image is input into the recall model for coarse screening, the target retrieval image can be screened according to the recall positive library and the recall reverse library in the recall model, and the candidate vehicle image can be obtained by sorting and extracting according to the similarity between the feature vector corresponding to the target retrieval image and the feature vector corresponding to the image in the recall reverse library, thereby realizing coarse screening of the target retrieval image; further, the feature vector corresponding to the candidate vehicle image and the feature vector obtained by inputting the sorting positive library feature extraction module are similarly calculated in the similarity calculation module in the sorting model, thereby ensuring the accuracy of the target retrieval image, thereby obtaining the vehicle re-identification image of the target retrieval image.

[0107] It can be understood that in this application, difficult sample data that appears in the target retrieval image re-identification process is obtained, and binary data is constructed for the difficult sample data; the binary data is used as the input of the sorting model, and the sorting model is fine-tuned based on the binary data.

[0108] Based on the analysis of the scenario and the Corner Case feedback from the production environment, we construct binary images, gradually improve the data construction strategy, and use the Backbone of the recall model as a pre-training model to fine-tune the sorting model.

[0109] Furthermore, difficult sample data appearing in the vehicle re-identification image is obtained, and bigram data is constructed for the difficult sample data; the bigram data is used as input of the sorting model, and the sorting model is fine-tuned according to the bigram data.

[0110] Furthermore, the binary data are respectively input into the sorting positive library feature extraction module to obtain two positive library feature vectors; the similarity between the two positive library feature vectors is calculated, and the similarity calculation module in the sorting model is fine-tuned according to the similarity between the positive library feature vectors.

[0111] Update the two split-deployment models of the twin network, splitting into two parts: obtaining embedding features and calculating the similarity between two embedding features. Simultaneously, update the sorting and sorting feature extraction module in the vehicle image library offline.

[0112] Optimizing the ranking model based on Corner Cases: After the vehicle re-ID system goes live, targeted bigram data is constructed based on online Corner Case feedback to enable rapid iterative optimization of the ranking model (this refers to retraining the ranking model after obtaining Corner Case difficult sample data). After updating the ranking model, the offline ranking positive database feature extraction module is re-updated.

[0113] like Figure 5 As shown, the vehicle re-ID system faces even more challenging optimization challenges after going live in a production environment. Corner cases are constantly being fed back in the production environment, and optimizing these corner cases is a key task after the system goes live. The shared twin network of the sorting model takes binary image data as input. Corner cases can be used to construct targeted, difficult sample data, and the recall model can be used as a pre-trained model to fine-tune the sorting model. The sorting model is iteratively optimized for corner cases in the production environment. This approach makes optimization of the entire system more agile and efficient.

[0114] In the embodiment of the present application, after the recall model and the ranking model in the vehicle re-identification system are constructed, the target retrieval image is input into the vehicle re-identification system to collect the corner cases, i.e., difficult sample data, that appear during the vehicle re-identification process, and a binary image suitable for the twin network model is constructed based on the difficult sample data; the binary image is input into the ranking model with the recall model as the pre-trained model for training and optimization, thereby realizing feedback optimization of the ranking model, which is conducive to gradually improving the data construction strategy and improving the accuracy of the ranking model.

[0115] In summary, this paper proposes a vehicle re-identification method that leverages the characteristics of two commonly used re-identification methods: the high generalization and high recall rate exhibited by representation learning, and the flexibility of targeted data construction, high optimization iteration and precision exhibited by metric learning. This method utilizes a funnel framework commonly used in search systems, employing a two-stage recall and ranking phase to improve the performance of the vehicle re-identification system. Furthermore, it uses a production-specific corner case to enable rapid iterative optimization of the vehicle re-identification system.

[0116] The following further describes the entire implementation process according to the steps of executing the vehicle re-identification method of this application, such as Figure 6 As shown:

[0117] Step S0, start and execute step S1;

[0118] Step S1, obtaining a first vehicle image dataset, and proceeding to step S2;

[0119] Step S2, training a classification model based on the first vehicle image dataset, and then proceeding to step S3;

[0120] Step S3: After the classification model is trained, a recall model is obtained, and step S4 is continued;

[0121] Step S4, constructing a twin network model based on the recall model, and proceeding to step S5;

[0122] Step S5, obtaining a second vehicle image dataset, and proceeding to step S6;

[0123] Step S6: training the twin network model according to the second vehicle image dataset to obtain a ranking model, and then proceeding to step S7;

[0124] Step S7: Input the vehicle image dataset to be detected into the recall model to obtain a feature vector corresponding to each vehicle image in the vehicle image dataset to be detected; construct a recall positive list based on the vehicle images in the vehicle image dataset to be detected and the feature vectors corresponding to the vehicle images, and construct a recall negative list based on the recall positive list, and then proceed to step S8;

[0125] Step S8, dividing the ranking model into a ranking positive library feature extraction module and a similarity calculation module, and proceeding to step S9;

[0126] Step S9: Acquire a target retrieval image, and input the target retrieval image into the recalled inverted database to obtain a recall feature vector corresponding to the target retrieval image; obtain the feature vector in the recalled inverted database, and obtain a candidate vehicle image based on the feature vector in the recalled inverted database and the recalled feature vector, and then proceed to step S10;

[0127] Step S10, inputting the target retrieval image into the ranking model, obtaining a target feature vector corresponding to the target retrieval image, and executing step S11;

[0128] Step S11, calculating the similarity between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image through the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image, and executing step S12;

[0129] Step S12, obtaining difficult sample data that appears during the target retrieval image re-identification process, and executing step S13;

[0130] Step S13: for the difficult sample data, update the second vehicle image dataset, construct bigram data, use the bigram data as input of the twin network model, fine-tune the twin network model according to the bigram data, and execute step S14;

[0131] Step S14, end.

[0132] In summary, the embodiments of the present application combine the two vehicle re-identification models with the search architecture to obtain a first-stage recall model and a second-stage ranking model, thereby constructing a vehicle re-identification system. By sequentially inputting the target retrieval image into the recall model and the ranking model for screening, a vehicle re-identification image of the target retrieval image can be quickly and accurately obtained. At the same time, by continuously providing feedback to the vehicle re-identification system through real-time collection of difficult corner case sample data in the application environment, the model can be further optimized and the accuracy of vehicle re-identification can be improved.

[0133] Next, a vehicle re-identification system according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0134] Figure 7 4 is a block diagram of a vehicle re-identification system according to an embodiment of the present application.

[0135] like Figure 7 As shown, the vehicle re-identification system 10 includes: a two-stage model training module 100 , a candidate vehicle image acquisition module 200 , a target feature vector generation module 300 and a re-identification image acquisition module 400 .

[0136] Specifically, the two-stage model training module is used to construct a recall model and a ranking model based on the first vehicle image training set and the second vehicle image training set, and to build a vehicle image database based on actual business needs, wherein the vehicle image database contains all the vehicle images to be retrieved required in actual business;

[0137] a candidate vehicle image acquisition module, configured to acquire a target retrieval image, input the target retrieval image into a recall model, and perform feature vector matching between the target retrieval image and each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image;

[0138] a target feature vector generating module, configured to input the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image;

[0139] The re-identification image acquisition module is used to calculate the similarity between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image through the ranking model to obtain the vehicle re-identification image corresponding to the target retrieval image.

[0140] Optionally, in one embodiment of the present application, the two-stage model training module 100 includes: a recall model training unit, a sorting model training unit, a first vehicle image data set vector generation unit, and a vehicle image database generation unit.

[0141] The recall model training unit is configured to obtain a first vehicle image training set and a second vehicle image training set, and train a classification model based on the first vehicle image training set to obtain a recall model, wherein the first vehicle image training set includes an open source dataset or a vehicle ID dataset constructed based on its own business, and the second vehicle image training set includes the vehicle ID dataset constructed based on its own business, the open source dataset, and a historical difficult sample dataset;

[0142] A sorting model training unit, configured to train the Siamese network model according to the second vehicle image training set to obtain a sorting model, wherein the backbone network of the Siamese network model is loaded with the backbone network of the recall model;

[0143] a first vehicle image dataset vector generation unit, configured to obtain a vehicle image dataset to be detected, collected based on actual business needs, and input the vehicle image dataset to be detected into the recall model to obtain a feature vector corresponding to each vehicle image in the vehicle image dataset to be detected;

[0144] A vehicle image database generation unit is used to construct a recalled positive-order library based on the vehicle images in the vehicle image data set to be detected and the feature vectors corresponding to the vehicle images, and to construct a recalled negative-order library based on the recalled positive-order library, wherein the recalled positive-order library and the recalled negative-order library are combined into the vehicle image database.

[0145] Optionally, in one embodiment of the present application, the recall model training unit includes: a classification model selection subunit.

[0146] The classification model selection subunit is used to select a classification model whose model backbone network is one of the Arcface model, SwinTransformer model, ConvNeX model, ResNet18 model or MobileNet model.

[0147] Optionally, in one embodiment of the present application, the candidate vehicle image acquisition module 200 includes: a recall feature vector generation unit and a candidate vehicle image generation unit.

[0148] The recall feature vector generating unit is used to obtain a target retrieval image, input the target retrieval image into the recall inverted database, and obtain a recall feature vector corresponding to the target retrieval image;

[0149] The candidate vehicle image generating unit is used to obtain the feature vector in the recalled inverted database and obtain the candidate vehicle image according to the feature vector in the recalled inverted database and the recalled feature vector.

[0150] Optionally, in one embodiment of the present application, the recall feature vector generation unit includes: a similarity comparison subunit and a recall feature vector generation subunit.

[0151] a similarity comparison subunit, configured to calculate the similarity between the recalled feature vector and the feature vectors in the recalled inverted database, and screen out candidate vehicle images corresponding to the feature vectors in the recalled inverted database whose similarity is greater than a first preset threshold;

[0152] Recall feature vector generation subunit, used for.

[0153] Optionally, in one embodiment of the present application, the target feature vector generation module 300 includes: a sorting model division unit.

[0154] The sorting model division unit is used to divide the sorting model into a sorting positive library feature extraction module and a similarity calculation module.

[0155] Optionally, in one embodiment of the present application, the ranking model division unit includes: a ranking positive library feature extraction module setting subunit and a similarity calculation module setting subunit.

[0156] The sorting positive library feature extraction module is provided with a subunit, which is used to divide the sorting model according to the feature vector generation layer in the sorting model, and set the part before the feature vector generation layer as the sorting positive library feature extraction module, wherein the sorting positive library feature extraction module is used to generate a corresponding feature vector from the vehicle image input into the sorting model;

[0157] The similarity calculation module setting subunit is used to set the part after the feature vector generation layer as a similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the feature vectors obtained after inputting the sorting model.

[0158] Optionally, in one embodiment of the present application, the target feature vector generation module 300 further includes: third and fourth feature vector generation units and a similarity calculation module training unit.

[0159] The third and fourth feature vector generating units are used to obtain two vehicle images from the vehicle image data set to be detected, and input the two vehicle images into the sorting positive library feature extraction module respectively to obtain two sorting feature vectors;

[0160] The similarity calculation module training unit is used to calculate the cosine similarity between two sorted feature vectors and train the similarity calculation module according to the cosine similarity.

[0161] Optionally, in one embodiment of the present application, the re-identification image acquisition module 400 includes: a plurality of similarity score generation units and a candidate vehicle image generation unit.

[0162] wherein, a plurality of similarity score generating units are configured to perform similarity calculation on the target feature vector and the candidate feature vector corresponding to the candidate vehicle image in the similarity calculation module of the ranking model to obtain a plurality of similarity scores;

[0163] A candidate vehicle image generation unit is used to extract a preset number of feature vectors from the candidate feature vectors in descending order according to the multiple similarity scores, and output a final vehicle image corresponding to the preset number of feature vectors in the candidate feature vectors, wherein the final vehicle image is a vehicle re-identification image corresponding to the target retrieval image.

[0164] Optionally, in one embodiment of the present application, the vehicle re-identification system of the embodiment of the present application further includes: a binary data construction unit.

[0165] The binary data construction unit is configured to obtain difficult sample data appearing in the vehicle re-identification image and construct binary data for the difficult sample data;

[0166] A sorting model fine-tuning unit, configured to take the bigram data as input to the sorting model and fine-tune the sorting model based on the bigram data

[0167] Optionally, in one embodiment of the present application, the sorting model fine-tuning unit includes: a sixth feature vector generating subunit and a sorting positive library feature extraction module fine-tuning unit.

[0168] The sixth feature vector generating subunit is used to input the two-tuple data into the sorting positive library feature extraction module to obtain two positive library feature vectors;

[0169] The sorting positive library feature extraction module fine-tuning unit is used to calculate the similarity between the two positive library feature vectors and fine-tune the similarity calculation module in the sorting model according to the similarity between the positive library feature vectors.

[0170] It should be noted that the aforementioned explanation of the embodiment of the vehicle re-identification method is also applicable to the vehicle re-identification system of this embodiment, and will not be repeated here.

[0171] According to the vehicle re-identification system proposed in the embodiment of the present application, the two vehicle re-identification models can be combined with the search architecture to obtain a first-stage recall model and a second-stage ranking model. In this way, the vehicle re-identification system is constructed. By sequentially inputting the target retrieval image into the recall model and the ranking model for screening, the vehicle re-identification image of the target retrieval image can be quickly and accurately obtained. At the same time, by continuously providing feedback to the vehicle re-identification system through the real-time collection of corner case difficult sample data in the application environment, the model can be further iterated and optimized to improve the accuracy of vehicle re-identification.

[0172] This solves the problem in related technologies that the iterative optimisation of the entire vehicle re-identification system cannot be considered from the perspective of vehicle re-identification system construction and in the application environment, resulting in low accuracy of the vehicle re-identification results.

[0173] Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0174] Memory 501 , processor 502 , and computer programs stored in the memory 501 and executable on the processor 502 .

[0175] When the processor 502 executes the program, the vehicle re-identification method provided in the above embodiment is implemented.

[0176] Furthermore, the terminal further includes:

[0177] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0178] The memory 501 is used to store computer programs that can be run on the processor 502 .

[0179] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0180] If the memory 501, processor 502, and communication interface 503 are implemented independently, the communication interface 503, memory 501, and processor 502 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EIS) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0181] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0182] The processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0183] This embodiment also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the above vehicle re-identification method is implemented.

[0184] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0185] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0186] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0187] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (not exhaustive) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0188] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logical functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0189] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0190] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0191] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

[0192] It should be understood that the application of this application is not limited to the above examples. For ordinary technicians in this field, they can make improvements or changes based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

Claims

1. A vehicle re-identification method, characterized in that: The vehicle re-identification method includes: A recall model and a ranking model are constructed based on the first vehicle image training set and the second vehicle image training set, and a vehicle image database is constructed based on actual business needs, wherein the vehicle image database contains all vehicle images to be retrieved required in actual business; The recall model and the ranking model are constructed based on the first vehicle image training set and the second vehicle image training set, specifically including: Obtaining a first vehicle image training set and a second vehicle image training set, and training a classification model based on the first vehicle image training set to obtain a recall model, wherein the first vehicle image training set includes an open source dataset or a vehicle ID dataset constructed based on its own business, and the second vehicle image training set includes the vehicle ID dataset constructed based on its own business, the open source dataset, and a historical difficult sample dataset; Training the twin network model according to the second vehicle image training set to obtain a ranking model, wherein the backbone network of the twin network model is loaded with the backbone network of the recall model; Acquire a target retrieval image, input the target retrieval image into a recall model, and perform feature vector matching between the target retrieval image and each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image; Inputting the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image; The target feature vector and the candidate feature vector corresponding to the candidate vehicle image are similarly calculated using the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image.

2. The vehicle re-identification method according to claim 1, characterized in that: The model backbone network of the classification model is one of the Arcface model, SwinTransformer model, ConvNeX model, ResNet18 model or MobileNet model.

3. The vehicle re-identification method according to claim 1, characterized in that: The construction of a vehicle image database based on actual business needs specifically includes: Obtaining a dataset of vehicle images to be detected that is collected based on actual business needs, and inputting the dataset into the recall model to obtain a feature vector corresponding to each vehicle image in the dataset; A recalled positive-order library is constructed based on the vehicle images in the vehicle image data set to be detected and the feature vectors corresponding to the vehicle images, and a recalled negative-order library is constructed based on the recalled positive-order library, wherein the recalled positive-order library and the recalled negative-order library are combined into the vehicle image database.

4. The vehicle re-identification method according to claim 3, characterized in that: The step of acquiring a target retrieval image, inputting the target retrieval image into a recall model, and performing feature vector matching between the target retrieval image and each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image specifically includes: Acquire a target retrieval image, and input the target retrieval image into the recall inverted library to obtain a recall feature vector corresponding to the target retrieval image; The feature vector in the recalled inverted database is obtained, and a candidate vehicle image is obtained according to the feature vector in the recalled inverted database and the recalled feature vector.

5. The vehicle re-identification method according to claim 4, characterized in that: The step of obtaining a candidate vehicle image according to the feature vector in the recalled inverted database and the recalled feature vector specifically includes: The similarity between the recalled feature vector and the feature vectors in the recalled inverted database is calculated, and candidate vehicle images corresponding to the feature vectors in the recalled inverted database whose similarity is greater than a first preset threshold are screened out.

6. The vehicle re-identification method according to claim 3, characterized in that: The step of inputting the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image further includes: The ranking model is divided into a ranking positive library feature extraction module and a similarity calculation module.

7. The vehicle re-identification method according to claim 6, characterized in that: The ranking model is divided into a ranking positive library feature extraction module and a similarity calculation module, specifically including: The sorting model is divided according to the feature vector generation layer in the sorting model, and the part before the feature vector generation layer is set as a sorting positive library feature extraction module, wherein the sorting positive library feature extraction module is used to generate corresponding feature vectors from the vehicle image input into the sorting model; The portion after the feature vector generation layer is set as a similarity calculation module, wherein the similarity calculation module is used to calculate the similarity between the feature vectors obtained after inputting the sorting model.

8. The vehicle re-identification method according to claim 7, characterized in that: The part after the feature vector generation layer is set as a similarity calculation module, and then further includes: Obtain two vehicle images from the to-be-detected vehicle image dataset, and input the two vehicle images into the sorting feature extraction module to obtain two sorting feature vectors; The cosine similarity between the two sorted feature vectors is calculated, and the similarity calculation module is trained according to the cosine similarity.

9. The vehicle re-identification method according to claim 6, characterized in that: The step of calculating similarity between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image using the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image specifically includes: performing similarity calculation on the target feature vector and the candidate feature vectors corresponding to the candidate vehicle images in the similarity calculation module of the ranking model to obtain a plurality of similarity scores; A preset number of feature vectors are extracted from the candidate feature vectors in descending order according to the multiple similarity scores, and a final vehicle image corresponding to the preset number of feature vectors in the candidate feature vectors is output, wherein the final vehicle image is a vehicle re-identification image corresponding to the target retrieval image.

10. The vehicle re-identification method according to claim 6, characterized in that: The method further includes: calculating the similarity between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image by using the ranking model to obtain a vehicle re-identification image corresponding to the target retrieval image; and then further including: Acquire difficult sample data appearing in the vehicle re-identification image, and construct binary data for the difficult sample data; The bigram data is used as input of the ranking model, and the ranking model is fine-tuned according to the bigram data.

11. The vehicle re-identification method according to claim 10, characterized in that: The fine-tuning of the sorting model according to the bigram data specifically includes: Inputting the two-tuple data into the sorting positive library feature extraction module respectively to obtain two positive library feature vectors; The similarity between the two positive ranking library feature vectors is calculated, and the similarity calculation module in the sorting model is fine-tuned according to the similarity between the positive ranking library feature vectors.

12. A vehicle re-identification system, characterized in that: The vehicle re-identification system is applied to the vehicle re-identification method according to any one of claims 1 to 11, and the vehicle re-identification system includes: A two-stage model training module is used to construct a recall model and a ranking model based on the first vehicle image training set and the second vehicle image training set, and to build a vehicle image database based on actual business needs, wherein the vehicle image database contains all the vehicle images to be retrieved required in actual business; a candidate vehicle image acquisition module, configured to acquire a target retrieval image, input the target retrieval image into a recall model, and perform feature vector matching between the target retrieval image and each vehicle image in the vehicle image database based on the recall model to obtain a candidate vehicle image; a target feature vector generating module, configured to input the target retrieval image into the ranking model to obtain a target feature vector corresponding to the target retrieval image; The re-identification image acquisition module is used to calculate the similarity between the target feature vector and the candidate feature vector corresponding to the candidate vehicle image through the ranking model to obtain the vehicle re-identification image corresponding to the target retrieval image.

13. A terminal, characterized in that: The terminal includes: a memory, a processor, and a vehicle re-identification program stored in the memory and executable on the processor. When the vehicle re-identification program is executed by the processor, the steps of the vehicle re-identification method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a vehicle re-identification program, and when the vehicle re-identification program is executed by a processor, the steps of the vehicle re-identification method according to any one of claims 1 to 11 are implemented.

Images