Vehicle Re-identification Method Based on License Plate Information

Abstract

This invention provides a vehicle re-identification method based on license plate information. The method includes the following steps: acquiring a target image; acquiring an image of the vehicle to be identified from the target image and performing global feature extraction to obtain appearance feature information; extracting the license plate region from the vehicle to be identified in the target image and aligning and transforming the license plate region to obtain a license plate image; extracting features from the license plate image to obtain license plate feature information; performing weighted fusion of appearance feature information and license plate feature information, and determining the target vehicle based on the weighted fusion result. The method introduces license plate information into the vehicle re-identification network to improve vehicle distinguishability. In addition, the method extracts vehicle appearance features and license plate features in parallel, and obtains the final result through a fusion module, avoiding errors caused by license plate matching failures in the progressive search framework. It is also robust to the case of blurred license plates in open scenes and can be widely applied in the field of computer vision technology.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a vehicle re-identification method based on license plate information. Background Technology

[0002] Vehicle re-identification technology based on computer vision identifies the same vehicle from large-scale images / videos captured by multiple cameras. It can be applied to traffic monitoring systems to perform promising tasks such as traffic flow analysis, arterial road travel time estimation, and suspicious vehicle search. Therefore, applying vehicle re-identification technology to Intelligent Transportation Systems (ITS) has significant practical value. On the other hand, with the development and construction of smart cities, more monitoring equipment will be deployed to cover a wider area. However, current manually operated monitoring systems are inefficient in detection and re-identification, and are labor-intensive, especially when operators are monitoring multiple video feeds simultaneously. When vehicle license plates are blurred or obscured, obtaining the vehicle's trajectory requires substantial manpower and resources. Thanks to recent advancements in graphics processing units (GPUs) and computer vision, automated video analysis has become possible, and some deep convolutional neural network-based methods have achieved excellent results in vehicle re-identification tasks.

[0003] In relevant technical solutions, deep learning-based vehicle re-identification methods mainly focus on extracting distinctive appearance features. However, in real-world urban surveillance scenarios, due to changes in viewing angle, lighting conditions, and the fact that many similar vehicles belong to the same or similar models, identifying the same vehicle across multiple cameras based solely on appearance features is challenging. Furthermore, in complex situations, such as when license plates are obscured, decorated, forged, or removed, the license plate image cannot be clearly captured; consequently, the identified target vehicle has significant uncertainty. Summary of the Invention

[0004] In view of this, in order to at least partially solve one of the above-mentioned technical problems or defects, the purpose of this invention is to provide a vehicle re-identification method based on license plate information and with more accurate identification results.

[0005] The technical solution of this application provides a vehicle re-identification method based on license plate information, including the following steps: acquiring a target image;

[0006] The image of the vehicle to be identified in the target image is obtained, and global feature extraction is performed to obtain appearance feature information;

[0007] The license plate region of the vehicle to be identified in the target image is extracted, and the license plate region is aligned and transformed to obtain a license plate image.

[0008] Feature extraction is performed on the license plate image to obtain license plate feature information;

[0009] The appearance feature information and the license plate feature information are weighted and fused, and the target vehicle is determined based on the weighted fusion result.

[0010] In one feasible embodiment of the solution in this application, the step of acquiring the vehicle image to be identified in the target image and performing global feature extraction to obtain appearance feature information includes:

[0011] The image of the vehicle to be identified is input into a first network model, and a first feature map is obtained by performing convolution operations through the first network model.

[0012] The first feature channel in the first feature map is compressed to obtain the first feature value;

[0013] The first weight value is obtained by performing a semilinear transformation based on the first feature value;

[0014] The first feature channel is weighted according to the first weight value, and the appearance feature information is output according to the weighting result.

[0015] In one feasible embodiment of the present application, the step of obtaining the vehicle image to be identified from the target image and performing global feature extraction to obtain appearance feature information further includes:

[0016] Determine the loss function for the first network model;

[0017] The model parameters of the first network model are adjusted based on the calculation result of the loss function, and the appearance feature information is updated by adjusting the first network model.

[0018] In one feasible embodiment of the solution in this application, the loss function is:

[0019] Loss = L CE +αL Tri

[0020] Among them, L CE Let L be the cross-entropy loss value, α be a hyperparameter, and L be the cross-entropy loss value Tri This is the triplet loss function.

[0021] In one feasible embodiment of the present application, the step of extracting the license plate region of the vehicle to be identified in the target image and aligning and transforming the license plate region to obtain a license plate image includes:

[0022] The vehicle image to be identified is extracted from the target image, and the license plate area is determined in the vehicle image to be identified;

[0023] The license plate area is input into the second network model, and the second feature map is obtained by the feedforward unit in the second network model.

[0024] In the second feature map, a cell center is selected, and a virtual plane is constructed around the cell center;

[0025] Construct the affine matrix of the virtual plane, and perform perspective transformation based on the affine matrix to obtain the license plate image.

[0026] In one feasible embodiment of the present application, selecting a cell center in the second feature map and constructing a virtual plane around the cell center includes:

[0027] The first target probability of each pixel unit is output according to the second feature channel of the second feature map, wherein the pixel unit includes a plurality of pixels;

[0028] When the first target probability is greater than the target probability threshold, the pixel unit is determined to be the unit center.

[0029] In one feasible embodiment of the present application, the step of extracting features from the license plate image to obtain license plate feature information includes:

[0030] The license plate image is input into the third network model, and the first feature matrix is ​​output.

[0031] The first eigenvector is obtained by performing a matrix transformation based on the first eigenvector.

[0032] The first feature vector is input into the trained classifier, and each license plate character is output. The license plate characters are then combined to obtain the license plate feature information.

[0033] In one feasible embodiment of the present application, the method further includes the following steps:

[0034] The appearance feature information and the license plate feature information are vector-concatenated to obtain the second feature vector;

[0035] The target vehicle is determined based on the second feature vector.

[0036] In one feasible embodiment of the present application, the step of performing weighted fusion of the appearance feature information and the license plate feature information, and determining the target vehicle based on the weighted fusion result, includes:

[0037] Obtain historical appearance information and determine a first feature distance between the appearance feature information and the historical appearance information;

[0038] Obtain historical license plate information and determine a second feature distance between the license plate feature information and the historical license plate information;

[0039] The target vehicle is determined based on a weighted fusion of the first feature distance and the second feature distance, and the weighted fusion result is used to determine the target vehicle.

[0040] In one feasible embodiment of the present application, the calculation formula for the weighted fusion is:

[0041] D total =(1-β)×D(x1,x2)+β×D(lp1,lp1)

[0042] Among them, D total β is the weighted fusion result of the first feature distance and the second feature distance; β is the weight of the second feature distance; D(x1,x2) is the first feature distance, where x1 represents the queried vehicle appearance feature information and x2 represents the historical appearance information; D(lp1,lp2) is the second feature distance, where lp1 represents the license plate feature information and lp2 represents the historical license plate information.

[0043] The advantages and beneficial effects of the present invention will be set forth in part in the following description, and the rest will become apparent from the specific embodiments thereof:

[0044] This application proposes a vehicle re-identification method that combines vehicle appearance features and license plate features into a dual-stream structure for joint license plate information. On the one hand, the method aims to introduce license plate information into the vehicle re-identification network to improve vehicle distinguishability. On the other hand, the method extracts vehicle appearance features and license plate features in parallel, and obtains the final result through a fusion module. This avoids the errors caused by license plate matching failures in the progressive search framework and is also robust to the case of blurred license plates in open scenarios. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the steps of the vehicle re-identification method based on license plate information in the technical solution of this application.

[0047] Figure 2 This is a schematic diagram of the vehicle re-identification model architecture based on the combination of channel attention and license plate information in the technical solution of this application;

[0048] Figure 3 A schematic diagram of a license plate feature extraction network based on Mobilenet in the technical solution of this application. Detailed Implementation

[0049] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0050] As noted in the technical background, existing deep learning-based vehicle re-identification methods primarily focus on extracting distinctive appearance features. However, in real-world urban surveillance scenarios, due to changes in viewing angle, lighting conditions, and the fact that many similar vehicles belong to the same or similar models, identifying the same vehicle across multiple cameras based solely on appearance features is challenging. Prominent areas in car images, such as rearview mirrors, headlights, taillights, and signs, are often blurred, and their positions change significantly with varying shooting angles.

[0051] On the other hand, license plate numbers serve as vehicle identification, and license plate recognition technology is relatively mature in restricted environments. Some vehicle re-identification methods consider license plate information, using license plate characters to confirm vehicle identity and re-identify the target vehicle from a large number of vehicle images. The problem with this approach is that in real traffic monitoring environments, multiple viewing angles, complex lighting, and different camera resolutions significantly impact license plate recognition. In extreme cases, such as when license plates are obscured, decorated, forged, or removed, the license plate image cannot be clearly captured. Therefore, re-identifying target vehicles by retrieving license plate information in traffic monitoring environments carries significant uncertainty. How to effectively utilize license plate information to improve vehicle re-identification performance in unrestricted urban monitoring remains a pressing issue.

[0052] Regarding the problems identified in the aforementioned technical solutions, in the first aspect, such as Figure 1 As shown, the technical solution of this application provides a vehicle re-identification method based on license plate information; the method includes steps S100-S500:

[0053] S100, Acquire the target image;

[0054] Specifically, in this embodiment, real-time traffic conditions or vehicle flow monitoring footage of the target road segment is collected using pre-set video acquisition or monitoring equipment. A specific frame from the video stream is selected as the target image in this embodiment. It should be noted that the target image obtained from the video stream, or the target image acquired through other real-time acquisition methods, should contain the target vehicle to be re-identified. Furthermore, the process of obtaining the target image by decomposing the video stream frame by frame in this embodiment can be performed using video image extraction techniques from relevant technical solutions, which will not be elaborated upon here.

[0055] S200: Obtain the image of the vehicle to be identified from the target image, and perform global feature extraction to obtain appearance feature information;

[0056] Specifically, in the proposed dual-stream structure model of vehicle appearance and license plate features, the vehicle appearance branch is responsible for extracting global appearance features. The backbone network used in this embodiment is a channel-focused ResNet50, which effectively focuses on vehicle appearance without relying on expensive manual annotation, explicitly modeling the relationships between feature channels. Furthermore, the network model used in this embodiment does not introduce new spatial dimensions for feature channel fusion; instead, it automatically learns the weights of each feature channel and then boosts or suppresses the corresponding features according to these weights.

[0057] S300: Extract the license plate region of the vehicle to be identified in the target image, align and transform the license plate region to obtain the license plate image;

[0058] Specifically, in this embodiment, the license plate of the vehicle to be identified (partial image) contained in the target image is first detected and located. Based on the license plate location result, the license plate image is further aligned and transformed to avoid introducing noise during the extraction of license plate feature information, and finally the transformed license plate image is obtained.

[0059] S400. Extract features from the license plate image to obtain license plate feature information;

[0060] Specifically, in the embodiments, a license plate feature extraction network based on the Mobilenet license plate detection network can be constructed. Through this network model, the feature information of the license plate image can be captured accurately and effectively.

[0061] S500 performs weighted fusion of appearance feature information and license plate feature information, and determines the target vehicle based on the weighted fusion result;

[0062] Specifically, in the embodiment, the appearance feature information of the vehicle to be identified obtained in step S200 and the license plate feature information obtained in step S400 are weighted and fused. For example, in the embodiment, the weighted fusion can be performed based on the confidence of the license plate positioning result and the proportion of the target vehicle in the target image, so that the re-identification result of the target vehicle is more robust when the license plate image of the target vehicle is relatively blurry.

[0063] In some feasible implementations, the step S200 of obtaining the image of the vehicle to be identified in the target image and performing global feature extraction to obtain appearance feature information may include steps S210-S240:

[0064] S210. Input the image of the vehicle to be identified into the first network model, and perform convolution operation through the first network model to obtain the first feature map;

[0065] The first network model is a network model for extracting vehicle appearance features. For example, in this embodiment, a ResNet-50 network based on channel focus can be constructed to extract comprehensive global features of the vehicle. Figure 2 As shown, in the dual-stream structure of vehicle appearance and license plate features in this embodiment, the vehicle appearance branch is responsible for extracting global appearance features. The backbone network used here is a channel-focused ResNet50, which effectively focuses on vehicle appearance without relying on expensive manual annotation, explicitly modeling the relationships between feature channels. Furthermore, no new spatial dimension is introduced for feature channel fusion. Instead, the weights of each feature channel are automatically obtained through learning, and then the corresponding features are boosted or suppressed according to these weights.

[0066] Specifically, in this embodiment, the SE module can be embedded in the convolutional module of ResNet-50 to obtain the first network model in this embodiment. After the residual module, the features are weighted together with the original features and the features before the residual through global pooling, fully connected, ReLU activation, fully connected, and Sigmoid activation operations to achieve channel focus and form the adopted SE-ResNet-50 backbone network.

[0067] S220. Compress the first feature channel in the first feature map to obtain the first feature value;

[0068] The first feature map is obtained by performing a series of convolution operations using the convolutional module of ResNet-50. Specifically, in this embodiment, after obtaining this feature map, feature compression is performed along the spatial dimension, transforming each multi-dimensional feature channel in the first feature map into a real number, i.e., the first feature value. This real number, to some extent, possesses a global receptive field, and the output dimension matches the number of input feature channels. It represents the global distribution of the response across the feature channels and allows layers closer to the input to also obtain a global receptive field.

[0069] S230. Perform a semilinear transformation based on the first feature value to obtain the first weight value;

[0070] Specifically, in this embodiment, after obtaining the first feature value, an activation operation is further performed on this feature value, which is a mechanism similar to the gate in a recurrent neural network. Weights are generated for each feature channel using the parameter w, where the parameter w is learned to explicitly model the correlation between feature channels.

[0071] S240. Perform a weighted operation on the first feature channel according to the first weight value, and output the appearance feature information according to the weighted operation result.

[0072] Specifically, in the embodiment, the final step is a reweighting operation, which takes the weights of the activation function output as the importance of each feature channel after feature selection, and then multiplies them channel by channel to weight the previous features, thus completing the recalibration of the original features in the channel dimension; the final appearance feature information is obtained by summing the weight values ​​based on the recalibrated original features.

[0073] In some alternative implementations, the step S200 of the embodiment method, which involves acquiring the image of the vehicle to be identified from the target image and performing global feature extraction to obtain appearance feature information, may further include steps S250-S260:

[0074] S250. Determine the loss function for the first network model;

[0075] S260. Adjust the model parameters of the first network model according to the calculation result of the loss value function, and update the appearance feature information through the adjusted first network model.

[0076] Specifically, in this embodiment, the loss function of the vehicle appearance feature extraction network consists of cross-entropy loss and triplet loss, where the cross-entropy loss is expressed as:

[0077]

[0078] Among them, y iThis represents the probability of image identity label verification, and N represents the number of images in a processing batch during model training.

[0079] The triplet loss function in another embodiment is formulated as follows:

[0080]

[0081] Where s represents the anchor image, n represents the negative sample image, p represents the positive sample image, λ represents the boundary distance between positive and negative samples, and N represents the number of images in a batch during model training. sn D represents the feature distance between the anchor image and the positive sample. sn The feature distance between the anchor image and the negative sample is represented by: In summary, the overall loss function of the vehicle appearance feature extraction network can be expressed as:

[0082] Loss = L CE +αL Tri

[0083] Where α is a hyperparameter.

[0084] In some alternative implementations, the step S300 of the embodiment method extracting the license plate region of the vehicle to be identified in the target image and aligning and transforming the license plate region to obtain the license plate image may include steps S310-S340:

[0085] S310. Extract the image of the vehicle to be identified from the target image, and determine the license plate area in the image of the vehicle to be identified;

[0086] S320. Input the license plate area into the second network model, and obtain the second feature map through the feedforward unit in the second network model.

[0087] S330. Select the center of the unit in the second feature map and construct a virtual plane around the center of the unit;

[0088] S340. Construct the affine matrix of the virtual plane, and perform perspective transformation based on the affine matrix to obtain the license plate image;

[0089] In this embodiment, in the license plate feature extraction branch of the dual-stream structure, the license plate image is first extracted based on the license plate detection results. Considering that in unconstrained scenarios, license plates in vehicle images from a monitoring perspective have a top-down angle and perspective distortion, a non-rectangular detection box is used to more accurately outline the license plate image in order to prevent background noise such as vehicle body color information from being introduced into the license plate image feature extraction.

[0090] Specifically, in certain non-limiting scenarios, license plates may be severely distorted due to tilted viewing angles. Therefore, the second network model in this embodiment can detect and correct multiple distorted license plates in a single image. For example, the ALPR model can be used as the second network model in this embodiment. The task of the ALPR model is to find and recognize license plates in an image, which can be divided into four sub-tasks: vehicle detection, license plate detection, character segmentation, and character recognition. The Twisted Plane Object Detection Network (WPOD-NET) searches for license plates. WPOD-NET is developed using the concepts of YOLO, SSD, and STN. YOLO and SSD perform fast multi-object detection and recognition, without considering spatial transformations, only generating rectangular bounding boxes for each detection.

[0091] In the embodiment, step S330, which involves selecting the center of a cell in the second feature map and constructing a virtual plane around that center, may further include steps S331-S332:

[0092] S331. Output the first target probability of each pixel unit according to the second feature channel of the second feature map; wherein, a pixel unit includes several pixels;

[0093] S332. When the probability of the first target is greater than the target probability threshold, determine the pixel unit as the unit center;

[0094] More specifically, in the implementation using WPOD-NET, a vehicle image resized by the vehicle detection module is first input into the network. The feedforward network obtains an 8-channel feature map, and calculates the target / non-target probability (2 feature maps) and regression affine transformation parameters (6 feature maps). To extract distorted license plates, a fictitious square of fixed size around the cell center (m, n) is first considered. If the probability of the cell being a target is higher than a given detection threshold, partial regression parameters are used to construct an affine matrix that transforms the fictitious square into the four vertices of the license plate. Then, a perspective transformation is used to correct the obtained license plate region to a front view.

[0095] In some feasible implementations, the step S400 of the embodiment method to extract features from the license plate image to obtain license plate feature information may include steps S410-S430:

[0096] S410. Input the license plate image into the third network model and output the first feature matrix.

[0097] S420. Obtain the first eigenvector by performing matrix transformation based on the first eigenvector;

[0098] S430. Input the first feature vector into the trained classifier and output each license plate character. Combine the license plate characters to obtain the license plate feature information.

[0099] The third network model can be, for example, MobileNet for license plate feature extraction. Figure 3 As shown, in this specific embodiment, license plate information is incorporated into vehicle re-identification. Unlike other license plate-based vehicle re-identification algorithms that directly verify license plates using SNNs, this approach treats license plate information as a re-identification problem within the vehicle re-identification framework, essentially adopting a re-identification strategy. This parallel framework avoids errors introduced into the re-identification system by license plate verification errors in progressive search. During model training, a license plate recognition network trained using cross-entropy loss can be used to extract license plate features, trained on the CCPC dataset. During inference, the last layer is removed, and the license plate character information is not regressed. A fully connected layer is then used to obtain a 1*1536 dimensional feature vector of the license plate.

[0100] For example, in this embodiment, the license plate feature extraction model based on MobileNet as the backbone network uses Softmax classification loss to train the license plate recognition network to extract license plate features. The license plate image is transformed into a 3×8×64-dimensional feature matrix through the MobileNet backbone network, which is the first feature matrix. After reshaping the aforementioned feature matrix to obtain the first feature vector, this feature vector is then input into a fully connected layer and predicted using seven classifiers, the number of classifiers being the same as the number of characters in the license plate. After training on the CCPC dataset, the final classification layer is removed during the network inference stage to prevent the regression of license plate character information. The license plate feature vector is obtained after reshaping. It is worth noting that not obtaining the license plate string during the inference stage is important for privacy protection. According to license plate rules, the last five characters of a regular license plate each contain 24 different letters or 10 numbers. The category dimension of license plate data is smaller than that of vehicle re-identification, so it is suitable to use classification loss to train the network without causing overfitting to the data domain.

[0101] In some feasible implementations, the step S500, which involves weighted fusion of appearance feature information and license plate feature information to determine the target vehicle based on the weighted fusion result, may include steps S510-S530:

[0102] S510. Obtain historical license plate information and determine the first feature distance between the appearance feature information and the historical appearance information;

[0103] S520. Obtain historical appearance information and determine the second feature distance between license plate feature information and historical license plate information;

[0104] S530. Perform weighted fusion based on the first feature distance and the second feature distance, and determine the target vehicle based on the weighted fusion result;

[0105] As mentioned earlier, license plate information in open scenarios is not always stable and reliable, and directly splicing features can introduce noise that affects the accuracy of re-identification. Therefore, the two branches calculate feature distances with the image library separately, and then weight them and sort them. The Weighted Feature Fusion (WFF) module weights feature distances based on the license plate location confidence and the original pixel area of ​​the license plate, and is also robust to the case of blurred license plates in open scenarios. Here, the confidence refers to the probability of the license plate target in the region during the license plate region detection process. Secondly, in constructing the dataset, the implementation found that the clarity of the license plate is basically positively correlated with the size of the pixel region occupied by the license plate in the original image.

[0106] Furthermore, the feature distance weighted fusion calculation formula in the embodiment is as follows:

[0107] D total =(1-β)×D(x1,x2)+β×D(lp1,lp2)

[0108] Among them, D total The first feature distance and the second feature distance are weighted fusion results; β is the weight of the second feature distance; D(x1,x2) is the first feature distance, where x1 represents the queried vehicle appearance feature information and x2 represents the historical appearance information; D(lp1,lp2) is the second feature distance, where lp1 represents the license plate feature information and lp2 represents the historical license plate information.

[0109] It should be noted that in the embodiment, D() represents the Euclidean distance between features; based on the Euclidean distance calculation formula, D(x1,x2) in the embodiment can be expressed as:

[0110]

[0111] in, It refers to the vehicle's exterior features. These are the vehicle appearance features in the image library. D(lp1,lp2) can be further represented as:

[0112]

[0113] in, It's a characteristic of the license plate. These are license plate features from the image library. It should be noted that this is an example. and Images from the same image library.

[0114] In some alternative implementations, the vehicle re-identification method based on license plate information may further include steps S600-S700:

[0115] S600: Concatenate the appearance feature information and the license plate feature information into a vector to obtain the second feature vector;

[0116] S700: Determine the target vehicle based on the second feature vector;

[0117] In a specific embodiment, after obtaining the license plate feature vector through license plate feature model inference, there are two operation methods: it can be directly concatenated with the feature vector of the vehicle appearance branch to form a new feature vector, which is the second feature vector; the target vehicle can be identified by a classifier or other network model based on this second feature vector.

[0118] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0119] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0120] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0121] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0122] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0123] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A vehicle re-identification method based on license plate information, characterized in that, Includes the following steps: Acquire the target image; The image of the vehicle to be identified in the target image is obtained, and global feature extraction is performed to obtain appearance feature information; The license plate region of the vehicle to be identified in the target image is extracted, and the license plate region is aligned and transformed to obtain a license plate image. Feature extraction is performed on the license plate image to obtain license plate feature information; The appearance feature information and the license plate feature information are weighted and fused, and the target vehicle is determined based on the weighted fusion result. Specifically, the weighted fusion is performed based on a first feature distance and a second feature distance. The first feature distance is determined based on historical appearance information and the appearance feature information, and the second feature distance is determined based on historical license plate information and the license plate feature information. The step of extracting the license plate region of the vehicle to be identified in the target image and aligning and transforming the license plate region to obtain a license plate image includes: The vehicle image to be identified is extracted from the target image, and the license plate area is determined in the vehicle image to be identified; The license plate area is input into the second network model, and the second feature map is obtained by the feedforward unit in the second network model. The first target probability of each pixel unit is output according to the second feature channel of the second feature map. When the first target probability is greater than the target probability threshold, the pixel unit is determined as the unit center, and a virtual plane is constructed around the unit center. The pixel unit includes a number of pixels, the size of the virtual plane is fixed, and the first target probability represents the probability that the pixel unit is a target. Construct the affine matrix of the virtual plane, and perform perspective transformation based on the affine matrix to obtain the license plate image; The step of extracting features from the license plate image to obtain license plate feature information includes: The license plate image is input into the third network model, and the first feature matrix is ​​output. The first eigenvector is obtained by performing a matrix transformation based on the first eigenvector. The first feature vector is input into the trained classifier, and each license plate character is output. The license plate characters are then combined to obtain the license plate feature information.

2. The vehicle re-identification method based on license plate information as described in claim 1, characterized in that, The step of acquiring the vehicle image to be identified from the target image and performing global feature extraction to obtain appearance feature information includes: The image of the vehicle to be identified is input into a first network model, and a first feature map is obtained by performing convolution operations through the first network model. The first feature channel in the first feature map is compressed to obtain the first feature value; The first weight value is obtained by performing a semilinear transformation based on the first feature value; The first feature channel is weighted according to the first weight value, and the appearance feature information is output according to the weighting result.

3. The vehicle re-identification method based on license plate information according to claim 2, characterized in that, The step of acquiring the vehicle image to be identified from the target image and performing global feature extraction to obtain appearance feature information further includes: Determine the loss function for the first network model; The model parameters of the first network model are adjusted based on the calculation result of the loss function, and the appearance feature information is updated by adjusting the first network model.

4. The vehicle re-identification method based on license plate information according to claim 3, characterized in that, The loss function is: in, This represents the cross-entropy loss value. For hyperparameters, This is the triplet loss function.

5. The vehicle re-identification method based on license plate information according to claim 1, characterized in that, The method further includes the following steps: The appearance feature information and the license plate feature information are vector-concatenated to obtain the second feature vector; The target vehicle is determined based on the second feature vector.

6. The vehicle re-identification method based on license plate information according to claim 1, characterized in that, The step of weighted fusion of the appearance feature information and the license plate feature information, and determining the target vehicle based on the weighted fusion result, includes: Obtain historical appearance information and determine a first feature distance between the appearance feature information and the historical appearance information; Obtain historical license plate information and determine a second feature distance between the license plate feature information and the historical license plate information; The target vehicle is determined based on a weighted fusion of the first feature distance and the second feature distance, and the weighted fusion result is used to determine the target vehicle.

7. The vehicle re-identification method based on license plate information according to claim 6, characterized in that, The formula for calculating the weighted fusion is as follows: in, This is the weighted fusion result of the first feature distance and the second feature distance; The weight of the second feature distance, The first feature distance, Represents querying appearance feature information. Represents historical appearance information; The second feature distance, Characteristic query of license plate feature information, Represents historical license plate information.

Images