Vehicle attribute recognition model training method, vehicle attribute recognition method and device

Abstract

The application provides a vehicle attribute recognition model training method, a vehicle attribute recognition method and device, and relates to the technical field of computer vision. The vehicle attribute recognition model training method comprises the following steps: extracting a base library feature vector of each picture in a vehicle attribute base library, and extracting a sample feature vector of a to-be-tested sample; each picture in the vehicle attribute base library comprises a license plate region; calculating a vector distance between each base library feature vector and the sample feature vector, respectively; determining pseudo-label information of the to-be-tested sample according to the vector distance; taking the target to-be-tested sample and the pseudo-label information of the target to-be-tested sample as a set of unlabeled samples, using multiple sets of unlabeled samples and the vehicle attribute base library as a training set to train a classification model, and obtaining a vehicle attribute recognition model. The application generates pseudo-labels for to-be-tested samples, obtains unlabeled samples, performs semi-supervised learning according to the unlabeled samples and the vehicle attribute base library comprising the license plate region, reduces calibration costs, and improves the training efficiency of the model.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for training a vehicle attribute recognition model, a vehicle attribute recognition method, and an apparatus. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] Vehicle attributes refer to inherent characteristics of a vehicle, such as brand, model, and color. Vehicle attribute recognition is widely used in the transportation sector, identifying different dimensions of vehicle information (brand, model, color, etc.) for traffic management, violation processing, and other operations. While license plate recognition performance is already very good, vehicle attribute recognition performance lags far behind. The process of vehicle attribute recognition is difficult due to challenges such as calibration and high noise levels (e.g., color is greatly affected by lighting conditions). Furthermore, the recognition rate of vehicle attributes is lower than that of license plate (ID-type) recognition, and the finer the granularity, the lower the performance. For example, there are tens of thousands of vehicle sub-brands. Vehicle attribute calibration is costly; improving performance requires a significant investment of calibration resources. Summary of the Invention

[0004] This application provides a vehicle attribute recognition model training method, a vehicle attribute recognition method and apparatus, to at least solve the problems of reducing vehicle attribute calibration costs and improving model training efficiency.

[0005] According to one aspect of this application, a method for training a vehicle attribute recognition model is provided, comprising: extracting base feature vectors of each image in a vehicle attribute base database, and extracting sample feature vectors of a test sample; wherein each image in the vehicle attribute base database includes a license plate area; calculating the vector distance between each base feature vector and the sample feature vector, and determining the pseudo-label information of the test sample based on the vector distance; using the target test sample and the pseudo-label information of the target test sample as a set of unlabeled samples, and using multiple sets of unlabeled samples and the vehicle attribute base database as a training set to train a classification model, thereby obtaining a vehicle attribute recognition model.

[0006] According to another aspect of this application, a vehicle attribute recognition method is provided, comprising: acquiring a target image; inputting the target image into a vehicle attribute recognition model to obtain attribute information of the vehicle in the target image; wherein the vehicle attribute recognition model is obtained by training using the above method.

[0007] According to another aspect of this application, a vehicle attribute recognition model training device is provided, comprising: a feature extraction module, used to extract the base feature vectors of each image in a vehicle attribute base database and extract the sample feature vectors of the test sample; wherein each image in the vehicle attribute base database includes a license plate area; a pseudo-label module, used to calculate the vector distance between each base feature vector and the sample feature vector, and determine the pseudo-label information of the test sample based on the vector distance; and a training module, used to take the target test sample and the pseudo-label information of the target test sample as a set of unlabeled samples, and use multiple sets of unlabeled samples and the vehicle attribute base database as a training set to train a classification model to obtain a vehicle attribute recognition model.

[0008] According to another aspect of this application, a vehicle attribute recognition device is provided, comprising: an acquisition module for acquiring a target image; and a determination module for inputting the target image into a vehicle attribute recognition model to obtain attribute information of the vehicle in the target image; wherein the vehicle attribute recognition model is obtained by training using the method described above.

[0009] According to another aspect of this application, an electronic device is also provided, comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method described above.

[0010] According to another aspect of this application, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to cause the computer to perform the method steps according to the above description.

[0011] According to another aspect of this application, a computer program product is also provided, wherein the computer program product includes a computer program that, when executed by a processor, implements the above-described method steps.

[0012] In this embodiment, the base feature vectors of each image in the vehicle attribute base database are extracted, and the sample feature vectors of the test sample are extracted. Each image in the vehicle attribute base database includes a license plate area. The vector distance between each base feature vector and the sample feature vector is calculated, and the pseudo-label information of the test sample is determined based on the vector distance. The target test sample and the pseudo-label information of the target test sample are used as a set of unlabeled samples. Multiple sets of unlabeled samples and the vehicle attribute base database are used as a training set to train a classification model, thereby obtaining a vehicle attribute recognition model. This embodiment of the invention generates pseudo-labels for the test sample by calculating the vector distance between each base feature vector and the sample feature vector, resulting in unlabeled samples. Semi-supervised learning is then performed based on the unlabeled samples and the vehicle attribute base database including the license plate area, reducing a large amount of time-consuming and laborious sample calibration work, lowering calibration costs, and improving the training efficiency of the vehicle attribute recognition model. Attached Figure Description

[0013] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0014] Figure 1 A flowchart of a vehicle attribute recognition model training method according to an exemplary embodiment of this application is shown;

[0015] Figure 2 A schematic diagram of the overall network structure according to an exemplary embodiment of this application is shown;

[0016] Figure 3 A schematic diagram of a vehicle ID recognition network according to an exemplary embodiment of this application is shown;

[0017] Figure 4 A schematic diagram of a pseudo-label generation process according to an exemplary embodiment of this application is shown;

[0018] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of this application is shown;

[0019] Figure 6 A schematic diagram of a fine-grained attribute recognition network according to an exemplary embodiment of this application is shown;

[0020] Figure 7 A schematic diagram of a coarse-grained attribute recognition network according to an exemplary embodiment of this application is shown;

[0021] Figure 8 A network block structure 1 according to an exemplary embodiment of this application is shown;

[0022] Figure 9 A second network block structure according to an exemplary embodiment of this application is shown;

[0023] Figure 10 A network block structure three according to an exemplary embodiment of this application is shown;

[0024] Figure 11 A network block structure four according to an exemplary embodiment of this application is shown;

[0025] Figure 12 A structural block diagram of a vehicle attribute recognition model training apparatus according to an exemplary embodiment of this application is shown. Detailed Implementation

[0026] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0027] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0028] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0031] Existing vehicle attribute recognition schemes include: Scheme 1: Local region-based supervised attribute recognition. This scheme relies on labeled data, extracting a complete vehicle image and at least one local image from a vehicle image, with different parts fed into different networks. Scheme 2: Full-image-based supervised attribute recognition. This scheme also relies on labeled data and focuses on extracting features from the vehicle itself for classification, without introducing other high-dimensional information. Scheme 3: Multi-label-based supervised attribute recognition. This method improves the recognition accuracy of the vehicle attribute recognition model.

[0032] Based on this, this application provides a vehicle attribute recognition model training method, vehicle attribute recognition method, and apparatus. The input is a complete image of the vehicle, which is directly fed into the network. It outputs recognition results end-to-end, without the concept of local regions. It utilizes features identified by the vehicle ID and relies on semi-supervised learning with a small number of labeled and a large number of unlabeled samples, reducing the time-consuming and laborious sample labeling work, lowering labeling costs, and improving the training efficiency of the vehicle attribute recognition model. Through the constraints of a hierarchical loss function, dimensionality reduction supervision of vehicle attribute feature learning can achieve a painless improvement effect.

[0033] First, the terms involved will be explained.

[0034] Vehicle ID: A vehicle with a license plate number has a unique ID.

[0035] Dimensionality reduction supervision: Train the model with finer-grained categories, and then use this feature to perform coarse-grained classification tasks.

[0036] According to one aspect of the present invention, a method for training a vehicle attribute recognition model is provided. Figure 1 The flowchart of the vehicle attribute recognition model training method provided in the embodiments of the present invention is as follows: Figure 1 As shown, the method includes the following steps:

[0037] Step S202: Extract the base feature vector of each image in the vehicle attribute base database, and extract the sample feature vector of the sample to be tested; each image in the vehicle attribute base database includes the license plate area;

[0038] In this step, the vehicle attribute database includes multiple photos. For example, assuming brand recognition is being performed, there are N brands, each corresponding to R photos, then the database consists of N*R photos. When collecting the R photos, factors such as shooting angle (relative to camera pitch, yaw, and rotation), vehicle orientation, and scene brightness (daytime, evening, night, etc.) can be considered. Each image in the vehicle attribute database includes the license plate area. The test sample can be an image that includes vehicle images. Feature extraction is performed on each image in the vehicle attribute database and the test sample, respectively, to obtain the database feature vector and the sample feature vector.

[0039] Step S204: Calculate the vector distance between each of the base database feature vectors and the sample feature vectors, and determine the pseudo-label information of the sample to be tested based on the vector distance;

[0040] In this step, the vector distance between each of the base database feature vectors and the sample feature vectors is calculated to achieve similarity comparison. Based on the degree of similarity, the base database feature vectors are selected as pseudo-label information for the test samples. By generating pseudo-label information for different test samples using the same steps, a large number of unlabeled samples can be obtained, avoiding extensive calibration work and reducing calibration difficulty and cost.

[0041] In one possible implementation, the pseudo-label information of the sample to be tested is determined based on the vector distance, which can be performed according to the following steps: if the first vector distance between the target base library feature vector and the sample feature vector is less than the second vector distance, then the target base library feature vector is used as the pseudo-label information of the sample to be tested; the second vector distance is the vector distance between any vector in the base library feature vector other than the target base library feature vector and the sample feature vector.

[0042] Step S206: The target test sample and its pseudo-label information are used as a set of unlabeled samples. Multiple sets of unlabeled samples and the vehicle attribute database are used as training sets to train a classification model and obtain a vehicle attribute recognition model.

[0043] In this step, each image in the vehicle attribute database includes the license plate area. Therefore, the vehicle attribute database can be used as labeled samples. After processing in steps S202 and S206, multiple sets of unlabeled samples are obtained. Semi-supervised learning is performed using a small number of labeled and a large number of unlabeled samples as training sets to obtain the vehicle attribute recognition model.

[0044] In this embodiment, the base feature vectors of each image in the vehicle attribute base database are extracted, and the sample feature vectors of the test sample are extracted. Each image in the vehicle attribute base database includes a license plate area. The vector distance between each base feature vector and the sample feature vector is calculated, and the pseudo-label information of the test sample is determined based on the vector distance. The target test sample and the pseudo-label information of the target test sample are used as a set of unlabeled samples. Multiple sets of unlabeled samples and the vehicle attribute base database are used as a training set to train a classification model, thereby obtaining a vehicle attribute recognition model. This embodiment of the invention generates pseudo-labels for the test sample by calculating the vector distance between each base feature vector and the sample feature vector, resulting in unlabeled samples. Semi-supervised learning is then performed based on the unlabeled samples and the vehicle attribute base database including the license plate area, reducing a large amount of time-consuming and laborious sample calibration work, lowering calibration costs, and improving the training efficiency of the vehicle attribute recognition model.

[0045] To ensure high feature extraction efficiency, in one optional implementation, the base feature vectors of each image in the vehicle attribute base database are extracted, and the sample feature vectors of the test sample are extracted. This can be done by following these steps: using a feature extraction network, the base feature vectors of each image in the vehicle attribute base database are extracted, and the sample feature vectors of the test sample are extracted.

[0046] In this optional implementation, see Figure 8 The first network block structure shown can adopt the backbone of Darknet (a relatively lightweight open-source deep learning framework). The backbone block consists of two convolutional networks (Cnn+Bn+LeakyReLU) and one residual network (Residual Block). Figure 8 M and N will change. See also Figure 9 The second network block structure shown in the diagram illustrates the Residual Block of the Darknet network.

[0047] See Figure 4 The diagram illustrates the pseudo-label generation process. An image, including the license plate area, from the vehicle attribute database is input into a feature extraction network. The network extracts license plate information and vehicle attribute information from the image to obtain the database feature vector. The test sample is then input into the feature extraction network, which extracts license plate information and vehicle attribute information from the image to obtain the sample feature vector. Finally, pseudo-labels are obtained by calculating the vector distance.

[0048] To improve the robustness of vehicle ID recognition, in one optional implementation, before extracting the base feature vectors of each image in the vehicle attribute base database using a feature extraction network, the following steps can be performed: training a target network using a first sample set to obtain the feature extraction network; each sample in the first sample set includes a vehicle image; the vehicle image includes a license plate region; the first sample set includes noise regions with random probability; and so on.

[0049] In this optional implementation, each sample in the first sample set includes a vehicle image; the vehicle image includes a license plate region. To prevent the training process from learning only certain regions, such as the license plate region, the first sample set can include noise regions with random probability. These noise regions occlude a designated region of the vehicle image. This designated region can be set according to actual needs, thus ensuring that the training process of the target network does not only learn certain specific regions, significantly improving robustness. For example, the designated region can be the license plate region, a region other than the license plate region, or a region other than the window region. If the designated region is the license plate region, it can be completely occluded with random probability. If the designated region is a non-license plate region, it can be completely unoccluded with random probability. If the designated region is a non-window region, it can occlude random regions other than the window with random probability. It should be noted that when the designated region is a non-window region, a random area Mask (0 < S) ​​can be used. mask <1 / 2*S car It can obstruct random areas other than the car windows.

[0050] In one optional implementation, training the target network using a first sample set to obtain a feature extraction network can be performed according to the following steps: generating a first calculation result using the first sample set and the target network; generating a second calculation result using the first calculation result and the metric learning network; adjusting the target network using the cross-entropy loss function and the first calculation result, and adjusting the metric learning network using the circle loss function and the second calculation result.

[0051] In this optional implementation, see Figure 3 The diagram shown illustrates a vehicle ID recognition network that incorporates a metric learning network. The principle of metric learning is to learn a mapping from the original features to a low-dimensional dense vector space (embedding space), such that objects of the same type are closer when the distance calculated using a distance function in the embedding space, while objects of different types are farther apart. The goal of metric learning is to shorten the distance between objects with the same ID and increase the distance between objects with different IDs. The metric learning network structure can adopt a Res-series backbone. The backbone block uses a Residual Block, see [link to documentation]. Figure 10 The network block structure shown is number three. The network consists of multiple backbone blocks, where the number of channels in the convolutions varies.

[0052] The loss function for the target network can be the focal loss function (cross-entropy loss), the principle of which can be found in the formula: L fl =-(1-p t ) γ log(p t ), where L flrepresents the value of the focal loss function, pt represents the probability that the t-th sample is predicted as a positive sample, and γ is a hyperparameter used to adjust the contribution of simple and difficult samples to the loss function.

[0053] The loss function for metric learning networks can be Circle Loss. Circle loss proposes that if a similarity score is far from the optimal center, it should be penalized more. The principle can be found in the formula: Among them, K intra-class similarity scores Within a class refers to vehicles with the same license plate, and the K similarity scores are within each class. Within a class refers to vehicles with the same license plate, S n S p This is the formula for calculating similarity scores. The formulas for calculating similarity scores within and between classes can be different. γ is the weight, which is set manually, and m is a constant.

[0054] In one alternative implementation, the circle loss function is calculated according to the following formula:

[0055]

[0056] Among them, L cl This is the value of the circle loss function, where γ is the first weight. Where (q=1,…,L) branch_same These are vehicles with different license plates but the same attributes. Where (u=1,...,L) branch_diff (L) refers to vehicles with different license plates and different attributes. branch_same +L branch_diff =L, β is the second weight, β>1, m is a constant, S n S p This is the formula used to calculate the similarity score. It is the similarity score within K classes. It is the similarity score between L classes.

[0057] In this optional implementation, the value of the circle loss function can be calculated based on this formula, thereby achieving similarity scores among the L classes in the formula. The optimization aims to narrow the similarity scores between vehicles with different IDs from the same main brand, and widen the similarity scores between vehicles with different IDs from different main brands. It's important to note that "within-class" refers to vehicles with the same ID, while "between-class" refers to vehicles with different IDs. This loss function optimizes the training process of the feature extraction network, thereby improving its performance.

[0058] To optimize the attribute recognition network and enable it to adapt to attribute recognition tasks in more granular scenarios, in one optional implementation, the classification model is trained using multiple sets of unlabeled samples and the vehicle attribute database as a training set. This can be performed according to the following steps: training the classification model using a hierarchical loss function; the hierarchical loss function penalizes different levels of classification errors in the classification results with varying degrees; the classification results include attributes at multiple levels.

[0059] In this optional implementation, the classification results include attributes at multiple levels. For example, the brand attribute of a vehicle can be divided into three different levels: main brand, sub-brand, and detailed model. If there is a classification error within the main brand level, the penalty will be different from the penalty for a classification error within the sub-brand level. The hierarchical loss function applies different penalties to classification errors between different levels in the classification results, making the trained classification model more sensitive to differences between categories.

[0060] It should be noted that the network structure of the classification model can adopt InceptionV2. InceptionV2 consists of multiple blocks, and the block structure... Figure 11 As shown, the conv block structure in this block is (Cnn+Bn+LeakyReLU) on the left side of the figure. Based on the features of vehicle ID recognition, it relies on semi-supervised learning with a small number of labeled and a large number of unlabeled samples. Leveraging the high performance of the vehicle ID recognition network, it utilizes a large amount of data to add pseudo-labels to aid in the training of the vehicle attribute recognition task.

[0061] Considering that vehicle brands can be divided into: main brand, sub-brand, and detailed model, the similarity between vehicles with the same "detailed model" is greater than the similarity between vehicles with different "detailed models" but the same sub-brand, which is greater than the similarity between vehicles with different "sub-brands" but the same "model year," which is greater than the similarity between vehicles with different "main brands." Therefore, the classification results can be subdivided into different levels. In one optional implementation, the classification results include attributes at a first level, attributes at a second level, and attributes at a third level; the attributes at the first level include the attributes at the second level, and the attributes at the second level include the attributes at the third level; the classification model can be trained using a hierarchical loss function, according to... The following steps are performed: If the classification model produces an attribute classification error between the first levels, a first penalty result is calculated using a first penalty coefficient; if the classification model produces an attribute classification error between the second levels, a second penalty result is calculated using a second penalty coefficient; the first penalty coefficient is greater than the second penalty coefficient; if the classification model produces an attribute classification error between the third levels, a third penalty result is calculated using a third penalty coefficient; the second penalty coefficient is greater than the third penalty coefficient; a hierarchical loss function is calculated using the first penalty result, the second penalty result, and the third penalty result, and the classification model is adjusted using the calculation results.

[0062] In this optional implementation, the classification model classifies vehicle attributes, and the classification result includes first-level attributes, second-level attributes, and third-level attributes. For example, the first-level attributes include attributes A and attributes B, wherein the second-level attributes of attribute A include A1 and A2, the second-level attributes of attribute B include B1 and B2, the third-level attributes of attribute A1 include A11 and A12, and the third-level attributes of A2 include A21 and A22. That is, the first-level attributes include the second-level attributes, and the second-level attributes include the third-level attributes.

[0063] If the classification model classifies attribute A1 as attribute B1, it incurs a classification error at the first level, and a first penalty is calculated using the first penalty coefficient. If the model classifies attribute A2 as attribute A1, it incurs a classification error at the second level, and a second penalty is calculated using the second penalty coefficient. If the model classifies attribute A21 as attribute A22, it incurs a classification error at the third level, and a third penalty is calculated using the third penalty coefficient. It should be noted that the values ​​of the first, second, and third penalty coefficients can be set according to actual needs, provided that the first penalty coefficient is greater than the second penalty coefficient and the second penalty coefficient is greater than the third penalty coefficient. The hierarchical loss function is used to divide attributes into levels, with different levels of penalty for classification errors. A specific example is provided below to illustrate this possible implementation:

[0064] Assumptions: The attributes are divided into {A, B, ..., N}, A can be further refined into {A1, ..., An}, A1 can be further refined into {A11, ..., A1n}, and B can be further refined into {B1, ..., Bn};

[0065] set up The penalty coefficient is γ. (ifPredict) label ≠GT label If Predict label =GT label And i≠m, then γ=2; ifPredict label =GT label If i = m and j ≠ n, then γ = 1. By constraining the hierarchical loss function, the gradual relationship of features between levels can be effectively utilized to better supervise attribute recognition.

[0066] Considering that sometimes coarse-grained recognition is needed and sometimes fine-grained recognition is needed, coarse-grained recognition refers to situations where the differences between categories are large, such as in an "animal category recognition task" where categories are divided into "cat," "dog," and "pig," etc., while fine-grained recognition refers to situations where some categories have small differences while others have large differences, such as in an "animal category recognition task" where categories are divided into "Persian cat," "orange cat," "Labrador," and "Samoyed," etc. For coarse-grained scenarios, see [reference needed]. Figure 7 The diagram shown illustrates a coarse-grained attribute recognition network, according to... Figure 7The training classification model shown is used for attribute recognition, and it runs quickly. To better suit fine-grained scenarios, metric learning can be introduced during the training of the classification model. Therefore, in one possible implementation, the classification model is trained using a hierarchical loss function, which can be performed as follows: generating a third calculation result using the training set and the classification model; generating a fourth calculation result using the third calculation result and the metric learning network; adjusting the classification model using the third calculation result and the hierarchical loss function, and adjusting the metric learning network using the fourth calculation result and the arcface loss function.

[0067] In this possible implementation, the classification model can also adopt the InceptionV2 structure, and the loss function for metric learning can be Arcface loss. Arcface influences the classification result by comparing angle values, and its formula is: Where N represents the number of samples, i represents the i-th sample, j represents the j-th class, yi represents the class of the i-th sample, and θ j This represents the angle between the feature vector of the i-th sample and the feature map of the j-th class in the base database. The base database is the feature map generated during model training. m represents the hyperparameter used to increase the penalty; the larger m is, the larger the loss function value. s() represents the score of the sample in the j-th class. See also Figure 6 The diagram shown illustrates a fine-grained attribute recognition network. During classification model training, a hierarchical loss function and Arcface are used as the loss functions for the classification model and the metric function, respectively. This results in a classification model with higher recognition performance in fine-grained scenarios.

[0068] This application provides a method for training a vehicle attribute recognition model, a vehicle attribute recognition method, and a device. (See also...) Figure 2 The diagram showing the overall network structure illustrates the training method for this vehicle attribute recognition model. By leveraging vehicle ID recognition features and relying on semi-supervised learning with a small number of labeled and a large number of unlabeled samples, it achieves high-precision vehicle attribute recognition. This method enables dimensionality reduction-supervised vehicle attribute feature learning, resulting in a seamless learning experience. Furthermore, this method is highly versatile and applicable to attribute recognition at various granularities for vehicles.

[0069] According to another aspect of the present invention, a vehicle attribute recognition method is also provided, the method comprising: acquiring a target image; inputting the target image into a vehicle attribute recognition model to obtain attribute information of a vehicle in the target image; wherein the vehicle attribute recognition model is obtained by training using any of the above-described vehicle attribute recognition model training methods.

[0070] According to another aspect of the present invention, a vehicle attribute recognition model training apparatus is also provided. Figure 12 This is a structural block diagram of the vehicle attribute recognition model training device provided in an embodiment of the present invention, as shown below. Figure 12 As shown, the vehicle attribute recognition model training device includes: a feature extraction module 1201, a pseudo-label module 1202, and a training module 1203. The vehicle attribute recognition model training device will be described in detail below.

[0071] The feature extraction module 1201 is used to extract the base feature vectors of each image in the vehicle attribute base database and extract the sample feature vectors of the test sample; each image in the vehicle attribute base database includes a license plate area; the pseudo-label module 1202 is used to calculate the vector distance between each base database feature vector and the sample feature vector, and determine the pseudo-label information of the test sample based on the vector distance; the training module 1203 is used to take the target test sample and the pseudo-label information of the target test sample as a set of unlabeled samples, and use multiple sets of unlabeled samples and the vehicle attribute base database as a training set to train a classification model to obtain a vehicle attribute recognition model.

[0072] It should be noted that the feature extraction module 1201, pseudo-label module 1202 and training module 1203 mentioned above correspond to steps S102 to S106 in the method embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment.

[0073] In one optional implementation, the base feature vectors of each image in the vehicle attribute base database are extracted, and the sample feature vectors of the test sample are extracted, including: using a feature extraction network to extract the base feature vectors of each image in the vehicle attribute base database and to extract the sample feature vectors of the test sample.

[0074] In one optional implementation, before using the feature extraction network to extract the base feature vectors of each image in the vehicle attribute base database, the method further includes: training a target network using a first sample set to obtain the feature extraction network; each sample in the first sample set includes a vehicle image; the vehicle image includes a license plate region; the first sample set includes a noise region with random probability; the noise region occludes a specified area of ​​the vehicle image.

[0075] In one optional implementation, the designated area is the license plate area, a non-license plate area, or a non-window area.

[0076] In one optional implementation, training a target network using a first sample set to obtain a feature extraction network includes: generating a first calculation result using the first sample set and the target network; generating a second calculation result using the first calculation result and a metric learning network; adjusting the target network using a cross-entropy loss function and the first calculation result; and adjusting the metric learning network using a circle loss function and the second calculation result.

[0077] In one alternative implementation, the circle loss function is calculated according to the following formula: Among them, L cl This is the value of the circle loss function, where γ is the first weight. Where (q=1,...,L) branch_same These are vehicles with different license plates but the same attributes. Where (u=1,…,L) branch_diff (L) refers to vehicles with different license plates and different attributes. branch_same +Lbranch_diff = L, p is the second weight, β > 1, m is a constant, S n S p This is the formula used to calculate the similarity score. It is the similarity score within K classes. It is the similarity score between L classes.

[0078] In one optional implementation, a classification model is trained using multiple sets of the unlabeled samples and the vehicle attribute database as a training set, including: training the classification model using a hierarchical loss function; the hierarchical loss function penalizes different levels of classification errors in the classification results; the classification results include attributes at multiple levels.

[0079] In one optional implementation, the classification result includes attributes at a first level, attributes at a second level, and attributes at a third level; the attributes at the first level include the attributes at the second level, and the attributes at the second level include the attributes at the third level; training the classification model using a hierarchical loss function includes: if the classification model produces an attribute classification error at the first level, calculating a first penalty result using a first penalty coefficient; if the classification model produces an attribute classification error at the second level, calculating a second penalty result using a second penalty coefficient; the first penalty coefficient is greater than the second penalty coefficient; if the classification model produces an attribute classification error at the third level, calculating a third penalty result using a third penalty coefficient; the second penalty coefficient is greater than the third penalty coefficient; calculating a hierarchical loss function using the first penalty result, the second penalty result, and the third penalty result, and adjusting the classification model using the calculation result.

[0080] In one optional implementation, training the classification model using a hierarchical loss function includes: generating a third calculation result using a training set and the classification model; generating a fourth calculation result using the third calculation result and a metric learning network; adjusting the classification model using the third calculation result and the hierarchical loss function; and adjusting the metric learning network using the fourth calculation result and the arcface loss function.

[0081] In one optional implementation, determining the pseudo-label information of the sample to be tested based on the vector distance includes: if the first vector distance between the target base library feature vector and the sample feature vector is less than the second vector distance, then the target base library feature vector is used as the pseudo-label information of the sample to be tested; the second vector distance is the vector distance between any vector in the base library feature vector other than the target base library feature vector and the sample feature vector.

[0082] According to another aspect of the present invention, a vehicle attribute recognition device is also provided, comprising: an acquisition module for acquiring a target image; and a determination module for inputting the target image into a vehicle attribute recognition model to obtain attribute information of a vehicle in the target image; wherein the vehicle attribute recognition model is obtained after training using the training method described above.

[0083] An exemplary embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this application.

[0084] An exemplary embodiment of this application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this application.

[0085] An exemplary embodiment of this application also provides a computer program product, wherein the computer program product includes a computer program that, when executed by a processor, implements the method of the embodiments of this application.

[0086] refer to Figure 5 The present invention describes a structural block diagram of an electronic device 500 that can serve as a server or client of this application, which is an example of a hardware device that can be applied to various aspects of this application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0087] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0088] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, output unit 507, storage unit 508, and communication unit 509. Input unit 506 can be any type of device capable of inputting information to electronic device 500. Input unit 506 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 507 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 508 may include, but is not limited to, disk and optical disk. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0089] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the above-described vehicle attribute recognition model training method or vehicle attribute recognition method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. In some embodiments, the computing unit 501 can be configured to perform the above-described vehicle attribute recognition model training method or vehicle attribute recognition method by any other suitable means (e.g., by means of firmware).

[0090] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0092] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0094] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0095] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A method for training a vehicle attribute recognition model, comprising: Extract the base feature vector of each image in the vehicle attribute base database, and extract the sample feature vector of the sample to be tested; each image in the vehicle attribute base database includes the license plate area; Calculate the vector distance between each of the base database feature vectors and the sample feature vectors, and determine the pseudo-label information of the sample to be tested based on the vector distance; The target test sample and its pseudo-label information are used as a set of unlabeled samples. Multiple sets of unlabeled samples and the vehicle attribute database are used as training sets to train a classification model and obtain a vehicle attribute recognition model. The process involves extracting the base feature vectors of each image in the vehicle attribute base database and extracting the sample feature vectors of the test samples, including: using a feature extraction network to extract the base feature vectors of each image in the vehicle attribute base database and extracting the sample feature vectors of the test samples. Before extracting the base feature vectors of each image in the vehicle attribute base database using the feature extraction network, the method further includes: training a target network using a first sample set to obtain the feature extraction network; each sample in the first sample set includes a vehicle image; the vehicle image includes a license plate region; the first sample set includes noise regions with random probability; the noise regions occlude a specified area of ​​the vehicle image. Training a target network using a first sample set to obtain a feature extraction network includes: generating a first calculation result using the first sample set and the target network; generating a second calculation result using the first calculation result and a metric learning network; adjusting the target network using a cross-entropy loss function and the first calculation result, and adjusting the metric learning network using a circle loss function and the second calculation result. The circle loss function is calculated according to the following formula: in, It is the value of the circle loss function. It is the first weight. ,in These are vehicles with different license plates but the same attributes. ,in, These are vehicles with different license plates and different attributes. , It is the second weight. , It is a constant. , This is the formula used to calculate the similarity score. yes Intra-class similarity score, yes Inter-class similarity score.

2. The method as described in claim 1, wherein, The designated area is the license plate area, an area other than the license plate area, or an area other than the window area.

3. The method as described in claim 1, wherein, The classification model is trained using multiple sets of unlabeled samples and the vehicle attribute database as a training set, including: The classification model is trained using a hierarchical loss function; the hierarchical loss function penalizes different levels of classification errors in the classification results; the classification results include attributes at multiple levels.

4. The method of claim 3, wherein, The classification results include first-level attributes, second-level attributes, and third-level attributes; the first-level attributes include the second-level attributes, and the second-level attributes include the third-level attributes; Training the classification model using a hierarchical loss function includes: If the classification model produces an attribute classification error in the first level, then the first penalty result is calculated using the first penalty coefficient; If the classification model produces an attribute classification error between the second levels, a second penalty result is calculated using a second penalty coefficient; the first penalty coefficient is greater than the second penalty coefficient. If the classification model produces an attribute classification error in the third level, a third penalty result is calculated using a third penalty coefficient; the second penalty coefficient is greater than the third penalty coefficient. The hierarchical loss function is calculated using the first penalty result, the second penalty result, and the third penalty result, and the classification model is adjusted using the calculation results.

5. The method of claim 3, wherein, Training the classification model using a hierarchical loss function includes: A third calculation result is generated using the training set and the classification model; A fourth calculation result is generated using the third calculation result and the metric learning network; The classification model is adjusted using the third calculation result and the hierarchical loss function, and the metric learning network is adjusted using the fourth calculation result and the arcface loss function.

6. The method according to any one of claims 1-5, wherein, Based on the vector distance, the pseudo-label information of the sample to be tested is determined, including: If the first vector distance between the target base database feature vector and the sample feature vector is less than the second vector distance, then the target base database feature vector is used as the pseudo-label information of the sample to be tested; the second vector distance is the vector distance between any vector in the base database feature vector other than the target base database feature vector and the sample feature vector.

7. A vehicle attribute recognition method, comprising: Obtain the target image; The target image is input into the vehicle attribute recognition model to obtain the attribute information of the vehicle in the target image; The vehicle attribute recognition model is obtained by training the method described in any one of claims 1-6.

8. A vehicle attribute recognition model training device, comprising: The feature extraction module is used to extract the base feature vectors of each image in the vehicle attribute base database and to extract the sample feature vectors of the sample to be tested. Each image in the vehicle attribute database includes the license plate area; The pseudo-label module is used to calculate the vector distance between each of the base database feature vectors and the sample feature vectors, and determine the pseudo-label information of the sample to be tested based on the vector distance. The training module is used to take the target test sample and the pseudo-label information of the target test sample as a set of unlabeled samples, and use multiple sets of unlabeled samples and the vehicle attribute database as a training set to train the classification model to obtain the vehicle attribute recognition model. The process involves extracting the base feature vectors of each image in the vehicle attribute base database and extracting the sample feature vectors of the test samples, including: using a feature extraction network to extract the base feature vectors of each image in the vehicle attribute base database and extracting the sample feature vectors of the test samples. Before extracting the base feature vectors of each image in the vehicle attribute base database using the feature extraction network, the method further includes: training a target network using a first sample set to obtain the feature extraction network; each sample in the first sample set includes a vehicle image; the vehicle image includes a license plate region; the first sample set includes noise regions with random probability; the noise regions occlude a specified area of ​​the vehicle image. Training a target network using a first sample set to obtain a feature extraction network includes: generating a first calculation result using the first sample set and the target network; generating a second calculation result using the first calculation result and a metric learning network; adjusting the target network using a cross-entropy loss function and the first calculation result, and adjusting the metric learning network using a circle loss function and the second calculation result. The circle loss function is calculated according to the following formula: in, It is the value of the circle loss function. It is the first weight. ,in These are vehicles with different license plates but the same attributes. ,in, These are vehicles with different license plates and different attributes. , It is the second weight. , It is a constant. , This is the formula used to calculate the similarity score. yes Intra-class similarity score, yes Inter-class similarity score.

9. A vehicle attribute recognition device, comprising: The acquisition module is used to acquire the target image; The prediction module is used to input the target image into the vehicle attribute recognition model to obtain the attribute information of the vehicle in the target image; The vehicle attribute recognition model is obtained by training the method described in any one of claims 1-6.

10. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-7.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

12. A computer program product, wherein, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images