Vehicle classification method, device and storage medium

Abstract

The present application relates to a vehicle classification method, device and storage medium, belonging to the field of image recognition technology. The method comprises: inputting a target image into a pre-trained basic classification model to obtain image features of the target image, classification results of vehicles in the target image and confidence levels of the classification results, wherein the classification results of each vehicle include m levels of classification information; obtaining a sample vehicle image belonging to the first level classification information when the confidence level is less than a preset threshold to obtain a support set; inputting the support set into a pre-trained small sample classification model to perform a meta-test task to obtain small sample features of each subclass under the first level classification information; determining the final classification result of the vehicle based on the image features and the small sample features of each subclass; the method can solve the problem that the classification model is insufficiently trained due to the small amount of training data, resulting in insufficient robustness of the classification model and easy errors in the vehicle classification results; and the accuracy of vehicle classification can be improved.

Description

Technical field

[0001] The present application relates to a vehicle classification method, device and storage medium, and belongs to the field of image recognition technology. [Background Technology]

[0002] With the development of Intelligent Transport System (ITS) technology, it is now possible to automatically identify vehicle categories from images. However, users may need to identify multiple vehicle categories. For example, when identifying a vehicle brand, it is necessary not only to identify the main brand, but also the sub-brands and model years of the main brand.

[0003] Traditional vehicle classification methods involve combining multiple classification labels for a vehicle image into a single overall label, then using the vehicle image and the overall label to train a neural network model. The trained classification model is then used to simultaneously identify multiple categories of the image, yielding multiple classifications of the vehicle in the image.

[0004] However, a small number of current vehicle models may be used in small quantities, resulting in a lack of training data for training neural network models. This will make the trained classification model less robust, leading to vehicle classification errors that affect subsequent behavior judgments. [Summary of the invention]

[0005] This application provides a vehicle classification method, device, and storage medium that can address the problem of insufficient training data for training classification models, resulting in insufficient robustness of the classification model and prone to errors in vehicle classification results. In this application, by integrating the features of the basic classification model with the features of the small sample classification model, an end-to-end vehicle classification method is designed to improve the overall accuracy of vehicle classification. This application provides the following technical solutions:

[0006] In a first aspect, a vehicle classification method is provided, the method comprising:

[0007] Acquire the target image;

[0008] Inputting the target image into a pre-trained basic classification model to obtain image features of the target image, classification results of vehicles in the target image, and confidence levels of the classification results, wherein the classification results of each vehicle include m-level classification information, where m is a positive integer greater than 1; the basic classification model is trained using a first training set, wherein the first training set includes sample vehicle images and m-level classification labels for each vehicle in the sample vehicle images;

[0009] When the confidence level is less than a preset threshold, obtaining sample vehicle images belonging to the first-level classification information in the classification result from a second training set to obtain a support set; the second training set includes vehicle sample images corresponding to each first-level classification information and a subclass label of each vehicle sample image under the corresponding first-level classification information;

[0010] Inputting the support set into a pre-trained small sample classification model to perform a meta-test task, thereby obtaining small sample features of each subclass under the first-level classification information and an updated small sample classification model; the small sample classification model is trained using the meta-training task constructed using the third training set;

[0011] A final classification result of the vehicle is determined based on the image features and the small sample features of each subclass.

[0012] Optionally, determining a final classification result of the vehicle based on the image features and the small sample features of each subclass includes:

[0013] Fusing the image features with the small sample features of each subclass respectively to obtain fused features of each subclass;

[0014] Inputting the target image into the updated small sample classification model to obtain small sample features of the target image;

[0015] The final classification result of the vehicle is determined based on the similarity between the small sample features of the target image and the fused features of each subclass.

[0016] Optionally, determining the final classification result of the vehicle based on the similarity between the small sample features of the target image and the fused features of each subclass includes:

[0017] When the similarity between the small sample feature of the target image and the fused features of each subclass is greater than a similarity threshold, determining the fused feature having the highest similarity to the small sample feature from the fused features of each subclass;

[0018] The subclass corresponding to the fused feature with the highest similarity and the first level classification information are used as the final classification result.

[0019] Optionally, the third training set includes a plurality of sample vehicle images and at least two classification results for each sample vehicle image;

[0020] The process of training the small sample classification model using the meta-training task constructed by the third training set includes:

[0021] Randomly determine N classification results from the at least two classification results; N is a positive integer;

[0022] For each classification result, K sample vehicle images are extracted from the sample vehicle images corresponding to the classification result as a support set, and P sample vehicle images are extracted from the remaining sample vehicle images corresponding to the classification result as a query set, to obtain the meta-training task; K and P are positive integers;

[0023] Iterative learning is performed on the meta-training task using a pre-created neural network model until the learned neural network model converges to obtain the small sample classification model.

[0024] Optionally, inputting the support set into a pre-trained small sample classification model to perform a meta-test task to obtain small sample features of each subclass under the first-level classification information and an updated small sample classification model includes:

[0025] Using the pre-trained small sample classification model to predict the support set, to obtain a predicted value;

[0026] Comparing the predicted value with the subclass label corresponding to the support set to obtain a prediction loss value;

[0027] Performing parameter updates on the pre-trained small sample classification model based on the predicted loss value to obtain the updated small sample classification model;

[0028] The updated small sample classification model is used to extract small sample features from the support set to obtain small sample features of each subclass under the first level classification information.

[0029] Optionally, the basic classification model includes a feature extraction network, a first branch network, a second branch network and a third branch network respectively connected to the feature extraction network, and a fusion layer connected to the first branch network, the second branch network and the third branch network;

[0030] The feature extraction network is used to extract features from the target image to obtain a feature map;

[0031] The first branch network is used to directly input the feature map output by the feature extraction network into the fusion layer;

[0032] The second branch network is used to extract features from feature maps of different channels and then cascade them to obtain a cascaded feature map;

[0033] The third branch network is used to assign weight information to different channels of the feature map according to the pre-learned channel weights to obtain an updated feature map;

[0034] The fusion layer is used to perform feature fusion on the feature map, the cascaded feature map and the updated feature map to obtain the image feature.

[0035] Optionally, the second branch network includes a global average feature extraction layer, a channel layer and a channel matching layer connected in sequence;

[0036] The global average feature extraction layer is used to extract the global average features of the feature map;

[0037] The channel layer is used to extract different channel features respectively and cascade the extracted features of different channels;

[0038] The channel matching layer is used to adjust the number of channels of the cascaded features so that the adjusted number of channels matches the number of channels of the first branch network and the number of channels of the third branch network.

[0039] Optionally, the third branch network includes a maximum pooling layer, a fully connected layer, an activation function layer and a channel matching layer connected in sequence;

[0040] The maximum pooling layer is used to reduce the size of the feature map;

[0041] The fully connected layer is used to arrange the reduced feature maps according to the number of channels;

[0042] The activation function layer is used to assign weight information to the feature maps of different channels obtained after arrangement;

[0043] The channel matching layer is used to adjust the number of channels of the weight-adjusted features so that the adjusted number of channels matches the number of channels of the first branch network and the number of channels of the second branch network.

[0044] In a second aspect, an electronic device is provided, comprising a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the vehicle classification method provided in the first aspect.

[0045] In a third aspect, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and when the program is executed by a processor, it is used to implement the vehicle classification method provided in the first aspect.

[0046] The beneficial effects of the present application include at least: by inputting the target image into a pre-trained basic classification model, the image features of the target image, the classification results of the vehicles in the target image and the confidence of the classification results are obtained, and the classification results of each vehicle include m levels of classification information; when the confidence is less than a preset threshold, the sample vehicle image belonging to the first level classification information in the classification result is obtained from the second training set to obtain a support set; the support set is input into the pre-trained small sample classification model to perform the meta-test task, and the small sample features of each subclass under the first level classification information and the updated small sample classification model are obtained; based on the image features and the small sample features of each subclass, the final classification result of the vehicle is determined; it can solve the problem that the training data of the classification model is small, resulting in insufficient robustness of the classification model and easy errors in the vehicle classification results; since in the case of inaccurate classification of the basic classification model, the small sample features can continue to be extracted by meta-learning. Since the small sample features extracted by the meta-learning method are more accurate, the final classification result is determined by combining the features of the basic classification model and the features of the small sample classification model, which can improve the accuracy of vehicle classification.

[0047] In addition, by fusing the features of the basic classification model and the features of the small sample classification model to determine the vehicle type classification, the classification results of the two models can be combined to determine the vehicle classification, which can further improve the accuracy of vehicle classification.

[0048] In addition, when the similarity between the small sample features of the target image and the fused features of each subclass is greater than the similarity threshold, the subclass and first-level classification information corresponding to the fused features with the highest similarity are used as the final classification result; this can avoid the problem of outputting incorrect classification results when the similarity between all fused features and the small sample features of the target image is small; it can ensure that the classification results corresponding to the fused features that are greater than the similarity threshold and have the highest similarity are output, thereby improving the accuracy of vehicle classification.

[0049] In addition, by designing a basic classification network consisting of three network branches, it is possible to obtain richer fine-grained features while maintaining the original input information, and at the same time assign weight information to each feature channel, thereby improving the accuracy of the vehicle classification model.

[0050] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement it in accordance with the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0051] Figure 1 This is a flow chart of a vehicle classification method provided by one embodiment of the present application;

[0052] Figure 2 is a schematic diagram of a basic classification model provided by an embodiment of the present application;

[0053] Figure 3 This is a distribution diagram of vehicle data with different classification results provided by an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a vehicle classification process provided by an embodiment of the present application;

[0055] Figure 5 is a block diagram of a vehicle classification device provided by one embodiment of the present application;

[0056] Figure 6 This is a block diagram of an electronic device provided by an embodiment of the present application. [Specific implementation method]

[0057] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0058] First, several terms involved in this application are introduced.

[0059] Meta-learning: Meta-learning leverages previously learned tasks to help learn new tasks. Therefore, meta-training is required to learn prior knowledge from previous tasks (in this case, a pre-trained small-sample classification model), and then leverage this prior knowledge to help learn the new meta-test task.

[0060] N-Way K-Shot Classification: N-Way stands for N-ways or N-classes, while K-Shot stands for K-times or K-individuals. It involves constructing a classification task using a small number of samples. It is primarily used in few-shot learning scenarios where sample data is insufficient, such as in meta-learning.

[0061] Optionally, the present application uses the vehicle classification method provided in each embodiment as an example for explanation in an electronic device, where the electronic device is a terminal or a server. The terminal can be a mobile phone, computer, tablet computer, scanner, electronic eye, surveillance camera, etc. This embodiment does not limit the type of electronic device.

[0062] Figure 1 This is a flow chart of a vehicle classification method provided by an embodiment of the present application, which includes at least the following steps:

[0063] Step 101: Acquire a target image.

[0064] The target image refers to the image to be classified as a vehicle. The target image can be an image obtained by capturing the vehicle driving environment, or a frame of image in a video stream obtained by shooting the vehicle driving scene. This embodiment does not limit the source of the target image.

[0065] Optionally, the target image may include an image of a vehicle, or may not include an image of a vehicle; in the case of including an image of a vehicle, the target image may include images of multiple vehicles, or may include an image of one vehicle.

[0066] In this application, the vehicle may be a motor vehicle, a bicycle, or an electric vehicle, and this embodiment does not limit the type of vehicle.

[0067] Step 102 : Input the target image into a pre-trained basic classification model to obtain image features of the target image, a classification result of the vehicle in the target image, and a confidence level of the classification result.

[0068] The classification result for each vehicle includes m levels of classification information, where m is a positive integer greater than 1. The first-level classification information includes m-1 subclasses, and there is at least one m-1 subclass. In other words, for each first-level classification information, the m-1 levels of classification information under that first-level classification information are all subclasses of that first-level classification information.

[0069] For example, the classification results of a vehicle include three levels of classification information: the first level of classification information is the major brand of the vehicle, the second level of classification information is the minor brand, and the third level of classification information is the model year.

[0070] The classification of major brands includes: A, B, and C;

[0071] The classification of small brands includes: X1 series and X2 series under A, E series and V series under B, A series and B series under C;

[0072] The model year classification includes: 2012 and 2013 models of small brand X1, 2020 model of small brand X2, 2010 E260L and 2010 E300L of small brand E series, 2018 and 2017 models of small brand V series, 2015 and 2016 models of small brand A series, and 2019 200T and 2019 280T of small brand B series.

[0073] The above vehicle classification results are only illustrative. In actual implementation, the vehicle classification results may also include other classifications, such as vehicle color, vehicle model, etc. This embodiment does not limit the vehicle classification method.

[0074] The basic classification model is trained using a first training set, which includes sample vehicle images and m-level classification labels for each vehicle in the sample vehicle images.

[0075] In this embodiment, the number of sample vehicle images corresponding to each m-level classification label in the first training set is greater than a preset number threshold, wherein the preset number threshold can be 50, 60, etc., and this embodiment does not limit the value of the preset number threshold.

[0076] Schematically, the process of obtaining the first training set includes: obtaining multiple sample vehicle images, classifying the multiple sample vehicle images, and obtaining m-level classification labels for each sample vehicle image; then, taking the classification results in which the number of sample vehicle images exceeds a preset number threshold as a large sample category to obtain the first training set.

[0077] The basic classification model is trained using the first training set. In other words, the basic classification model is trained using a large number of sample categories.

[0078] Optionally, the base classification model is an improvement to the traditional classification model. Specifically, the base classification model is designed to improve the feature expression of the large-sample vehicle classification model. It is mainly based on an improved residual module and adds a unique attention module to obtain more fine-grained features.

[0079] refer to Figure 2 The basic classification model includes a feature extraction network 21, a first branch network 22, a second branch network 23 and a third branch network 24 respectively connected to the feature extraction network 21, and a fusion layer 25 connected to the first branch network 22, the second branch network 23 and the third branch network 24.

[0080] The feature extraction network is used to extract features from the target image to obtain a feature map. Optionally, the feature extraction network can be called a backbone network, and the feature extraction network can be composed of multiple residual modules.

[0081] The first branch network is used to directly input the feature map output by the feature extraction network into the fusion layer. The first branch network can also be called the original branch network. The first branch network connects the feature extraction network to the fusion layer, thus maintaining the integrity of the feature map.

[0082] The second branch network is used to extract features from feature maps of different channels and then cascade them to obtain cascaded feature maps. The second branch network can also be called a multi-feature network. Figure 2 ,The second branch network includes a global average feature extraction layer, a channel layer and a channel matching layer connected in sequence.

[0083] The global average feature extraction layer is used to extract the global average features of the feature map. Schematically, the global feature extraction layer includes a 1*1 convolution layer and an average pooling layer connected to the 1*1 convolution layer.

[0084] The channel layer is used to extract different channel features and cascade the extracted features of different channels. Schematically, the channel layer includes multiple convolutional layers connected to the global average feature extraction layer, and the number of convolution kernels in different convolutional layers is different, such as Figure 2 The number of different convolution kernels is represented by ×1, ×2, ×3 and ×4; after extracting the features of the corresponding channels at different convolution layers, the features of each channel are cascaded. Figure 2 The cascaded features of y1, y2, y3, and y4 are obtained. In this way, the second branch network can ultimately obtain richer fine-grained features. This second branch network can effectively improve the representation of fine-grained features such as vehicle logo and body color in vehicle model classification.

[0085] The channel matching layer is used to adjust the number of channels of the cascaded features so that the adjusted number of channels matches the number of channels of the first branch network and the number of channels of the third branch network. Schematically, the channel matching layer is implemented by a 1×1 convolutional layer.

[0086] The third branch network is used to assign weight information to different channels of the feature map according to the pre-learned channel weights to obtain the updated feature map. Figure 2 , the third branch network includes a maximum pooling layer, a fully connected layer, an activation function (sigmoid) layer and a channel matching layer connected in sequence.

[0087] Among them, the maximum pooling layer is used to reduce the size of the feature map, and the fully connected layer is used to arrange the reduced feature map according to the number of channels.

[0088] The activation function layer is used to assign weight information to the feature maps of different channels obtained after permutation. The activation function layer pre-learns the weight expression for each channel.

[0089] The channel matching layer is used to adjust the number of channels of the weighted features so that the adjusted number of channels matches the number of channels of the first branch network and the number of channels of the second branch network. Schematically, the channel matching layer is implemented by a 1×1 convolutional layer.

[0090] The fusion layer is used to fuse the feature map, the cascaded feature map and the updated feature map to obtain image features.

[0091] From the above content, it can be seen that the basic classification model provided by this embodiment can obtain richer fine-grained features while maintaining the original input information, and at the same time assign weight information to each feature channel, thereby improving the accuracy of the vehicle classification model.

[0092] Optionally, since the basic classification model in this step is mainly based on large sample data to fit the real distribution of vehicle classification, the test is better for large sample data, but in fact the distribution of vehicle data is as follows Figure 3 As shown in the figure, these data are obviously distributed with an extremely long tail, which will inevitably lead to a worse classification effect of the tail data, i.e., small sample data.

[0093] Based on this, in this embodiment, the confidence level (or standard deviation) output by the softmax layer of the basic classification model is also used to distinguish between large sample data and small sample data. If the confidence level is less than a preset threshold (e.g., 0.9), it indicates that the classification error is large and the input target image is a small sample, and step 103 is executed. If the confidence level is greater than or equal to the preset threshold (e.g., 0.9), it indicates that the classification error is small and the input target image is a large sample, and the classification result is directly output. In other words, when the confidence level is greater than or equal to the preset threshold, the classification result output by the basic classification model is used as the final classification result for the vehicle.

[0094] In this embodiment, the preset threshold value of 0.9 is used as an example for explanation. In actual implementation, the preset threshold value may also be other values, such as 0.85, etc. This embodiment does not limit the value of the preset threshold value.

[0095] Step 103: When the confidence level is less than a preset threshold, sample vehicle images belonging to the first-level classification information in the classification result are obtained from the second training set to obtain a support set; the second training set includes vehicle sample images corresponding to each first-level classification information, and the subclass label of each vehicle sample image under the corresponding first-level classification information.

[0096] Since the first-level classification information output by the basic classification network is usually more accurate, based on this, in order to reduce the computational complexity and accuracy of the small sample classification model, in this embodiment, the sample vehicle images of each subclass under the first-level classification information output by the basic classification model are used as the support set.

[0097] The types of subclass labels under the first-level classification information in the second training set correspond to the label types used when training the basic classification model.

[0098] Step 104: Input the support set into the pre-trained small sample classification model to perform the meta-test task, and obtain the small sample features of each subclass under the first-level classification information and the updated small sample classification model; the small sample classification model is trained using the meta-training task constructed using the third training set.

[0099] Due to the limited availability of small-sample vehicle model data, this embodiment also designs a small-sample vehicle model classification method. Small-sample classification primarily collects large amounts of multi-type data and uses a K-way-N-shot meta-learning approach. This approach is primarily processed on the data side of the classification model, primarily by randomly selecting K categories and N training data. This composition is used to train small-sample classification. The goal is to adapt the classification model to this small-sample data format. Furthermore, when inputting this composition during verification, more suitable features can be extracted.

[0100] Specifically, the third training set includes multiple sample vehicle images and at least two classification results for each sample vehicle image; the process of training the small sample classification model using the meta-training task constructed using the third training set includes: randomly determining N classification results from at least two classification results; for each classification result, extracting K sample vehicle images from the sample vehicle images corresponding to the classification result as a support set, and extracting P sample vehicle images from the remaining sample vehicle images corresponding to the classification result as a query set to obtain a meta-training task; using a pre-created neural network model to iteratively learn on the meta-training task until the learned neural network model converges to obtain a small sample classification model.

[0101] N, K, and P are positive integers. Typically, the value of K is small, such as 5 or 10 images. In other words, the value of K is much smaller than the number of samples used in large-sample data training.

[0102] The classification results in the third training set may be of the same or different types than those in the second training set. For example, if the classification results in the third training set are of the type major brand, minor brand, model 1, and year, and the classification results in the second training set are of the type major brand, model 1, and year, then the classification results in the third training set are of different types than those in the second training set.

[0103] There are multiple meta-training tasks, each of which is divided into batches and fed into a pre-created neural network model for iterative learning. Each meta-training task includes N classification results, each of which corresponds to a support set and a query set.

[0104] Optionally, the pre-created neural network model may be a small mobilenet model (ie, a lightweight CNN), or may be other lightweight networks. This embodiment does not limit the type of the neural network model.

[0105] Specifically, the support set is input into a pre-trained small sample classification model to perform a meta-test task to obtain the small sample features of each subclass under the first-level classification information and the updated small sample classification model, including: using the pre-trained small sample classification model to predict the support set to obtain a predicted value; comparing the predicted value with the subclass label corresponding to the support set to obtain a prediction loss value; updating the parameters of the pre-trained small sample classification model based on the prediction loss value to obtain an updated small sample classification model; using the updated small sample classification model to extract small sample features from the support set to obtain small sample features of each subclass under the first-level classification information.

[0106] Step 105 : Determine the final classification result of the vehicle based on the image features and the small sample features of each subclass.

[0107] Optionally, the final classification result of the vehicle is determined based on the image features and the small sample features of each subclass, including: fusing the image features with the small sample features of each subclass respectively to obtain the fused features of each subclass; inputting the target image into the updated small sample classification model to obtain the small sample features of the target image; and determining the final classification result of the vehicle based on the similarity between the small sample features of the target image and the fused features of each subclass.

[0108] Among them, the similarity between the small sample features and the fused features of each subclass can be expressed by the Euclidean distance of the features, and the Euclidean distance is negatively correlated with the similarity.

[0109] In one example, a final classification result of a vehicle is determined based on the similarity between the small sample features of a target image and the fused features of each subclass, including: when the similarity between the small sample features of a target image and the fused features of each subclass is greater than a similarity threshold, determining the fused features with the highest similarity to the small sample features from the fused features of each subclass; and taking the subclass and first-level classification information corresponding to the fused features with the highest similarity as the final classification result.

[0110] Optionally, when the similarities between the small sample features of the target image and the fused features of each subclass are less than or equal to a similarity threshold, first-level classification information is output, or a classification failure prompt is output.

[0111] refer to Figure 4 An example of the vehicle classification process shown in the figure is that after the target image is input into the basic classification model, the classification result and the confidence of the classification result are obtained; if the confidence is greater than 0.9, the classification result is output; if the confidence is less than or equal to 0.9, the first-level classification information in the classification result is obtained ( Figure 3The sample vehicle images corresponding to each subclass in (taking the car logo as an example) are used to obtain the support set; the image features obtained from the support set and the basic classification model are input into the pre-trained small sample classification model for feature fusion to obtain the final classification result.

[0112] In summary, the vehicle classification method provided in this embodiment obtains image features of the target image, classification results of vehicles in the target image, and confidence of the classification results by inputting the target image into a pre-trained basic classification model. The classification results of each vehicle include m-level classification information. When the confidence is less than a preset threshold, sample vehicle images belonging to the first-level classification information in the classification result are obtained from the second training set to obtain a support set. The support set is input into the pre-trained small sample classification model to perform a meta-test task to obtain small sample features of each subclass under the first-level classification information and an updated small sample classification model. The final classification result of the vehicle is determined based on the image features and the small sample features of each subclass. This can solve the problem that the classification model training data is insufficient, resulting in insufficient robustness of the classification model and easy errors in the vehicle classification results. Since the small sample features extracted by the meta-learning method are more accurate when the basic classification model is inaccurate, the final classification result can be determined by combining the features of the basic classification model and the features of the small sample classification model, which can improve the accuracy of vehicle classification.

[0113] In addition, by fusing the features of the basic classification model and the features of the small sample classification model to determine the vehicle type classification, the classification results of the two models can be combined to determine the vehicle classification, which can further improve the accuracy of vehicle classification.

[0114] In addition, when the similarity between the small sample features of the target image and the fused features of each subclass is greater than the similarity threshold, the subclass and first-level classification information corresponding to the fused features with the highest similarity are used as the final classification result; this can avoid the problem of outputting incorrect classification results when the similarity between all fused features and the small sample features of the target image is small; it can ensure that the classification results corresponding to the fused features that are greater than the similarity threshold and have the highest similarity are output, thereby improving the accuracy of vehicle classification.

[0115] In addition, by designing a basic classification network consisting of three network branches, it is possible to obtain richer fine-grained features while maintaining the original input information, and at the same time assign weight information to each feature channel, thereby improving the accuracy of the vehicle classification model.

[0116] Figure 51 is a block diagram of a vehicle classification device provided by an embodiment of the present application. The device includes at least the following modules: an image acquisition module 510, a first classification module 520, a data acquisition module 530, a meta-test module 540, and a second classification model 550.

[0117] An image acquisition module 510 is used to acquire a target image;

[0118] a first classification module 520 configured to input the target image into a pre-trained basic classification model to obtain image features of the target image, a classification result of the vehicle in the target image, and a confidence level of the classification result, wherein the classification result of each vehicle includes m-level classification information, where m is a positive integer greater than 1; the basic classification model is trained using a first training set, wherein the first training set includes sample vehicle images and m-level classification labels for each vehicle in the sample vehicle images;

[0119] A data acquisition module 530 is configured to acquire, from a second training set, sample vehicle images belonging to the first-level classification information in the classification result, to obtain a support set when the confidence level is less than a preset threshold; the second training set includes vehicle sample images corresponding to each first-level classification information and a subclass label for each vehicle sample image under the corresponding first-level classification information;

[0120] A meta-testing module 540 is configured to input the support set into a pre-trained small sample classification model to perform a meta-testing task, thereby obtaining small sample features of each subclass under the first-level classification information and an updated small sample classification model; the small sample classification model is trained using the meta-training task constructed using the third training set;

[0121] The second classification model 550 is used to determine the final classification result of the vehicle based on the image features and the small sample features of each subclass.

[0122] For relevant details, please refer to the above method embodiment.

[0123] It should be noted that the vehicle classification device provided in the above embodiment only uses the aforementioned functional modules as an example for vehicle classification. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, that is, the internal structure of the vehicle classification device can be divided into different functional modules to perform all or part of the functions described above. In addition, the vehicle classification device provided in the above embodiment and the vehicle classification method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0124] Figure 6 6 is a block diagram of an electronic device provided in one embodiment of the present application. The device includes at least a processor 601 and a memory 602.

[0125] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0126] Memory 602 may include one or more computer-readable storage media, which may be non-transitory. Memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 602 is used to store at least one instruction, which is executed by processor 601 to implement the vehicle classification method provided in the method embodiment of the present application.

[0127] In some embodiments, the electronic device may optionally include a peripheral device interface and at least one peripheral device. The processor 601, memory 602, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Illustratively, the peripheral devices include, but are not limited to, a radio frequency circuit, a touchscreen display, an audio circuit, and a power supply.

[0128] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0129] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored. The program is loaded and executed by a processor to implement the vehicle classification method of the above method embodiment.

[0130] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, in which a program is stored. The program is loaded and executed by a processor to implement the vehicle classification method of the above method embodiment.

[0131] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A vehicle classification method, characterized in that: The method comprises: Acquire the target image; Inputting the target image into a pre-trained basic classification model to obtain image features of the target image, classification results of vehicles in the target image, and confidence levels of the classification results, wherein the classification results of each vehicle include m-level classification information, where m is a positive integer greater than 1; the basic classification model is trained using a first training set, wherein the first training set includes sample vehicle images and m-level classification labels for each vehicle in the sample vehicle images; When the confidence level is less than a preset threshold, obtaining sample vehicle images belonging to the first-level classification information in the classification result from a second training set to obtain a support set; the second training set includes vehicle sample images corresponding to each first-level classification information and a subclass label of each vehicle sample image under the corresponding first-level classification information; Inputting the support set into a pre-trained small-sample classification model to perform a meta-test task, thereby obtaining small-sample features of each subclass under the first-level classification information and an updated small-sample classification model; the small-sample classification model is trained using a meta-training task constructed using a third training set; the third training set includes a plurality of sample vehicle images and at least two classification results for each sample vehicle image; A final classification result of the vehicle is determined based on the image features and the small sample features of each subclass.

2. The method according to claim 1, characterized in that Determining a final classification result of the vehicle based on the image features and the small sample features of each subclass includes: Fusing the image features with the small sample features of each subclass respectively to obtain fused features of each subclass; Inputting the target image into the updated small sample classification model to obtain small sample features of the target image; The final classification result of the vehicle is determined based on the similarity between the small sample features of the target image and the fused features of each subclass.

3. The method according to claim 2, characterized in that Determining a final classification result of the vehicle based on the similarity between the small sample features of the target image and the fused features of each subclass includes: When the similarity between the small sample feature of the target image and the fused features of each subclass is greater than a similarity threshold, determining the fused feature having the highest similarity to the small sample feature from the fused features of each subclass; The subclass corresponding to the fused feature with the highest similarity and the first level classification information are used as the final classification result.

4. The method according to claim 1, wherein The process of training the small sample classification model using the meta-training task constructed by the third training set includes: Randomly determine N classification results from the at least two classification results; N is a positive integer; For each classification result, K sample vehicle images are extracted from the sample vehicle images corresponding to the classification result as a support set, and P sample vehicle images are extracted from the remaining sample vehicle images corresponding to the classification result as a query set, to obtain the meta-training task; K and P are positive integers; Iterative learning is performed on the meta-training task using a pre-created neural network model until the learned neural network model converges to obtain the small sample classification model.

5. The method according to claim 1, wherein The step of inputting the support set into a pre-trained small sample classification model to perform a meta-test task, thereby obtaining small sample features of each subclass under the first-level classification information and an updated small sample classification model, includes: Using the pre-trained small sample classification model to predict the support set, to obtain a predicted value; Comparing the predicted value with the subclass label corresponding to the support set to obtain a prediction loss value; Performing parameter updates on the pre-trained small sample classification model based on the predicted loss value to obtain the updated small sample classification model; The updated small sample classification model is used to extract small sample features from the support set to obtain small sample features of each subclass under the first level classification information.

6. The method according to claim 1, characterized in that The basic classification model includes a feature extraction network, a first branch network, a second branch network and a third branch network respectively connected to the feature extraction network, and a fusion layer connected to the first branch network, the second branch network and the third branch network; The feature extraction network is used to extract features from the target image to obtain a feature map; The first branch network is used to directly input the feature map output by the feature extraction network into the fusion layer; The second branch network is used to extract features from feature maps of different channels and then cascade them to obtain a cascaded feature map; The third branch network is used to assign weight information to different channels of the feature map according to the pre-learned channel weights to obtain an updated feature map; The fusion layer is used to perform feature fusion on the feature map, the cascaded feature map and the updated feature map to obtain the image feature.

7. The method according to claim 6, characterized in that The second branch network includes a global average feature extraction layer, a channel layer and a channel matching layer connected in sequence; The global average feature extraction layer is used to extract the global average features of the feature map; The channel layer is used to extract different channel features respectively and cascade the extracted features of different channels; The channel matching layer is used to adjust the number of channels of the cascaded features so that the adjusted number of channels matches the number of channels of the first branch network and the number of channels of the third branch network.

8. The method according to claim 6, characterized in that The third branch network includes a maximum pooling layer, a fully connected layer, an activation function layer and a channel matching layer connected in sequence; The maximum pooling layer is used to reduce the size of the feature map; The fully connected layer is used to arrange the reduced feature maps according to the number of channels; The activation function layer is used to assign weight information to the feature maps of different channels obtained after arrangement; The channel matching layer is used to adjust the number of channels of the weight-adjusted features so that the adjusted number of channels matches the number of channels of the first branch network and the number of channels of the second branch network.

9. An electronic device, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the vehicle classification method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The storage medium stores a program, and when the program is executed by the processor, it is used to implement the vehicle classification method according to any one of claims 1 to 8.

Images