Vehicle identification method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a vehicle identification method, device, equipment and storage medium, the method comprising: acquiring an image containing a target vehicle; extracting features from the image through a first model to obtain a first image feature; extracting features from the image through a second model to obtain a second image feature, the vehicle category that can be identified by the first model is the first category, the vehicle included in the training sample image of the second model belongs to the second category, and the first category matches the second category. The first image feature and the second image feature are fused to obtain a third image feature, and the category of the target object is determined according to the third image feature to ensure that the category of the target vehicle in the image is accurately identified.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a vehicle recognition method, device, equipment and storage medium. Background Art

[0002] Image recognition is required in many application scenarios, such as identifying the category of objects contained in an image. For example, identifying what type of vehicle a motor vehicle in an image is; identifying what type of animal an image contains, and so on.

[0003] At present, in order to complete the above vehicle recognition tasks, the general approach is to collect training sample sets, complete the training of vehicle recognition models based on the training sample sets, and use the trained vehicle recognition models to perform vehicle recognition processing on the images to be recognized. Taking the motor vehicle classification scenario as an example, during the vehicle recognition model training phase, pictures of various motor vehicles are collected in advance, such as cars, muck trucks, buses, mixer trucks, buses, etc., and the category labels are annotated for each collected picture.

[0004] However, in reality, there may be a situation where the number of images collected for certain types of motor vehicles is much smaller than the number of images collected for other types of motor vehicles. This will result in poor recognition ability of the trained vehicle recognition model for motor vehicle categories with a small number of samples, ultimately leading to poor vehicle recognition accuracy. Summary of the invention

[0005] Embodiments of the present invention provide a vehicle identification method, apparatus, device and storage medium for improving the accuracy of vehicle type identification results.

[0006] In a first aspect, an embodiment of the present invention provides a vehicle identification method, the method comprising:

[0007] Acquire an image containing a target vehicle;

[0008] Extracting features from the image using a first model to obtain first image features; wherein the vehicle categories that can be identified by the first model include the first category;

[0009] Extracting features from the image using a second model to obtain second image features; wherein the vehicles included in the training sample image of the second model belong to a second category, and the first category matches the second category;

[0010] fusing the first image feature and the second image feature to obtain a third image feature;

[0011] The category of the target vehicle is determined according to the third image feature.

[0012] In a second aspect, an embodiment of the present invention provides a vehicle identification device, the device comprising:

[0013] An acquisition module, used for acquiring an image containing a target vehicle;

[0014] An extraction module, configured to extract features from the image using a first model to obtain first image features; and extract features from the image using a second model to obtain second image features; wherein the vehicle categories that can be identified by the first model include a first category, the vehicles included in the training sample images of the second model belong to a second category, and the first category matches the second category;

[0015] The recognition module is used to fuse the first image feature and the second image feature to obtain a third image feature; and determine the category of the target vehicle according to the third image feature.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the vehicle identification method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the vehicle identification method described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present invention provides a vehicle identification method, the method comprising:

[0019] In response to a request from a user device to call a vehicle identification service interface, the following steps are performed using processing resources corresponding to the vehicle identification service interface:

[0020] Acquire an image containing a target vehicle;

[0021] Extracting features from the image using a first model to obtain first image features; wherein the vehicle categories that can be identified by the first model include the first category;

[0022] Extracting features from the image using a second model to obtain second image features; wherein the vehicles included in the training sample image of the second model belong to a second category, and the first category matches the second category;

[0023] fusing the first image feature and the second image feature to obtain a third image feature;

[0024] The category of the target vehicle is determined according to the third image feature.

[0025] In the solution provided by the embodiment of the present invention, when it is necessary to identify the category of a target vehicle contained in an image, first, the first image feature is extracted by the first model, and the second image feature is extracted by the second model. Then, the first image feature and the second image feature are fused to obtain the third image feature, and finally the category of the target vehicle is determined according to the third image feature. The second model is used differently from the first model, that is, the tasks performed by the two models are different, and the two models are models with different parameters. Assuming that the multiple vehicle categories that can be identified by the first model are called the first category, the relationship between the second model and the first model is: the first category matches the second category corresponding to the vehicle contained in the training sample image for training the second model, that is, assuming that the vehicle contained in the training sample image of the second model belongs to the second category, then the second category matches the first category that can be identified by the first model, for example, the first category contains the second category.

[0026] Based on the solution provided by the embodiment of the present invention, even if the number of training sample images corresponding to certain vehicle categories is insufficient when training the first model, the accuracy of the vehicle category recognition result of this category can be guaranteed. Because the feature extraction result of the second model on the input image is fused on the basis of the image features extracted by the first model, the fused image features have rich semantic information, which can ensure that the category of the target vehicle in the image is accurately recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 A flow chart of a vehicle identification method provided by an embodiment of the present invention;

[0029] Figure 2 It is a schematic diagram of data statistics in a motor vehicle data set;

[0030] Figure 3 This is a schematic diagram of data statistics in the muck truck data set;

[0031] Figure 4 A schematic diagram of an application of a vehicle identification method provided by an embodiment of the present invention;

[0032] Figure 5A schematic diagram of the execution process of a vehicle identification method provided by an embodiment of the present invention;

[0033] Figure 6 A flowchart of a semi-supervised learning method provided by an embodiment of the present invention;

[0034] Figure 7 A schematic diagram of an application of a vehicle identification method provided by an embodiment of the present invention;

[0035] Figure 8 A schematic diagram of the structure of a vehicle identification device provided by an embodiment of the present invention;

[0036] Fig. 9 For Figure 8 A schematic diagram of the structure of an electronic device corresponding to the vehicle identification device provided in the illustrated embodiment. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0039] The vehicle identification method provided in the embodiment of the present invention can be executed by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, etc., or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server or a server cluster.

[0040] Figure 1 A flow chart of a vehicle identification method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the following steps may be included:

[0041] 101. Acquire an image containing a target vehicle.

[0042] 102. Perform feature extraction on the image through a first model to obtain first image features, and perform feature extraction on the image through a second model to obtain second image features. The vehicle category that can be identified by the first model is the first category, and the vehicles included in the training sample image of the second model belong to the second category. The first category matches the second category.

[0043] 103. The first image feature and the second image feature are integrated to obtain a third image feature.

[0044] 104. Determine the category of the target vehicle according to the third image feature.

[0045] In this embodiment, the image containing the target vehicle is the image to be identified, and the purpose of vehicle identification is to identify the category corresponding to the target vehicle in the image.

[0046] The vehicle recognition task can be classified as a single classification task problem, that is, a classification task, but multiple categories can be set for the classification task, the purpose of which is to identify which category the vehicle contained in the input image belongs to.

[0047] For example, suppose a single classification task is to identify the category of motor vehicles contained in an image, that is, multiple categories of motor vehicles are pre-set, such as mixer trucks, buses, cars, muck trucks, forklifts, coaches, etc. For an image containing a certain motor vehicle, it is necessary to identify which category the motor vehicle contained in the image belongs to.

[0048] In order to complete the vehicle recognition task, a large number of training sample images are collected in advance, and supervised training of the first model is performed based on the category labels marked on each training sample image to obtain the first model. In practical applications, the first model can be a neural network model, such as a convolutional neural network model, but of course, it is not limited to this.

[0049] In the embodiment of the present invention, the multiple categories that the first model can identify in the process of completing the single classification task of vehicle category identification, that is, the multiple category labels marked in the process of training the first model, are called the first category. Based on the example of the above motor vehicle category identification scenario, the first category includes mixer trucks, buses, cars, muck trucks, forklifts, coaches and other categories.

[0050] In the traditional scheme, after the first model is trained and converged, the first model can be used to classify and identify related images. For example, an image containing a certain motor vehicle is obtained. After the image is input into the first model, the first model can finally output the category corresponding to the vehicle contained in the image, such as a sedan.

[0051] However, it is understandable that, in fact, in the process of collecting the training sample set corresponding to the first model, the number of training sample images corresponding to various motor vehicle categories that can be obtained may be unbalanced. Since the training sample images that can be collected when performing the motor vehicle category recognition task are affected by various actual conditions at the time, such as Figure 2As shown in , by collecting, organizing, and merging vehicle images of various categories, the number of training sample images corresponding to the various categories shown in the figure is obtained (different numbers are represented by different heights of the bar graph), and it can be found that there is an imbalance in the number of training samples between categories. The recognition accuracy of the first model trained based on such a training sample image set cannot be well guaranteed, especially for categories with a small number of training sample images, the recognition accuracy of the first model for this category may not be good.

[0052] In order to ensure the accuracy of the category recognition result of the target vehicle in the image, a multi-source data feature fusion mechanism is introduced in the embodiment of the present invention.

[0053] In the multi-source data feature fusion mechanism, another model is introduced: the second model. The second model has a different purpose from the first model, that is, the first model and the second model are models for completing different tasks, and the model parameters are different. However, the second model is not a model that has nothing to do with the first model. In an embodiment of the present invention, there is a correlation between the training sample set corresponding to the second model and the training sample set corresponding to the first model. The correlation is mainly reflected in: the multiple vehicle categories that can be recognized by the first model are collectively referred to as the first category, that is, the multiple vehicle category labels contained in the training sample set corresponding to the first model are called the first category, and assuming that the vehicle contained in the training sample image of the second model belongs to the second category, then the first category and the second category are matched. In other words, the first category that can be recognized by the first model matches the second category to which the vehicle contained in the training sample image of the second model belongs.

[0054] The match may be that: the first category includes the second category, or the first category and the second category are different subcategories under the same parent category.

[0055] For ease of understanding, for example, assume that the first model is a model for identifying motor vehicle categories, and the second model can be a model for classifying and identifying structured attributes of muck trucks. Among them, the classification and identification of structured attributes of muck trucks is actually a multi-classification task problem. In simple terms, it is to set multiple attribute dimensions (i.e., multiple classification tasks) for muck trucks, and classify and identify multiple category labels in each attribute dimension.

[0056] For ease of understanding, combined Figure 3 To illustrate this. Figure 3In the above, it is assumed that four classification tasks are set: the classification tasks corresponding to the vehicle body perspective, vehicle body color, license plate occlusion, and roof light. For the classification task of vehicle body perspective, four category labels are set: front, body, rear, and others, which are used to identify which category the shooting perspective of the dump truck in the input image corresponds to. For the classification task of vehicle body color, the colors of dump trucks specified in different regions are counted, and multiple color categories are set as shown in the figure to identify the color category of the dump truck in the input image. For the classification task of license plate occlusion, four category labels are set: yes, no, cannot be judged, and others, which are used to identify whether the license plate of the dump truck in the input image is blocked. For the classification task of roof light, four category labels are set: yes, no, cannot be judged, and others, which are used to identify whether the dump truck has a roof light in the input image.

[0057] From the above examples, it can be seen that no matter which classification task the second model is used to process the muck truck images, the training sample set of the second model contains images of muck trucks. The multiple categories corresponding to the first training sample set corresponding to the first model happen to include muck trucks. It can be seen that from the perspective of vehicle type, the second category of muck trucks is included in the first category that can be recognized by the first model. This is a situation where the first category matches the second category. This situation corresponds to the fact that the category recognition ability of the first model can cover the training sample images corresponding to the second model, that is, the first model can be used to identify the category of the object contained in the training sample images of the second model. In other words, there is a situation where the data categories of the two models overlap.

[0058] In addition, the situation where the first category matches the second category may also include: the first category and the second category belong to different subcategories under the same parent category. For example, suppose the first category includes: cars, buses, forklifts, etc., but does not include muck trucks, and the second model is used to perform multi-classification task recognition processing on muck truck images, that is, the training sample images corresponding to the second model all correspond to the vehicle category of muck trucks (the second category). Although the second category is not included in the first category due to incomplete sample collection, in fact, the first category and the second category both correspond to various vehicle categories included in motor vehicles and belong to the same parent category of motor vehicles.

[0059] From the above examples, we can see that, in fact, there are some common image features between the training sample images corresponding to the first model and the training sample images corresponding to the second model. For example, various types of motor vehicles have common features such as multiple wheels and steering wheels. Based on this condition, the second model can be used to assist in completing the vehicle recognition task that the first model should have completed.

[0060] Specifically, for an image containing a target object, on the one hand, the image can be input into the first model, and features of the image can be extracted by the first model to obtain first image features; on the other hand, the image can be input into the second model, and features of the image can be extracted by the second model to obtain second image features. After that, the first image features and the second image features are fused to obtain the third image features, and finally the category of the target object is determined according to the third image features.

[0061] In practice, a third model may be provided, and the third image feature may be input into the third model so as to determine the category of the target object through the third model. In practical applications, the scale of the third model is smaller than that of the first model and the second model.

[0062] Taking the motor vehicle category recognition scenario as an example, combined with Figure 4 The vehicle identification process in this scenario is illustrated in the following example. In practical applications, for example, if an organization wants to count the traffic conditions of various types of motor vehicles on a certain road within a certain period of time, the organization can collect the road video corresponding to the corresponding road section within the corresponding period of time, in which the vehicle traffic conditions are recorded, and then divide the video into several image frames, and perform motor vehicle type identification processing on each frame. Figure 4 As shown in , for any frame image, this image is input into the first model, which extracts the first image feature, and this image is input into the second model, which extracts the second image feature. The first image feature and the second image feature are fused into the third image feature, and the third image feature is input into the third model. The third model outputs the category corresponding to the motor vehicle in this image based on the third image feature, as shown in Figure 4 As shown in , it is assumed to be a sedan.

[0063] In practical applications, the first image features and the second image features extracted by the first model and the second model may include multiple image features such as edge contours, textures, corner points, and colors. The fused third image features can be considered to include the first image features and the second image features. It is understandable that, although the first image features and the second image features may contain some common types of features in terms of feature types, the first model and the second model originally perform different tasks. Therefore, even for the same type of features, the extraction results of the two models may be different, and fusion together will have a reinforcing effect.

[0064] The above embodiment introduces the principle of the vehicle identification scheme. Figure 5 To illustrate an optional specific implementation process of the vehicle identification solution.

[0065] like Figure 5As shown in , the first model and the second model are both assumed to be convolutional neural network models, denoted as CNN_A and CNN_B, respectively. However, since the two models were originally used to perform different tasks, the model composition structure (such as the number of hidden layers) and related model parameters are different. For the image X containing the target vehicle, after CNN_A performs feature extraction, it is assumed that the extracted first image feature is 1x1024-dimensional, and the first image feature can be further dimensionally transformed (reshaped) to be transformed into an image feature of 1x32x32 dimensions, recorded as image feature f1. At the same time, the image X is input into CNN_B, and after CNN_B performs feature extraction, it is assumed that the extracted second image feature is 1x1024-dimensional, and the second image feature can be further dimensionally transformed (reshaped) to be transformed into an image feature of 1x32x32 dimensions, recorded as image feature f2.

[0066] In addition, image X is a color image with three channels of RGB, such as Figure 5 As shown in , assuming that the original resolution of image X is wxh, optionally, image X can be converted into a grayscale image (1xwxh) first, and then the grayscale image is compressed to obtain a grayscale image of a set scale, which matches the scale corresponding to the above-mentioned image features f1 and f2, and is also a 1x32x32-dimensional image feature, recorded as image feature f3. Afterwards, image features f1, f2, and f3 are fused (concatenated) together to obtain a third image feature of 3x32x32 dimensions, and the third image feature is input into a third model, recorded as CNN_C, and the category of the target object in image X is determined based on the output result of CNN_C. Since image features f1 and f2 are already features extracted by CNN_A and CNN_B, CNN_C can be set as a lightweight model, and finally complete the single classification task processing, and output the corresponding category confidence and category label.

[0067] exist Figure 5 In the illustrated embodiment, in addition to fusing the image features extracted by the first model and the second model, the image features of the original image (ie, image X) are also fused, making the features richer and making the fused features more expressive.

[0068] In summary, compared with the original recognition scheme that only uses one model to predict the target vehicle category, the scheme provided by the embodiment of the present invention obtains more recognizable image features through feature extraction and feature fusion of three models, making the final category recognition result more accurate.

[0069] In fact, if Figure 2 and Figure 3As shown in , the number of training sample images corresponding to different category labels in the training sample set of a certain model often shows an imbalance. Figure 2 The number of vehicle images of different categories is unbalanced. Figure 3 The number of vehicle images corresponding to each category label in each classification task also shows an imbalance. In view of this phenomenon, the embodiment of the present invention also proposes a semi-supervised learning method to alleviate the problem of category imbalance in the training sample set, that is, the problem of the imbalance in the number of training sample images corresponding to different category labels in the training sample set. Figure 6 To illustrate the execution process of this semi-supervised learning method.

[0070] Figure 6 A flowchart of a semi-supervised learning method provided by an embodiment of the present invention is as follows: Figure 6 As shown, the following steps may be included:

[0071] 601. Determine a training sample image corresponding to a second category from a first training sample set corresponding to a first model, where the second category is a category to which the training sample image corresponding to the second model belongs among a plurality of vehicle categories that can be identified by the first model.

[0072] 602. Input the training sample image into a second model to obtain an output result of the second model.

[0073] 603. If the output result meets the set condition, determine the label corresponding to the training sample image in the second training sample set corresponding to the second model according to the output result, and add the training sample image to the second training sample set.

[0074] The multiple vehicle categories that can be identified by the first model are collectively referred to as the first category, the training sample set corresponding to the first model is referred to as the first training sample set, and the training sample set corresponding to the second model is referred to as the second training sample set. A prerequisite assumed in this embodiment is that the category to which the training sample images in the second training sample set belong under the category recognition task corresponding to the first model (referred to as the second category) is included in the first category, that is, included in the multiple vehicle categories that can be identified by the first model.

[0075] For example, assuming that the first training sample set corresponds to multiple vehicle categories including Figure 1 The training sample images included in the second training sample set are all images of muck trucks. The muck truck vehicle category is included in Figure 1 Among the multiple vehicle categories shown in .

[0076] In order to train the second model to be able to complete multiple classification tasks, for each classification task, each training sample image contained in the second training sample set needs to be annotated with a relevant category label so as to complete the training of the second model under the supervision of the category label.

[0077] However, if Figure 3 As shown in , for a certain classification task, the number of training sample images corresponding to the multiple category labels is different, and some show great differences. This significant imbalance in the number of training sample images corresponding to different category labels will have an adverse impact on the performance of the second model.

[0078] The second training sample set can be expanded to solve this problem. Specifically, if the category labels that originally correspond to a small number of samples can be expanded with more training sample images, the imbalance in the number of samples between different category labels can be reduced.

[0079] In summary, the idea of ​​the solution provided in this embodiment is: training sample images corresponding to the above-mentioned second category are screened out from the first training sample set corresponding to the first model, and these screened training sample images can obtain corresponding prediction output results after prediction by the second model. The training sample images corresponding to the higher confidence levels in these results can be selected, and their true value labels can be determined. Finally, the training sample images originally belonging to the first training sample set with the annotated labels automatically determined will be expanded to the second training sample set for use in optimizing the training of the second model. This method is a semi-supervised learning method.

[0080] Specifically, since a corresponding category label (which is the category label corresponding to the first model) is associated with each training sample image in the process of forming the first training sample set, training sample images specifically belonging to the second category can be screened out from the first training sample set based on the marked category label.

[0081] Since for the second model, the training sample images selected from the first training sample set are still samples without labeled information, that is, these training sample images do not yet have the supervisory information (that is, labeled labels) required for training the second model, it is necessary to automatically complete the labeling process for these training sample images so as to add the training sample images with labeled labels corresponding to the second model to the second training sample set.

[0082] For ease of description, assume that N training sample images corresponding to the second category are screened out from the first training sample set, N is greater than 1, and any of the training sample images is called training sample image i. Taking training sample image i as an example, the process of determining the annotation label corresponding to training sample image i in the second training sample set is introduced:

[0083] First, the training sample image i is input into the second model, and the second model extracts features from the training sample image i, and obtains the final output result based on the extracted features. It can be understood that the output result of the second model is related to the task that the second model is used to perform. For example, assuming that the second model is used to complete Figure 3 The classification tasks corresponding to the various attributes of the muck truck shown in the figure, then the output result of the second model is the classification recognition result for each classification task. The classification recognition result of any classification task can be finally expressed as: the category label of the confidence target.

[0084] Secondly, based on the output result corresponding to the training sample image i output by the second model, if the output result meets the set conditions, the annotation label corresponding to the training sample image i in the second training sample set can be determined based on the output result, and the training sample image i associated with the annotation label can be added to the second training sample set.

[0085] In summary, the output result includes the confidence level corresponding to the vehicle contained in the training sample image i under the set label. Therefore, if the confidence level meets the set conditions, it can be determined that the annotation label corresponding to the training sample image i in the second training sample set is the set label.

[0086] For example, assuming that the second model also performs a single classification task, such as the second model is used to identify the color of a muck truck, then the output result of the second model for the training sample image i is actually: the confidence corresponding to the muck truck in the training sample image i under multiple preset color category labels, based on which, the color category label with the target confidence can be determined as the final recognition result of the color of the muck truck in the training sample image i. At this time, for example, the above-mentioned setting condition is: the target confidence is greater than the set threshold, then assuming that the color of the muck truck in the training sample image i is finally identified as the confidence corresponding to the red category label is the target, and the confidence corresponding to red is greater than the set threshold, then it can be determined that the annotation label corresponding to the attribute of the muck truck color of the training sample image i should be the red category label. In practical applications, the target confidence can be the confidence ranked first.

[0087] The above example is a case where the second model is used to perform a single classification task. In fact, as mentioned above, the second model can also be a model used to complete multiple classification tasks. In this case, the output result of the second model for the training sample image i will be the category labels and confidences corresponding to the vehicles contained in the training sample image i under these multiple classification tasks.

[0088] In the case where the second model is a model for completing multiple classification tasks, in an optional embodiment, determining the annotation label corresponding to the training sample image i in the second training sample set according to the output result of the second model can be implemented as follows:

[0089] For a target classification task among multiple classification tasks, if the confidence level of the vehicle included in the training sample image i corresponding to the target category label under the target classification task meets the set condition, it is determined that the annotation label corresponding to the training sample image i in the second training sample set includes the target category label, wherein the target category label is one of the multiple category labels preset under the target classification task. The target classification task is any one of the multiple classification tasks.

[0090] For example, suppose the second model is used to complete Figure 3 The model of multiple classification tasks corresponding to multiple attributes of a dump truck is shown in the figure. Assume that the current target classification task is: to identify whether the dump truck has a roof light. Assume that the output result of the second model for the training sample image i under this task branch is: the confidence that the dump truck in the training sample image i has a roof light is 0.95. Then, if the above-mentioned setting condition is a preset confidence threshold, and 0.95 is greater than the threshold, it can be determined that the annotation label corresponding to the training sample image i may include: having a roof light. Similarly, for other classification tasks, the annotation labels corresponding to the training sample image i under other classification tasks can be determined.

[0091] It should be noted that if under a certain classification task, the confidence levels of the category labels corresponding to the training sample image i do not meet the set conditions (for example, they are greater than the set threshold), then it can be considered that the training sample image i is not distinguishable under the classification task, and the training sample image i will not be associated with the annotation label under the classification task.

[0092] The above examples of setting conditions are all greater than the set threshold. In fact, the setting conditions can also be implemented in other ways. For example, as mentioned above, assuming that N training sample images corresponding to the second category are screened out from the first training sample set, for a certain target classification task, the second model can output the category labels corresponding to the target confidence of each of the N training sample images under the target classification task, which are called target category labels for description purposes. The N training sample images can be sorted according to the target confidence corresponding to each of the N training sample images under the target classification task, and the topK of them are selected, where K is a preset value, such as 2000, and K is less than N. In this way, the selected K training sample images can be associated with their respective corresponding target category labels to be added to the second training sample set. By performing the above processing for multiple classification tasks respectively, the training samples with labeled labels can be expanded for various classification tasks in the second training sample set.

[0093] It should be noted that the K value set for each classification task can be different. For example, if the number of training samples with labeled labels under a certain classification task in the second training sample set is much higher than the number of training samples with labeled labels under other classification tasks, then the corresponding K value under the classification task can be set smaller to balance the number of training samples under different classification tasks.

[0094] In addition, optionally, the above sorting strategy can also be implemented as follows: for the target confidences and corresponding category labels of the N training sample images under the target classification task, the N training sample images can also be grouped according to the same category label, and the training sample images contained in the same group all correspond to the same category label. After that, topK is selected within each group, and the K values ​​corresponding to different groups can be different. This can balance the problem of unbalanced number of training samples corresponding to different category labels under the same classification task.

[0095] In addition to the above optional embodiments, when the second model is a model for completing multiple classification tasks, in another optional embodiment, determining the annotation label corresponding to the training sample image i in the second training sample set according to the output result of the second model can be implemented as follows:

[0096] Determine, according to the output results, target category labels corresponding to the vehicle i contained in the training sample image under multiple classification tasks, wherein the target category label corresponding to each classification task is a category label with target confidence under the corresponding classification task;

[0097] If the target confidences corresponding to the vehicles included in the training sample image i under multiple classification tasks are all greater than the first set threshold, and / or the average value of the target confidences corresponding to the vehicles included in the training sample image i under multiple classification tasks is greater than the second set threshold, then it is determined that the annotation labels corresponding to the training sample image i in the second training sample set include the target category labels corresponding to the vehicles included in the training sample image i under multiple classification tasks. The second set threshold is greater than the first set threshold.

[0098] The starting point of the solution provided in this embodiment is the target confidence corresponding to the training sample image i under each classification task. If the target confidence of the vehicle contained in the training sample image i under each classification task is greater than the first set threshold (for example, 0.6), and the average of multiple target confidences corresponding to the training sample image i is greater than the second set threshold (for example, 0.7), then the training sample image i is considered to be a good sample for each classification task. The training sample image i is associated with the category label corresponding to each target confidence and added to the second training sample set to expand the second training sample set so that the second training sample set has more samples of good quality.

[0099] In summary, by supplementing the second training sample set with training sample images from the first training sample set, and automatically determining the annotation labels of the training sample images added to the second training sample set through a semi-supervised learning method, the cost and time of manual labeling can be avoided, which also helps to improve the imbalance in the number of samples between classes in the second training sample set, and can also make the second training sample set contain more samples of good quality.

[0100] As mentioned above, the vehicle identification method provided by the present invention can be executed in the cloud, and a number of computing nodes can be deployed in the cloud, each of which has computing, storage and other processing resources. In the cloud, multiple computing nodes can be organized to provide a certain service. Of course, a computing node can also provide one or more services. The way to provide the service in the cloud can be to provide a service interface to the outside, and the user calls the service interface to use the corresponding service. The service interface includes software development kits (Software Development Kit, referred to as SDK), application programming interfaces (Application Programming Interface, referred to as API), and other forms.

[0101] According to the solution provided in the embodiment of the present invention, the cloud can provide a service interface for vehicle identification service, and the user calls the vehicle identification service interface through the user device to trigger a request to call the vehicle identification service interface to the cloud. The cloud determines the computing node that responds to the request, and uses the processing resources in the computing node to perform the following steps:

[0102] Acquire an image containing a target vehicle;

[0103] Extracting features from the image using a first model to obtain first image features; wherein the vehicle categories that can be identified by the first model include the first category;

[0104] Extracting features from the image using a second model to obtain second image features; wherein the vehicles included in the training sample image of the second model belong to a second category, and the first category matches the second category;

[0105] fusing the first image feature and the second image feature to obtain a third image feature;

[0106] The category of the target vehicle is determined according to the third image feature.

[0107] The detailed process of the vehicle identification service interface using processing resources to perform vehicle identification processing can refer to the relevant descriptions in the above other embodiments, which will not be repeated here. In practical applications, the above request can directly carry an image containing the target object, and the cloud parses the image from the request. In addition, the cloud can pre-store the above first model, second model, and third model to call these models to complete the corresponding processing.

[0108] For ease of understanding, combined Figure 7 To illustrate this. Figure 7 In the embodiment, after the user obtains an image containing the target vehicle, the vehicle identification service interface is called in the user device E1 to send a call request to the cloud computing node E2, and the call request includes the image containing the target vehicle. After receiving the call request, the cloud computing node E2 extracts features from the image through the first model to obtain a first image feature; extracts features from the image through the second model to obtain a second image feature, fuses the first image feature and the second image feature to obtain a third image feature, and determines the category of the target vehicle according to the third image feature. The detailed execution process is described in the aforementioned embodiment and will not be repeated here. Afterwards, the cloud computing node E2 sends the category recognition result of the target object to the user device E1, and the user device E1 records the category recognition result.

[0109] In the above embodiments, only the category recognition of motor vehicles is used as an example to introduce a category recognition method provided by an embodiment of the present invention. In fact, it can be understood that the category recognition method is not limited to the vehicle category recognition scenario, and the category recognition scenarios of other objects are still applicable. At this time, the relationship between the first model and the second model can be expressed as follows: the category label that the first model can recognize matches the category label to which the object contained in the training sample of the second model belongs, wherein the category label to which the object contained in the training sample of the second model belongs is defined from the perspective of the classification task performed by the first model.

[0110] The following will describe in detail the vehicle identification device of one or more embodiments of the present invention. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0111] Figure 8 A schematic diagram of the structure of a vehicle identification device provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, the device includes: an acquisition module 11, an extraction module 12, and an identification module 13.

[0112] The acquisition module 11 is used to acquire an image containing a target vehicle.

[0113] The extraction module 12 is used to extract features from the image through a first model to obtain first image features; and to extract features from the image through a second model to obtain second image features; wherein the vehicle categories that can be identified by the first model include a first category, the vehicles included in the training sample images of the second model belong to a second category, and the first category matches the second category.

[0114] The recognition module 13 is used to fuse the first image feature and the second image feature to obtain a third image feature; and determine the category of the target vehicle according to the third image feature.

[0115] Optionally, the first category includes the second category, or the first category and the second category are different subcategories under the same parent category.

[0116] Optionally, the recognition module 13 is also used to: convert the image into a grayscale image; compress the grayscale image to obtain a grayscale image of a set scale; perform dimension transformation processing on the first image feature and the second image feature respectively to transform the first image feature and the second image feature into the set scale; and fuse the first image feature, the second image feature after dimension transformation, and the grayscale image of the set size to obtain a third image feature.

[0117] Optionally, the recognition module 13 is further used to: input the third image feature into a third model to determine the category of the target vehicle through the third model, and the scale of the third model is smaller than the first model and the second model.

[0118] Optionally, the first category includes the second category, and the device also includes: a training optimization module, used to determine the training sample image corresponding to the second category from the first training sample set corresponding to the first model; input the training sample image into the second model to obtain the output result of the second model; if the output result meets the set conditions, determine the annotation label corresponding to the training sample image in the second training sample set corresponding to the second model according to the output result, and add the training sample image to the second training sample set.

[0119] Among them, optionally, the output result includes the confidence corresponding to the vehicle contained in the training sample image under the set label. Based on this, the training optimization module can be specifically used to: if the confidence meets the set conditions, determine that the annotation label corresponding to the training sample image in the second training sample set is the set label.

[0120] Optionally, the second model is a model for completing multiple classification tasks, and the output results include the category labels and confidences corresponding to the vehicles contained in the training sample images under the multiple classification tasks. Based on this, the training optimization module can be specifically used for: for the target classification task among the multiple classification tasks, if the confidence corresponding to the target category label of the vehicle contained in the training sample image under the target classification task meets the set conditions, then determine that the annotation label corresponding to the training sample image in the second training sample set includes the target category label, wherein the target category label is one of the multiple category labels preset under the target classification task.

[0121] Optionally, the second model is a model for completing multiple classification tasks, and the output result includes the category labels and confidences corresponding to the vehicles contained in the training sample images under the multiple classification tasks. Based on this, the training optimization module can be specifically used to: determine the target category labels corresponding to the vehicles contained in the training sample images under the multiple classification tasks according to the output result, and the target category label corresponding to each classification task is a category label with a target confidence under the corresponding classification task;

[0122] If the target confidences corresponding to the vehicles included in the training sample image under the multiple classification tasks are all greater than the first set threshold, and / or the average value of the target confidences corresponding to the vehicles included in the training sample image under the multiple classification tasks is greater than the second set threshold, then it is determined that the annotation labels corresponding to the training sample image in the second training sample set include the target category labels corresponding to the vehicles included in the training sample image under the multiple classification tasks.

[0123] Figure 8 The device shown can execute the steps introduced in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments, which will not be repeated here.

[0124] In one possible design, the above Figure 8 The structure of the vehicle identification device shown can be implemented as an electronic device, such as Fig. 9 As shown, the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable codes, and when the executable codes are executed by the processor 21, the processor 21 can at least implement the vehicle identification method provided in the above-mentioned embodiment.

[0125] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the vehicle identification method provided in the aforementioned embodiment.

[0126] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0127] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle identification method, characterized in that: include: Acquire an image containing a target vehicle; Extracting features from the image using a first model to obtain first image features; wherein the vehicle category that can be identified by the first model is a first category; Extracting features from the image using a second model to obtain second image features; wherein the vehicle included in the training sample image of the second model belongs to a second category, the first category includes the second category, or the first category and the second category are different subcategories under the same parent category; fusing the first image feature and the second image feature to obtain a third image feature; The category of the target vehicle is determined according to the third image feature.

2. The method according to claim 1, characterized in that The method further comprises: converting the image into a grayscale image; Compressing the grayscale image to obtain a grayscale image of a set scale; The fusing the first image feature and the second image feature to obtain a third image feature includes: Performing dimension transformation processing on the first image feature and the second image feature respectively, so as to transform the first image feature and the second image feature into the set scale; The first image feature after the dimension transformation, the second image feature, and the grayscale image of the set scale are fused to obtain a third image feature.

3. The method according to claim 1, characterized in that Determining the category of the target vehicle according to the third image feature includes: The third image feature is input into a third model to determine the category of the target vehicle through the third model, and the scale of the third model is smaller than the first model and the second model.

4. The method according to claim 1, characterized in that: The first category includes the second category, and the method further includes: Determine a training sample image corresponding to the second category from a first training sample set corresponding to the first model; Inputting the training sample image into the second model to obtain an output result of the second model; If the output result meets the set condition, the label corresponding to the training sample image in the second training sample set corresponding to the second model is determined according to the output result, and the training sample image is added to the second training sample set.

5. The method according to claim 4, characterized in that The output result includes the confidence level corresponding to the vehicle included in the training sample image under the set label; If the output result meets the set condition, determining the label corresponding to the training sample image in the second training sample set corresponding to the second model according to the output result, including: If the confidence meets the set condition, it is determined that the annotation label corresponding to the training sample image in the second training sample set is the set label.

6. A vehicle identification method, characterized in that: include: In response to a request from a user device to call a vehicle identification service interface, the following steps are performed using processing resources corresponding to the vehicle identification service interface: Acquire an image containing a target vehicle; Extracting features from the image using a first model to obtain first image features; wherein the vehicle categories that can be identified by the first model include the first category; Extracting features from the image using a second model to obtain second image features; wherein the vehicle included in the training sample image of the second model belongs to a second category, the first category includes the second category, or the first category and the second category are different subcategories under the same parent category; fusing the first image feature and the second image feature to obtain a third image feature; The category of the target vehicle is determined according to the third image feature.

7. A vehicle identification device, characterized in that: include: An acquisition module, used for acquiring an image containing a target vehicle; an extraction module, configured to extract features from the image using a first model to obtain first image features; and extract features from the image using a second model to obtain second image features; wherein the vehicle categories that can be identified by the first model include the first category, the vehicles included in the training sample images of the second model belong to the second category, the first category includes the second category, or the first category and the second category are different subcategories under the same parent category; The recognition module is used to fuse the first image feature and the second image feature to obtain a third image feature; and determine the category of the target vehicle according to the third image feature.

8. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the vehicle identification method as described in any one of claims 1 to 5.

9. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the vehicle identification method according to any one of claims 1 to 5.

Images