Method and device for classifying vehicles in image, computer equipment and readable storage medium

Abstract

The invention provides a vehicle classification method and device in an image, computer equipment and a computer readable storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: carrying out image classification on each image in a report image data set to obtain each first category image and each second category image of the report image data set; carrying out vehicle classification on the first class image to obtain a vehicle classification result of each first vehicle detection frame in the first class image; identifying the first class image to obtain a first vehicle color of a vehicle detection frame marked in the first class image and a second vehicle color of three vehicle detection frames in the first class image; and obtaining a vehicle classification result of each second vehicle detection frame in the second category image based on classification of the first vehicle color and the second vehicle color. According to the method, the distinguishing accuracy of the three vehicles and the target vehicle can be improved, and the difficulty of post-processing processes such as responsibility determination and loss assessment of the follow-up target vehicle and the three vehicles is reduced to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, apparatus, computer device and computer-readable storage medium for classifying vehicles in an image. Background Art

[0002] At present, the number of cars in use is gradually increasing every year, and the incidence of traffic accidents is also increasing. Vehicle insurance refers to a commercial insurance that is responsible for compensation for personal injury or property damage caused by natural disasters or accidents to motor vehicles. For vehicles that have purchased vehicle insurance, the owner can apply for insurance compensation after the accident. In the related technology, when applying for insurance compensation, the owner needs to upload the image of the vehicle related to the accident, and through the automatic recognition model based on computer vision technology, the subject vehicle and the third-party vehicle in the uploaded image are combined to determine the responsibility, damage, etc. to complete the claim process. However, some of the images of the accident-related vehicles uploaded by the owner do not contain license plate information, so it is difficult to distinguish the subject vehicle and the third-party vehicle in the image, which increases the difficulty of post-processing processes such as determining responsibility and damage for the subject vehicle and the third-party vehicle. Summary of the invention

[0003] The present application provides a method, device, computer equipment and computer-readable storage medium for classifying vehicles in an image, which belongs to the field of computer vision technology and can improve the accuracy of distinguishing between third-party vehicles and target vehicles, and reduce the difficulty of subsequent post-processing processes such as determining liability and damage for target vehicles and third-party vehicles to a certain extent.

[0004] In a first aspect, the present application provides a method for classifying vehicles in an image, the method comprising:

[0005] Obtain the crime report image dataset;

[0006] Performing image classification on each image in the case reporting image data set to obtain first category images and second category images in the case reporting image data set, wherein the vehicle information completeness of the first category images is greater than the vehicle information completeness of the second category images;

[0007] Performing vehicle classification on the first category image to obtain vehicle classification results of each first vehicle detection frame in the first category image, wherein the vehicle classification result is a vehicle detection frame of a target vehicle as the target vehicle detection frame, and the vehicle classification result is a vehicle detection frame of three vehicles as the three vehicle detection frame;

[0008] Performing vehicle color recognition on the first category image to obtain a first vehicle color of a target vehicle detection frame in the first category image and a second vehicle color of a third vehicle detection frame in the first category image;

[0009] Based on the first vehicle color and the second vehicle color, the second category image is classified into vehicle categories to obtain vehicle classification results for each second vehicle detection frame in the second category image.

[0010] In a second aspect, the present application provides a device for classifying vehicles in an image, the device comprising:

[0011] An acquisition unit, used for acquiring a data set of crime report images;

[0012] an image classification unit, configured to perform image classification on each image in the case reporting image data set to obtain first category images and second category images in the case reporting image data set, wherein the vehicle information completeness of the first category images is greater than the vehicle information completeness of the second category images;

[0013] A first classification unit is used to perform vehicle classification on the first category image to obtain a vehicle classification result of each first vehicle detection frame in the first category image, wherein the vehicle classification result is a vehicle detection frame of a target vehicle as the target vehicle detection frame, and the vehicle classification result is a vehicle detection frame of three vehicles as the three vehicle detection frame;

[0014] an identification unit, configured to perform vehicle color identification on the first category image to obtain a first vehicle color of a target vehicle detection frame in the first category image and a second vehicle color of a third vehicle detection frame in the first category image;

[0015] The second classification unit is used to perform vehicle classification on the second category image based on the first vehicle color and the second vehicle color, and obtain a vehicle classification result of each second vehicle detection frame in the second category image.

[0016] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the method for classifying vehicles in an image when executing the computer program.

[0017] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the method for classifying vehicles in an image.

[0018] The present application obtains each first category image and each second category image of the report image data set by first classifying each image in the report image data set; on the one hand, for the first category images with relatively high vehicle information completeness, the image information can be directly identified to distinguish the target vehicle and the third party vehicle, and the vehicle classification of the first category images can be completed quickly and accurately; on the other hand, for the second category images with relatively low vehicle information completeness, the target vehicle and the third party vehicle are distinguished by comparing the first vehicle color of the target vehicle detection frame in the first category image with the second vehicle color of the third party vehicle detection frame in the first category image, and the vehicle distinguishing information of the first category image can be used to complete the distinction between the target vehicle and the third party vehicle in the second category image, thereby avoiding the problem that the third party vehicle and the target vehicle are difficult to distinguish due to incomplete image information (such as some images do not contain license plate information), improving the accuracy of distinguishing the third party vehicle and the target vehicle, and reducing the difficulty of subsequent post-processing processes such as determining liability and damage for the target vehicle and the third party vehicle to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 It is a flow chart of a method for classifying vehicles in an image provided by an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram for explaining how to classify vehicles by associating a first category image with a second category image using color information in an embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of an embodiment of the flow chart of the training of the first image generation model in the embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of an embodiment of the flow chart of the training of the second image generation model in the embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of a principle structure framework of the first preset generation model in the embodiment of the present application;

[0025] Figure 6 is a schematic diagram of the structure of an embodiment of a vehicle classification device in an image provided by an embodiment of the present application;

[0026] Figure 7 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0030] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0031] Embodiments of the present application provide a method, apparatus, computer device, and computer-readable storage medium for classifying vehicles in an image.

[0032] The executor of the method for classifying vehicles in an image in an embodiment of the present application may be the device for classifying vehicles in an image provided in an embodiment of the present application, or the computer device provided in an embodiment of the present application, wherein the device for classifying vehicles in an image may be implemented in hardware or software.

[0033] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] See also Figure 1 , Figure 11 is a flow chart of a method for classifying vehicles in an image provided by an embodiment of the present application. The method for classifying vehicles in an image includes steps 101 to 105, wherein:

[0035] 101. Obtain a dataset of crime report images.

[0036] Among them, the report image dataset is a collection of vehicle images uploaded in claim cases. Figure 2 The report image dataset includes image a (e.g., a complete image of the target vehicle), image b (e.g., a complete image of the third vehicle), image c (e.g., an image of both the target vehicle and the third vehicle), image d (e.g., a detailed image of the target vehicle), and image e (e.g., a detailed image of the third vehicle).

[0037] 102. Perform image classification on each image in the case reporting image dataset to obtain first category images and second category images in the case reporting image dataset.

[0038] Among them, the vehicle information completeness of the first category image is greater than the vehicle information completeness of the second category image.

[0039] The first category of images is images containing license plate information.

[0040] The second category of images is images that do not contain license plate information.

[0041] Exemplarily, a training sample set can be constructed using vehicle images containing license plate information and vehicle images without license plate information; a preset image classification model is trained using the training sample set to obtain a trained image classification model, at which point the image can be classified into a first category image or a second category image using the trained image classification model.

[0042] In some embodiments, in step 102, first, a vehicle detection model can be used to detect each image i in the report image data set respectively to obtain a vehicle detection frame in each image i. Then, if there are two or more vehicle detection frames in image i, it can be considered as a multi-vehicle scene, and image i is divided into a first category image. If image i has only one vehicle detection frame, then based on whether the detection frame of image i is connected to the boundary, it is judged whether the current vehicle in image i is complete and whether it is a vehicle detail image; if the vehicle detection frame of image i is connected to the boundary, image i is divided into a second category image (such as a detail image, incomplete vehicle information), otherwise image i is divided into a first category image (such as a complete vehicle body image, complete vehicle information)

[0043] In some embodiments, in step 102, the trained image classification model can be used to determine whether each image i in the crime report image dataset contains license plate information. When image i contains license plate information, image i is regarded as a first category image, and when image i does not contain license plate information, image i is regarded as a second category image. In this way, each first category image and each second category image of the crime report image dataset can be obtained. For example, referring to Figure 2 , assuming that the crime report image data set includes image a, image b, image c, image d, and image e, the trained image classification model can be used to determine whether image a contains license plate information. Since image a contains license plate information, image a can be used as a first category image; the trained image classification model can be used to determine whether image b contains license plate information. Since image b contains license plate information, image b can be used as a first category image; the trained image classification model can be used to determine whether image c contains license plate information. Since image c contains license plate information, image c can be used as a first category image; the trained image classification model can be used to determine whether image d contains license plate information. Since image d does not contain license plate information, image d can be used as a second category image; the trained image classification model can be used to determine whether image e contains license plate information. Since image e does not contain license plate information, image e can be used as a second category image; and so on, the first category images and second category images of the crime report image data set are obtained.

[0044] There are many ways to “determine whether each image i in the report image dataset contains license plate information”, including, for example:

[0045] (1) In some embodiments, when the vehicle information integrity of one of the vehicle detection frames in the image is greater than a preset integrity threshold, the image is determined to contain the license plate information; otherwise, the image is determined not to contain the license plate information. In this case, the process of "determining whether each image i in the report image data set contains the license plate information" can be as follows: first, vehicle detection is performed on each image i in the report image data set to obtain multiple vehicle detection frames in image i; then, vehicle information integrity is determined for each vehicle detection frame in image i to obtain the vehicle information integrity of each vehicle detection frame in image i; finally, if the vehicle information integrity of at least one vehicle detection frame in image i is greater than a preset integrity threshold, it is determined that image i contains the license plate information; if the vehicle information integrity of all vehicle detection frames in image i is less than or equal to the preset integrity threshold, it is determined that image i does not contain the license plate information.

[0046] (2) In some embodiments, when the vehicle information integrity of all vehicle detection frames in the image is greater than a preset integrity threshold, the image is determined to contain license plate information; otherwise, the image is determined not to contain license plate information. In this case, the process of "determining whether each image i in the report image data set contains license plate information" can be as follows: first, vehicle detection is performed on each image i in the report image data set to obtain multiple vehicle detection frames in image i; then, vehicle information integrity is determined for each vehicle detection frame in image i to obtain the vehicle information integrity of each vehicle detection frame in image i; finally, if the vehicle information integrity of all vehicle detection frames in image i is greater than a preset integrity threshold, image i is determined to contain license plate information; if the vehicle information integrity of one of the vehicle detection frames in image i is less than or equal to the preset integrity threshold, image i is determined not to contain license plate information.

[0047] 103. Perform vehicle classification on the first category image to obtain a vehicle classification result for each first vehicle detection frame in the first category image.

[0048] Among them, the vehicle classification result is the vehicle detection frame of the target vehicle as the target vehicle detection frame, and the vehicle classification result is the vehicle detection frame of the three vehicles as the three vehicle detection frame.

[0049] The first vehicle detection frame refers to a vehicle detection frame detected in the first category image.

[0050] The second vehicle detection frame refers to a vehicle detection frame detected in the second category image.

[0051] The area map of the first vehicle detection frame refers to the image area where the first vehicle detection frame is located in the first category image.

[0052] The area map of the second vehicle detection frame refers to the image area where the second vehicle detection frame is located in the second category image.

[0053] There are many ways to classify vehicles in step 103, illustratively including:

[0054] (1) In some embodiments, for example, when the vehicle information completeness of each first vehicle detection frame in the first category image is greater than a preset completeness threshold, step 103 may include the following steps (1.1) to (1.6):

[0055] (1.1) Start vehicle classification, set j = 1, and enter (1.2).

[0056] (1.2) First, the license plate number is recognized for the first vehicle detection frame j in the first category image using the trained license plate recognition model (refer to Figure 2In “License Plate Recognition”), obtain the license plate number information of the first vehicle detection frame j in the first category image and enter (1.3).

[0057] (1.3) Then, match the license plate number information of the first vehicle detection frame j in the first category image with the license plate number of the reported target vehicle to determine whether the first vehicle detection frame j in the first category image matches the license plate number of the reported target vehicle; if they match, proceed to (1.4); if not, proceed to (1.5).

[0058] (1.4) If the first vehicle detection frame j in the first category image matches the license plate number of the reported target vehicle, the target vehicle is taken as the vehicle classification result of the first vehicle detection frame j in the first category image and enters (1.6).

[0059] (1.5) If the first vehicle detection frame j in the first category image does not match the license plate number of the reported target vehicle, the three vehicles are taken as the vehicle classification result of the first vehicle detection frame j in the first category image and enter (1.6).

[0060] (1.6) Is j equal to the total number n1 of the first vehicle detection frames in the first category image? If not, proceed to (1.7); if equal, proceed to (1.8).

[0061] (1.7) If j is not equal to the total number n1 of the first vehicle detection frames in the first category image, set j=j+1 and proceed to (1.2).

[0062] (1.8) The vehicle classification is completed and the vehicle classification result of the first vehicle detection frame j (1≤j≤n1) in the first category image is obtained.

[0063] (2) In some embodiments, for example, when each first vehicle detection frame in the first category image includes a vehicle detection frame whose vehicle information integrity is greater than a preset integrity threshold, and a vehicle detection frame whose vehicle information integrity is less than or equal to the preset integrity threshold, step 103 may include the following steps (2.1) to (2.8):

[0064] (2.1) Start vehicle classification, set j = 1, and enter (2.2).

[0065] (2.2) First, the license plate number is recognized for the first vehicle detection frame j in the first category image through the trained license plate recognition model, and the license plate number information of the first vehicle detection frame j in the first category image is obtained, and then enter (2.3).

[0066] (2.3) Then, the license plate number information of the first vehicle detection frame j in the first category image and the license plate number of the reported target vehicle are matched to determine whether the first vehicle detection frame j in the first category image matches the license plate number of the reported target vehicle; if they match, proceed to (2.4); if not, proceed to (2.5).

[0067] (2.4) If the first vehicle detection frame j in the first category image matches the license plate number of the reported target vehicle, the target vehicle is used as the vehicle classification result of the first vehicle detection frame j in the first category image, and the three vehicles are used as the vehicle classification results of other first vehicle detection frames in the first category image (here, other first vehicle detection frames in the first category image refer to the first vehicle detection frames in the first category image except the first vehicle detection frame j), and go to (2.8).

[0068] (2.5) If the first vehicle detection frame j in the first category image does not match the license plate number of the reported target vehicle, then the three vehicles are taken as the vehicle classification results of the first vehicle detection frame j in the first category image, and it is detected whether j is equal to the total number n1 of the first vehicle detection frames in the first category image. If not, proceed to (2.6); if equal, proceed to (2.7).

[0069] (2.6) If j is not equal to the total number n1 of the first vehicle detection frames in the first category image, then set j=j+1 and proceed to (2.2).

[0070] (2.7) If j is equal to the total number n1 of the first vehicle detection frames in the first category image, proceed to step (2.8).

[0071] (2.8) The vehicle classification is completed and the vehicle classification result of the first vehicle detection frame j (1≤j≤n1) in the first category image is obtained.

[0072] 104. Perform vehicle color recognition on the first category image to obtain a first vehicle color of a target vehicle detection frame in the first category image and a second vehicle color of a third vehicle detection frame in the first category image.

[0073] The first vehicle color refers to the color of the vehicle corresponding to the target vehicle detection frame in the first category image.

[0074] The second vehicle color refers to the color of the vehicle corresponding to the three-vehicle detection frame in the first category image.

[0075] The third vehicle color refers to the color of the vehicle corresponding to the second vehicle detection frame in the second category image.

[0076] In some embodiments, the vehicle color recognition of the target vehicle detection frame can be as follows: first, the target vehicle detection frame in the first category image is cropped out to obtain a cropped target vehicle region map of the first category image; then, the cropped target vehicle region map of the first category image is input into a trained color recognition model, and the color recognition model performs color recognition based on the cropped target vehicle region map of the first category image to obtain the first vehicle color of the target vehicle detection frame in the first category image. For example, Figure 2 As shown, taking the first category image as image c as an example, assuming that in step 103, when performing vehicle classification on image c, a target vehicle detection frame is detected (reference Figure 2 The target vehicle detection frame (i.e., detection frame 2) is cropped out as the cropped target vehicle region map of image c; the cropped target vehicle region map of image c is input into the trained color recognition model, and the color recognition model performs color recognition based on the cropped target vehicle region map of image c to obtain the target vehicle detection frame in image c (reference Figure 2 The detection box 1) in image c corresponds to the color of the vehicle (eg, white).

[0077] In some embodiments, the vehicle color recognition of the three-car detection frame can be as follows: first, the three-car detection frame in the first category image is cropped to obtain a cropped three-car region map of the first category image; then, the cropped three-car region map of the first category image is input into a trained color recognition model, and the color recognition model performs color recognition based on the cropped three-car region map of the first category image to obtain the first vehicle color of the three-car detection frame in the first category image. For example, Figure 2 As shown, taking the first category image as image c as an example, assuming that in step 103, when performing vehicle classification for image c, three vehicle detection frames are detected (reference Figure 2 The three-car detection frame (i.e., detection frame 2) is cropped out as the cropped three-car region map of image c; the cropped three-car region map of image c is input into the trained color recognition model, and the color recognition model performs color recognition according to the cropped three-car region map of image c to obtain the three-car detection frame in image c (reference Figure 2 The detection box 2) in image c corresponds to the color of the vehicle (eg, red).

[0078] 105. Perform vehicle classification on the second category image based on the first vehicle color and the second vehicle color to obtain a vehicle classification result for each second vehicle detection frame in the second category image.

[0079] In step 105, there are multiple ways to classify the second category image into vehicles, illustratively including:

[0080] (1) Using color information association. Compare the first vehicle color, the second vehicle color and the third vehicle color respectively to determine the vehicle classification result of the second vehicle detection frame. At this time, step 105 may specifically include the following steps 1051A to 1053A:

[0081] 1051A. Perform vehicle color recognition on the second category image to obtain a third vehicle color of each of the second vehicle detection frames.

[0082] In some embodiments, the vehicle color recognition of the second vehicle detection frame can be as follows: first, the second vehicle detection frame in the second category image is cropped out to obtain a cropped vehicle area map of the second category image; then, the cropped vehicle area map of the second category image is input into a trained color recognition model, and color recognition is performed according to the cropped vehicle area map of the second category image by the color recognition model to obtain a third vehicle color of the second vehicle detection frame in the second category image.

[0083] For example, Figure 2 As shown, taking the second category image as image d as an example, assuming that a second car detection frame is detected for image d (reference Figure 2 The second vehicle detection frame (i.e., detection frame 3) is cropped out as the cropped vehicle region map of image d; the cropped vehicle region map of image d is input into the trained color recognition model, and the color recognition model performs color recognition based on the cropped vehicle region map of image d to obtain the second vehicle detection frame in image d (reference Figure 2 The detection box 3) in image d corresponds to the color of the vehicle (eg, white).

[0084] 1052A. Obtain a first color similarity between the first vehicle color and the third vehicle color of each second vehicle detection frame, and a second color similarity between the second vehicle color and the third vehicle color of each second vehicle detection frame.

[0085] 1053A. Determine a vehicle classification result for each of the second vehicle detection frames according to the first color similarity and the second color similarity.

[0086] Exemplarily, if the first color similarity is greater than the second color similarity, the target vehicle is used as the vehicle classification result of the second vehicle detection frame; or, if the second color similarity is greater than the first color similarity, the three vehicles are used as the vehicle classification results of the second vehicle detection frame in the second category image.

[0087] For example, Figure 2 As shown, taking the first category image as image c and the second category image as image d as an example, assuming that a second vehicle detection frame is detected for image d (reference Figure 2 The first vehicle color (such as white) of the target vehicle detection frame (such as detection frame 1) in the first category image (such as image c), the second vehicle color (such as red) of the third vehicle detection frame (such as detection frame 2) in the first category image (such as image c), and the third vehicle color (such as white) of the second vehicle detection frame (such as detection frame 3) in the second category image (such as image d) are compared. The similarity between the white color of detection frame 1 of image c and the white color of detection frame 3 of image d is used as the first color similarity (such as 90%); the similarity between the red color of detection frame 2 of image c and the white color of detection frame 3 of image d is used as the second color similarity (such as 0%); since the first color similarity (such as 90%) is greater than the second color similarity (such as 0%), the target vehicle is taken as the vehicle classification result of the second vehicle detection frame (i.e., detection frame 3) in the second category image (such as image d).

[0088] For example, Figure 2 As shown, taking the first category image as image c and the second category image as image e as an example, assuming that a second car detection frame is detected for image e (reference Figure 2 4), the first vehicle color (such as white) of the target vehicle detection frame (such as detection frame 1) in the first category image (such as image c), the second vehicle color (such as red) of the three-vehicle detection frame (such as detection frame 2) in the first category image (such as image c), and the third vehicle color (such as red) of the second vehicle detection frame (such as detection frame 4) in the second category image (such as image e), then the similarity between the white color of the detection frame 1 of the contrast image c and the red color of the detection frame 4 of the image e is taken as the first color similarity (such as 0%); the similarity between the red color of the detection frame 2 of the image c and the red color of the detection frame 4 of the image e is taken as the second color similarity (such as 90%); since the second color similarity (such as 90%) is greater than the first color similarity (such as 0%), the three-vehicle is taken as the vehicle classification result of the second vehicle detection frame (i.e., detection frame 4) in the second category image (such as image e).

[0089] (2) Use color information and generate image information to associate. If the similarity between the first vehicle color and the second vehicle color is less than the preset similarity threshold, then the first vehicle color, the second vehicle color and the third vehicle color are compared respectively through steps 1051B to 1053B to determine the vehicle classification result of the second vehicle detection frame. If the similarity between the first vehicle color and the second vehicle color is greater than or equal to the preset similarity threshold, then the target vehicle reference image and the three vehicle reference images are generated through steps 1054B to 1056B for comparison to determine the vehicle classification result of the second vehicle detection frame. At this time, step 105 can specifically include the following steps 1051B to 1056B:

[0090] 1051B. If the similarity between the first vehicle color and the second vehicle color is less than a preset similarity threshold, vehicle color recognition is performed on the second category image to obtain a third vehicle color of each second vehicle detection frame.

[0091] 1052B. Obtain a first color similarity between the first vehicle color and the third vehicle color of each second vehicle detection frame, and a second color similarity between the second vehicle color and the third vehicle color of each second vehicle detection frame.

[0092] 1053B. Determine a vehicle classification result for each of the second vehicle detection frames according to the first color similarity and the second color similarity.

[0093] Steps 1051B to 1053B are implemented similarly to steps 1051A to 1053A. For details, please refer to the relevant descriptions in the previous text and will not be repeated here.

[0094] 1054B. If the similarity between the first vehicle color and the second vehicle color is greater than or equal to a preset similarity threshold, generate a target vehicle reference image for each of the second vehicle detection frames and a three-vehicle reference image for each of the second vehicle detection frames.

[0095] In some embodiments, Figure 3 As shown in the figure, the process of "generating a target vehicle reference image of each second vehicle detection frame" can be as follows: steps 301 to 305:

[0096] 301. Obtain vehicle orientation information of each second vehicle detection frame in the second category image.

[0097] The vehicle position information is used to indicate the vehicle part within the vehicle detection frame, such as the right door, front, and rear of the vehicle.

[0098] Exemplarily, the region map of each second vehicle detection frame k can be classified using a trained orientation detection model to obtain the vehicle orientation information of each second vehicle detection frame k.

[0099] 302. Obtain first vehicle model information of the target vehicle detection frame.

[0100] Among them, vehicle model information includes but is not limited to vehicle brand, vehicle model, etc.

[0101] Among them, the first model information is the model information of the target vehicle.

[0102] In some embodiments, the first vehicle information may be extracted using the case report data corresponding to the case report image data set.

[0103] In some embodiments, vehicle type detection may be performed on the target vehicle detection frame to obtain first vehicle type information of the target vehicle.

[0104] Steps 303 to 304 can be implemented by the first image generation model trained by the following steps B1 to B4.

[0105] 303. Perform text feature extraction based on the vehicle orientation information of each second vehicle detection frame and the first vehicle model information through a feature encoder of the first image generation model to obtain a first text feature of each second vehicle detection frame.

[0106] Exemplarily, the orientation information and the first vehicle model information of each second vehicle detection frame k are converted into an embedding vector of fixed dimension through the feature encoder of the first image generation model as the first text feature of each second vehicle detection frame k.

[0107] 304. Perform visual feature extraction based on the area map of the target vehicle detection frame and the area map of each of the second vehicle detection frames through the feature encoder of the first image generation model to obtain a first visual feature of each of the second vehicle detection frames.

[0108] Exemplarily, first, the region map of the target vehicle detection frame is vectorized by the feature encoder of the first image generation model to obtain the visual vector of the target vehicle detection frame; and the region map of each second vehicle detection frame k is vectorized by the feature encoder of the first image generation model to obtain the visual vector of each second vehicle detection frame k. Then, the visual vector of the target vehicle detection frame and the visual vector of each second vehicle detection frame k are connected as the first visual feature of each second vehicle detection frame k.

[0109] 305. Generate a target vehicle reference image of each second vehicle detection frame according to the first text feature of each second vehicle detection frame and the first visual feature of each second vehicle detection frame through the generation layer of the first image generation model.

[0110] Exemplarily, the first text feature of each second vehicle detection frame k and the first visual feature of each second vehicle detection frame k can be input into the generation layer of the first image generation model, and the target vehicle reference image of each second vehicle detection frame k is generated through the generation layer of the image generation model.

[0111] Therefore, through steps 301 to 305, on the first hand, the visual information and text information can be integrated to generate the target vehicle reference image of the second vehicle detection frame, thereby improving the generation accuracy of the target vehicle reference image of the second vehicle detection frame k; on the second hand, since the text based on vehicle orientation information, vehicle model information, etc. can reflect the situation of the vehicle image to a certain extent, the generation accuracy of the target vehicle reference image of the second vehicle detection frame k is improved.

[0112] In some embodiments, Figure 4 As shown in the figure, the process of "generating a three-vehicle reference image of each second vehicle detection frame" can be as follows: steps 401 to 405:

[0113] 401. Obtain vehicle position information of each second vehicle detection frame in the second category image;

[0114] 402. Obtain second vehicle model information of the three-vehicle detection frame;

[0115] 403. Perform text feature extraction based on the vehicle orientation information and the second vehicle model information of each second vehicle detection frame through a feature encoder of the second image generation model to obtain a second text feature of each second vehicle detection frame;

[0116] 404. Perform visual feature extraction based on the region maps of the three vehicle detection frames and the region map of each second vehicle detection frame by using the feature encoder of the first image generation model to obtain a second visual feature of each second vehicle detection frame;

[0117] 405. Generate a three-vehicle reference image of each second vehicle detection frame according to the first text feature of each second vehicle detection frame and the second visual feature of each second vehicle detection frame through the generation layer of the first image generation model.

[0118] The implementation of steps 401 to 405 is similar to that of steps 301 to 305 , and the details may refer to the above related descriptions, which will not be repeated here.

[0119] 1055B. Obtain a first image similarity between the target vehicle reference image of each second vehicle detection frame and the regional image of each second vehicle detection frame, and a second image similarity between the three-vehicle reference image of each second vehicle detection frame and the regional image of each second vehicle detection frame.

[0120] Step 1055B is implemented similarly to step 1052A. For details, please refer to the relevant instructions in the previous text and will not be repeated here.

[0121] 1056B. Determine a vehicle classification result for each of the second vehicle detection frames according to the first image similarity and the second image similarity.

[0122] In some embodiments, if the similarity of the first image is greater than the similarity of the second image, the target vehicle is used as the vehicle classification result of the second vehicle detection frame. If the similarity of the second image is greater than the similarity of the first image, the three vehicles are used as the vehicle classification result of the second vehicle detection frame.

[0123] In some embodiments, the vehicle classification result of the second vehicle detection frame may be determined by combining the comparison result of the generated image and the comparison result of the touch position. At this time, step 1056B may specifically include the following steps A1 to A5:

[0124] A1. Obtain the collision position of the target vehicle and the collision positions of the three vehicles.

[0125] A2. Obtain the target orientation of each of the second vehicle detection frames.

[0126] A3. Determine a first classification result of each second vehicle detection frame and a weight of the first classification result based on the target position, the target vehicle contact position and the three-vehicle contact positions.

[0127] Among them, the first classification result is a vehicle classification result of each second vehicle detection frame determined based on the target position, the target vehicle contact position and the third vehicle contact position.

[0128] In some embodiments, a first orientation similarity between the target orientation and the orientation of the target vehicle touching, and a second orientation similarity between the target orientation and the orientation of the three vehicles touching are obtained. If the first orientation similarity is greater than the second orientation similarity, the target vehicle is used as the first classification result of the second vehicle detection frame, and the first orientation similarity is used as the weight of the first classification result. If the second orientation similarity is greater than the first orientation similarity, the three vehicles are used as the first classification result of the second vehicle detection frame, and the second orientation similarity is used as the weight of the first classification result.

[0129] A4. Determine a second classification result of each second vehicle detection frame and a weight of the second classification result according to the first image similarity and the second image similarity.

[0130] The second classification result is a vehicle classification result of each second vehicle detection frame determined based on the first image similarity and the second image similarity.

[0131] In some embodiments, if the similarity of the first image is greater than the similarity of the second image, the target vehicle is used as the second classification result of the second vehicle detection frame, and the similarity of the first image is used as the weight of the second classification result. If the similarity of the second image is greater than the similarity of the first image, the three vehicles are used as the second classification result of the second vehicle detection frame, and the similarity of the second image is used as the weight of the second classification result.

[0132] A5. Determine a vehicle classification result of each of the second vehicle detection frames based on the weight of the first classification result and the weight of the second classification result.

[0133] Exemplarily, if the first classification result and the second classification result are different, and the weight of the first classification result is greater than the weight of the second classification result, the first classification result is used as the vehicle classification result of the second vehicle detection frame. If the first classification result and the second classification result are different, and the weight of the second classification result is greater than the weight of the first classification result, the second classification result is used as the vehicle classification result of the second vehicle detection frame.

[0134] Therefore, through steps 1051B to 1056B, when the similarity between the first vehicle color and the second vehicle color is greater than or equal to the preset similarity threshold, that is, the target vehicle and the three vehicles are relatively close in color, a reference image of the target vehicle and a reference image of the three vehicles are generated, which are compared with the area image of the second vehicle detection frame respectively, and the target vehicle and the three vehicles are distinguished by the image similarity results, so that when the target vehicle and the three vehicles are relatively close in color, the target vehicle and the three vehicles can be accurately distinguished, thereby improving the classification accuracy of the target vehicle and the three vehicles.

[0135] In some embodiments, the first image generation model can be learned by referring to the following steps B1 to B4:

[0136] B1. Perform feature encoding based on the multimodal information of the first sample image through a feature encoder of the first preset generation model to obtain text features of each first sample detection frame in the first sample image and visual features of each first sample detection frame.

[0137] The first sample image is a sample image used to learn and acquire a first image generation model.

[0138] The first sample detection frame is a vehicle detection frame detected in the first sample image.

[0139] Among them, the multimodal information of the first sample image includes the vehicle orientation information of each first sample detection frame p in the first sample image, the vehicle model information of each first sample detection frame p, the area map of each first sample detection frame p, and the target vehicle annotation area map of each first sample detection frame p.

[0140] The labeled vehicle region map of each first sample detection frame p refers to the labeled labeled vehicle region map of each first sample detection frame p.

[0141] Exemplarily, a feature encoder of the first preset generation model can be used to perform text feature extraction based on the vehicle orientation information of each first sample detection frame p and the vehicle model information of the first sample detection frame p to obtain the text feature of each first sample detection frame p; a feature encoder of the first preset generation model can be used to perform visual feature extraction based on the target vehicle annotation area map of the first sample detection frame p and the area map of each first sample detection frame p to obtain the visual feature of each first sample detection frame p.

[0142] Exemplarily, the feature encoder of the first preset generation model may adopt a VAE (Vertical Autoencoder) structure.

[0143] B2. Generate a predicted target vehicle reference image of each first sample detection frame based on the text features of each first sample detection frame and the visual features of each first sample detection frame through the generation layer of the first preset generation model.

[0144] In step B2, the predicted target vehicle reference image of the first sample detection frame p is generated based on the text features of each first sample detection frame p and the visual features of each first sample detection frame p. Figure 5 As shown in the “Prediction Reference Figure Y'”.

[0145] Exemplarily, the generation layer of the first preset generation model may adopt a conditional VAE (Conditional VAE) structure.

[0146] B3. Based on the first loss between the predicted target vehicle reference image of each first sample detection frame and the labeled target vehicle reference image of each first sample detection frame, the first preset generation model is trained to obtain a first trained generation model.

[0147] The first trained generated model is a model obtained by training the first preset generated model.

[0148] There are many ways to implement step B3, illustratively including:

[0149] (1) In some embodiments, training is performed based on the loss of the generated graph (i.e., the first loss L1). In this case, step B3 may specifically include: obtaining a reference graph of the predicted target vehicle (reference Figure 5 The predicted reference image Y' shown in the figure) and the labeled reference image of each first sample detection frame (reference Figure 5 A first loss is determined between the prediction reference image Y shown in the “prediction reference image Y” in the figure); based on the first loss, the model parameters of the generation layer and the feature coding layer in the first preset generation model are adjusted until the preset training stop condition is met, and the first trained generation model is obtained.

[0150] (2) In some embodiments, training is performed based on the loss of the generated graph (i.e., the first loss L1) and the hidden feature loss between the image and the multimodal information (i.e., the second loss L2). At this time, step B3 may specifically include the following steps B31 to B35:

[0151] B31. Based on the text features of each of the first sample detection frames and the visual features of each of the first sample detection frames, obtain the first hidden features of each of the first sample detection frames.

[0152] The first hidden feature of the first sample detection frame p is a hidden feature obtained by further processing (such as fusion, Gaussian sampling, etc.) the text feature of the first sample detection frame p and the visual feature of the first sample detection frame p.

[0153] Please refer to Figure 5 For example, first, the text features of the first sample detection frame p and the visual features of the first sample detection frame p can be fused to obtain the fused features Zx of the first sample detection frame p. Figure 5 Then, Gaussian sampling is performed on the fused features of the first sample detection frame p to obtain the Gaussian sampled features as the first hidden features of the first sample detection frame p. Figure 5 As shown in the “Hidden feature h1” in .

[0154] B32. Perform image encoding based on the labeled vehicle reference image of each first sample detection frame through the image encoder of the first preset generation model to obtain the labeled vehicle reference image features of each first sample detection frame.

[0155] In step B32, the labeled vehicle reference image of the first sample detection frame p refers to the labeled vehicle reference image of the first sample detection frame p.

[0156] Among them, the reference image feature of the target vehicle of the first sample detection frame p (reference Figure 5 ”Zy” in the figure is the image feature obtained by encoding the reference image of the labeled vehicle of the first sample detection frame p.

[0157] B33. Based on the target vehicle reference image feature of each of the first sample detection frames, obtain the second hidden feature of each of the first sample detection frames.

[0158] The second hidden feature of the first sample detection frame p is a hidden feature obtained by further processing (such as Gaussian sampling, etc.) the target vehicle reference image feature Zy of the first sample detection frame p.

[0159] Please refer to Figure 5, exemplarily, first, in step B32, the image encoder of the first preset generation model is used to perform image encoding based on the labeled vehicle reference image of each first sample detection frame p, and obtain the vehicle reference image feature Zy of each first sample detection frame p. Then, Gaussian sampling is performed on the vehicle reference image feature Zy of the first sample detection frame p to obtain the Gaussian sampled feature as the second hidden feature h2 of the first sample detection frame p, and the reference feature is obtained. Figure 5 As shown in the “Hidden Features h2” in .

[0160] B34. Obtain a second loss between a first hidden feature of each of the first sample detection frames and a second hidden feature of each of the first sample detection frames.

[0161] B35. Based on the first loss and the second loss, the first preset generation model is trained to obtain a first trained generation model.

[0162] It can be seen that by constructing the second loss between the first hidden feature of the first sample detection frame and the second hidden feature of the first sample detection frame (reference Figure 5 ), which is used to train the first preset generation model, thereby constructing the feature loss between the hidden space of the target vehicle reference image and the hidden space of the multimodal information of the first sample image, so that the first trained generation model can learn to capture the common features of the multimodal information of the target vehicle reference image and the first sample image, thereby improving the generation accuracy of the target vehicle reference image, and then the first image generation model extracted based on the first trained generation model can also capture the common features between the multimodal information of the target vehicle reference image and the first sample image, and is used to generate the target vehicle reference image of each second vehicle detection frame in the second category image, thereby improving the generation accuracy of the target vehicle reference image, thereby improving the image similarity judgment accuracy, and improving the vehicle classification result of each second vehicle detection frame, which to a certain extent solves the problem of being unable to accurately distinguish the target vehicle and the third vehicle in the image due to the close color of the target vehicle and the third vehicle, and improves the accuracy of distinguishing the target vehicle and the third vehicle in the image.

[0163] B4. Construct the first image generation model based on the feature encoder in the first trained generation model and the generation layer in the first trained generation model.

[0164] Exemplarily, the model parameters of the feature encoder in the first trained generative model can be extracted as the model parameters of the feature encoder of the first image generation model; the model parameters of the generation layer in the first trained generative model can be extracted as the generation model parameters of the first image generation model, thereby building the first image generation model.

[0165] In some embodiments, the first image generation model can be learned by referring to the following steps C1 to C4:

[0166] C1. Perform feature encoding based on the multimodal information of the second sample image through a feature encoder of a second preset generation model to obtain text features of each second sample detection box in the second sample image and visual features of each second sample detection box.

[0167] C2. Generate a predicted three-car reference image of each second sample detection frame based on the text features of each second sample detection frame and the visual features of each second sample detection frame through the generation layer of the second preset generation model.

[0168] C3. Based on the third loss between the predicted three-car reference image of each second sample detection frame and the labeled three-car reference image of each second sample detection frame, the second preset generation model is trained to obtain a second trained generation model.

[0169] C4. Construct the second image generation model based on the feature encoder in the second trained generation model and the generation layer in the second trained generation model.

[0170] The implementation of steps C1 to C4 is similar to that of steps B1 to B4. For details, please refer to the relevant descriptions in the previous text and will not be repeated here.

[0171] For example, in some embodiments, at this time, step C3 may specifically include the following steps C31 to C35:

[0172] C31. Based on the text features of each second sample detection frame and the visual features of each second sample detection frame, obtain the first hidden feature of each second sample detection frame.

[0173] The first hidden feature of the second sample detection frame q is a hidden feature obtained by further processing (such as fusion, Gaussian sampling, etc.) the text feature of the second sample detection frame q and the visual feature of the second sample detection frame q.

[0174] C32. Perform image encoding based on the annotated three-car reference image of each second sample detection frame through the image encoder of the second preset generation model to obtain the three-car reference image features of each second sample detection frame.

[0175] In step C32, the annotated three-car reference image of the second sample detection frame q refers to the annotated three-car reference image of the second sample detection frame q.

[0176] Among them, the three-car reference image features of the second sample detection frame q (reference Figure 5(shown as “Zy” in the figure) is an image feature obtained by encoding the labeled three-car reference image of the second sample detection box q.

[0177] C33. Based on the three-car reference image features of each second sample detection frame, obtain the second hidden feature of each second sample detection frame.

[0178] The second hidden feature of the second sample detection frame q is a hidden feature obtained by further processing (such as Gaussian sampling, etc.) the three-car reference image feature Zy of the second sample detection frame q.

[0179] C34. Obtain a fourth loss between the first hidden feature of each of the second sample detection frames and the second hidden feature of each of the second sample detection frames.

[0180] C35. Based on the third loss and the fourth loss, the second preset generation model is trained to obtain a second trained generation model.

[0181] It can be seen that by constructing the second loss between the first hidden feature of the second sample detection frame and the second hidden feature of the second sample detection frame (reference Figure 5 ), which is used to train the second preset generation model, thereby constructing the feature loss between the hidden space of the three-car reference image and the hidden space of the multimodal information of the second sample image, so that the second trained generation model can learn to capture the common features of the multimodal information of the three-car reference image and the second sample image, thereby improving the generation accuracy of the three-car reference image, and then the second image generation model extracted based on the second trained generation model can also capture the common features between the multimodal information of the three-car reference image and the second sample image, and is used to generate the three-car reference image of each second vehicle detection frame in the second category image, thereby improving the generation accuracy of the three-car reference image, thereby improving the image similarity discrimination accuracy, and improving the vehicle classification result of each second vehicle detection frame, which to a certain extent solves the problem of being unable to accurately distinguish the target car and the three-car in the image due to the close color of the target car and the three-car, and improves the accuracy of distinguishing the target car and the three-car in the image.

[0182] From the above content, it can be seen that by first classifying each image in the report image data set, the first category images and the second category images of the report image data set are obtained; on the one hand, for the first category images with relatively high vehicle information completeness, the image information can be directly identified to distinguish the target vehicle and the third vehicle, and the vehicle classification of the first category images can be completed quickly and accurately; on the other hand, for the second category images with relatively low vehicle information completeness, the first vehicle color of the target vehicle detection frame in the first category image is compared with the second vehicle color of the third vehicle detection frame in the first category image to distinguish the target vehicle and the third vehicle. The vehicle distinction information of the first category image can be used to complete the distinction between the target vehicle and the third vehicle in the second category image, avoiding the problem that the third vehicle and the target vehicle are difficult to distinguish due to incomplete image information (such as some images do not contain license plate information), improving the accuracy of distinguishing the third vehicle and the target vehicle, and reducing the difficulty of subsequent post-processing processes such as liability determination and damage assessment for the target vehicle and the third vehicle to a certain extent.

[0183] In addition, in order to better implement the method for classifying vehicles in an image in the embodiment of the present application, based on the method for classifying vehicles in an image, an apparatus for classifying vehicles in an image is also provided in the embodiment of the present application, such as Figure 6 FIG. 1 is a schematic diagram of a structure of an embodiment of a vehicle classification device in an image provided by an embodiment of the present application. The vehicle classification device 600 in the image includes:

[0184] An acquisition unit 601 is used to acquire a data set of crime report images;

[0185] An image classification unit 602 is used to perform image classification on each image in the reported image data set to obtain each first category image and each second category image in the reported image data set, wherein the vehicle information completeness of the first category image is greater than the vehicle information completeness of the second category image;

[0186] A first classification unit 603 is used to perform vehicle classification on the first category image to obtain a vehicle classification result of each first vehicle detection frame in the first category image, wherein the vehicle classification result is a vehicle detection frame of a target vehicle as the target vehicle detection frame, and the vehicle classification result is a vehicle detection frame of three vehicles as the three vehicle detection frame;

[0187] The recognition unit 604 is used to perform vehicle color recognition on the first category image to obtain a first vehicle color of a target vehicle detection frame in the first category image and a second vehicle color of a third vehicle detection frame in the first category image;

[0188] The second classification unit 605 is used to perform vehicle classification on the second category image based on the first vehicle color and the second vehicle color, and obtain a vehicle classification result of each second vehicle detection frame in the second category image.

[0189] In some embodiments, the second classification unit 605 is specifically used to:

[0190] If the similarity between the first vehicle color and the second vehicle color is less than a preset similarity threshold, performing vehicle color recognition on the second category image to obtain a third vehicle color of each second vehicle detection frame;

[0191] Acquire a first color similarity between the first vehicle color and the third vehicle color of each second vehicle detection frame, and a second color similarity between the second vehicle color and the third vehicle color of each second vehicle detection frame;

[0192] A vehicle classification result of each of the second vehicle detection frames is determined according to the first color similarity and the second color similarity.

[0193] In some embodiments, the second classification unit 605 is specifically used to:

[0194] If the similarity between the first vehicle color and the second vehicle color is greater than or equal to a preset similarity threshold, generating a target vehicle reference image of each second vehicle detection frame and a three-vehicle reference image of each second vehicle detection frame;

[0195] Obtaining a first image similarity between a target vehicle reference image of each second vehicle detection frame and a region image of each second vehicle detection frame, and a second image similarity between a three-vehicle reference image of each second vehicle detection frame and a region image of each second vehicle detection frame;

[0196] A vehicle classification result of each of the second vehicle detection frames is determined according to the first image similarity and the second image similarity.

[0197] In some embodiments, the second classification unit 605 is specifically used to:

[0198] Acquire vehicle orientation information of each second vehicle detection frame in the second category image;

[0199] Obtaining first vehicle model information of the target vehicle detection frame;

[0200] Using a feature encoder of the first image generation model, extracting text features based on the vehicle orientation information of each second vehicle detection frame and the first vehicle model information, to obtain a first text feature of each second vehicle detection frame;

[0201] Extracting visual features based on the region map of the target vehicle detection frame and the region map of each of the second vehicle detection frames through a feature encoder of the first image generation model to obtain a first visual feature of each of the second vehicle detection frames;

[0202] A target vehicle reference image of each second vehicle detection frame is generated through the generation layer of the first image generation model according to the first text feature of each second vehicle detection frame and the first visual feature of each second vehicle detection frame.

[0203] In some embodiments, the vehicle classification device in an image further includes a training unit (not shown in the figure), wherein the training unit is specifically used to:

[0204] Performing feature encoding based on the multimodal information of the first sample image through a feature encoder of the first preset generation model to obtain text features of each first sample detection frame in the first sample image and visual features of each first sample detection frame;

[0205] Generate a predicted target vehicle reference image of each first sample detection frame based on the text features of each first sample detection frame and the visual features of each first sample detection frame through the generation layer of the first preset generation model;

[0206] Based on a first loss between a predicted target vehicle reference image of each first sample detection frame and a labeled target vehicle reference image of each first sample detection frame, training the first preset generation model to obtain a first trained generation model;

[0207] The first image generation model is constructed based on the feature encoder in the first trained generation model and the generation layer in the first trained generation model.

[0208] In some embodiments, the training unit is specifically used for:

[0209] Based on the text features of each first sample detection frame and the visual features of each first sample detection frame, obtaining a first hidden feature of each first sample detection frame;

[0210] Performing image encoding based on the labeled vehicle reference image of each first sample detection frame by using the image encoder of the first preset generation model to obtain the labeled vehicle reference image features of each first sample detection frame;

[0211] Based on the target vehicle reference image feature of each of the first sample detection frames, obtaining a second hidden feature of each of the first sample detection frames;

[0212] Obtaining a second loss between a first hidden feature of each of the first sample detection frames and a second hidden feature of each of the first sample detection frames;

[0213] Based on the first loss and the second loss, the first preset generation model is trained to obtain a first trained generation model.

[0214] In some embodiments, the second classification unit 605 is specifically used to:

[0215] Obtain the collision position of the target vehicle and the collision position of the third vehicle;

[0216] Obtaining a target orientation of each of the second vehicle detection frames;

[0217] Determine a first classification result of each second vehicle detection frame and a weight of the first classification result based on the target position, the target vehicle contact position and the three-vehicle contact position;

[0218] Determining a second classification result and a weight of the second classification result for each second vehicle detection frame according to the first image similarity and the second image similarity;

[0219] Based on the weight of the first classification result and the weight of the second classification result, a vehicle classification result of each second vehicle detection frame is determined.

[0220] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units can refer to the vehicle classification method embodiment in the previous image, which will not be repeated here.

[0221] See also Figure 7 , Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0222] like Figure 7 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0223] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the vehicle classification methods in the image.

[0224] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0225] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the vehicle classification methods in the image.

[0226] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0227] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0228] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned method for classifying vehicles in an image may be accomplished through instructions, or through instruction-controlled related hardware, and the instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0229] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any one of the vehicle classification methods in an image provided by the embodiment of the present application.

[0230] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0231] In the above-described vehicle classification device and computer-readable storage medium embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the vehicle classification device, computer-readable storage medium and its corresponding units in the above-described image can refer to the description of the vehicle classification method in the image in the above embodiment, and will not be repeated here.

[0232] The above is a detailed introduction to a method, device, computer equipment and computer-readable storage medium for classifying vehicles in an image provided by the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application. The non-company software tools or components that appear in the embodiments of the present application are only examples and do not represent actual use.

Claims

1. A method for classifying vehicles in an image, characterized in that: The method comprises: Obtain the crime report image dataset; Performing image classification on each image in the case reporting image data set to obtain first category images and second category images in the case reporting image data set, wherein the vehicle information completeness of the first category images is greater than the vehicle information completeness of the second category images; Performing vehicle classification on the first category image to obtain vehicle classification results of each first vehicle detection frame in the first category image, wherein the vehicle classification result is a vehicle detection frame of a target vehicle as the target vehicle detection frame, and the vehicle classification result is a vehicle detection frame of three vehicles as the three vehicle detection frame; Performing vehicle color recognition on the first category image to obtain a first vehicle color of a target vehicle detection frame in the first category image and a second vehicle color of a third vehicle detection frame in the first category image; Based on the first vehicle color and the second vehicle color, the second category image is classified into vehicle categories to obtain vehicle classification results for each second vehicle detection frame in the second category image.

2. The method for classifying vehicles in an image according to claim 1, characterized in that: The performing vehicle classification on the second category image based on the first vehicle color and the second vehicle color to obtain a vehicle classification result of each second vehicle detection frame in the second category image includes: If the similarity between the first vehicle color and the second vehicle color is less than a preset similarity threshold, performing vehicle color recognition on the second category image to obtain a third vehicle color of each second vehicle detection frame; Acquire a first color similarity between the first vehicle color and the third vehicle color of each second vehicle detection frame, and a second color similarity between the second vehicle color and the third vehicle color of each second vehicle detection frame; A vehicle classification result of each of the second vehicle detection frames is determined according to the first color similarity and the second color similarity.

3. The method for classifying vehicles in an image according to claim 1, characterized in that: The performing vehicle classification on the second category image based on the first vehicle color and the second vehicle color to obtain a vehicle classification result of each second vehicle detection frame in the second category image includes: If the similarity between the first vehicle color and the second vehicle color is greater than or equal to a preset similarity threshold, generating a target vehicle reference image of each second vehicle detection frame and a three-vehicle reference image of each second vehicle detection frame; Obtaining a first image similarity between a target vehicle reference image of each second vehicle detection frame and a region image of each second vehicle detection frame, and a second image similarity between a three-vehicle reference image of each second vehicle detection frame and a region image of each second vehicle detection frame; A vehicle classification result of each of the second vehicle detection frames is determined according to the first image similarity and the second image similarity.

4. The method for classifying vehicles in an image according to claim 3, characterized in that: The target vehicle reference image of each second vehicle detection frame is generated in the following manner: Acquire vehicle orientation information of each second vehicle detection frame in the second category image; Obtaining first vehicle model information of the target vehicle detection frame; Extracting text features based on the vehicle orientation information of each second vehicle detection frame and the first vehicle model information through a feature encoder of the first image generation model to obtain a first text feature of each second vehicle detection frame; Extracting visual features based on the region map of the target vehicle detection frame and the region map of each of the second vehicle detection frames through a feature encoder of the first image generation model to obtain a first visual feature of each of the second vehicle detection frames; A target vehicle reference image of each second vehicle detection frame is generated through the generation layer of the first image generation model according to the first text feature of each second vehicle detection frame and the first visual feature of each second vehicle detection frame.

5. The method for classifying vehicles in an image according to claim 4, characterized in that: The first image generation model is obtained by: Performing feature encoding based on the multimodal information of the first sample image through a feature encoder of the first preset generation model to obtain text features of each first sample detection frame in the first sample image and visual features of each first sample detection frame; Generate a predicted target vehicle reference image of each first sample detection frame based on the text features of each first sample detection frame and the visual features of each first sample detection frame through the generation layer of the first preset generation model; Based on a first loss between a predicted target vehicle reference image of each first sample detection frame and a labeled target vehicle reference image of each first sample detection frame, training the first preset generation model to obtain a first trained generation model; The first image generation model is constructed based on the feature encoder in the first trained generation model and the generation layer in the first trained generation model.

6. The method for classifying vehicles in an image according to claim 5, characterized in that: The first preset generation model is trained based on a first loss between a predicted target vehicle reference image of each first sample detection frame and a labeled target vehicle reference image of each first sample detection frame to obtain a first trained generation model, including: Based on the text features of each first sample detection frame and the visual features of each first sample detection frame, obtaining a first hidden feature of each first sample detection frame; Performing image encoding based on the labeled vehicle reference image of each first sample detection frame by using the image encoder of the first preset generation model to obtain the labeled vehicle reference image features of each first sample detection frame; Based on the target vehicle reference image feature of each of the first sample detection frames, obtaining a second hidden feature of each of the first sample detection frames; Obtaining a second loss between a first hidden feature of each of the first sample detection frames and a second hidden feature of each of the first sample detection frames; Based on the first loss and the second loss, the first preset generation model is trained to obtain a first trained generation model.

7. The method for classifying vehicles in an image according to claim 3, characterized in that: The determining, according to the first image similarity and the second image similarity, a vehicle classification result of each second vehicle detection frame comprises: Obtain the collision position of the target vehicle and the collision position of the third vehicle; Obtaining a target orientation of each of the second vehicle detection frames; Determine a first classification result of each second vehicle detection frame and a weight of the first classification result based on the target position, the target vehicle contact position and the three-vehicle contact position; Determining a second classification result and a weight of the second classification result for each second vehicle detection frame according to the first image similarity and the second image similarity; Based on the weight of the first classification result and the weight of the second classification result, a vehicle classification result of each second vehicle detection frame is determined.

8. A vehicle classification device in an image, characterized in that: The vehicle classification device in the image comprises: An acquisition unit, used for acquiring a data set of crime report images; an image classification unit, configured to perform image classification on each image in the case reporting image data set to obtain first category images and second category images in the case reporting image data set, wherein the vehicle information completeness of the first category images is greater than the vehicle information completeness of the second category images; A first classification unit is used to perform vehicle classification on the first category image to obtain a vehicle classification result of each first vehicle detection frame in the first category image, wherein the vehicle classification result is a vehicle detection frame of a target vehicle as the target vehicle detection frame, and the vehicle classification result is a vehicle detection frame of three vehicles as the three vehicle detection frame; an identification unit, configured to perform vehicle color identification on the first category image to obtain a first vehicle color of a target vehicle detection frame in the first category image and a second vehicle color of a third vehicle detection frame in the first category image; The second classification unit is used to perform vehicle classification on the second category image based on the first vehicle color and the second vehicle color, and obtain a vehicle classification result of each second vehicle detection frame in the second category image.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the method for classifying vehicles in an image according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the method for classifying vehicles in an image as claimed in any one of claims 1 to 7.

Images