Vehicle body color identification method and device based on deep learning model

Abstract

The invention provides a vehicle body color recognition method and device based on a deep learning model, and solves the problem of low vehicle body color recognition accuracy in the prior art. A deep learning network model is pre-trained and trained by successively constructing a data set for pre-training and a data set for training, and the similarity between a vehicle image uploaded by a user and each standard color block image in a standard color block image database is calculated through the trained deep learning network model. And according to the obtained similarity, the to-be-recognized vehicle body color recognition result is output, and the accuracy of vehicle body recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a vehicle body color recognition method and device based on deep learning model. BACKGROUND

[0002] At present, computer vision is widely applied to various industries, especially the online intelligent examination of materials in the government field, and the correctness and running performance of the algorithm directly affect the intelligent examination result. Color as an image feature point needs to be identified and distinguished in many scenes. For example, whether the color of the vehicle on the picture matches the color marked in the certificate in the vehicle examination; whether the background of the ID photo meets the current requirements of the materials, etc. At present, the color recognition of the image mainly uses a model to locate the target area, and then uses the RGB or HSV color model to identify the pixel points in the area.

[0003] The color of the image taken by the camera depends on the photosensitive device, and the corresponding color of the same object in different environments and with different shooting devices is different when converted into RGB or HSV values. It is difficult to handle non-single color targets, and the final recognition result has a large error. If a large number of algorithms and rules are used to increase the processing steps to increase the recognition accuracy, the running performance will be affected.

[0004] At present, the general method for target color recognition is to first use a model to locate the target area, and then intercept the target area. Considering that the colors in the target area are not single colors, the target area needs to be divided into blocks, and after the division, the RGB or HSV values are directly used to judge the color of the image for each block. Finally, the final color type is judged based on the rules, and the rule generally uses the pixel point with the most colors to determine the color of the block. This method has low accuracy, does not consider the influence of environment and shooting conditions on the color of the target, and pixel point traversal also affects the running efficiency. SUMMARY

[0005] The present application provides a vehicle body color recognition method and device based on deep learning model to solve the problem of low accuracy of vehicle body color recognition in the prior art.

[0006] In a first aspect, the present application provides a vehicle body color recognition method based on deep learning model, which specifically includes the following steps:

[0007] Step S1, constructing an image pair data set for pre-training of a deep learning model;

[0008] Step S2, a deep learning network model is constructed, and the deep learning network model is pre-trained to form a pre-trained deep learning network model; wherein the deep learning network model comprises a VIT (Vision Transformer, vision encoder) image encoder and a fully connected linear neural network connected in sequence;

[0009] Step S3, an image pair data set for color recognition is constructed.

[0010] Step S4, the image pair data set for color recognition is input into the pre-trained deep learning network model for model training to form a trained deep learning network model.

[0011] Step S5, a color patch database containing a plurality of color patches is constructed, and a similarity between each color patch and a vehicle image is formed according to the color patch database, the VIT image encoder and the user uploaded vehicle image.

[0012] Step S6, according to the similarity between each color patch and the vehicle image, an identification result of the vehicle body color in the vehicle image is formed.

[0013] Preferably, in step S1, an image pair data set for deep learning model pre-training is constructed, specifically comprising the following steps:

[0014] Step S101, obtaining a vehicle image to be recognized, a plurality of (in this application, the "plurality" means at least two) environment background images not containing a vehicle, and a plurality of pure color background images;

[0015] Step S102, labeling the vehicle region in the vehicle image, taking any one of the plurality of environment background images or the plurality of pure color backgrounds as the background to form images of the same vehicle in different backgrounds;

[0016] Step S103, constructing an image pair according to any two images of the same vehicle in different backgrounds; wherein the image pair comprises a positive sample image pair and a negative sample image pair, the positive sample image pair represents two images containing the same vehicle and different backgrounds, and the negative sample image pair represents one image containing a vehicle and one image not containing a vehicle, and the backgrounds of the two images are the same;

[0017] Step S104, repeating steps S101-S103 to form a plurality of vehicle image pairs in different backgrounds to form a pre-trained image pair data set.

[0018] Preferably, in step S2, a deep learning network model is constructed, and the deep learning network model is pre-trained to form a pre-trained deep learning network model, specifically including the following steps:

[0019] Step S201, constructing a deep learning network model;

[0020] Step S202, inputting an image pair (positive sample image pair or negative sample image pair) into the VIT image encoder to form a feature vector h = {h i ,h j};

[0021] Step S203, inputting the feature vector h into a fully connected linear neural network to form a vector z = {z i ,z j};

[0022] Step S204, calculating the similarity and contrast loss of the current image pair according to the vector z, and sequentially performing back propagation through the fully connected linear neural network and the VIT image encoder;

[0023] Step S205, repeatedly performing steps S202-S204 to minimize the contrast loss, and optimizing the contrast loss of the positive sample image pair in the pre-training image pair data set until all image pairs in the pre-training image pair data set are inputted to form a pre-trained deep learning network model.

[0024] Wherein, i, j are positive integers.

[0025] Preferably, in step S204, the calculation of the similarity is specifically represented as follows:

[0026]

[0027] Wherein, t represents a proportional adjustment parameter, t>1; i, j, k are positive integers, 1≤i,j,k≤2N; z i and z j represent the vector space of the positive sample image pair; 1 [k≠i and k≠j] is 1 when k is not equal to i and j, and 0 when k is equal to i or j; 2N represents the total number of samples, N is a positive integer; sim(z i ,z j ) represents the similarity between the vectors formed by the images in the positive sample image pair, and sim(z i ,z j ) represents the similarity between the vectors formed by the images in the negative sample image pair; l i,j represents the contrast loss of the i-th image and the j-th image which can form a positive sample image pair with the i-th image.

[0028] Preferably, in step S3, the image pair data set for color recognition is constructed, specifically comprising the following steps:

[0029] Step S301, label and intercept the vehicle body color block to form color block data;

[0030] Step S302, pair the color block data and the image containing the vehicle to form an image pair data set for color recognition; wherein the image pair data set for color recognition includes positive sample color image pairs and negative sample color image pairs, the positive sample color image pairs represent that the vehicle body color and the color block color are the same, and the negative sample color image pairs represent that the vehicle body color and the color block color are different.

[0031] Preferably, in step S301, the vehicle body color block is labeled and intercepted to form color block data, specifically comprising the following steps:

[0032] Step S301a, label the first region capable of representing the vehicle body color through labeling software;

[0033] Step S301b, when the maximum inscribed rectangle proportion of the first region exceeds 60%, the vehicle body color block data is formed by intercepting through the maximum inscribed rectangle, otherwise, the vehicle body color block data is formed by traversing the pixel points of the first region.

[0034] Preferably, in order to increase the diversity of data and improve the generalization ability of the model, color block data can also be generated according to the preset RGB value or HSV value.

[0035] Preferably, in step S4, the image pair data set for color recognition is input into the pre-trained deep learning network model for model training to form a trained deep learning network model, specifically comprising the following steps:

[0036] Step S401, input a sample color image pair (positive sample color image pair or negative sample color image pair) into the VIT image encoder to form a feature vector h={h i ,h j};

[0037] Step S402, input the feature vector h into the fully connected linear neural network to form a vector z={z i ,z j};

[0038] Step S403, calculate the similarity and contrast loss of the current image pair according to the vector z, and sequentially perform back propagation through the fully connected linear neural network and the VIT image encoder;

[0039] Step S404, steps S401-S403 are repeatedly performed to minimize the contrast loss, and the contrast loss of the positive sample color image pair in the image pair data set for color recognition is optimized until all the image pairs in the image pair data set for color recognition are input, forming a trained deep learning network model.

[0040] Preferably, in step S5, a color block database containing a plurality of color blocks is constructed, and similarity between each color block and the vehicle image is formed according to the color block database, the deep learning network model and the vehicle image uploaded by the user, specifically including the following steps:

[0041] Step S501, a color block database containing a plurality of color blocks is constructed, and each color block is input into the VIT image encoder to form a plurality of color block feature vectors;

[0042] Step S502, the vehicle image uploaded by the user is input into the VIT image encoder to form a feature vector ha;

[0043] Step S503, the feature vector ha and each color block feature vector are respectively calculated for similarity to form similarity between each color block and the vehicle image.

[0044] Preferably, in step S6, the recognition result of the vehicle body color in the vehicle image is formed according to the similarity between each color block and the vehicle image, specifically including the following steps:

[0045] Step S601, similarity between each color block and the vehicle image is sorted from high to low;

[0046] Step S602, the top three similarities are normalized by a softmax function and a threshold is set, and the color of the color block corresponding to the normalized value greater than the threshold is taken as the vehicle body color in the vehicle image (i.e., the recognition result of the vehicle body color in the vehicle image is formed).

[0047] In a second aspect, the present application also provides a vehicle body color recognition device based on a deep learning model, specifically including the following modules:

[0048] A model pre-training data set construction module is configured to construct an image pair data set for pre-training of a deep learning model;

[0049] The model construction and pre-training module is configured to construct a deep learning network model and pre-train the deep learning network model to form a pre-trained deep learning network model; wherein the deep learning network model comprises a VIT (Vision Transformer) image encoder and a fully connected linear neural network connected in sequence.

[0050] The model training data set construction module is configured to construct an image pair data set for color recognition.

[0051] The model training module is configured to input the image pair data set for color recognition into the pre-trained deep learning network model for model training to form a trained deep learning network model.

[0052] The similarity calculation module is configured to construct a color patch database comprising a plurality of color patches, and form a similarity between each color patch and a vehicle image according to the color patch database, the VIT image encoder and the vehicle image uploaded by a user.

[0053] The color recognition result generation module is configured to form a recognition result of a vehicle body color in the vehicle image according to the similarity between each color patch and the vehicle image.

[0054] Preferably, the model pre-training data set construction module specifically comprises the following sub-modules:

[0055] The model pre-training data set construction first sub-module is configured to obtain a vehicle image to be recognized, a plurality of environment background images not containing vehicles and a plurality of pure color background images.

[0056] The model pre-training data set construction second sub-module is configured to label a vehicle region in the vehicle image, and form images of the same vehicle in different backgrounds by taking any one of the plurality of environment background images or the plurality of pure color backgrounds as a background.

[0057] The model pre-training data set construction third sub-module is configured to construct an image pair according to any two images of the same vehicle in different backgrounds; wherein the image pair comprises a positive sample image pair and a negative sample image pair, the positive sample image pair represents two images containing the same vehicle and having different backgrounds, and the negative sample image pair represents one image containing a vehicle and one image not containing a vehicle, and the backgrounds of the two images are the same.

[0058] The model pre-training data set construction fourth sub-module is configured to repeatedly execute the model pre-training data set construction first sub-module, the model pre-training data set construction second sub-module and the model pre-training data set construction third sub-module to form a plurality of image pairs of vehicles in different backgrounds and form a pre-trained image pair data set.

[0059] Preferably, the model building and pre-training module specifically includes the following submodules:

[0060] Model construction and pre-training submodule 1, used to build deep learning network models;

[0061] The second submodule of model construction and pre-training is used to input an image pair (positive sample image pair or negative sample image pair) into the VIT image encoder to form a feature vector h = {h i ,h j};

[0062] The third submodule of model construction and pre-training is used to input the feature vector h into a fully connected linear neural network to form a vector z={z i ,z j};

[0063] The fourth submodule of model construction and pre-training is used to calculate the similarity and contrast loss of the current image pair based on the vector z, and perform backpropagation through the fully connected linear neural network and the VIT image encoder in sequence;

[0064] The fifth submodule of model construction and pre-training is used to repeatedly execute the second submodule of model construction and pre-training, the third submodule of model construction and pre-training, and the fourth submodule of model construction and pre-training to minimize the contrast loss and optimize the contrast loss of the positive sample image pairs in the image pair dataset for pre-training until all image pairs in the image pair dataset for pre-training are input to form a pre-trained deep learning network model.

[0065] Among them, i and j are both positive integers.

[0066] Preferably, in the fourth submodule of model construction and pre-training, the calculation of the similarity is specifically expressed as follows:

[0067]

[0068] Where t represents the proportional adjustment parameter, t>1; i, j, k are all positive integers, 1≤i,j,k≤2N; z i and z j Represents the vector space of positive sample image pairs; 1 [k≠i] It means that when k is not equal to i, it is 1, and when k is equal to i, it is 0; 2N represents the total number of samples, N is a positive integer; sim(z i ,z j ) represents the similarity between the vectors formed by each image in the positive sample image pair, sim(z i ,z j) represents the similarity between the vectors formed by the images in the negative sample image pair; l i,j represents the contrast loss of the i-th image and the j-th image that can form a positive sample image pair.

[0069] Preferably, the model training dataset construction module specifically comprises the following sub-modules:

[0070] The model training dataset construction first sub-module is configured to label and intercept the vehicle body color patches to form color patch data.

[0071] The model training dataset construction second sub-module is configured to pair the color patch data with the images containing vehicles to form an image pair dataset for color recognition. The image pair dataset for color recognition comprises positive sample color image pairs and negative sample color image pairs. The positive sample color image pairs represent the same vehicle body color and color patch color, and the negative sample color image pairs represent different vehicle body color and color patch color.

[0072] Preferably, the model training dataset construction first sub-module specifically comprises the following units:

[0073] The first unit is configured to label the first region capable of representing the vehicle body color by using a labeling software.

[0074] The second unit is configured to intercept the first region by using the maximum inscribed rectangle to form the vehicle body color patch data when the maximum inscribed rectangle proportion of the first region exceeds 60%, or to form the vehicle body color patch data by traversing the pixel points of the first region.

[0075] Preferably, in order to increase the diversity of data and improve the generalization ability of the model, color patch data can also be generated according to the preset RGB value or HSV value.

[0076] Preferably, the model training module specifically comprises the following sub-modules:

[0077] The model training first sub-module is configured to input one sample color image pair (positive sample color image pair or negative sample color image pair) into the VIT image encoder to form a feature vector h = {h i ,h j}.

[0078] The model training second sub-module is configured to input the feature vector h into a fully connected linear neural network to form a vector z = {z i ,z j}.

[0079] The model training third sub-module is configured to calculate the similarity and the contrast loss of the current image pair according to the vector z, and sequentially perform back propagation through the fully connected linear neural network and the VIT image encoder.

[0080] The model training fourth sub-module is configured to repeatedly execute the model training first sub-module, the model training second sub-module and the model training third sub-module to minimize the contrast loss, optimize the contrast loss of the positive sample color image pair in the image pair data set for color recognition, until all the image pairs in the image pair data set for color recognition are input, and a trained deep learning network model is formed.

[0081] Preferably, the similarity calculation module specifically comprises the following sub-modules:

[0082] The similarity calculation first sub-module is configured to construct a color block database containing a plurality of color blocks, and input each color block into the VIT image encoder to form a plurality of color block feature vectors.

[0083] The similarity calculation second sub-module is configured to input the user-uploaded vehicle image into the VIT image encoder to form a feature vector ha.

[0084] The similarity calculation third sub-module is configured to perform similarity calculation on the feature vector ha and each color block feature vector respectively to form the similarity between each color block and the vehicle image.

[0085] Preferably, the color recognition result generation module specifically comprises the following sub-modules:

[0086] The color recognition result generation first sub-module is configured to sort the similarity between each color block and the vehicle image from high to low.

[0087] The color recognition result generation second sub-module is configured to normalize the top three similarities by using a softmax function and set a threshold, and take the color of the color block corresponding to the normalized value greater than the threshold as the vehicle body color in the vehicle image.

[0088] In a third aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the vehicle body color recognition method based on the deep learning model according to any one of the first aspect.

[0089] In a fourth aspect, the present application further provides an electronic device, comprising a memory storing a computer program, and a processor connected to the memory and configured to execute the computer program to implement the method for identifying vehicle body color based on a deep learning model according to any one of the first aspect.

[0090] Compared with the prior art, the present application has the following obvious and substantial features and advantages:

[0091] The present application provides a method and device for identifying vehicle body color based on a deep learning model, which solves the problem of low accuracy of vehicle body color identification in the prior art. The deep learning network model is pre-trained and trained by successively constructing a data set for pre-training and a data set for training, and the similarity between a vehicle image uploaded by a user and each standard color block image in a standard color block image database is calculated by using the trained deep learning network model. According to the obtained similarity, the identification result of the vehicle body color to be identified is output, thereby improving the accuracy of vehicle body identification. BRIEF DESCRIPTION OF DRAWINGS

[0092] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments thereof and their descriptions serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0093] Figure 1 is a flow chart of a method for identifying vehicle body color based on a deep learning model according to a preferred embodiment of the present application.

[0094] Figure 2 is a schematic diagram of an image pair data set for pre-training of a deep learning model according to a preferred embodiment of the present application.

[0095] Figure 3 is a schematic diagram of a positive and negative sample image pair data according to a preferred embodiment of the present application.

[0096] Figure 4 is a schematic diagram of a deep learning network model structure according to a preferred embodiment of the present application.

[0097] Figure 5 is a schematic diagram of a color block generated based on an RGB value according to a preferred embodiment of the present application.

[0098] Figure 6 is a schematic diagram of color block extraction according to a preferred embodiment of the present application.

[0099] Figure 7 is a schematic diagram of an image pair of a color identification sample according to a preferred embodiment of the present application.

[0100] Figure 8A structure schematic diagram of a vehicle body color recognition device based on a deep learning model is a preferred embodiment of the present application.

[0101] Figure 9 A structure schematic diagram of a model pre-training data set construction module is a preferred embodiment of the present application.

[0102] Figure 10 A structure schematic diagram of a model construction and pre-training module is a preferred embodiment of the present application.

[0103] Figure 11 A structure schematic diagram of a model training data set construction module is a preferred embodiment of the present application.

[0104] Figure 12 A structure schematic diagram of a first sub-module of a model training data set construction module is a preferred embodiment of the present application.

[0105] Figure 13 A structure schematic diagram of a model training module is a preferred embodiment of the present application.

[0106] Figure 14 A structure schematic diagram of a similarity calculation module is a preferred embodiment of the present application.

[0107] Figure 15 A structure schematic diagram of a color recognition result generation module is a preferred embodiment of the present application.

[0108] Reference signs:

[0109] 100, model pre-training data set construction module; 200, model construction and pre-training module; 300, model training data set construction module; 400, model training module; 500, similarity calculation module; 600, color recognition result generation module.

[0110] 110, first sub-module of model pre-training data set construction; 120, second sub-module of model pre-training data set construction; 130, third sub-module of model pre-training data set construction; 140, fourth sub-module of model pre-training data set construction;

[0111] 210, first sub-module of model construction and pre-training; 220, second sub-module of model construction and pre-training; 230, third sub-module of model construction and pre-training; 240, fourth sub-module of model construction and pre-training; 250, fifth sub-module of model construction and pre-training;

[0112] 310, first sub-module of model training data set construction; 320, second sub-module of model training data set construction;

[0113] 311, first unit; 312, second unit;

[0114] 410、model training first sub-module; 420、model training second sub-module; 430、model training third sub-module; 440、model training fourth sub-module;

[0115] 510、similarity calculation first sub-module; 520、similarity calculation second sub-module; 530、similarity calculation third sub-module;

[0116] 610、color recognition result generation first sub-module; 620、color recognition result generation second sub-module. DETAILED DESCRIPTION

[0117] The application provides a vehicle body color recognition method and device based on a deep learning model. To make the purpose, technical solution and effect of the application more clear and explicit, the application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0118] It should be noted that the terms "first", "second" and the like in the specification and claims of the application and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0119] Example 1:

[0120] As shown in the figure, the vehicle body color recognition method based on the deep learning model described in the embodiment specifically includes steps S1-S6. Figures 1-7

[0121] Step S1, constructing an image pair data set for pre-training of a deep learning model.

[0122] Optionally, the step S1 specifically includes steps S101-S104.

[0123] Step S101, obtaining a vehicle image to be recognized, a plurality of environment background images not containing vehicles and a plurality of pure color background images, as shown in the figure. Figure 2

[0124] Step S102, labeling the vehicle region in the vehicle image, taking any one of the plurality of environment background images or the plurality of pure color backgrounds as the background, and forming the image of the same vehicle in different backgrounds.​​

[0125] Step S103, constructing an image pair according to any two images of the same vehicle in different backgrounds; wherein the image pair includes a positive sample image pair and a negative sample image pair, the positive sample image pair represents two images containing the same vehicle and different backgrounds, and the negative sample image pair represents one image containing a vehicle and one image not containing a vehicle, and the backgrounds of the two images are the same, as shown in Figure 3 .

[0126] Step S104, repeatedly performing steps S101-S103 to form a plurality of image pairs of vehicles in different backgrounds to form a pre-trained image pair dataset.

[0127] Step S2, constructing a deep learning network model and pre-training the deep learning network model to form a pre-trained deep learning network model; wherein the deep learning network model includes a VIT image encoder and a fully connected linear neural network connected in sequence, as shown in Figure 4 .

[0128] Optionally, the step S2 specifically includes steps S201-S205.

[0129] Step S201, constructing a deep learning network model.

[0130] Step S202, inputting an image pair (positive sample image pair or negative sample image pair) into the VIT image encoder (in the specific implementation of the embodiment, before the image pair is input into the VIT image encoder, all images can also be scaled, and small-pixel images can accelerate the training and inference speed of the subsequent deep learning network model, the images are scaled to a uniform size, and the size is preferably (224, 224)). i j ; wherein i and j are positive integers.

[0131] Step S203, inputting the feature vector h into the fully connected linear neural network to form a vector z = {z i j .

[0132] Step S204, calculating the similarity and contrast loss of the current image pair according to the vector z, and sequentially performing back propagation through the fully connected linear neural network and the VIT image encoder.

[0133] In the specific implementation of the embodiment, the calculation of the similarity is specifically represented as follows:

[0134] ​​

[0135] wherein t represents a proportion adjustment parameter, t>1; i, j, k are all positive integers, 1≤i,j,k≤2N; z i and z j represent a vector space of a positive sample image pair; 1 [k≠i and k≠j] represents 1 when k is not equal to i and j, and 0 when k is equal to i or j; 2N represents the total number of samples, N is a positive integer; sim(z i ,z j ) represents the similarity between the vectors formed by the images in the positive sample image pair, sim(z i ,z j ) represents the similarity between the vectors formed by the images in the negative sample image pair; l i,j represents the contrast loss of the i-th image and the j-th image which can form a positive sample image pair with the i-th image, and the contrast loss decreases (decreases) when the feature vectors from the positive sample image pair are similar.

[0136] Step S205, steps S202-S204 are repeatedly executed to minimize the contrast loss, and the contrast loss of the positive sample image pair in the image pair data set for pre-training is optimized until all image pairs in the image pair data set for pre-training are input, forming a pre-trained deep learning network model.

[0137] Step S3, constructing an image pair data set for color recognition.

[0138] Optionally, the step S3 specifically comprises steps S301-S302.

[0139] Step S301, labeling and intercepting a car body color block to form a color block data. In the specific implementation of the embodiment, the region capable of representing the car body color can be labeled and intercepted by labeling software.

[0140] Optionally, the step S301 specifically comprises steps S301a-S301b.

[0141] Step S301a, labeling the first region capable of representing the car body color by labeling software.

[0142] Step S301b, when the proportion of the largest inscribed rectangle of the first region exceeds 60%, the car body color block data is formed by intercepting the largest inscribed rectangle, otherwise, the car body color block data is formed by traversing the pixel points of the first region.

[0143] In step S302, the color patch data and the image containing the vehicle are paired to form an image pair data set for color recognition. The image pair data set for color recognition includes positive sample color image pairs and negative sample color image pairs. The positive sample color image pairs represent the same color between the vehicle body color and the color patch color. The negative sample color image pairs represent different colors between the vehicle body color and the color patch color.

[0144] Optionally, as shown in Figure 5 To increase the diversity of data and improve the generalization ability of the model, color patch data can also be generated according to the preset RGB value or HSV value.

[0145] In addition, in the specific implementation of the embodiment, the color patch can also add some images close to the whole vehicle, as shown in Figure 6 The background is as little as possible in the color patch, which helps to improve the accuracy of subsequent vehicle body color recognition.

[0146] In step S4, the image pair data set for color recognition is input into the pre-trained deep learning network model for model training to form a trained deep learning network model.

[0147] Optionally, the step S4 has a specific implementation process, which is similar to the step S2 (i.e., the pre-training step of the deep learning network model is similar, and the input image pair data is different), and specifically includes steps S401-S404.

[0148] In step S401, a sample color image pair (positive sample color image pair or negative sample color image pair (as shown in Figure 7 )) is input into the VIT image encoder to form a feature vector h = {h i ,h j}.

[0149] In step S402, the feature vector h is input into a fully connected linear neural network to form a vector z = {z i ,z j}.

[0150] In step S403, the similarity and contrast loss of the current image pair are calculated according to the vector z, and the back propagation is sequentially performed through the fully connected linear neural network and the VIT image encoder.

[0151] In step S404, steps S401-S403 are repeatedly executed to minimize the contrast loss, optimize the contrast loss of the positive sample color image pairs in the image pair data set for color recognition, until all the image pairs in the image pair data set for color recognition are input to form a trained deep learning network model.

[0152] Step S5, constructing a color tile database containing a plurality of color tiles, forming a similarity between each color tile and the vehicle image according to the color tile database, the VIT image encoder and the vehicle image uploaded by the user.

[0153] Optionally, the step S5 specifically comprises steps S501-S503.

[0154] Step S501, constructing a color tile database containing a plurality of color tiles, inputting each color tile into the VIT image encoder respectively to form a plurality of color tile feature vectors.

[0155] Step S502, inputting the vehicle image uploaded by the user into the VIT image encoder to form a feature vector ha.

[0156] Step S503, respectively performing similarity calculation on the feature vector ha and each color tile feature vector to form a similarity between each color tile and the vehicle image.

[0157] Step S6, forming a recognition result of the vehicle body color in the vehicle image according to the similarity between each color tile and the vehicle image.

[0158] Optionally, the step S6 specifically comprises steps S601-S602.

[0159] Step S601, sorting the similarity between each color tile and the vehicle image from high to low.

[0160] Step S602, performing normalization processing on the top three similarities by using a softmax function and setting a threshold, and taking the color of the color tile corresponding to the normalized result greater than the threshold as the vehicle body color in the vehicle image.

[0161] In the specific implementation of the embodiment, if the output vehicle body color is one kind, the top 3 results with the highest similarity (several can be adjusted according to actual conditions) are normalized by using a softmax function, and finally the normalized result is obtained. For example, the top three scores are [0.9 (black), 0.85 (black), 0.6 (blue)], and after normalization by using a softmax function, they are [0.372 (black), 0.353 (black), 0.275 (blue)], that is, the score of black is 0.725 and the score of blue is 0.275. A threshold is set, and we think that the scores greater than 0.3 are all the vehicle body color of the user, and the result is that the vehicle body color in the vehicle image uploaded by the user is black.

[0162] If the output vehicle body color can be multiple (such as 3), that is, only three colors with larger area are recorded for multi-color vehicles. For multi-color vehicles, corresponding data can be added in the color block database. The similarity score can be processed by taking the highest score of each color block. A threshold value such as 0.5 is set. As long as the threshold value is exceeded, it is considered that the vehicle body contains the color. The color recorded on the certificate is considered to match the color on the vehicle picture uploaded by the user and the color marked in the certificate, which meets the requirements of the certificate.

[0163] Example 2:

[0164] As shown in Figures 8-15 The vehicle body color recognition device based on the deep learning model comprises a model pre-training data set construction module 100, a model construction and pre-training module 200, a model training data set construction module 300, a model training module 400, a similarity calculation module 500, and a color recognition result generation module 600.

[0165] The model pre-training data set construction module 100 is used to construct an image pair data set for deep learning model pre-training.

[0166] The model pre-training data set construction module 100 comprises a model pre-training data set construction first sub-module 110, a model pre-training data set construction second sub-module 120, a model pre-training data set construction third sub-module 130, and a model pre-training data set construction fourth sub-module 140.

[0167] The model pre-training data set construction first sub-module 110 is used to obtain a vehicle image to be recognized, a plurality of environment background images not containing a vehicle, and a plurality of pure color background images.

[0168] The model pre-training data set construction second sub-module 120 is used to label the vehicle region in the vehicle image, and use any one of the plurality of environment background images or the plurality of pure color backgrounds as a background to form an image of the same vehicle in different backgrounds.

[0169] The model pre-training data set construction third sub-module 130 is used to construct an image pair according to any two images of the same vehicle in different backgrounds. The image pair comprises a positive sample image pair and a negative sample image pair. The positive sample image pair indicates that it comprises two images containing the same vehicle and different backgrounds. The negative sample image pair indicates that it comprises an image containing a vehicle and an image not containing a vehicle, and the backgrounds of the two images are the same.

[0170] The model pre-training data set construction fourth sub-module 140 is used for repeatedly executing the model pre-training data set construction first sub-module 110, the model pre-training data set construction second sub-module 120 and the model pre-training data set construction third sub-module 130, forms the image pair of a plurality of vehicles in different backgrounds, and forms the pre-trained image pair data set.

[0171] The model construction and pre-training module 200 is used for constructing a deep learning network model, pre-training the deep learning network model, and forming a pre-trained deep learning network model; wherein the deep learning network model comprises a VIT image encoder and a fully connected linear neural network connected in sequence.

[0172] The model construction and pre-training module 200 comprises a model construction and pre-training first sub-module 210, a model construction and pre-training second sub-module 220, a model construction and pre-training third sub-module 230, a model construction and pre-training fourth sub-module 240 and a model construction and pre-training fifth sub-module 250.

[0173] The model construction and pre-training first sub-module 210 is used for constructing a deep learning network model.

[0174] The model construction and pre-training second sub-module 220 is used for inputting one image pair (a positive sample image pair or a negative sample image pair) into the VIT image encoder to form a feature vector h={h i ,h j}; wherein i and j are positive integers.

[0175] The model construction and pre-training third sub-module 230 is used for inputting the feature vector h into a fully connected linear neural network to form a vector z={z i ,z j}.

[0176] The model construction and pre-training fourth sub-module 240 is used for calculating the similarity and contrast loss of the current image pair according to the vector z, and sequentially performing back propagation through the fully connected linear neural network and the VIT image encoder.

[0177] The similarity is calculated as follows:

[0178]

[0179] Wherein t represents a proportional adjustment parameter, t>1; i, j and k are positive integers, 1≤i,j,k≤2N; z i and z j represent the vector space of the positive sample image pair; 1 [k≠i]sim(z i ,z j ) represents the similarity between the vectors formed by the images in the positive sample image pair, and sim(z i ,z j ) represents the similarity between the vectors formed by the images in the negative sample image pair; l i,j represents the contrast loss of the i-th image and the j-th image that can form a positive sample image pair with it.

[0180] The model construction and pre-training fifth sub-module 250 is configured to repeatedly execute the model construction and pre-training second sub-module 220, the model construction and pre-training third sub-module 230, and the model construction and pre-training fourth sub-module 240 to minimize the contrast loss and optimize the contrast loss of the positive sample image pairs in the image pair data set for pre-training until all image pairs in the image pair data set for pre-training are input to form a pre-trained deep learning network model.

[0181] The model training data set construction module 300 is configured to construct an image pair data set for color recognition.

[0182] The model training data set construction module 300 specifically includes a model training data set construction first sub-module 310 and a model training data set construction second sub-module 320.

[0183] The model training data set construction first sub-module 310 is configured to label and intercept a vehicle body color block to form color block data.

[0184] The model training data set construction first sub-module 310 specifically includes a first unit 311 and a second unit 312.

[0185] The first unit 311 is configured to label a first region capable of representing a vehicle body color by using a labeling software.

[0186] The second unit 312 is configured to intercept a maximum inscribed rectangle of the first region to form vehicle body color block data when the maximum inscribed rectangle of the first region accounts for more than 60%, and otherwise, to traverse pixel points of the first region to form vehicle body color block data.

[0187] Optionally, in order to increase the diversity of data and improve the generalization ability of the model, color block data can also be generated according to preset RGB values or HSV values.

[0188] The model training data set construction second sub-module 320 is configured to pair the color block data and the image containing the vehicle to form an image pair data set for color recognition; wherein the image pair data set for color recognition includes positive sample color image pairs and negative sample color image pairs, the positive sample color image pairs represent that the vehicle body color and the color block color are the same, and the negative sample color image pairs represent that the vehicle body color and the color block color are different.

[0189] The model training module 400 is configured to input the image pair data set for color recognition into the pre-trained deep learning network model to perform model training, and form a trained deep learning network model.

[0190] The model training module 400 specifically includes a model training first sub-module 410, a model training second sub-module 420, a model training third sub-module 430 and a model training fourth sub-module 440.

[0191] The model training first sub-module 410 is configured to input one sample color image pair (a positive sample color image pair or a negative sample color image pair) into the VIT image encoder to form a feature vector h={h i ,h j}.

[0192] The model training second sub-module 420 is configured to input the feature vector h into a fully connected linear neural network to form a vector z={z i ,z j}.

[0193] The model training third sub-module 430 is configured to calculate the similarity and contrast loss of the current image pair according to the vector z, and sequentially perform back propagation through the fully connected linear neural network and the VIT image encoder.

[0194] The model training fourth sub-module 440 is configured to repeatedly execute the model training first sub-module 410, the model training second sub-module 420 and the model training third sub-module 430 to minimize the contrast loss, optimize the contrast loss of the positive sample color image pairs in the image pair data set for color recognition, and form a trained deep learning network model until all the image pairs in the image pair data set for color recognition are input.

[0195] The similarity calculation module 500 is configured to construct a color block database containing a plurality of color blocks, and form the similarity between each color block and the vehicle image according to the color block database, the VIT image encoder and the vehicle image uploaded by the user.

[0196] The similarity calculation module 500 specifically comprises a similarity calculation first submodule 510, a similarity calculation second submodule 520 and a similarity calculation third submodule 530.

[0197] The similarity calculation first submodule 510 is configured to construct a color block database comprising a plurality of color blocks, input each color block into the VIT image encoder respectively, and form a plurality of color block feature vectors.

[0198] The similarity calculation second submodule 520 is configured to input the user-uploaded vehicle image into the VIT image encoder and form a feature vector ha.

[0199] The similarity calculation third submodule 530 is configured to perform similarity calculation on the feature vector ha and each color block feature vector respectively, and form the similarity between each color block and the vehicle image.

[0200] The color recognition result generation module 600 is configured to form the recognition result of the vehicle body color in the vehicle image according to the similarity between each color block and the vehicle image.

[0201] The color recognition result generation module 600 specifically comprises a color recognition result generation first submodule 610 and a color recognition result generation second submodule 620.

[0202] The color recognition result generation first submodule 610 is configured to sort the similarity between each color block and the vehicle image from high to low.

[0203] The color recognition result generation second submodule 620 is configured to normalize the top three similarities by using a softmax function and set a threshold, and take the color of the color block corresponding to the normalized value greater than the threshold as the vehicle body color in the vehicle image.

[0204] The above describes the specific embodiments of the present application in detail, but it is only as an example, and the present application is not limited to the above-described specific embodiments. For those skilled in the art, any equivalent modification and replacement of the present application are also within the scope of the present application. Therefore, any equivalent transformation and modification without departing from the spirit and scope of the present application should be covered within the scope of the present application.

Claims

1. A vehicle body color recognition method based on a deep learning model, characterized in that: The specific steps include: Step S1: construct an image pair dataset for pre-training of a deep learning model; Step S2: constructing a deep learning network model and pre-training the deep learning network model to form a pre-trained deep learning network model; wherein the deep learning network model includes a VIT image encoder and a fully connected linear neural network connected in sequence; Step S3: constructing an image pair dataset for color recognition; Step S4: inputting the image pair data set for color recognition into the pre-trained deep learning network model for model training to form a trained deep learning network model; Step S5: constructing a color patch database comprising a plurality of color patches, and calculating a similarity between each color patch and the vehicle image based on the color patch database, the VIT image encoder, and the vehicle image uploaded by the user; Step S6: generating a recognition result of the vehicle body color in the vehicle image based on the similarity between each color block and the vehicle image.

2. The vehicle body color recognition method based on a deep learning model according to claim 1, characterized in that: In step S1, an image pair dataset for deep learning model pre-training is constructed, which specifically includes the following steps: Step S101: obtaining a vehicle image to be identified, a plurality of environment background images excluding the vehicle, and a plurality of images of a solid color background; Step S102: marking the vehicle area in the vehicle image, using any one of the multiple environmental background images or the multiple pure color background images as a background, to form images of the same vehicle under different backgrounds; Step S103: constructing an image pair based on any two images of the same vehicle in different backgrounds; wherein the image pair includes a positive sample image pair and a negative sample image pair, wherein the positive sample image pair includes two images containing the same vehicle but with different backgrounds, and the negative sample image pair includes one image containing the vehicle and one image not containing the vehicle, and the two images have the same background; Step S104 : Repeat steps S101 to S103 to generate multiple image pairs of vehicles under different backgrounds, thereby forming a pre-trained image pair dataset.

3. The vehicle body color recognition method based on a deep learning model according to claim 1, characterized in that: In step S2, a deep learning network model is constructed and pre-trained to form a pre-trained deep learning network model, which specifically includes the following steps: Step S201: construct a deep learning network model; Step S202: Input an image pair into the VIT image encoder to form a feature vector h = {h i ,h j }; Step S203: Input the feature vector h into a fully connected linear neural network to form a vector z={z i ,z j }; Step S204: Calculate the similarity and contrast loss of the current image pair based on the vector z, and perform backpropagation through the fully connected linear neural network and the VIT image encoder in sequence; Step S205: Repeat steps S202 to S204 to minimize the contrast loss and optimize the contrast loss of the positive sample image pairs in the image pair dataset for pre-training until all image pairs in the image pair dataset for pre-training are input to form a pre-trained deep learning network model; Among them, i and j are both positive integers.

4. The vehicle body color recognition method based on a deep learning model according to claim 3 is characterized in that: In step S204, the calculation of the similarity is specifically represented as follows: Where t represents the proportional adjustment parameter, t>1; i, j, k are all positive integers, 1≤i,j,k≤2N; z i and z j Represents the vector space of positive sample image pairs; 1 [k≠i] It means that when k is not equal to i, it is 1, and when k is equal to i, it is 0; 2N represents the total number of samples, N is a positive integer; sim(z i ,z j ) represents the similarity between the vectors formed by each image in the positive sample image pair, sim(z i ,z j ) represents the similarity between the vectors formed by each image in the negative sample image pair; l i,j It represents the contrast loss between the i-th image and the j-th image that can form a positive sample image pair with it.

5. The vehicle body color recognition method based on a deep learning model according to claim 1, characterized in that: In step S3, an image pair dataset for color recognition is constructed, which specifically includes the following steps: Step S301: marking and intercepting the vehicle body color block to form color block data; Step S302: Pair the color patch data with the image containing the vehicle to form an image pair dataset for color recognition; wherein the image pair dataset for color recognition includes a positive sample color image pair and a negative sample color image pair, the positive sample color image pair indicates that the vehicle body color and the color patch color are the same, and the negative sample color image pair indicates that the vehicle body color and the color patch color are different.

6. The vehicle body color recognition method based on a deep learning model according to claim 5 is characterized in that: In step S301, the vehicle body color block is marked and intercepted to form color block data, which specifically includes the following steps: Step S301a: marking a first area representing the color of the vehicle body using marking software; Step S301b: when the maximum inscribed rectangle of the first area accounts for more than 60%, the vehicle body color block data is formed by intercepting the maximum inscribed rectangle; otherwise, the vehicle body color block data is formed by traversing the pixels of the first area.

7. The vehicle body color recognition method based on a deep learning model according to claim 1, characterized in that: In step S4, the image pair dataset for color recognition is input into the pre-trained deep learning network model for model training to form a trained deep learning network model, which specifically includes the following steps: Step S401: Input a sample color image pair into the VIT image encoder to form a feature vector h = {h i ,h j }; Step S402: Input the feature vector h into a fully connected linear neural network to form a vector z={z i ,z j }; Step S403: Calculate the similarity and contrast loss of the current image pair based on the vector z, and perform backpropagation through the fully connected linear neural network and the VIT image encoder in sequence; Step S404: Repeat steps S401 to S403 to minimize the contrast loss and optimize the contrast loss of the positive sample color image pairs in the image pair dataset for color recognition until all image pairs in the image pair dataset for color recognition are input to form a trained deep learning network model.

8. The vehicle body color recognition method based on a deep learning model according to claim 1, characterized in that: In step S5, a color patch database containing multiple color patches is constructed, and the similarity between each color patch and the vehicle image is calculated based on the color patch database, the deep learning network model, and the vehicle image uploaded by the user. Specifically, the following steps are included: Step S501: construct a color block database containing multiple color blocks, and input each color block into the VIT image encoder to form multiple color block feature vectors; Step S502: Input the vehicle image uploaded by the user into the VIT image encoder to form a feature vector ha; Step S503 : performing similarity calculation on the feature vector ha and the feature vector of each color block respectively, to form the similarity between each color block and the vehicle image.

9. The vehicle body color recognition method based on a deep learning model according to claim 1, characterized in that: In step S6, based on the similarity between each color block and the vehicle image, a recognition result of the vehicle body color in the vehicle image is formed, which specifically includes the following steps: Step S601: sort the similarities between each color block and the vehicle image from high to low; Step S602: normalize the top three similarities using a softmax function and set a threshold, and use the color of the color block that is larger than the threshold after normalization as the body color in the vehicle image.

10. A vehicle body color recognition device based on a deep learning model, characterized in that: Specifically, it includes the following modules: Model pre-training dataset construction module, used to build image pair datasets for deep learning model pre-training; A model construction and pre-training module is used to construct a deep learning network model and pre-train the deep learning network model to form a pre-trained deep learning network model; wherein the deep learning network model includes a VIT image encoder and a fully connected linear neural network connected in sequence; Model training dataset construction module, used to construct image pair dataset for color recognition; A model training module is used to input the image pair dataset for color recognition into the pre-trained deep learning network model for model training to form a trained deep learning network model; a similarity calculation module, configured to construct a color patch database comprising a plurality of color patches, and calculate a similarity between each color patch and the vehicle image based on the color patch database, the VIT image encoder, and the vehicle image uploaded by the user; The color recognition result generating module is used to form a recognition result of the vehicle body color in the vehicle image according to the similarity between each color block and the vehicle image.

Images