A vehicle color recognition method under white light conditions

Abstract

The present invention discloses a vehicle color recognition method under white light conditions. The present invention mainly includes two parts: (1) A vehicle color correction method under white light conditions is proposed. The method combines the advantages of the RetineX and darkChannel methods, can effectively alleviate the impact caused by low illumination, backlighting and other phenomena, and reduce the problem of reduced recognition rate caused by these adverse environmental factors. (2) A network model based on the spatial attention mechanism is designed. The model allows the network model to automatically focus on the effective area of ​​the vehicle color, increase the weight of the effective area of ​​the vehicle body color, and reduce the weight of the invalid area of ​​the vehicle color, and then use a deep neural network to accurately identify the vehicle color.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a vehicle color recognition method under white light conditions. Background Art

[0002] In recent years, with the rapid development of computer and internet technologies, the number of motor vehicles has skyrocketed, and various types of information, including traffic-related information, have experienced explosive growth. To manage this information more safely and efficiently, intelligent transportation systems have emerged. These systems can be powerfully used in toll booths, parking lots, criminal tracking, and other areas. Vehicle information captured in video images includes license plate numbers, color, and model. Vehicle color recognition plays a crucial role in roadside monitoring and is an indispensable component of vehicle information. When license plate numbers are unrecognizable or indistinguishable, other vehicle information, such as vehicle color, becomes a basis for distinguishing vehicles. For example, in the identification of vehicles with duplicate license plates, non-license plate information must be used to distinguish vehicles with the same license plate, and vehicle color is the primary method.

[0003] The current vehicle color recognition methods are mainly the following:

[0004] (1) Direct classification method [1]: First, the vehicle position is detected, then the color features of the vehicle area are directly extracted, and finally the classifier is used for classification. This method cannot find the vehicle area to be identified, so the windows, tires, background, etc. will have a huge impact on the results.

[0005] (2) Methods for selecting vehicle color regions based on image processing [2][3]. First, the license plate position and size are located, and then the vehicle color recognition region is preliminarily selected based on prior knowledge. Then, the vehicle color recognition region is selected based on information such as gradient and brightness, and finally, the vehicle color is recognized within the region. This method requires a lot of manual experience, is not very robust, and is greatly affected by the environment.

[0006] (3) Method based on segmenting vehicle color regions[4]. A large amount of data on valid vehicle color regions is labeled, and semantic segmentation is used to detect the valid regions for vehicle color recognition. Vehicle color recognition is then performed within these regions. This method requires a huge workload for data labeling, and recognition requires semantic segmentation, which results in a huge computational workload.

[0007] (4) Grayscale elimination method [5]: This method removes interference areas such as window glass and vehicle shadows by using the difference between the maximum and minimum values ​​of the RGB space, and distinguishes between color vehicles and black and white vehicles by the ratio of the number of pixels. This method is based on image processing and is easily affected by ambient light and is not very robust.

[0008] (5) The scene classification method [6] first classifies the scene of the vehicle to be identified, and then uses the color classification model of the scene to classify the vehicle color. This method requires a large amount of data and cannot find the area where the vehicle is to be identified.

[0009] In summary, the existing technology has the following problems to be solved: (1) Regarding vehicle color, based on the intuition of the human eye, only certain areas can actually be used for vehicle color recognition. For example, windows, lights, tires, etc. do not belong to the vehicle color judgment area and need to be excluded. The effective vehicle color area is generally the vehicle hood, doors, etc. If the algorithm cannot accurately and effectively find the effective area, the recognition rate will be reduced.

[0010] (2) Due to the influence of factors such as different lighting, different weather, and different angles in natural scenes, even if the car itself is of the same color, the image will often show color cast, reflection, etc., resulting in a lower algorithm recognition rate.

[0011] The relevant references are as follows:

[0012] [1] Vehicle Color Recognition Method and Device, Publication No. CN 107844745 A.

[0013] [2] A method and device for positioning vehicle body color recognition area, publication number CN 106529553 B.

[0014] [3] A vehicle body color recognition method in monitoring scenarios, publication number CN 109741406 A.

[0015] [4] Automatic vehicle color recognition method, electronic device, computer equipment and medium, Publication No. CN111325211 A.

[0016] [5] A vehicle body color recognition method, publication number CN 105005766 A.

[0017] [6] Vehicle color classification model training method, device and vehicle color recognition method, publication number CN 110348505A. Summary of the Invention

[0018] To address the shortcomings of existing technologies, this paper provides a method for vehicle color correction under white light conditions. This method combines the advantages of the RetineX and DarkChannel methods, effectively mitigating the effects of low illumination, backlighting, and other conditions, reducing the problem of reduced recognition rates caused by these adverse environmental factors. A network model based on a spatial attention mechanism is also designed. This model automatically focuses on the valid areas of vehicle color, increasing the weight of valid areas of vehicle color and decreasing the weight of invalid areas of vehicle color. A deep neural network is then used to accurately identify the vehicle color.

[0019] In order to achieve the above object, the present invention provides a technical solution: a vehicle color recognition method under white light conditions, comprising the following steps:

[0020] Step 1: Collect a large amount of vehicle image data collected under white light conditions and mark each vehicle with its own color.

[0021] Step 2: Design a vehicle color recognition network model;

[0022] The vehicle color recognition model includes a backbone module, an attention module, a BAP module, an FC module and a softmax module;

[0023] Step 3: Provide the collected vehicle data to the vehicle color recognition network model for repeated iterative training;

[0024] Step 4: Using a vehicle detector to detect vehicle position information in the image;

[0025] Step 5: crop the vehicle image from the original image using the detected vehicle position;

[0026] Step 6: Perform color correction on the cropped vehicle image;

[0027] Step 7: Send the color-corrected vehicle image to the vehicle color recognition network to predict the vehicle color category.

[0028] Furthermore, the backbone module in the vehicle color recognition model described in step 2 uses the first four residual structures of mobilenetV2; the attention module obtains a spatial weight mask from the original features extracted by the backbone module, which is used to weight different positions; the BAP module weights the original features extracted by the backbone module with the weight mask obtained by the attention module to obtain weighted new features; the FC module is used to match the new feature dimension with the number of output categories; and the softmax module is used to output the probability of each category.

[0029] Furthermore, the specific processing of the Attention module is as follows;

[0030] Use M 1x1 convolution kernels to perform convolution operation on the feature_raw extracted by the backbone module to obtain Mx14x14 mask images, and normalize each mask to between 0 and 1;

[0031] The normalization formula is:

[0032] Where, maskSrc j Represents the original j-th mask, maskDst j represents the jth mask after normalization,

[0033] They represent the i-th point of the original j-th mask and the i-th point of the normalized j-th mask respectively;

[0034] min(maskSrc j ),max(maskSrc j ) represent the minimum and maximum values ​​of the original j-th mask respectively.

[0035] Furthermore, the BAP module performs a dot product operation on the M mask maps in the attention module and the feature_raw extracted by the backbone module to obtain a 64xMx14x14 feature map, and then performs an average pooling operation on each feature map to obtain a 64xMx1x1 feature map.

[0036] Furthermore, the Softmax module performs a softmax operation on the N prediction results output by the FC module in order to predict the probability of each class. The softmax formula is: where z i represents the predicted value of the i-th category, p i represents the predicted probability of the i-th class.

[0037] Furthermore, the specific implementation process of step 6 is as follows;

[0038] (1) Crop the detected vehicle image block, denoted as P;

[0039] (2) Calculate the maximum and minimum values ​​of P, Vmax and Vmin;

[0040] (3) Linearly stretching the P obtained in step (2) to obtain P1 = 255*(P-Vmin) / (Vmax-Vmin);

[0041] (4) Transform P1 obtained in step (3) into the logarithmic domain to obtain P2 = log(P1+1) / log(256);

[0042] (5) Based on P2 obtained in step (4), calculate the dark primary color channel of P2 to obtain P3;

[0043] (6) Based on P2 obtained in step (4), calculate the brightness map of each color channel of P2 to obtain P4;

[0044] (7) Calculate the color-corrected image based on P3 and P4 obtained in steps (5) and (6), and obtain P5 = (P2-P3) / (P4-P3);

[0045] (8) The color correction image obtained in step (7) is restored to the normal image, and P6 = P5*255 is obtained.

[0046] Furthermore, the dark primary channel of P2 is calculated in step (5) as follows: Among them J dark It is called the dark primary color channel of the image, Ω(x) represents the local block area centered on x, c represents one color channel among the three channels r, g, and b, and J c is the c channel of the input image; x refers to a pixel point on the logarithmic domain image P2, and y is a pixel point in the local block area Ω(x).

[0047] Furthermore, the calculation method of the brightness map of each color channel of P2 in step (6) is: in It is called the brightness map of the c channel of the image, Ω(x) represents the local block area centered on x, c represents one of the three channels r, g, b, J c is the c channel of the input image; x refers to a pixel point on the logarithmic domain image P2, and y is a pixel point in the local block area Ω(x).

[0048] Compared with the existing technology, the advantages and benefits of the present invention are as follows: (1) A vehicle color correction method under white light conditions is proposed, which can reduce the impact of low illumination and backlighting on the reduced recognition rate. (2) A network model based on the spatial attention mechanism is designed, which can automatically focus on the effective area of ​​the vehicle body color, eliminating the need to rely on manual experience to locate the area to be recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0050] Figure 2This is a color correction flow chart of the present invention.

[0051] Figure 3 This is a color correction effect diagram of the present invention.

[0052] Figure 4 This is the vehicle color classification network model diagram of the present invention. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] like Figure 1 As shown, the present invention provides a vehicle color recognition method under white light conditions, comprising the following steps:

[0055] Step 1: Collect a large amount of vehicle image data collected under white light conditions and mark each vehicle with its color.

[0056] In step 1, a large amount of vehicle image data under white light conditions is collected through the camera at the checkpoint and the parking lot. Then, the vehicle position information in the image is detected by the vehicle detector. The position information here includes (x1, y1, x2, y2), where x1, y1, x2, y2 represent the horizontal and vertical coordinates of the upper left corner and lower right corner of the vehicle respectively. Then, the vehicle image block is cropped from the image data according to the vehicle position information, and finally the vehicle color in the vehicle image block is manually marked.

[0057] Step 2: Design a vehicle color recognition network model.

[0058] The vehicle color recognition model designed in step 2 is as follows Figure 4 As shown in the figure, it includes modules such as backbone, attention, BAP, FC, and softmax. The backbone uses the first four residual structures of mobilenetV2. Attention generates a spatial weight mask from the original features extracted by the backbone, which is used to weight different locations. BAP weights the original features extracted by the backbone using the weight mask obtained by attention to obtain new weighted features. The FC layer matches the new feature dimensions with the number of output categories. Softmax outputs the probability of each category.

[0059] The vehicle color recognition network of the present invention adds a spatial attention mechanism to the traditional classification network model. The function of the spatial attention module is to automatically assign different weights to different areas in the vehicle image, increase the weight of the vehicle color effective area, and reduce the weight of the vehicle invalid area, so as to improve the classification performance of the network model. The entire network structure is composed of backbone, attention, BAP, FC and softmax modules. The network model structure is as follows: Figure 4 shown.

[0060] Backbone module: This invention uses the first four residual structures of mobilenetV2 stacked together, with an input size of 3x224x224 and an output size of 64x14x14. The output is recorded as feature_raw.

[0061] Attention module: This paper uses M 1x1 convolution kernels to convolve the feature_raw extracted by the backbone to obtain Mx14x14 mask images, and normalizes each mask to between 0 and 1. The default value of M is 16. The normalization formula is: Where maskSrc j Represents the original j-th mask, maskDst j represents the jth mask after normalization, They represent the i-th point of the original j-th mask and the i-th point of the normalized j-th mask respectively. min(maskSrc j ),max(maskSrc j ) represent the minimum and maximum values ​​of the original j-th mask respectively.

[0062] BAP module (Bilinear Attention Pooling): This paper uses the dot product operation of the M mask maps in the attention and the feature_raw extracted by the backbone to obtain a 64xMx14x14 feature map, and then performs an average pooling operation on each feature map to obtain a 64xMx1x1 feature map, which is recorded as feature_matrix.

[0063] FC module: The present invention performs N-class FC full connections on the feature_matrix extracted by BAP to output the prediction results of each class.

[0064] Softmax module, the present invention performs softmax operation on the N prediction results output by FC, the purpose is to predict the probability of each class. The softmax formula is: where z irepresents the predicted value of the i-th category, p i represents the predicted probability of the i-th class.

[0065] Step 3: Provide the collected vehicle data to the vehicle color recognition network model for repeated iterative training.

[0066] The parameters for network training in step 3 are: lr = 0.001, moment = 0.9, weight_decay = 1e-5, epoch = 50, batchsize = 64. The network optimization adopts SGD gradient descent method, and the learning rate adopts fixed compensation decay, multiplied by 0.9 every 2 epochs; where lr is the learning rate, moment is the momentum, weight_decay is the decay coefficient, epoch is the total number of iterations of the training set, and batchsize is the number of samples in one iteration.

[0067] Step 4: Use a vehicle detector to detect the vehicle position information in the image.

[0068] The vehicle detector in step 4 uses the yolov5 framework and is trained using the COCO+VOC joint dataset.

[0069] Step 5: Crop the vehicle image from the original image using the detected vehicle position.

[0070] Step 6: Perform color correction on the cropped vehicle image.

[0071] The flowchart of color correction in step 6 is as follows Figure 2 As shown in the figure, the steps include: linear stretching, logarithmic domain transformation, dark original color channel extraction, each color channel brightness map extraction, each color channel correction, etc. Through these steps, the original vehicle image color is corrected. This invention combines the advantages of RetineX and darkChannel. Through experiments, it is found that the effect is superior for vehicle color correction in harsh environments. The flow chart is as follows Figure 2 As shown, the effect is as follows Figure 3 As shown, the left side is the original image, and the right side is the image after color correction. The steps are as follows:

[0072] (1), crop the detected vehicle image block, denoted as P.

[0073] (2) Calculate the maximum and minimum values ​​of P, Vmax and Vmin.

[0074] (3) Linearly stretch the P obtained in step (2) to obtain P1 = 255*(P-Vmin) / (Vmax-Vmin).

[0075] (4) Transform P1 obtained in step (3) into the logarithmic domain to obtain P2 = log(P1+1) / log(256).

[0076] (5) According to P2 obtained in step (4), the dark primary color channel of P2 is calculated to obtain P3. The process of obtaining the dark primary color channel is as follows: the minimum component of the three channels R, G, and B of each pixel of the original image is extracted to obtain a single-channel dark primary color image, and then the local area minimum filtering of the dark primary color image is performed using Marcel van Herk's fast algorithm, that is, grayscale corrosion operation, to obtain the dark primary color channel.

[0077] (6) According to P2 obtained in step (4), the brightness map of each color channel of P2 is calculated to obtain P4. The process of obtaining the brightness map of each color channel is as follows: the original image is decomposed in the RGB space to obtain three channel maps of R, G, and B, and then the Marcel van Herk fast algorithm is used to perform local area maximum filtering on the three channel maps of R, G, and B, that is, grayscale expansion operation, to obtain the brightness map of each channel.

[0078] (7) Calculate the color-corrected image based on P3 and P4 obtained in steps (5) and (6), and obtain P5 = (P2-P3) / (P4-P3).

[0079] (8) The color correction image obtained in step (7) is restored to the normal image, and P6 = P5*255 is obtained.

[0080] The dark primary color channel of P2 in step (5) is calculated as follows: Among them J dark It is called the dark primary color channel of the image, Ω(x) represents the local block area centered on x, c represents one color channel among the three channels r, g, and b, and J c is the c channel of the input image.

[0081] The calculation method for calculating the brightness map of each color channel of P2 in step (6) is:

[0082] in It is called the brightness map of the c channel of the image, Ω(x) represents the local block area centered on x, c represents one of the three channels r, g, b, J c is the c channel of the input image.

[0083] Step 7: Send the color-corrected vehicle image block to the vehicle color recognition network to predict the vehicle color category.

[0084] The specific embodiments described herein are merely illustrative of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the scope of the present invention or exceeding the scope of the appended claims.

Claims

1. A vehicle color recognition method under white light conditions, characterized in that: The steps include: Step 1: Collect a large amount of vehicle image data collected under white light conditions and mark each vehicle with its own color. Step 2: Design a vehicle color recognition network model; The vehicle color recognition model includes a backbone module, an attention module, a BAP module, an FC module and a softmax module; The backbone module in the vehicle color recognition model described in step 2 uses the first four residual structures of mobilenetV2. The attention module obtains a spatial weight mask from the original features extracted by the backbone module, which is used to weight different positions. The BAP module weights the original features extracted by the backbone module with the weight mask obtained by the attention module to obtain weighted new features. The FC module is used to match the new feature dimension with the number of output categories. The softmax module is used to output the probability of each category. Step 3: Provide the collected vehicle data to the vehicle color recognition network model for repeated iterative training; Step 4: Using a vehicle detector to detect vehicle position information in the image; Step 5: crop the vehicle image from the original image using the detected vehicle position; Step 6: Perform color correction on the cropped vehicle image; The specific implementation process of step 6 is as follows; (1) Crop the detected vehicle image block, denoted as P; (2) Calculate the maximum and minimum values ​​of P, Vmax and Vmin; (3) Linearly stretching the P obtained in step (2) to obtain P1 = 255*(P-Vmin) / (Vmax-Vmin); (4) Transform P1 obtained in step (3) into the logarithmic domain to obtain P2 = log(P1+1) / log(256); (5) Based on P2 obtained in step (4), calculate the dark primary color channel of P2 to obtain P3; (6) Based on P2 obtained in step (4), calculate the brightness map of each color channel of P2 to obtain P4; (7) Calculate the color-corrected image based on P3 and P4 obtained in steps (5) and (6), and obtain P5 = (P2-P3) / (P4-P3); (8) Restore the color correction image obtained in step (7) to the normal image, and obtain P6 = P5*255; Step 7: Send the color-corrected vehicle image to the vehicle color recognition network to predict the vehicle color category.

2. The vehicle color recognition method under white light conditions according to claim 1, characterized in that: The specific processing process of the attention module is as follows; Use M 1x1 convolution kernels to perform convolution operation on the feature_raw extracted by the backbone module to obtain Mx14x14 mask images, and normalize each mask to between 0 and 1; The normalization formula is: Where, maskSrc j Represents the original j-th mask, maskDst j represents the jth mask after normalization, They represent the i-th point of the original j-th mask and the i-th point of the normalized j-th mask respectively; min(maskSrc j ),max(maskSrc j ) represent the minimum and maximum values ​​of the original j-th mask respectively.

3. The vehicle color recognition method under white light conditions according to claim 2, characterized in that: The BAP module performs a dot product operation on the M mask maps in the attention module and the feature_raw extracted by the backbone module to obtain a 64xMx14x14 feature map, and then performs an average pooling operation on each feature map to obtain a 64xMx1x1 feature map.

4. The vehicle color recognition method under white light conditions according to claim 1, characterized in that: The Softmax module performs a softmax operation on the N prediction results output by the FC module in order to predict the probability of each class. The softmax formula is: where z i represents the predicted value of the i-th category, p i represents the predicted probability of the i-th class.

5. The vehicle color recognition method under white light conditions according to claim 1, characterized in that: The dark primary color channel of P2 in step (5) is calculated as follows: Among them J dark It is called the dark primary color channel of the image, Ω(x) represents the local block area centered on x, c represents one color channel among the three channels r, g, and b, and J c is the c channel of the input image; x refers to a pixel point on the logarithmic domain image P2, and y is a pixel point in the local block area Ω(x).

6. The vehicle color recognition method under white light conditions according to claim 1, characterized in that: The calculation method for calculating the brightness map of each color channel of P2 in step (6) is: in It is called the brightness map of the c channel of the image, Ω(x) represents the local block area centered on x, c represents one of the three channels r, g, b, J c is the c channel of the input image; x refers to a pixel point on the logarithmic domain image P2, and y is a pixel point in the local block area Ω(x).

Images