A vehicle color recognition method based on a deep learning fusion model

Abstract

The present invention discloses a vehicle color recognition method based on a deep learning fusion model, including: constructing a highway vehicle face image dataset; performing image enhancement on the highway vehicle face image data by using the YUV-FM-Retinex method; constructing a vehicle color recognition model VCR-VGG16 based on deep learning to obtain a one-dimensional feature vector F V ; constructing a vehicle color recognition model VCR-Xception based on deep learning to obtain a one-dimensional feature vector F X ; constructing a vehicle color recognition model VCR-DenseNet201 based on deep learning to obtain a one-dimensional feature vector F D ; fusing the feature vectors F V , F X , F D to construct a vehicle color recognition fusion model VCR-FDCNN based on deep learning for vehicle color recognition. The present invention proposes an image enhancement preprocessing method and constructs a deep learning fusion model, so as to more accurately realize the recognition of vehicle colors and provide technical support for the perception of vehicle information in highway scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation and intelligent highway research, and particularly relates to a vehicle color recognition method based on a deep learning fusion model. Background Technique

[0002] With the wide popularization of the non-stop toll collection technology, new requirements are put forward for vehicle information perception. Vehicle information perception is developing towards the perception of vehicle holographic attributes that integrates features such as vehicle license plates, brands, colors, and types. Compared with vehicle license plate recognition, there is less research on the field of vehicle color recognition at present. Moreover, as a basic attribute of a vehicle, vehicle color accounts for a large proportion, is not easily blocked or modified, and is not sensitive to image quality. It can largely make up for the deficiencies of license plate recognition.

[0003] In color recognition research, traditional machine learning methods rely on feature engineering, have low computational efficiency, and are difficult to apply to actual intelligent systems. While deep learning reduces computational complexity by adaptively extracting features and has stronger generalization ability at the same time. The fusion network combines the structural characteristics of each single model, makes up for the defects of a single model, and integrates the advantages of each model, which can greatly improve the comprehensive performance of the model. Therefore, the present invention proposes a vehicle color recognition method based on a deep learning fusion model. Summary of the Invention

[0004] Object of the Invention: In order to overcome the deficiencies existing in the prior art, the present invention provides a vehicle color recognition method based on a deep learning fusion model, which uses the YUV-FM-Retinex algorithm for image preprocessing and effectively recognizes and classifies vehicle colors by using a deep fusion neural network, and can provide technical support for vehicle information perception in highway scenarios.

[0005] Technical Solution: To achieve the above object, the present invention provides a vehicle color recognition method based on a deep learning fusion model, including the following steps:

[0006] S1: Construct a highway vehicle face image dataset;

[0007] S2: Perform YUV-FM-Retinex image enhancement on the highway vehicle face image dataset;

[0008] S3: Construct a vehicle color recognition model VCR-VGG16 based on deep learning to obtain a one-dimensional feature vector F V ;

[0009] S4: Construct a vehicle color recognition model VCR-Xception based on deep learning to obtain a one-dimensional feature vector F X ;

[0010] S5: Construct a vehicle color recognition model VCR-DenseNet201 based on deep learning and obtain a one-dimensional feature vector F D ;

[0011] S6: Fuse the feature vectors F V , F X , F D to construct a vehicle color recognition fusion model VCR-FDCNN based on deep learning for vehicle color recognition.

[0012] Furthermore, the specific method for constructing the highway vehicle face image dataset in step S1 is as follows:

[0013] S1-1: Collect 7,200 highway vehicle face images, including 6 color categories of black, white, red, yellow, blue, and green, and 3 vehicle types of cars, buses, and trucks, to construct a highway vehicle face image dataset. Among them, there are 2,400 images for each vehicle type, and the images contain changes in different time periods and different lighting conditions;

[0014] S1-2: Before training each model, randomly select image data as the training set, validation set, and test set.

[0015] Furthermore, the specific steps for performing YUV-FM-Retinex image enhancement on the highway vehicle face image data in step S2 are as follows:

[0016] S2-1: Switch to the YUV color space and convert the original image in the RGB color space to the YUV color space;

[0017] S2-2: Separate channels, separate the YUV color space image to obtain the luminance component Y and the chrominance components U and V;

[0018] S2-3: FM-Retinex enhancement, only perform FM-Retinex algorithm enhancement on the luminance component Y;

[0019] S2-4: Channel fusion, fuse the enhanced luminance channel Y with the chrominance channels U and V;

[0020] S2-5: Switch to the RGB color space and convert the image in the YUV color space back to the RGB color space;

[0021] S2-6: Luminance normalization, adjust the gray values of each channel of the image to be normalized and then restore them to the range of (0, 225) proportionally.

[0022] Furthermore, the specific content of the FM-Retinex algorithm enhancement in step S2-3 is:

[0023] ① Convert the pixel values in the Y channel from the integer domain to the logarithmic domain and add 1 to the whole:

[0024] y'(x, y) = log(1 + Y(x, y))

[0025] ② For an image with a resolution of m×n, let the distance between the two farthest comparison points from the current pixel be X:

[0026] X = 2 P

[0027] P = fix[log2 min(m, n) - 1]

[0028] ③ If X > 0, then let:

[0029]

[0030] If X < 0, then let:

[0031]

[0032] ④ After each step of the above calculation is completed, change the comparison point spacing:

[0033] X = -X / 2

[0034] ⑤ When |X| < 1, obtain the final output result array r n+1 (x, y), and perform linear stretching:

[0035]

[0036] Among them, max and min respectively represent the maximum and minimum values of r n+1 (x, y).

[0037] Further, the specific content of the gray value normalization in step S2-6 is:

[0038]

[0039] Among them, G(x, y) and G′(x, y) are the gray values before and after normalization respectively; Max and Min are the maximum and minimum values of the gray levels of the original image respectively.

[0040] Further, in step S3, construct a vehicle color recognition model VCR-VGG16 based on deep learning to obtain a one-dimensional feature vector F VThe specific method is as follows: Based on the VGG16 model, a global average pooling layer, a Dropout layer with a dropout rate of 0.5, and a fully connected layer are added. The network can identify 6 categories, namely black, white, red, yellow, blue, and green, to obtain the VCR-VGG16 vehicle color recognition model based on deep learning. After training is completed, the last fully connected layer and the Dropout layer are deleted, and finally a one-dimensional feature vector F of 1×1×512 is output V 。

[0041] Furthermore, in step S4, a vehicle color recognition model VCR-Xception based on deep learning is constructed to obtain a one-dimensional feature vector F X The specific method is as follows: Based on the Xception model, a global average pooling layer, a Dropout layer, and a fully connected layer are added. The network can identify 6 categories, namely black, white, red, yellow, blue, and green, to obtain the VCR-Xception vehicle color recognition model based on deep learning. After training is completed, the fully connected layer and the Dropout layer are deleted to obtain a one-dimensional feature vector F of 1×1×2048 X 。

[0042] Furthermore, in step S5, a vehicle color recognition model VCR-DenseNet201 based on deep learning is constructed to obtain a one-dimensional feature vector F D The specific method is as follows: Based on the DenseNet201 model, a global average pooling layer, a Dropout layer, and a fully connected layer are added. The network can identify 6 categories, namely black, white, red, yellow, blue, and green, to obtain the VCR-DenseNet201 vehicle color recognition model based on deep learning. After training is completed, the fully connected layer and the Dropout layer are deleted to obtain a one-dimensional feature vector F of 1×1×1920 D 。

[0043] Furthermore, in step S6, the feature vectors F V 、F X 、F D are fused. The specific method for constructing the vehicle color recognition fusion model VCR-FDCNN based on deep learning is as follows: For the three models of VCR-VGG16, VCR-Xception, and VCR-DenseNet201, the original structure's fully connected layer and Dropout layer are removed, and the feature vectors F V 、F X and F D are extracted. The obtained one-dimensional feature vectors are fused in parallel to obtain a feature of 1×1×4480. Then, the merged one-dimensional feature vector is used as the input to retrain the fully connected layer, and the Dropout layer is added again to prevent overfitting. Finally, the fusion model VCR-FDCNN is obtained

[0044] Beneficial effects: Compared with the prior art, the present invention realizes the automatic recognition of vehicle colors based on deep learning technology, improves the recognition accuracy, model stability and generalization ability, and can provide technical support for the perception of vehicle information in highway scenarios. Description of the drawings

[0045] Figure 1 It is a flowchart of the YUV-FM-Retinex algorithm.

[0046] Figure 2 It is a schematic diagram of the VCR-FDCNN model structure. Detailed implementation manners

[0047] The present invention will be further clarified below in conjunction with the drawings and specific embodiments.

[0048] The present invention provides a vehicle color recognition method based on deep learning, including the following steps:

[0049] S1: Construct a highway vehicle face image dataset, specifically:

[0050] S1-1: Collect 7,200 highway vehicle face images, including 6 color categories of black, white, red, yellow, blue, and green, and 3 vehicle types of cars, buses, and trucks, to construct a highway vehicle face image dataset. Among them, there are 2,400 images for each vehicle type, and the images include changes in different time periods such as early morning, daytime, dusk, and night, and different lighting conditions such as sunny, rainy, and foggy days. The composition of the dataset is shown in Table 1;

[0051] Table 1 Composition of the vehicle face image dataset

[0052] Black White Red Yellow Blue Green Total Car 600 400 400 318 400 166 2284 Bus 0 400 400 441 400 517 2158 Truck 600 400 400 441 400 517 2758 Total 1200 1200 1200 1200 1200 1200 7200

[0053] S1-2: Before training each model, randomly extract image data in a ratio of 4:1:1 as the training set, validation set, and test set.

[0054] S2: Perform YUV-FM-Retinex image enhancement on the highway vehicle face image dataset, specifically:

[0055] S2-1: Switch the YUV color space: Convert the initial image in the RGB color space to the YUV color space;

[0056] S2-2: Separate channels: Separate the YUV color space image to obtain the luminance component Y and the chrominance components U and V;

[0057] S2-3: FM-Retinex enhancement: Only perform FM-Retinex algorithm enhancement on the luminance component Y;

[0058] ① Convert the pixel values of the Y channel from the integer domain to the logarithmic domain and add 1 to the whole:

[0059] y'(x, y) = log(1 + Y(x, y))

[0060] ② For an image with a resolution of m×n, let the distance between the two farthest comparison points from the current pixel be X:

[0061] X = 2 P

[0062] P = fix[log2 min(m, n) - 1]

[0063] ③ If X > 0, then let:

[0064]

[0065] If X < 0, then let:

[0066]

[0067] ④ After each step of the above calculation is completed, change the comparison point spacing:

[0068] X = -X / 2

[0069] ⑤ When |X| < 1, obtain the final output result array r n+1 (x, y), and perform linear stretching to obtain:

[0070]

[0071] where max and min respectively represent the maximum and minimum values of r n+1 (x, y).

[0072] S2-4: Channel fusion: fuse the enhanced luminance channel Y with the chrominance channels U and V;

[0073] S2-5: Switch to the RGB color space: convert the image in the YUV color space back to the RGB color space;

[0074] S2-6: Luminance normalization: adjust the gray values of each channel of the image to be normalized and then restore them proportionally to the interval (0, 225):

[0075] Let the gray values of each channel:

[0076]

[0077] where G(x, y) and G′(x, y) are the gray values before and after normalization respectively; Max and Min are the maximum and minimum values of the gray levels of the original image. Then expand proportionally to (0, 225) to obtain the luminance normalization result.

[0078] S3: Construct a vehicle color recognition model VCR-VGG16 based on deep learning to obtain a one-dimensional feature vector F V : Based on the classical convolutional neural network VGG16 model, successively add a global average pooling layer, a Dropout layer with a dropout rate of 0.5, and a fully connected layer. Set the model output so that the network recognition categories are divided into 6 categories: black, white, red, yellow, blue, and green. Obtain a vehicle color recognition model VCR-VGG16 based on deep learning through training. After training is completed, delete the fully connected layer and the Dropout layer at the end of the network to obtain a one-dimensional feature vector F with an output of 1×1×512 V 。

[0079] S4: Construct a vehicle color recognition model VCR-Xception based on deep learning to obtain a one-dimensional feature vector F X : Based on the classical convolutional neural network Xception model, successively add a global average pooling layer, a Dropout layer, and a fully connected layer. Set the model output so that the network recognition categories are divided into 6 categories: black, white, red, yellow, blue, and green. Obtain a vehicle color recognition model VCR-Xception based on deep learning through training. After training is completed, delete the fully connected layer and the Dropout layer to obtain a one-dimensional feature vector F with an output of 1×1×2048 X 。

[0080] S5: Construct a vehicle color recognition model VCR-DenseNet201 based on deep learning to obtain a one-dimensional feature vector F D : Based on the classical convolutional neural network DenseNet201 model, successively add a global average pooling layer, a Dropout layer, and a fully connected layer. Set the model output so that the network recognition categories are divided into 6 categories: black, white, red, yellow, blue, and green. Obtain a vehicle color recognition model VCR-DenseNet201 based on deep learning through training. After training is completed, delete the fully connected layer and the Dropout layer to obtain a one-dimensional feature vector F with an output of 1×1×1920 D 。

[0081] S6: Fuse the feature vectors F V 、F X 、F D to construct a vehicle color recognition fusion model VCR-FDCNN based on deep learning to recognize vehicle colors: The three vehicle color recognition models VCR-VGG16, VCR-Xception, and VCR-DenseNet201 adopt a parallel fusion rule to form a VCR-FDCNN vehicle color recognition model as shown in Figure 2As shown. For the three models of VCR-VGG16, VCR-Xception, and VCR-DenseNet201, after completing the model training, remove the original fully connected layer and Dropout layer of the structure, and extract the feature vectors F V , F X and F D . Concatenate the obtained one-dimensional feature vectors to fuse them into a new one-dimensional feature vector with a size of 1×1×4480. Then, use the merged one-dimensional feature vector as the input, and add a Dropout layer with a dropout rate of 0.5 again to prevent overfitting. Retrain the fully connected layer. Finally, obtain the fusion model VCR-FDCNN, and output 6 recognition results of black, white, red, yellow, blue, and green.

[0082] To verify the superiority of the YUV-FM-Retinex enhancement algorithm, a comparative experiment was conducted in this case. Based on 7200 face images of vehicles on highways, the training set, validation set, and test set were divided according to a ratio of 4:1:1. The original image, brightness normalization, switching the Lab color space, histogram equalization, and the YUV-FM-Retinex algorithm were used for image preprocessing respectively, and a neural network was trained. The recall rates on the test set are shown in Table 2. It can be seen that the test results of the model using the YUV-FM-Retinex algorithm are significantly better than other preprocessing methods, especially the recall rate is significantly improved compared with the original image. It shows that the YUV-FM-Retinex enhancement algorithm can effectively process images and reduce the interference of different lights and time periods on the images.

[0083] Table 2 Comparison of preprocessing methods

[0084] Black White Red Yellow Blue Green Average recall rate Original image 96 97 98.5 95.5 96.5 97 96.75 Brightness normalization 97 96.5 98.5 96.5 96.5 97 97.00 Switch to Lab color space 97 96 98.5 96 97.5 97.5 97.08 Histogram equalization 98 99.5 98.5 97 95.5 94 97.08 YUV-FM-Retinex algorithm 99 97 99 98.5 98.5 96.5 98.08

[0085] To verify the superiority of the fusion model (VCR-FDCNN), a comparative experiment was also conducted. Based on 7200 face images of vehicles in highway scenes, the training set, validation set, and test set were divided according to a ratio of 4:1:1. The YUV-FM-Retinex algorithm was used for image preprocessing, and four vehicle color recognition models of VCR-VGG16, VCR-Xception, VCR-DenseNet201, and VCR-FDCNN were trained respectively. The accuracies of different models were compared and analyzed. The results are shown in Table 3. It can be seen that the vehicle color recognition rates of the fusion model (VCR-FDCNN) are as high as 100% and 99.5% on the training set and validation set, which are significantly higher than other single models; at the same time, compared with the single models, the training speed of the fusion model is extremely fast on the basis of a single network.

[0086] Table 3 Comparison of fusion models

[0087] VGG16 Xception DenseNet201 FDCNN-VTR Training accuracy 99.55 99.8 100 100 Validation accuracy 98.67 99.3 99.08 99.5 Training time consumption 44s / epoch 40s / epoch 42s / epoch 0.56s / epoch

[0088] To further demonstrate the effectiveness of the VCR-FDCNN model, the confusion matrix of the recognition results of the test set was counted in this case, as shown in Table 4. The results show that the FDCNN-VCR fusion model has good recognition performance for vehicle colors, and the recognition accuracy of the test sets of all six colors is not less than 98.5%, and the average recognition accuracy reaches 99.50%.

[0089] Table 4 Confusion matrix of the VCR-FDCNN fusion model

[0090] Black White Red Yellow Blue Green Black 99.50％ 0 0.50％ 0 0 0 White 0 100％ 0 0 0 0 Red 0 0 99.50％ 0 0 0 Yellow 0 0 0 99.50％ 0 1.00％ Blue 0 0 0 0.50％ 100％ 0.50％ Green 0.50％ 0 0 0 0 98.50％

[0091] The technical means disclosed in the solution of the present invention are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A vehicle color recognition method based on a deep learning fusion model, characterized in that: It includes the following steps: S1: Construct a highway vehicle face image dataset; S2: Perform YUV-FM-Retinex image enhancement on the highway vehicle face image dataset; S3: Build a vehicle color recognition model VCR-VGG16 based on deep learning and obtain a one-dimensional feature vector F V ; S4: Construct a vehicle color recognition model VCR-Xception based on deep learning to obtain a one-dimensional feature vector F X ; S5: Construct a vehicle color recognition model VCR-DenseNet201 based on deep learning and obtain a one-dimensional feature vector F D ; S6: Combine the feature vectors F V , F X , F D to build a vehicle color recognition fusion model VCR-FDCNN based on deep learning for vehicle color recognition; The specific steps for performing YUV-FM-Retinex image enhancement on the highway vehicle face image data in step S2 are as follows: S2-1: Switch the YUV color space and convert the initial image in the RGB color space to the YUV color space; S2-2: Separate the channels, separate the YUV color space image to obtain the luminance component Y and the chrominance components U and V; S2-3: FM-Retinex enhancement, only perform FM-Retinex algorithm enhancement on the luminance component Y; S2-4: Channel fusion, fuse the enhanced luminance channel Y and the chrominance channels U and V; S2-5: Switch the RGB color space and convert the image in the YUV color space back to the RGB color space; S2-6: Luminance normalization, adjust the gray values of each channel of the image to be normalized and then restore them proportionally to the range of (0, 225); In step S6, for the three models of VCR-VGG16, VCR-Xception, and VCR-DenseNet201, the fully connected layer and the Dropout layer of the original structure are removed, and the obtained one-dimensional feature vectors F V , F X , F D are fused in parallel to obtain a feature of 1×1×4480. Then, the merged one-dimensional feature vector is used as the input to retrain the fully connected layer, and the Dropout layer is added again to prevent overfitting. Finally, the fused model VCR-FDCNN is obtained.

2. The vehicle color recognition method based on a deep learning fusion model according to claim 1, characterized in that: The specific method for constructing the highway vehicle face image dataset in step S1 is as follows: S1-1: Collect 7200 highway vehicle face images, including 6 color categories of black, white, red, yellow, blue, and green, and 3 vehicle types of cars, buses, and trucks, to construct a highway vehicle face image dataset; among them, there are 2400 images for each vehicle type, and the images include changes in different time periods and different lighting conditions; S1-2: Before training each model, randomly extract image data to divide the training set, validation set, and test set.

3. A vehicle color recognition method based on a deep learning fusion model according to claim 1, characterized in that: The specific content of the FM-Retinex algorithm enhancement in step S2-3 is: ① Convert the pixel values of the Y channel from the integer domain to the logarithmic domain and add 1 to the whole: Y'(x,y) = log(1 + Y(x,y)) ② For an image with a resolution of m×n, let the distance between the two farthest comparison points from the current pixel point be X: X＝2 P P = fix[log2 min(m,n) - 1] ③ If X > 0, then let: If X < 0, then let: ④ After each step of the above calculation is completed, change the comparison point spacing: X = -X / 2 ⑤ When |X| < 1, the final output result array r is obtained n+1 (x, y) is subjected to linear stretching: where max and min represent the maximum and minimum values of r n+1 (x, y), respectively.

4. The vehicle color recognition method based on the deep learning fusion model according to claim 1, wherein: The specific content of the gray value normalization in step S2-6 is: Let the gray values of each channel: Among them, G(x, y) and G ′ (x, y) are the gray values before and after normalization respectively; Max and Min are the maximum and minimum values of the gray levels of the original image respectively.

5. A vehicle color recognition method based on a deep learning fusion model according to claim 1, characterized in that: In step S3, a vehicle color recognition model VCR-VGG16 based on deep learning is constructed to obtain a one-dimensional feature vector F V The specific method is as follows: based on the VGG16 model, a global average pooling layer, a Dropout layer with a dropout rate of 0.5, and a fully connected layer are added. The network recognition categories are divided into 6 categories: black, white, red, yellow, blue, and green, to obtain the VCR-VGG16 vehicle color recognition model based on deep learning. After training, the last fully connected layer and the Dropout layer are deleted, and finally a one-dimensional feature vector F of 1×1×512 is output V .

6. The vehicle color recognition method based on the deep learning fusion model according to claim 1, characterized in that: In step S4, a vehicle color recognition model VCR-Xception based on deep learning is constructed to obtain a one-dimensional feature vector F X The specific method is as follows: based on the Xception model, a global average pooling layer, a Dropout layer, and a fully connected layer are added. The network recognition categories are divided into 6 categories: black, white, red, yellow, blue, and green, to obtain the VCR-Xception vehicle color recognition model based on deep learning. After training is completed, the fully connected layer and the Dropout layer are deleted to obtain a one-dimensional feature vector F of 1×1×2048 X .

7. A vehicle color recognition method based on a deep learning fusion model according to claim 1, characterized in that: In step S5, a vehicle color recognition model VCR-DenseNet201 based on deep learning is constructed to obtain a one-dimensional feature vector F D The specific method is as follows: based on the DenseNet201 model, a global average pooling layer, a Dropout layer, and a fully connected layer are added. The network recognition categories are divided into 6 categories: black, white, red, yellow, blue, and green, to obtain the VCR-DenseNet201 vehicle color recognition model based on deep learning. After training is completed, the fully connected layer and the Dropout layer are deleted to obtain a one-dimensional feature vector F of 1×1×1920 D .

Images