A method of identifying a color of a vehicle

Abstract

The embodiment of the application provides a method for identifying the color of a vehicle, which comprises the following steps: obtaining a target vehicle picture, and processing the target vehicle picture to obtain a target vehicle body region picture; inputting the target vehicle body region picture into a network of a deep learning model with multi-label color sequence identification, to obtain a color sequence regression result; calculating the color sequence identification result; sorting the calculated color sequence identification result, and outputting a result greater than a threshold value, which is mapped to a corresponding color sequence, so as to obtain the color of the vehicle. The embodiment of the application solves the technical problems of the current vehicle body color method based on a local region, such as easy overfitting, poor robustness, and color difference caused by color distribution difference, illumination intensity, and shadow.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of image processing technology and computer vision, and particularly relate to a method for identifying vehicle color. BACKGROUND

[0002] Computer vision technology is a technology that uses a camera and a computer to replace the human eye and brain to identify, track and measure targets and other processes. The technology includes image processing, pattern recognition, image analysis and image understanding, etc. Computer vision research related theories and technologies, trying to establish an artificial intelligence system that can obtain information from images or multidimensional data.

[0003] With the development of computer vision technology, it has been widely applied in various fields, such as medical image processing, automatic driving, public security video monitoring, intelligent transportation system, and card system. In road video monitoring, in addition to license plate information, vehicle color information as the most intuitive and easy information has extremely important significance in management, positioning, retrieval, analysis of target intelligent research and judgment.

[0004] Regarding vehicle color recognition patents, they are divided into local and global from color region. From the recognition method, they are divided into traditional algorithm and deep learning algorithm.

[0005] In the existing method, the local region color extraction mainly considers that the color of the region near the license plate or the region near the car face is generally consistent with the color of the car body. The traditional algorithm recognition generally converts the pixel color to other more stable color space, such as HSV color space, according to the corresponding color of the hsv color interval value, counts the number of various color points, and sorts out the final color conclusion. Or normalize the picture to a certain size, then divide the region and traverse the color histogram, and use svm training classification. There are also deep learning methods, such as in "a vehicle color recognition system based on multi-task deep convolutional neural network", fastercnn is used to locate the vehicle, then the vehicle picture is color corrected, and then input into the cnn network training model.

[0006] In the above local area-based vehicle body color method, due to the differences in color distribution near license plates of different vehicle models, it is easy to make a generalization, and it is not applicable to side vehicles; according to the color space distribution of the pixel points, the number of each color point is counted, and due to the complex color space distribution and the influence of light intensity and shadow, color difference is easy to appear, so this method is suitable for stable light scenes, and needs to divide the interval value by experience; the svm training color histogram method is easy to overfit due to the differences in vehicle color distribution, color difference, etc., and has poor robustness. In the method of directly inputting the vehicle picture into the cnn network by adjusting the network structure and parameters, due to the differences in vehicle body color distribution of different vehicle models, some vehicles have multiple color combinations and multi-angle color distribution differences, there are problems of huge data volume and more network parameters, and it is not easy to fit. SUMMARY

[0007] The embodiment of the present application provides a method for identifying vehicle color, which solves the technical problems of the current local area-based vehicle body color method, such as the differences in color distribution near license plates of different vehicle models, easy to make a generalization, the complex color space distribution, and the influence of light intensity and shadow, easy to appear color difference, the differences in vehicle body color distribution of different vehicle models, some vehicles have multiple color combinations and multi-angle color distribution differences, huge data volume and more network parameters, and not easy to fit, and poor robustness.

[0008] The embodiment of the present application provides a method for identifying vehicle color, which solves the technical problems of the current local area-based vehicle body color method, such as the differences in color distribution near license plates of different vehicle models, easy to make a generalization, the complex color space distribution, and the influence of light intensity and shadow, easy to appear color difference, the differences in vehicle body color distribution of different vehicle models, some vehicles have multiple color combinations and multi-angle color distribution differences, huge data volume and more network parameters, and not easy to fit, and poor robustness.

[0009] Obtaining a target vehicle picture, and processing the target vehicle picture to obtain a target vehicle body area picture;

[0010] Inputting the target vehicle body area picture into a network of a deep learning model with multi-label color sequence identification to obtain a color sequence regression result;

[0011] Obtaining a color sequence identification result by calculating the proportion of each color sequence in the color sequence regression result;

[0012] Sorting the color sequence identification result, outputting a result greater than a threshold value, mapping the result to a corresponding color sequence, and obtaining a vehicle color.

[0013] Preferably, the obtaining of the target vehicle picture and the processing of the target vehicle picture to obtain the target vehicle body area picture specifically comprises:

[0014] Using a deep learning SSD to detect a target vehicle in the target vehicle picture, and obtaining the target vehicle body area picture through a target tracking algorithm.

[0015] Preferably, the deep learning model for recognizing the multi-label color sequence is a multi-task multi-label deep learning model with preset training data.

[0016] Preferably, the preset training data includes type labeling and color labeling on the target vehicle body region picture.

[0017] Preferably, the type labeling specifically operates as follows: according to the target vehicle body region picture, the target vehicle is labeled as one of the following eight types: bus, sedan, minivan, minivan, pickup truck, business and off-road vehicle, and truck.

[0018] Preferably, the colors in the color labeling are pre-set as red, blue, yellow, black, green, brown, gray, pink, white, orange, and purple.

[0019] Using a color sequence labeling tool, the part of the target vehicle body region picture that has been type labeled, except for the part of the window, windshield, wheel, and air intake grid, is labeled with color. After completion, import the labeling software and select the first ten colors in order.

[0020] If the color of the target vehicle in the target vehicle body region picture does not belong to any of the pre-set ten colors in the color labeling, skip the color labeling; connect the closed color contour of the target vehicle in the target vehicle body region picture with a dotted line, and then calculate the proportion of each color in the total area of all labeled colors through the labeling software to generate labeling information.

[0021] Preferably, the deep learning model is multi-label learning in multi-task, the multi-task can learn shared representation of multiple tasks, the shared representation has strong abstract ability, can adapt to multiple different but related indicators, and through learning of related tasks, the main task can obtain better generalization ability.

[0022] Preferably, the multi-task includes two tasks: vehicle color sequence regression and vehicle type classification.

[0023] Preferably, the deep learning model obtains label labeling data through a Data layer, and divides the label labeling data into two parts through a Slice layer, one part is a color sequence vector, and the other part is a vehicle type category.

[0024] The label labeling data divided into a color sequence vector and a vehicle type category by the Slice layer is input into the pooling layer and the convolution layer of the big core, so that the global color ability of the network is improved, and the main network structure of the network adopts the VGG structure of 3x3, 1x1 convolution and 2x2 pooling layer.

[0025] Finally, the full connection layer is divided into two full connection layers, one corresponding to the color sequence vector and the other corresponding to the vehicle type category, and then the two full connection layers are respectively connected to the SoftmaxWithLoss layer to obtain the loss.

[0026] Preferably, the method for identifying the vehicle color is used for identifying the vehicle color in a video stream obtained by road monitoring video.

[0027] The embodiment of the present application provides a method for identifying a vehicle color.

[0028] From the above technical solution, the embodiment of the present application has the following advantages:

[0029] The embodiment of the present application provides a method for identifying a vehicle color, obtains a target vehicle picture, and processes the target vehicle picture to obtain a target vehicle body region picture; the target vehicle body region picture is input into a network of a deep learning model with multi-label color sequence identification to obtain a color sequence regression result; the color sequence recognition result is obtained by calculating the proportion of each color sequence in the color sequence regression result; the color sequence recognition result is sorted, and the result greater than a threshold value is output, which is mapped to a corresponding color sequence, to obtain a vehicle color, a multi-label deep learning model of vehicle type annotation is added, and the internal expression of the hidden layer can be better learned. Meanwhile, the color sequence is used to annotate the color block of the vehicle region, and the color proportion is calculated, so that the model can better learn the fine-grained features. The method has strong robustness in the complex conditions such as multiple angle changes, strong light and backlight in the road video monitoring environment. The technical problems that the current vehicle body color method based on a local region is prone to be biased, the color space distribution is complex, the color difference is prone to occur due to the influence of light intensity and shadow, the vehicle body color distribution of different vehicle types is different, some vehicles have multiple color combinations, and the multi-angle color distribution difference exists, a large amount of data and a large number of network parameters are required, and the method is not easy to fit and has poor robustness are solved. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 FIG. 1 is a schematic diagram of an annotation operation of a method for identifying a vehicle color in the embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiment of the present application provides a method for identifying the color of a vehicle, which solves the technical problems that the current vehicle body color method based on a local area is prone to be biased, color space distribution is complex, color difference is prone to occur due to the influence of light intensity and shadow, the color distribution of the vehicle body of different vehicle models is different, some vehicles have multiple color combinations and multi-angle color distribution differences, a large amount of data and many network parameters are required, and the method is not easy to fit and has poor robustness.

[0032] In order to make the application purpose, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the following described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0033] One embodiment of the method for identifying the color of a vehicle provided by the present application comprises the following steps:

[0034] A target vehicle picture is obtained, and a target vehicle body area picture is obtained by processing the target vehicle picture; the target vehicle body area picture is input into a network of a deep learning model with multi-label color sequence recognition to obtain a color sequence regression result; the color sequence recognition result is obtained by calculating the proportion of each color sequence in the color sequence regression result; the color sequence recognition result is sorted, and the result greater than a threshold value is output to map the corresponding color sequence, thereby obtaining the color of the vehicle.

[0035] Specifically, the method for obtaining the target vehicle body area picture is to use a deep learning SSD to detect the target vehicle in the target vehicle picture, and to obtain the target vehicle body area picture by using a target tracking algorithm.

[0036] In the embodiment, the deep learning model with multi-label color sequence recognition is a multi-task multi-label deep learning model with preset training data. The preset training data includes type labeling and color labeling on the target vehicle body area picture. The type labeling is specifically performed as follows: the target vehicle is labeled as one of eight types, including a large bus, a sedan, a small bus, a small truck, a micro van, a pickup truck, a business and off-road vehicle, and a large truck, according to the target vehicle body area picture. The color in the color labeling is pre-set to have ten types, including red, blue, yellow, black, green, brown, gray, pink, white, orange and purple.

[0037] Using the color sequence labeling tool, the part of the target vehicle body area picture that has been type-labeled except the part of the vehicle window, windshield, wheel and air intake grid is labeled with color. After completion, import the labeling software and select the first ten colors in order;

[0038] If the color of the target vehicle in the target vehicle body area picture does not belong to any of the ten preset colors, skip the color labeling; connect the closed color contour of the target vehicle in the target vehicle body area picture with a dotted line, and then calculate the proportion of each color in the total area of all labeled colors through the labeling software to generate the labeling information.

[0039] Specifically, the deep learning model is multi-label learning in a multi-task, the multi-task can learn a shared representation of multiple tasks through learning, the shared representation has strong abstract ability and can adapt to multiple different but related indicators, and through learning of related tasks, the main task can obtain better generalization ability.

[0040] In this embodiment, the multi-task includes two tasks of vehicle color sequence regression and vehicle type classification.

[0041] Specifically, the deep learning model obtains label-labeled data through a Data layer, and divides the label-labeled data into two parts through a Slice layer, one part is a color sequence vector and the other part is a vehicle type category.

[0042] The label-labeled data divided into a color sequence vector and a vehicle type category through the Slice layer is input into a pooling layer and a convolution layer of a big core, so that the global color ability of the network is improved, and the main network structure of the network adopts a VGG structure of 3x3, 1x1 convolution and 2x2 pooling layer.

[0043] Finally, the full connection layer is divided into two full connection layers, one corresponding to the color sequence vector and the other corresponding to the vehicle type category, and then the two full connection layers are respectively connected to the SoftmaxWithLoss layer to calculate the loss.

[0044] In this embodiment, the method for identifying the color of the vehicle can be used for identifying the color of the vehicle in the video stream obtained from the road monitoring video.

[0045] Figure 1 The annotation operation schematic diagram of the method for identifying the color of the vehicle in this embodiment comprises:

[0046] 1. Training data preparation

[0047] The vehicle region is detected from the road monitoring video stream, the vehicle body picture is cut, and the vehicle body picture is divided into 8 categories according to the vehicle type: bus, sedan, minibus, minivan, micro van, pickup truck, business and off-road vehicle, and truck.

[0048] The color labeling is divided into 10 categories: red, blue, yellow, black, green, brown, gray, pink, white, orange, and purple. The color sequence labeling tool is used to label the color of the above classified picture. The labeling software is opened, the picture is imported, the first 10 colors are selected in sequence, if the selected color does not exist in the vehicle, the labeling is skipped, and the parts such as the window, windshield, wheel and air intake grid are not labeled into color. The closed color contour of the target vehicle is connected with a dotted line, the software will calculate the proportion of each color in the total area of all labeled colors respectively, and then generate the labeling information.

[0049] 2. Model training

[0050] In the model design, multi-label learning in deep learning multi-task is adopted. Multi-task learning can learn the shared representation of multiple tasks, and the shared representation has strong abstract ability and can adapt to multiple different but related indicators. The learning of related tasks can make the main task have better generalization ability.

[0051] As an inductive transfer method, multi-label, color and type label can use additional information to improve the learning performance of the current task, including improving the generalization accuracy, learning speed and reducing the model parameters. In the embodiment, the multi-task is vehicle color sequence regression and vehicle type classification.

[0052] The label annotation data is obtained through the Data layer, and then divided into two parts through the Slice layer. One part is the color sequence vector, and the other part is the vehicle type category.

[0053] The data is input into the pooling layer and the convolution layer of the big core, so that the global color ability of the network is improved. The main network structure adopts the VGG structure of 3x3, 1x1 convolution and 2x2 pooling layer. The last full connection layer is divided into two full connection layers, one full connection layer corresponds to the color sequence vector, and the other full connection layer corresponds to the type category. Finally, the SoftmaxWithLoss layer is connected to calculate the loss.

[0054] 3. Model testing

[0055] The video stream obtained from the road monitoring video uses a deep learning SSD to detect a vehicle target, and outputs a vehicle body region of the target vehicle through a target tracking algorithm. The vehicle body region is input into a network to obtain a regression result from a color sequence corresponding full connection layer. The result is sorted, and the result greater than a threshold is output, which is mapped to a corresponding color sequence. The embodiments of the application are tested in an actual scene, and the average accuracy of all colors is 95%, which is a method for identifying vehicle color with high accuracy and strong robustness.

[0056] The opening degree. For the case of non-frontal face, the opening degree of the human eye of the embodiments of the application only processes the side face with a small angle. First, the nose bridge is positioned according to the candidate region roughly dividing the human eye, and then the human face image is binarized, and the number of pixels with a pixel value of 0 in the human eye region is counted, which is used as a measurement index of the opening degree of the human eye.

[0057] The embodiments of the application are designed for the large difference in color distribution of different vehicle types in the road monitoring video, the various vehicle angles, the multiple colors of a single vehicle body, and the complex situations such as strong light, backlight interference, etc. A deep learning model for multi-label color sequence recognition is designed. The following advantages are provided:

[0058] (1) The vehicle color sample increases the vehicle type classification, adopts a multi-label deep model for training, and helps the shared hidden layer to learn a better mapping relationship through additional information.

[0059] (2) Instead of simply performing color classification, the vehicle color block is finely annotated, the proportion of each color sequence is calculated, and the color sequence recognition result is regressed.

[0060] The method proposed in the embodiments of the application can better learn the internal expression of the hidden layer due to the multi-label deep learning model with vehicle type annotation. At the same time, the color block of the vehicle region is annotated using the color sequence, and the color proportion is calculated, which can make the model better learn the fine-grained features. The embodiments of the application have strong robustness in the complex situations such as various angle changes, strong light, and backlight in the road video monitoring environment.

[0061] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the foregoing embodiments of the present application have been described in detail, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of identifying a color of a vehicle, the method comprising: The method comprises the following steps: acquiring a target vehicle picture and processing the target vehicle picture to obtain a target vehicle body region picture; inputting the target vehicle body region picture into a deep learning model network with multi-label color sequence recognition to obtain a color sequence regression result; the deep learning model is multi-task multi-label learning, the multi-task can learn a shared representation of multiple tasks, and the shared representation can adapt to multiple different but related indicators; the multi-task comprises two tasks of vehicle color sequence regression and vehicle type classification; the color sequence regression calculates the proportion of each color in the total area of all labeled colors by labeling the closed color contour of the part of the target vehicle body region except the window, windshield, wheel and air intake grid part region, and generates labeling information; color sequence recognition results are obtained by calculating the proportions of the color sequences in the color sequence regression results; sorting the color sequence recognition results and outputting results greater than a threshold value to map corresponding color sequences to obtain vehicle colors.

2. The method of identifying a color of a vehicle of claim 1, wherein, The method of acquiring a target vehicle picture and processing the target vehicle picture to obtain a target vehicle body region picture specifically comprises: using a deep learning SSD to detect a target vehicle in the target vehicle picture, and obtaining the target vehicle body region picture through a target tracking algorithm.

3. The method of identifying a color of a vehicle of claim 1, wherein, The deep learning model for multi-label color sequence recognition is a multi-task multi-label deep learning model with preset training data.

4. The method of identifying a color of a vehicle of claim 3, wherein, The preset training data comprises type labeling and color labeling of the target vehicle body region picture.

5. The method of identifying a color of a vehicle of claim 4, wherein, The type labeling specifically comprises labeling a target vehicle as one of eight types of large bus, sedan, small bus, small truck, mini-van, pickup truck, business and off-road vehicle, and large truck according to the target vehicle body region picture.

6. The method of identifying a color of a vehicle of claim 5, wherein, The colors in the color labeling are pre-set as red, blue, yellow, black, green, brown, gray, pink, white and orange. A color sequence labeling tool is used to label the part of the target vehicle body region picture except the window, windshield, wheel and air intake grid part region after the type labeling, and the first ten colors are selected in sequence after importing the labeling software. If the color of the target vehicle in the target vehicle body region picture does not belong to any of the pre-set ten colors in the color labeling, the color labeling is skipped, the closed color contour of the target vehicle in the target vehicle body region picture is connected with a dotted line, and then the proportion of each color in the total area of all labeled colors is calculated through the labeling software to generate labeling information.

7. The method of identifying a color of a vehicle of claim 1, wherein, The deep learning model acquires label labeling data through a Data layer and divides the label labeling data into two parts through a Slice layer, one part being a color sequence vector and the other part being a vehicle type category. The label annotation data divided into two parts of color sequence vectors and vehicle type categories by the Slice layer is input into a pooling layer and a convolution layer of a big core, and a main network structure of the network adopts a VGG structure of 3x3, 1x1 convolution and 2x2 pooling layer; Finally, full connection layers are divided into two full connection layers, one corresponding to the color sequence vectors and the other corresponding to the vehicle type categories, and then the two full connection layers are respectively connected to SoftmaxWithLoss layers to obtain losses.

8. The method of identifying a color of a vehicle according to any one of claims 1-7, wherein, The method for recognizing vehicle colors is used for recognizing vehicle colors in a video stream obtained by road monitoring video.

Images