Vehicle body color recognition method, storage medium and terminal
By combining semantic segmentation and Mask-RCNN with center filling, the accuracy problem of vehicle body color recognition in harsh environments was solved, achieving high-accuracy vehicle body color recognition.
Patent Information
- Application Number
- CN202110554335.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-20
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-05-20
AI Technical Summary
Existing technologies for vehicle body color recognition are affected by factors such as lighting, weather, and background interference, resulting in low recognition accuracy, difficulty in distinguishing the background color of the area outside the vehicle body, and easy to cause category overlap and misjudgment in harsh environments.
Mask regions of vehicle body, windows and license plate are extracted by semantic segmentation. Mask-RCNN multi-task convolutional neural network is used for target detection and color classification. The color histogram of candidate vehicle body regions is processed by center filling method. Convolutional neural networks such as MobileNet or ResNet are used for color recognition.
It improves the accuracy of vehicle body color recognition, and can maintain high recognition accuracy even in adverse environments such as color blocking, reflection, obstruction, and tree shadows.
Smart Images

Figure CN113239836B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the graphic recognition technology in the field of transportation, and in particular relates to a vehicle body color recognition method, a storage medium and a terminal. Background Art
[0002] The body color of a vehicle is an important information of a motor vehicle and plays an important role in vehicle-related algorithm applications such as automatic driving, criminal investigation and judgment by the public security, road management by the traffic police, and punishment for violations. Since color information is continuous information, it is restricted by various factors such as lighting, weather, background interference, and camera imaging, making it difficult to obtain a high accuracy rate in recognition.
[0003] Patent CN106384117A proposes a method of obtaining a region of interest (ROI) by dividing the vehicle into a cross shape, and achieving color recognition by statistically analyzing the RGB color histogram and the HSV color histogram. However, this method cannot distinguish the background color of the area outside the vehicle body within the vehicle ROI, and the analysis method of statistical color histogram is sensitive to light, which cannot effectively increase the distance between color categories, and easily causes category overlap and misjudgment.
[0004] Patent CN108563976A proposes a method of obtaining local color recognition candidate areas of the vehicle face and body through window positioning for color recognition. This method can exclude pseudo-recognition areas outside the vehicle body to a certain extent, but the candidate local area is too small, and the overall color judgment cannot be obtained in the case of multiple colors of the vehicle, sunlight reflected by the vehicle body, large angle of the vehicle, tree shadows, building shadows, occlusion, etc. Summary of the invention
[0005] In order to overcome the deficiencies of the prior art, an object of the present invention is to provide a vehicle body color recognition method, a storage medium and a terminal, which can solve the above-mentioned problems.
[0006] Purpose of the invention: By obtaining the actual vehicle body area through image segmentation, the overall vehicle body area can be extracted for color judgment and analysis, and it has a high recognition accuracy in harsh environments such as color matching, reflection, occlusion, and tree shadows.
[0007] Design:
[0008] A vehicle body color recognition method, the method comprising the following steps:
[0009] Step S1, extracting the mask area of the vehicle body, window and license plate, wherein the mask area is extracted by performing a semantic segmentation detector on the monitoring image;
[0010] Step S2, obtaining a vehicle body candidate area image to be color recognized;
[0011] Step S3, filling the body candidate area image;
[0012] Step S4: identifying the color. The identification of the vehicle body color includes:
[0013] 4.1 Establish a multi-task convolutional neural network target detection model Mask-RCNN; the multi-task loss function in the target detection model Mask-RCNN is: L = L cls +L box +L mask +L clr ;
[0014] Where L represents the total loss of the multi-task loss function, L cls represents the target classification loss, L box Represents the rectangular area loss of target detection, L mask represents the target mask loss, L clr represents the target color classification loss;
[0015] 4.2 Input the vehicle body candidate area image after color filling in step S3 into a convolutional neural network color classification model (including but not limited to Mobilenet, ResNet, etc.) to obtain the recognition probability Pj of the vehicle body color;
[0016] 4.3 Result fusion to obtain the final recognized color category C and confidence P.
[0017] Preferably, in step S2, the body candidate region image Im to be color recognized is obtained by taking the union of the mask regions of the body, the window and the license plate, and then taking the intersection with the original image, that is, I m =I s ∩(M b ∪M w ∪M p );
[0018] In the formula, I m Represents the candidate area image of the vehicle body to be color recognized, I s Represents the original captured image, M b Represents the body mask area, M w Represents the window mask area, M p Represents the license plate mask area.
[0019] Preferably, in step S3, the vehicle body candidate area image is filled in a center filling manner, which specifically includes the following steps:
[0020] Step S31, dividing the vehicle body candidate area image into m image blocks at equal angles from the center point, where m is a positive integer ≥ 2;
[0021] Step S32, solving the color histogram of each image block;
[0022] Step S33, solving the average color grayscale Di of the color histogram of the first n image blocks:
[0023] Where D i Represents the average color grayscale of the i-th angle image block, H j Represents the grayscale value of the jth color, G(H j ) represents the number of pixels of the grayscale value, and n represents the grayscale index of the first n colors with the largest number in the color histogram;
[0024] Step 34: For each angle of the image block, fill the area outside the mask with the average color of the angle.
[0025] Preferably, m is 10 and n is 20.
[0026] The present invention also provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed, the above method is executed.
[0027] A terminal comprises a memory and a processor, wherein the memory stores a graphic data set comprising a training set and a test set and computer instructions that can be run on the processor, and the processor executes the aforementioned method when running the computer instructions.
[0028] Compared with the prior art, the beneficial effect of the present invention is that the vehicle body color can be quickly identified through this solution, thereby improving the accuracy of identification. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A schematic diagram of the process of the vehicle body color recognition method of the present invention;
[0030] Figure 2 Extracting schematic diagrams for mask regions;
[0031] Figure 3 Fill the schematic diagram with the car body color;
[0032] Figure 4 Schematic diagram of training and testing the car body image dataset. DETAILED DESCRIPTION
[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0035] First embodiment
[0036] A vehicle body color recognition method, see Figure 1 , the method comprises the following steps:
[0037] Step S1: extract the mask area of the vehicle body, window and license plate, see Figure 2 The mask area is extracted by performing a semantic segmentation detector on the surveillance image. The semantic segmentation detector can be a conventional RCNN semantic segmentation detector, which will not be described here.
[0038] Step S2: obtaining a vehicle body candidate region image to be color recognized.
[0039] Specifically, in step S2, the body candidate region image Im to be color recognized is obtained by taking the union of the mask regions of the body, window and license plate and then taking the intersection with the original image:
[0040] I m =I s ∩(M b ∪M w ∪M p )…………………………………………Formula 1;
[0041] In the formula, I m Represents the candidate area image of the vehicle body to be color recognized, I s Represents the original captured image, M b Represents the body mask area, M w Represents the window mask area, M p Represents the license plate mask area.
[0042] Step S3: Fill the body candidate area image, see Figure 3Specifically, in step S3, the vehicle body candidate area image is filled in a center filling manner, which specifically includes the following steps:
[0043] Step S31: Divide the vehicle body candidate region image into m image blocks at equal angles from the center point, where m is a positive integer ≥2.
[0044] The image is divided into m equal blocks, and the angle of each block is 360 / m. The larger the m, the finer the block division, and the smoother the filling of the area outside the vehicle body mask, but the fence effect is increased and the calculation burden is increased; the smaller the m, the less obvious the fence effect is, but the filling of the area outside the vehicle body mask is not smooth enough. Experiments show that when m is 10, the algorithm has good effect and strong versatility.
[0045] Step S32: Calculate the color histogram of each image block.
[0046] Step S33, solving the average color grayscale Di of the color histogram of the first n image blocks:
[0047]
[0048] Where D i Represents the average color grayscale of the i-th angle image block, H j Represents the grayscale value of the jth color, G(H j ) represents the number of pixels of this grayscale value, and n represents the first n color grayscale indexes with the largest number in the color histogram.
[0049] The larger the n, the wider the range of grayscale involved in the calculation of the average color value, the weight of the majority color will be reduced, and the calculation burden will increase; the smaller the n, the greater the randomness of the average color. Experiments show that when n is 20, the algorithm has good effect and strong versatility.
[0050] Step 34: For each angle of the image block, fill the area outside the mask with the average color of the angle.
[0051] Step S4: identifying the color. The identification of the vehicle body color includes:
[0052] 4.1 Establish a multi-task convolutional neural network target detection model Mask-RCNN.
[0053] The multi-task loss function in the target detection model Mask-RCNN is:
[0054] L=L cls +L box +L mask +L clr …………………………Formula 3;
[0055] In the formula, L represents the total loss of the multi-task loss function, Lcls represents the target classification loss, L box Represents the rectangular area loss of target detection, L mask represents the target mask loss, L clr represents the target color classification loss.
[0056] 4.2 Input the vehicle body candidate area image after color filling in step S3 into the convolutional neural network color classification model (including but not limited to conventional color classification models such as Mobilenet, ResNet, etc., in any form) to obtain the recognition probability Pj of the vehicle body color; specifically, input the vehicle image after uniform color filling into the classification network 2 to obtain the color recognition probability Pj.
[0057] In step 4.1, see Figure 4 , the car body color recognition of the convolutional neural network target detection model Mask-RCNN includes the following steps:
[0058] Step 421: Acquire a vehicle body image dataset with the vehicle body candidate area image as the detection target.
[0059] Step 412: Label the vehicle body color, thereby dividing the vehicle body image dataset into a training set and a test set.
[0060] Step 413, the dataset is imported into the target detection model Mask-RCNN, and the training set of the dataset is preprocessed and semi-supervised model training is performed through the model to obtain a trained color classification model; the test set of the dataset is subjected to color classification reasoning and color classification through the trained color classification model.
[0061] Step 414, label matching output result, after color inference, uses the vehicle color candidate area and the vehicle color output result to do matching, realizes the classification of data classification labels, and performs weighted fusion to obtain the final output result.
[0062] 4.3 Result fusion to obtain the final recognized color category C and confidence P.
[0063] Second embodiment
[0064] The present invention also provides a computer-readable storage medium on which computer instructions are stored, and when the computer instructions are executed, the steps of the above method are executed. The method is described in detail in the above part and will not be described here.
[0065] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carriers.
[0066] Third embodiment
[0067] A terminal includes a memory and a processor, wherein the memory stores a graphic data set including a training set and a test set and a computer instruction that can be run on the processor, and the processor executes the method of the first embodiment when running the computer instruction. The method is described in detail in the above section and will not be described here.
[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A vehicle body color recognition method, characterized in that: The method comprises the following steps: Step S1, extracting the mask area of the vehicle body, window and license plate, wherein the mask area is extracted by performing a semantic segmentation detector on the monitoring image; Step S2, obtaining a vehicle body candidate area image to be color recognized; Step S3, filling the vehicle body candidate area image by center filling method, including the following steps: Step S31, dividing the vehicle body candidate area image into m image blocks at equal angles from the center point, where m is a positive integer ≥ 2; Step S32, solving the color histogram of each image block; Step S33, solving the average color grayscale Di of the color histogram of the first n image blocks: Where D i Represents the average color grayscale of the i-th angle image block, H j Represents the grayscale value of the jth color, G(H j ) represents the number of pixels of the grayscale value, and n represents the grayscale index of the first n colors with the largest number in the color histogram; Step 34: For each image block at each angle, fill the area outside the mask with the average color of the angle; Step S4: identifying the color. The identification of the vehicle body color includes: 4.1 Establish a multi-task convolutional neural network target detection model Mask-RCNN; the multi-task loss function in the target detection model Mask-RCNN is: L = L cls +L box +L mask +L clr ; Where L represents the total loss of the multi-task loss function, L cls represents the target classification loss, L box Represents the rectangular area loss of target detection, L mask represents the target mask loss, L clr represents the target color classification loss; 4.2 Input the vehicle body candidate area image after color filling in step S3 into the convolutional neural network color classification model to obtain the recognition probability Pj of the vehicle body color; 4.3 Result fusion to obtain the final recognized color category C and confidence P.
2. The method according to claim 1, characterized in that: In step S2, the body candidate region image Im to be color recognized is obtained by taking the union of the mask regions of the body, window and license plate and then taking the intersection with the original image, i.e., I m =I s ∩(M b ∪M w ∪M p ); In the formula, I m Represents the candidate area image of the vehicle body to be color recognized, I s Represents the original captured image, M b Represents the body mask area, M w Represents the window mask area, M p Represents the license plate mask area.
3. The method according to claim 1, characterized in that: m is 10 and n is 20.
4. The method according to claim 1, characterized in that: In step 4.1, the vehicle body color recognition of the convolutional neural network target detection model Mask-RCNN includes the following steps: Step 411, obtaining a vehicle body image data set with a vehicle body candidate area image as a detection target; Step 412: marking the vehicle body color, thereby dividing the vehicle body image data set into a training set and a test set; Step 413: The dataset is imported into the target detection model Mask-RCNN. After the training set of the dataset is preprocessed, semi-supervised model training is performed through the model to obtain a trained color classification model; the test set of the dataset is subjected to color classification reasoning and color classification through the trained color classification model; Step 414, label matching output result, after color inference, uses the vehicle color candidate area and the vehicle color output result to do matching, realizes the classification of data classification labels, and performs weighted fusion to obtain the final output result.
5. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed, the method according to any one of claims 1 to 4 is executed.
6. A terminal comprising a memory and a processor, characterized in that: The memory stores a graphic data set including a training set and a test set, and computer instructions that can be executed on the processor, and the processor executes the method according to any one of claims 1 to 4 when executing the computer instructions.
