Vehicle image recognition method, electronic device and computer readable storage medium

Abstract

The present invention discloses a vehicle image recognition method, an electronic device and a computer-readable storage medium, wherein the method comprises constructing a set of vehicle images to be filtered according to vehicle images acquired in real time; filtering the set of vehicle images to be filtered to obtain images to be recognized; performing front and rear vehicle body recognition on the images to be recognized to obtain corresponding front vehicle body images and rear vehicle body images; inputting the images to be recognized into a preset deep learning detection model for storing the correspondence between image features and vehicle body area features to obtain a number of set vehicle body area feature images; performing feature fusion on the front vehicle body image and the vehicle body area feature image related to the front vehicle body image, and performing feature fusion on the rear vehicle body image and the vehicle body area feature image related to the rear vehicle body image; and outputting the fused image result. The vehicle image recognition method provided by the present invention improves the accuracy of vehicle image recognition by configuring a vehicle image recognition strategy based on a deep learning model.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle detection, and in particular to a vehicle image recognition method, electronic equipment and a computer-readable storage medium. Background Art

[0002] With the continuous development of urban construction, the traffic management department has equipped each road checkpoint with video surveillance facilities to photograph and record all motor vehicles passing through the checkpoint, aiming to identify the corresponding vehicle information through the captured vehicle images.

[0003] Considering that the environmental conditions of the checkpoint are greatly affected by lighting, and the checkpoint located in the transportation hub has a large traffic volume and a fast speed, it is necessary to recognize and process the vehicle images obtained by the checkpoint. However, in current practical applications, some more traditional recognition methods are used to recognize vehicle images, such as HOG, LBP, Haar and other recognition methods. The advantages of these traditional methods are fast recognition speed, easy transplantation, and better practicality on embedded platforms. However, the recognition accuracy of vehicle images is not high, and it is difficult to accurately recognize the characteristics of the vehicle body area, resulting in the actual recognition effect being less than ideal. Summary of the invention

[0004] The present invention provides a vehicle image recognition method, an electronic device and a computer-readable storage medium to solve the technical problem of low precision of existing vehicle image recognition methods. By configuring a vehicle image recognition strategy based on a deep learning model, the accuracy of vehicle image recognition is improved.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a vehicle image recognition method, the method comprising:

[0006] Constructing a vehicle image set to be filtered according to the vehicle images acquired in real time;

[0007] Filtering the vehicle image set to be filtered based on image exposure to obtain an image to be identified;

[0008] Performing front and rear vehicle body recognition on the image to be recognized to obtain corresponding front vehicle body images and rear vehicle body images;

[0009] Inputting the image to be identified into a preset deep learning detection model for storing the correspondence between image features and vehicle body area features, to obtain a plurality of set vehicle body area feature maps;

[0010] Performing feature fusion on the front body image and the body area feature image related to the front body image, and performing feature fusion on the rear body image and the body area feature image related to the rear body image;

[0011] Output the fused image results respectively.

[0012] As one of the preferred solutions, the vehicle image set to be filtered is constructed based on the vehicle images acquired in real time, specifically:

[0013] Acquire vehicle images in real time;

[0014] Adding different degrees of random noise to the vehicle image respectively to obtain sample images;

[0015] A vehicle image set to be filtered is constructed using the sample images.

[0016] As one preferred solution, after acquiring the vehicle image in real time, the method further includes:

[0017] The vehicle image is preprocessed, wherein the preprocessing includes cropping and rotation transformation.

[0018] As one preferred solution, after adding different degrees of random noise to the vehicle images to obtain sample images, the method further includes:

[0019] The resolution of each of the sample images is adjusted respectively.

[0020] As one of the preferred solutions, filtering the vehicle image set to be filtered based on image exposure to obtain the image to be identified is specifically as follows:

[0021] Set the filter range of image exposure;

[0022] Calculating the exposure of each of the sample images;

[0023] The sample images whose exposure is outside the range are screened and filtered to obtain the corresponding images to be identified.

[0024] As one of the preferred solutions, the front and rear vehicle body recognition is performed on the image to be recognized to obtain the corresponding front vehicle body image and rear vehicle body image, specifically:

[0025] The image to be identified is input into the front and rear body classification model for analysis, and the corresponding front body image and rear body image are output, wherein the front and rear body classification model is one of the following: ZfNet model, ResNet model, MobileNet model.

[0026] As one of the preferred solutions, the set vehicle body area characteristic map at least includes front and rear wiper characteristic maps, front and rear light characteristic maps, front and rear window glass characteristic maps and an air intake grille characteristic map.

[0027] As one of the preferred solutions, the front body image and the body area feature map related to the front body image are feature-fused, and the rear body image and the body area feature map related to the rear body image are feature-fused, specifically:

[0028] Respectively extracting image features of the front-shot vehicle body image, the front wiper feature image, the front headlight feature image, the front window glass feature image, and the air intake grille feature image, and outputting corresponding front-shot probability images;

[0029] Perform weighted fusion based on the confidence of each of the preceding probability maps; and,

[0030] Respectively extracting image features of the rear-shot vehicle body image, the rear wiper feature image, the rear lamp feature image, and the rear window glass feature image, and outputting corresponding rear-shot probability images;

[0031] Weighted fusion is performed based on the confidence of each of the subsequent shot probability maps.

[0032] Another embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the vehicle image recognition method as described above when executing the computer program.

[0033] Yet another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle image recognition method as described above.

[0034] Compared with the prior art, the beneficial effects of the embodiments of the present invention lie in at least one of the following: first, a set of vehicle images to be filtered is constructed based on the vehicle images acquired in real time, thereby improving the richness of samples and providing accurate data support for subsequent deep learning-based image processing strategies; then, based on the judgment of exposure, vehicle images whose image quality obviously does not meet expectations are filtered out, thereby reducing the computational complexity of subsequent detection models; then, front and rear shot recognition is performed on the vehicle image, thereby dividing the vehicle image into two corresponding recognition areas, providing double guarantees for accurately identifying the vehicle's body features; thirdly, relevant feature areas of the vehicle image are extracted, and the extracted feature areas are fused with the corresponding recognition areas, so that the fused vehicle image has accurate body area features, and the dual perspective division of the front and rear shots of the vehicle can guarantee comprehensive, multi-category body area features, thereby improving the quality of the final output image, thereby improving the precision and accuracy of vehicle image recognition, and ensuring the practicality and reliability of vehicle image recognition at the bayonet. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic flow chart of a vehicle image recognition method in one embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of a process for constructing a vehicle image set to be filtered in one embodiment of the present invention;

[0037] Figure 3 It is a schematic diagram of the filtering process in one embodiment of the present invention;

[0038] Figure 4 is a schematic diagram of the fusion process in one embodiment of the present invention;

[0039] Figure 5 is a schematic diagram of the identification process in one embodiment of the present invention;

[0040] Figure 6 is a schematic structural diagram of an electronic device in one embodiment of the present invention;

[0041] Among them, 10, processor; 20, memory; A, computer program. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0044] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0045] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0046] An embodiment of the present invention provides a method for recognizing a vehicle image. For details, see Figure 1 , Figure 1 The figure is a flow chart of a method for recognizing a vehicle image in one embodiment of the present invention, wherein the method for recognizing a vehicle image specifically comprises:

[0047] S1. Construct a set of vehicle images to be filtered based on the vehicle images acquired in real time. In this embodiment, the vehicle images acquired in real time are key frame vehicle images acquired by the video surveillance equipment of each checkpoint, such as a vehicle body area image with a resolution of 160*300. Of course, the resolution specifications of the vehicle images acquired by each checkpoint are determined by the specifications of the shooting components of the checkpoint, which will not be repeated here.

[0048] S2. Filter the vehicle image set to be filtered based on image exposure to obtain images to be identified. In order to first screen out vehicle images that are obviously not of the correct quality, this embodiment filters the vehicle image set to be filtered based on image exposure, thereby reducing the amount of data processed and the consumption of running memory of the subsequent detection model.

[0049] S3. Perform front and rear vehicle body recognition on the image to be recognized to obtain corresponding front vehicle body images and rear vehicle body images; dividing the vehicle image into front and rear images is beneficial to constructing dual image guarantees, thereby improving the accuracy of vehicle recognition; on the other hand, the division of the front and rear perspectives is also beneficial to the subsequent fusion of the corresponding vehicle body area features with the vehicle image of the corresponding perspective, thereby providing accurate data support for accurately identifying the vehicle image.

[0050] S4. Input the image to be identified into a preset deep learning detection model for storing the correspondence between image features and vehicle body area features, and obtain a number of set vehicle body area feature maps; of course, the set vehicle body area feature maps are obtained in order to better identify vehicle images and prevent the loss of most vehicle body features due to overexposure or too dark environmental factors.

[0051] S5, feature fusion of the front vehicle body image and the vehicle body area feature map related to the front vehicle body image, and feature fusion of the rear vehicle body image and the vehicle body area feature map related to the rear vehicle body image; the multi-task image convolution feature extraction stage can share weights, and the classification stage can adapt to multiple different but related targets and has good generalization ability. The present invention will use the shared weight features to feature fuse the related vehicle body images, thereby improving the recognition precision and accuracy.

[0052] S6. Output the fused image results respectively.

[0053] It should be noted that in the checkpoint vehicle detection scenario, the camera needs to collect information about passing vehicles for identification. Due to the complex lighting conditions and the large number of vehicles in the checkpoint environment, the captured vehicle body area is usually overexposed or too dark, resulting in loss of features in the vehicle body area. The vehicle detection and identification system at the checkpoint will fail to detect the vehicle body area, or fail to identify the vehicle or misidentify the vehicle. Currently, there is no effective method for vehicle detection and identification at checkpoints. Vehicle identification requires strong robustness and requires classification and regression of body area features. For example, the inherent window glass in the window area is too dark, the headlights are overexposed due to the illumination of the lights, the night checkpoint environment is too dark, causing the body image to be too dark, and the body area features are completely lost and mixed with the background, resulting in failure of body detection or outputting the judgment result as the background as the detection result. At present, the commonly used image features are HOG, LBP, and Haar. Among them, HOG (Histogram of Oriented Gradient) is a feature used to calculate and count the gradient direction histogram of the local area of ​​the image; LBP (Local Binary Pattern) is an operator used to describe the local texture features of the image, with the advantages of rotation invariance and grayscale invariance; Haar features are composed of linear features, edge features, center features, and diagonal features to form a feature template. Commonly used combinations include HOG+SVM (Support Vector Machines), DPM (Deformable Part Model), etc. French researcher Dalal proposed the HOG+SVM method for pedestrian detection at the CVPR in 2005. This combination is the basis of many improved pedestrian detection algorithms. The DPM algorithm was proposed by Felzenszwalb in 2008. It uses the improved HOG feature, SVM classifier and sliding window detection ideas, and has strong robustness to target deformation. Paul Viola and Miachael Jones proposed the Adaboost (Adaptive Boosting) + Haar face detection algorithm at the CVPR in 2001, which also achieved good results.However, the inventors have found through research that the advantages of the above-mentioned traditional method are fast speed and easy transplantation, but it requires manual feature design and is not very robust. In the detection and recognition of vehicle markers, a single category detection is usually performed on the category of the window marker, and there are problems such as low detection accuracy and insufficient categories. In order to improve the above-mentioned shortcomings, the embodiment of the present invention divides the vehicle image into two corresponding recognition areas, providing a double guarantee for accurately identifying the body features of the vehicle; secondly, the relevant feature areas of the vehicle image are extracted, and the extracted feature areas are fused with the corresponding recognition areas, so that the fused vehicle image has accurate body area features, and the dual perspective division of the front and rear shots of the vehicle can guarantee comprehensive, multi-category body area features, thereby improving the quality of the final output image, thereby improving the precision and accuracy of vehicle image recognition, and ensuring the practicality and reliability of bayonet vehicle image recognition.

[0054] Further, in the above embodiment, for step S1: constructing a vehicle image set to be filtered based on the vehicle images acquired in real time, for details, please refer to Figure 2 , Figure 2 The flowchart of constructing a vehicle image set to be filtered in one embodiment of the present invention is shown, which includes:

[0055] S11, acquiring vehicle images in real time;

[0056] S12, adding different degrees of random noise to the vehicle image to obtain sample images;

[0057] S13: construct a vehicle image set to be filtered using the sample images.

[0058] It should be noted that in the process of random noise addition, the existing technology can be selected to use python to generate additive zero-mean Gaussian noise, add a random number to the gray value of each point, and use the Box-Muller algorithm, which uses two sets of independent random numbers generated by uniform distribution to generate normally distributed random variables X and Y through the following formula. In addition, in the concept of noise, the signal-noise ratio (SNR) is usually used to measure image noise. The different degrees of random noise in this embodiment are judged based on the indicator of the signal-noise ratio. Of course, it is also necessary to consider the different requirements of each checkpoint for vehicle image recognition, which will not be repeated here.

[0059] Furthermore, in the above embodiment, after the real-time acquisition of the vehicle image, the method further includes: preprocessing the vehicle image, wherein the preprocessing includes cropping and rotation transformation, such as random cropping of the image, and rotation of 0 to 5°. Of course, considering that different recognition systems have different requirements for image resolution, preferably, in the above embodiment, after adding different degrees of random noise to the vehicle image to obtain each sample image, the method further includes: adjusting the resolution of each sample image respectively, wherein the image can be compressed to a resolution of 128*128, 64*64, etc. according to the requirements of the deep learning detection model to improve the model training speed and recognition speed.

[0060] Further, in the above embodiment, for step S2: filtering the vehicle image set to be filtered based on the image exposure to obtain the image to be identified, for details, please refer to Figure 3 , Figure 3 The figure is a schematic diagram of the filtering process in one embodiment of the present invention, which specifically includes:

[0061] S21, setting a screening range of image exposure;

[0062] S22, calculating the exposure of each of the sample images;

[0063] S23, screening and filtering the sample images whose exposure is outside the range to obtain the corresponding images to be identified.

[0064] Preferably, in this embodiment, whether the vehicle image is overexposed or too dark is judged based on the image exposure, so as to filter out vehicle body images with poor image quality.

[0065] Further, in the above embodiment, for step S3: performing front and rear vehicle body recognition on the image to be identified to obtain corresponding front vehicle body images and rear vehicle body images, specifically: inputting the image to be identified into the front and rear vehicle body classification model for analysis, and outputting the corresponding front vehicle body images and rear vehicle body images, wherein the front and rear vehicle body classification model is one of the following: ZfNet model, ResNet model, MobileNet model.

[0066] Furthermore, in the above embodiment, the set body area feature map at least includes front and rear wiper feature maps, front and rear lamp feature maps, front and rear window glass feature maps and air intake grille feature maps. Considering that the recognition requirements of different bayonet ports are different, the body area features can also select other vehicle features, such as the roof, wheels, rearview mirrors, etc. Of course, the setting of specific body area features also needs to be combined with specific vehicle models, and different vehicle models can also be set to correspond to different body area features.

[0067] In addition, in this embodiment, see Figure 5 , Figure 5 A schematic diagram of the identification process in one of the embodiments of the present invention is shown, wherein the acquisition of the set body area feature map includes two steps of detecting the set body area features and identifying the set body area features. The detection of the set body area features refers to detecting the wipers, lights, window glass, and air intake grille of the input body map. Preferably, the detection-related technology can select SSD, YOLO, fast-rcnn and other common target detection networks based on deep learning technology; the identification of the set body area features refers to identifying the detected feature areas, and the output is seven categories including front wipers, rear wipers, front lights, rear lights, front window glass, rear window glass, and air intake grille.

[0068] Further, in the above embodiment, for step S5: the front body image and the body area feature map related to the front body image are feature-fused, and the rear body image and the body area feature map related to the rear body image are feature-fused. For details, please refer to Figure 4 , Figure 4 The figure is a schematic diagram of the fusion process in one embodiment of the present invention, which specifically includes:

[0069] S51, respectively extracting image features of the front-shot vehicle body image, the front wiper feature image, the front lamp feature image, the front window glass feature image, and the air intake grille feature image, and outputting corresponding front-shot probability images;

[0070] S52, performing weighted fusion based on the confidence of each of the preceding shot probability maps; and,

[0071] S53, respectively extracting image features of the rear-shot vehicle body image, the rear wiper feature image, the rear lamp feature image, and the rear window glass feature image, and outputting corresponding rear-shot probability images;

[0072] S54: Perform weighted fusion based on the confidence of each of the subsequent shot probability maps.

[0073] From the above, it can be seen that the multiple convolutional neural networks built can greatly improve the anti-interference and robustness of vehicle front and rear shot recognition through multi-scale feature fusion. Multiple convolutional neural networks can perform classification and recognition at the same time, integrating image features of different scales to improve model recognition accuracy and recall rate, and integrating the output results and confidence of the two tasks to judge the image quality, thereby obtaining high-precision front and rear body images, which improves the detection rate and recognition accuracy of vehicle front and rear shot detection and recognition.

[0074] Another embodiment of the present invention provides an electronic device. For details, see Figure 6 , Figure 6 A schematic diagram of the structure of an electronic device in one embodiment of the present invention is shown, which includes a processor 10, a memory 20, and a computer program A stored in the memory 20 and configured to be executed by the processor 10, and when the processor 10 executes the computer program A, the vehicle image recognition method as described above is implemented.

[0075] Yet another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle image recognition method as described above.

[0076] The vehicle image recognition method, electronic device and computer-readable storage medium provided by the embodiments of the present invention have the beneficial effects of at least one of the following: first, a set of vehicle images to be filtered is constructed based on the vehicle images acquired in real time, thereby improving the richness of the samples and providing accurate data support for the subsequent deep learning-based image processing strategy; then, based on the judgment of the exposure, the vehicle images whose image quality obviously does not meet the expectations are filtered out, thereby reducing the calculation amount of the subsequent detection model; then, the front and rear shots of the vehicle image are recognized, thereby dividing the vehicle image into two corresponding recognition areas, providing double protection for accurately identifying the vehicle body features; then, the relevant feature areas of the vehicle image are extracted, and the extracted feature areas are fused with the corresponding recognition areas, so that the fused vehicle image has accurate body area features, and the dual perspective division of the front and rear shots of the vehicle can ensure comprehensive, multi-category body area features, thereby improving the quality of the final output image, thereby improving the precision and accuracy of vehicle image recognition, and ensuring the practicality and reliability of vehicle image recognition at the bayonet.

[0077] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A vehicle image recognition method, characterized in that: The method comprises: Constructing a vehicle image set to be filtered according to the vehicle images acquired in real time; Filtering the vehicle image set to be filtered based on image exposure to obtain an image to be identified; Performing front and rear vehicle body recognition on the image to be recognized to obtain corresponding front vehicle body images and rear vehicle body images; Inputting the image to be identified into a preset deep learning detection model for storing the correspondence between image features and vehicle body area features, to obtain a plurality of set vehicle body area feature maps; Performing feature fusion on the front body image and the body area feature image related to the front body image, and performing feature fusion on the rear body image and the body area feature image related to the rear body image; Output the fused image results respectively; The set vehicle body area characteristic map at least includes front and rear wiper characteristic maps, front and rear lamp characteristic maps, front and rear window glass characteristic maps and air intake grille characteristic maps; The feature fusion of the front body image and the body area feature image related to the front body image, and the feature fusion of the rear body image and the body area feature image related to the rear body image, is specifically as follows: Respectively extracting image features of the front-shot vehicle body image, the front wiper feature image, the front headlight feature image, the front window glass feature image, and the air intake grille feature image, and outputting corresponding front-shot probability images; Perform weighted fusion based on the confidence of each of the preceding probability maps; and, Respectively extract the image features of the rear-shot vehicle body image, rear wiper feature image, rear light feature image, and rear window glass feature image, and output the corresponding rear-shot probability images; Weighted fusion is performed based on the confidence of each of the subsequent shot probability maps.

2. The vehicle image recognition method according to claim 1, characterized in that: The step of constructing a vehicle image set to be filtered based on the vehicle images acquired in real time is specifically as follows: Acquire vehicle images in real time; Adding different degrees of random noise to the vehicle image to obtain sample images; A vehicle image set to be filtered is constructed using the sample images.

3. The vehicle image recognition method according to claim 2, characterized in that: After acquiring the vehicle image in real time, the method further includes: The vehicle image is preprocessed, wherein the preprocessing includes cropping and rotation transformation.

4. The vehicle image recognition method according to claim 2, characterized in that: After adding different degrees of random noise to the vehicle images to obtain sample images, the method further includes: The resolution of each of the sample images is adjusted respectively.

5. The vehicle image recognition method according to claim 2, characterized in that: The filtering of the vehicle image set to be filtered based on the image exposure to obtain the image to be identified is specifically: Set the filter range of image exposure; Calculating the exposure of each of the sample images; The sample images whose exposure is outside the range are screened and filtered to obtain the corresponding images to be identified.

6. The vehicle image recognition method according to claim 1, characterized in that: The front and rear vehicle body recognition is performed on the image to be recognized to obtain the corresponding front vehicle body image and rear vehicle body image, specifically: The image to be identified is input into the front and rear body classification model for analysis, and the corresponding front body image and rear body image are output, wherein the front and rear body classification model is one of the following: ZfNet model, ResNet model, MobileNet model.

7. An electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the vehicle image recognition method as described in any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle image recognition method according to any one of claims 1 to 6.

Images