Vehicle intelligent recognition method and device based on AI vision and electronic equipment

Abstract

The application provides a vehicle intelligent recognition method and device based on AI vision and electronic equipment, the method comprises the following steps: m1, acquiring a vehicle image without a license plate as a to-be-recognized image, and acquiring a scene where the to-be-recognized image is located based on a scene classification model; m2, extracting vehicle attribute features, vehicle face global features and vehicle face local features from the to-be-recognized image based on a feature extraction model associated with the scene obtained in step m1; m3, performing image search in a sample image library associated with the scene obtained in step m1 based on the vehicle attribute features and the vehicle face global features, and obtaining a first search result; m4, selecting one or more recommended regions from nine vehicle face sub-images of the to-be-recognized image based on a nine-square grid and an attention map; and m5, performing image search in the first search result based on the vehicle face local features of the recommended regions, and obtaining a vehicle recognition result; the method can reduce the amount of similarity measurement calculation and improve the accuracy of similarity measurement.

Description

TECHNICAL FIELD

[0001] The present application relates to a vehicle identification method, in particular to an AI vision-based vehicle intelligent identification method, device and electronic equipment. BACKGROUND

[0002] In order to regulate the governance of oversize and overloaded transport of freight vehicles, and effectively protect the safety of highway traffic and the safety of the lives and property of the general public, the Ministry of Transport of the People's Republic of China has promulgated a series of regulations to govern the phenomenon of oversize and overloaded transport of freight vehicles. With the continuous advancement of the governance of oversize and overloaded transport of freight vehicles in various provinces, individual freight vehicle drivers have adopted the illegal behavior of deliberately obscuring license plates to evade detection and punishment, which has brought many difficulties to the work of law enforcement departments.

[0003] Artificial intelligence is a branch of computer science, which has four elements: algorithm, computing power, data, and application scenario. With the progress and enrichment of the above four elements, the application field of artificial intelligence is also expanding, such as machine vision, automatic planning, intelligent control, language and image understanding, etc. Al vision, as one of the core technologies in the field of artificial intelligence, is the most direct information source of the machine vision system, and its importance to robots is also like the human eye. Al vision uses visual sensors and computers to replace the human eye, so that the machine has the function of segmentation, classification, recognition, tracking, and decision-making of the target, so as to make the system realize the ability of simulating human "mind map", i.e. the ability of human thinking logic

[0004] At present, the vehicle identification technology based on AI vision is more and more widely used in intelligent traffic monitoring, intelligent traffic information system, etc. Vehicle identification includes moving vehicle detection, vehicle type classification, vehicle tracking, license plate recognition, etc. The invention patent with publication number CN107729818B discloses a multi-feature fusion vehicle re-identification method based on deep learning, which designs a vehicle re-identification method that fuses the license plate identification vector, vehicle expressive feature vector, and vehicle type attribute feature vector. This method not only can determine the illegal situation of license plate fraud, but also can obtain many attribute information of the vehicle such as color, brand, type, etc., and can effectively improve the accuracy of vehicle re-identification in monitoring video.

[0005] However, in the patent, each time needs to be in a large-scale database with different scenes, and the data volume is large; and the existing deep learning detection method is usually applied in good light during the day, and they have many challenges in night scenes: weak light environment makes vehicle features easy to be hidden, data set is relatively lacking, network does not learn enough vehicle features, and single detection method can only reflect part of the target situation. These make it difficult for existing deep learning methods to perform detection as well as in daytime scenes. Similarly, under different exposure conditions, the feature vector of the same vehicle will be biased due to environmental factors and exposure levels, for example, in rainy and foggy weather, the quality of the license plate picture will be affected; at night, under the action of the headlight, there will be some black and white blocks in the image of the license plate, and the above situations will also affect the effect of vehicle recognition.

[0006] In order to solve the above problems, people have been seeking an ideal technical solution. SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art and provide a vehicle intelligent recognition method, device and electronic equipment based on AI vision.

[0008] In order to achieve the above purpose, the technical scheme adopted by the present application is: a vehicle intelligent recognition method based on AI vision, comprising the following steps:

[0009] Step m1, scene classification

[0010] Obtain a vehicle image without a license plate as a to-be-recognized image, classify the to-be-recognized image based on a scene classification model, and obtain the scene where the to-be-recognized image is located;

[0011] Step m2, feature extraction

[0012] Detect a target vehicle image from the to-be-recognized image based on a target detection algorithm, detect and segment a vehicle face global image from the target vehicle image, and divide the vehicle face global image into nine vehicle face sub-images according to a nine-square grid;

[0013] Extract vehicle attribute features, vehicle face global features and vehicle face local features from the target vehicle image, the vehicle face global image and each vehicle face sub-image in sequence based on a feature extraction model associated with the scene obtained in step m1;

[0014] Step m3, first image search

[0015] Perform image search in a sample image library associated with the scene obtained in step m1 based on the vehicle attribute features and the vehicle face global features, and obtain a first search result;

[0016] Step m4, region recommendation

[0017] selecting one or more recommended regions from nine vehicle face sub-images of the to-be-identified image based on the nine-square grid and the attention map;

[0018] Step m5, secondary image search

[0019] performing image search on the vehicle face local features based on the recommended regions in the first search result, and obtaining a vehicle identification result.

[0020] Based on the above, the specific steps of selecting one or more recommended regions from nine vehicle face sub-images of the to-be-identified image based on the nine-square grid and the attention map are as follows:

[0021] processing the vehicle face global image based on the attention map, and extracting attention map layer data as an attention feature matrix;

[0022] quantizing each value of the attention feature matrix, taking the median value of the attention feature matrix as a threshold, assigning all values greater than the threshold in the attention feature matrix to 1, and assigning all values less than the threshold in the attention feature matrix to 0 to obtain a quantized feature image M;

[0023] dividing the feature image M according to the nine-square grid to obtain nine sub-feature images, each sub-feature image being denoted as M i , where i = 0, 1, …, 8;

[0024] calculating the sum S i of all values in each sub-feature image;

[0025] sorting S i in descending order, and selecting the top three S i The corresponding sub-feature images correspond to the corresponding vehicle face sub-images in the vehicle face global image, and are used as recommended regions.

[0026] Based on the above, the specific steps of step m1 are as follows:

[0027] m1.1, real-time acquisition of vehicle images, judgment of whether there is a license plate number in the vehicle image, if not, the vehicle image is used as a to-be-identified image;

[0028] m1.2, using a hue saturation value (HSV) color model and a day-night classifier constructed by openCV to perform day-night classification on the to-be-identified image, and determining whether the to-be-identified image is a daytime image or a nighttime image;

[0029] m1.3, converting the to-be-identified image into a grayscale image, calculating the mean and variance of the grayscale image, and performing exposure classification on the to-be-identified image based on the mean and variance to determine whether the to-be-identified image is a normally exposed image, an overexposed image, or an underexposed image;

[0030] m1.4, determining, according to the identification result of m1.2, m1.3, a scene in which the to-be-identified image is located as a daytime overexposure scene, a nighttime overexposure scene, a daytime underexposure scene, a nighttime underexposure scene, a daytime normal exposure scene or a nighttime normal exposure scene.

[0031] Based on the above, after obtaining the vehicle identification result, each vehicle identification result is synchronously viewed with the to-be-identified image.

[0032] The application further provides an AI vision-based vehicle intelligent identification device, which comprises:

[0033] An image acquisition module is configured to acquire a license plate-free vehicle image and take the license plate-free vehicle image as a to-be-identified image.

[0034] A scene selection module is configured to classify the to-be-identified image based on a scene classification model and acquire a scene in which the to-be-identified image is located.

[0035] A feature extraction module is internally provided with a plurality of feature extraction models under different scenes; a target vehicle image is detected from the to-be-identified image based on a target detection algorithm, a vehicle face global image is detected and segmented from the target vehicle image, the vehicle face global image is divided into nine vehicle face sub-images according to a nine-square grid, and vehicle attribute features, vehicle face global features and vehicle face local features are extracted from the target vehicle image, the vehicle face global image and each vehicle face sub-image in sequence through the feature extraction models under corresponding scenes.

[0036] A primary image search module is configured to perform image search in a sample image library under a corresponding scene based on vehicle attributes and vehicle face global features, and obtain a first search result.

[0037] A region recommendation module is configured to select one or more recommended regions from the nine vehicle face sub-images of the to-be-identified image based on the nine-square grid and an attention map.

[0038] A secondary image search module is configured to perform image search in the first search result based on vehicle face local features of the recommended regions, and obtain a vehicle identification result.

[0039] The application further provides an electronic device, which comprises:

[0040] A memory is configured to store executable instructions.

[0041] A processor is configured to execute the executable instructions stored in the memory, and implement the vehicle intelligent identification method.

[0042] The application has prominent substantial features and significant progress compared with the prior art, and specifically,

[0043] (1) The present invention performs scene analysis on the image to be identified, and selects a feature extraction model of the corresponding scene based on the analysis result to extract the vehicle attributes and vehicle face features of the image to be identified, thereby accurately extracting the vehicle attributes and vehicle face features of the image to be identified; further, after extracting the vehicle attributes and vehicle face features of the image to be identified, the present invention performs similarity measurement in the sample image library of the corresponding scene based on the vehicle attributes and vehicle face features. Since it is not necessary to perform similarity measurement on the sample image library under all scenes, the computational complexity of the similarity measurement can be greatly reduced; at the same time, searching in similar scenes can improve the accuracy of the similarity measurement;

[0044] (2) When performing feature extraction, the present invention extracts the vehicle attributes, global features of the vehicle face, and local features of the vehicle face in the global image required for the vehicle intelligent recognition process in advance, thereby reducing the extraction time of extracting the vehicle attributes, global features, and local features of the vehicle face one by one and improving the extraction efficiency;

[0045] (3) The present invention adopts a two-step image search method. The first step is to perform similarity search based on vehicle attributes and global features of the vehicle face, and find vehicle images with similar vehicle attributes and global features of the vehicle face, so as to narrow the scope of subsequent secondary search; the second step is to perform secondary search based on local features of the vehicle face, and filter out vehicle images with mismatched local features of the vehicle face, thereby improving the accuracy of the vehicle feature intelligent recognition results;

[0046] (4) The present invention intelligently recommends one or more sub-regions for local region similarity measurement based on the nine-square grid, attention map and prior library, which can improve the accuracy of local region similarity measurement. At the same time, since similarity measurement does not need to be performed on all regions, the amount of calculation can be reduced;

[0047] (5) The present invention extracts vehicle attributes, vehicle face regions, global features of the vehicle face, and nine-square grid local features of the vehicle face region from vehicle images with license plates, and stores the vehicle attributes, vehicle face regions, global features of the vehicle face, and local features of the vehicle face in a sample library, thereby continuously learning and expanding the sample library.

[0048] (6) After obtaining the vehicle recognition results, the present invention synchronizes and links each vehicle recognition result with the image to be recognized. When one of the images is magnified, the other image is automatically magnified and positioned in the same visible area. This can reduce manual operation costs, improve the efficiency of comparison and confirmation of artificial intelligence recognition results, and help quickly generate recognition result reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of the vehicle intelligent identification method of the present invention.

[0050] Figure 2 is the flowchart of the region recommendation of the present application.

[0051] Figure 3 is the result schematic diagram of the region recommendation of the present application.

[0052] Figure 4 is the flowchart of the synchronous linkage viewing of the present application. DETAILED DESCRIPTION

[0053] The technical solutions of the present application will be described in further detail below through specific embodiments.

[0054] The present application is based on a large number of researches on local short-distance transport vehicle groups, which have the characteristics of frequent routes of the same road at different times (especially at night), no blocking of license plates when not overweight, and blocking of license plates when overweight. Based on the above characteristics, the present application establishes a clear and complete vehicle space-time chain when the vehicle has a license plate when not overweight. Based on the above space-time chain, the present application compares the vehicle without a license plate, effectively improving the accuracy and reliability of the vehicle without a license plate recognition, and solving the typical and most dangerous behavior in the overload and out-of-gauge group.

[0055] Embodiment 1

[0056] As shown in Figure 1 , the present embodiment provides an AI vision-based vehicle intelligent identification method, comprising the following steps:

[0057] Step m1, scene classification

[0058] Obtain an image of a vehicle without a license plate as a to-be-identified image, classify the to-be-identified image based on a scene classification model, and obtain the scene where the to-be-identified image is located;

[0059] Step m2, feature extraction

[0060] Based on a target detection algorithm, a target vehicle image is detected from the to-be-identified image, a vehicle face global image is detected and segmented from the target vehicle image, and the vehicle face global image is divided into nine vehicle face sub-images according to a nine-square grid;

[0061] Based on a feature extraction model associated with the scene obtained in step m1, vehicle attribute features, vehicle face global features and vehicle face local features are extracted from the target vehicle image, the vehicle face global image and each vehicle face sub-image in turn;

[0062] Step m3, first image search

[0063] Based on the vehicle attribute features and the vehicle face global features, image search is performed in a sample image library associated with the scene obtained in step m1, and a first search result is obtained; preferably, the present embodiment adopts an image retrieval method based on image similarity measurement;

[0064] Step m4, region recommendation

[0065] Select one or more recommended regions from the nine face sub-images of the vehicle to be identified based on the nine-square grid and the attention map;

[0066] Step m5, secondary image search

[0067] Based on the local features of the recommended region, the image search is performed in the first search result to obtain the vehicle identification result.

[0068] The present application is aimed at identifying vehicles with obscured license plates by using data of the vehicles without obscured license plates at other time periods to determine the license plate number corresponding to the vehicle with obscured license plate. Since the vehicles with obscured license plates are mostly large trucks overloaded, and the vehicle itself may change significantly due to the uncertain time span, the face region of the vehicle is used for identifying the vehicle with obscured license plate.

[0069] Since the feature vector of the same vehicle may deviate due to environmental factors and exposure levels in different scenarios, such as overexposure in the daytime and insufficient exposure at night, the feature vector of the same vehicle may deviate greatly, resulting in the inability to accurately retrieve the same vehicle. Therefore, the present embodiment first automatically distinguishes the scenario in which the image to be analyzed is located, and then performs subsequent identification based on the corresponding feature extraction model of the automatically selected scenario, for example, if the image to be identified is in a normal daytime exposure scenario, the image retrieval is preferably performed from the normal daytime exposure scenario to accurately retrieve similar images. At the same time, since only the daytime scenario library is retrieved, the number of comparison samples can be reduced, further improving the retrieval efficiency.

[0070] It can be understood that in the vehicle intelligent identification method, a clear and complete scenario sample library is established by using images of vehicles with license plates when the vehicles are not overloaded, and feature extraction models in different scenarios are pre-trained, including vehicle attribute extraction models, face global feature extraction models, and face local feature extraction models. The face local feature extraction model needs to divide the face global image into nine-square grids, and then extract the local features of each face sub-image.

[0071] The vehicle attribute features include vehicle color, vehicle logo, vehicle type, vehicle light, and number of vehicle mirrors. When training the vehicle attribute extraction model, a vehicle color identification model, a vehicle logo identification model, a vehicle type identification model, a vehicle light identification model, and a vehicle mirror identification model can be trained respectively, or a vehicle attribute extraction model integrating vehicle color identification, vehicle logo identification, vehicle light identification, vehicle type identification, and vehicle mirror identification can be trained.

[0072] Since the vehicle attribute features are relatively prominent, in the specific implementation, in step m3, a coarse-grained image search can be first performed in the sample image library of the corresponding scene based on the vehicle attribute features, and then a fine-grained image search is performed on the search results based on the car face global features, so as to obtain the first search results. For example, data with incomplete car faces is first excluded; then car face global images that do not meet the requirements are removed by detecting the number of car lights and car mirrors in the car face global images; and then car face global images that do not meet the requirements are removed by taking the car logo, car model and vehicle color as the prominent attribute features of the vehicle, so as to improve the accuracy of subsequent similarity measurement based on the car face global features.

[0073] In another embodiment, step m3 can also perform similarity measurement based on both the vehicle attribute features and the car face global features.

[0074] In the specific implementation, the specific steps of step m1 are as follows:

[0075] m1.1, real-time collection of vehicle images, judgment of whether there is a license plate number in the vehicle image, if not, the vehicle image is taken as a to-be-identified image;

[0076] m1.2, daytime and night classification of the to-be-identified image is performed by using a hue saturation value (HSV) color model and a daytime and night classifier constructed by openCV, to determine whether the to-be-identified image is a daytime image or a night image;

[0077]

[0078] The hue saturation value (HSV) is a substitute color model of RGB, the hue (H) is three primary colors and three secondary colors in the color, the saturation (S) is the purity and intensity of the color, and the lower the value, the closer the color is to gray; the value (V) refers to the relative brightness or darkness of the color, and each of these values has a limit; H is from 0 to 360, S and V are from 0 to 100; when used, a best threshold value can be found through basic derivation of training images for daytime and night classification;

[0079] m1.3, the to-be-identified image is converted into a gray image, the mean value and variance of the gray image are calculated, and exposure classification is performed on the to-be-identified image based on the mean value and variance, to determine whether the to-be-identified image is a normal exposure image, an overexposure image or an underexposure image;

[0080] m1.4, according to the identification results of m1.2 and m1.3, it is determined that the scene where the to-be-identified image is located is a daytime overexposure scene, a night overexposure scene, a daytime underexposure scene, a night underexposure scene, a daytime normal exposure scene or a night normal exposure scene.

[0081] It should be noted that in step m1.1, if it is judged that there is a license plate number in the vehicle image, the license plate number in the vehicle image is recognized, and is compared with the license plate number in the vehicle record information reported by the data collection service. If the license plate numbers are consistent, steps m1.2, m1.3, and m1.4 are continued to obtain the scene in which the current vehicle image is located, to obtain the vehicle face global image, the vehicle attribute feature, the vehicle face global feature, and the vehicle face local feature of each vehicle face sub-image based on the feature extraction model associated with the scene in which the current vehicle image is located, and to store the current vehicle image, the vehicle face global image, the vehicle attribute feature, the vehicle face global feature, and the vehicle face local feature into the sample image library of the corresponding scene. If the license plate numbers are inconsistent, the current vehicle image is stored into the sample library of suspected wrong license plate.

[0082] The above steps can continuously expand the sample library by using the image of the vehicle with a license plate when the vehicle is not overloaded, so as to establish a clear and complete vehicle feature space-time chain, and effectively improve the accuracy and reliability of the license plateless vehicle recognition.

[0083] Further, it can be understood that in the process of determining the scene in which the to-be-recognized image is located, whether the similarity of the scene in which the to-be-recognized image is located to the daytime overexposure scene, the nighttime overexposure scene, the daytime underexposure scene, the nighttime underexposure scene, the daytime normal exposure scene, or the nighttime normal exposure scene reaches a threshold value can also be determined. If it reaches, it is determined to belong to the daytime overexposure scene, the nighttime overexposure scene, the daytime underexposure scene, the nighttime underexposure scene, the daytime normal exposure scene, or the nighttime normal exposure scene. If it does not reach, it is determined that the current to-be-recognized image does not belong to any of the above scenes.

[0084] When it is determined that the current to-be-recognized image does not belong to any of the above scenes, the whole vehicle texture feature and the key point texture feature of the to-be-recognized image are directly extracted by using the trained texture feature extraction model, and a similarity search is performed in the pre-trained texture sample library to give an image recommendation result.

[0085] For example, when it is judged that the license plate in the current vehicle image is consistent with the license plate number in the vehicle record information reported by the data collection service, and it is determined that the current vehicle image does not belong to any of the above scenes, the whole vehicle texture feature and the key point texture feature of the current vehicle image are directly extracted by using the trained texture feature extraction model, and the vehicle image with the license plate, the whole vehicle texture feature, and the key point texture feature are added to the texture sample library to continuously expand the texture sample library. In subsequent vehicle retrieval, a similarity search is performed based on the texture sample library.

[0086] In a specific implementation, the texture feature extraction step is as follows: the color influence in the vehicle image to be extracted is removed through an image binarization operation, and the whole vehicle texture feature and the texture feature of each key point (vehicle lamp, rearview mirror, and annual inspection mark) are extracted from the binary image by using a data enhancement method.

[0087] It should be noted that, due to the similarity between vehicles of the same brand and model, it is difficult to accurately retrieve the vehicle image according to the vehicle face global feature vector, and therefore the local feature vector can be further compared through the stickers, annual inspection marks, ornaments, scratches, and the like on the vehicle face; however, if all local features are compared, the data volume is large and the time consumption is long.

[0088] Therefore, the embodiment selects one or more recommended regions from the nine vehicle face sub-images of the image to be recognized based on the nine-square grid and the attention map for joint similarity measurement, as shown in Figure 2 The specific steps are as follows:

[0089] The vehicle face global image is processed based on the attention map, and the attention map layer data is extracted as an attention feature matrix;

[0090] Each value of the attention feature matrix is quantized, the median value of the attention feature matrix is taken as a threshold value, all values greater than the threshold value in the attention feature matrix are assigned as 1, and all values less than the threshold value in the attention feature matrix are assigned as 0, to obtain a quantized feature image M;

[0091] The feature image M is divided according to the nine-square grid, to obtain nine sub-feature images, each sub-feature image is denoted as M i , where i = 0, 1,..., 8.

[0092] The sum S i of all values in each sub-feature image is calculated.

[0093] S i is sorted in descending order, and the top three S i are selected. Figure 3 The corresponding sub-feature image in the vehicle face global image is the corresponding vehicle face sub-image in the vehicle face global image, as a recommended region, as shown in

[0094] It can be seen that the importance of the sub-region recommended based on the attention map and the nine-square grid is higher than that of the non-recommended sub-region, and therefore the local region similarity measurement based on the sub-region with high importance can improve the accuracy of the local region similarity measurement, and at the same time, since all regions do not need to be measured for similarity, the amount of calculation can be reduced.

[0095] After obtaining the recommended regions, the algorithm then determines whether the local features of the vehicle face in each recommended region are included in the prior salient feature library. If so, the recommended region is retained; otherwise, it is discarded. By filtering out regions containing salient features of interest to the user based on the prior library, the accuracy of the joint image similarity metric can be improved.

[0096] Example 2

[0097] The difference between this embodiment and embodiment 1 is that before or simultaneously selecting one or more recommended regions from the nine vehicle face sub-images of the image to be recognized based on the nine-square grid and the attention map, it is determined whether the local features of each vehicle face sub-image are located in the prior salient feature library. If so, the vehicle face sub-image is used as the prior recommended region.

[0098] After selecting one or more recommended regions from the nine vehicle face sub-images of the image to be recognized based on the nine-square grid and the attention map, it is determined whether each recommended region belongs to the prior recommended region. If so, the recommended region is retained; otherwise, the recommended region is discarded.

[0099] Example 3

[0100] Through artificial intelligence algorithms and feature vector retrieval, the vehicle information with the highest similarity and unobstructed license plates can be obtained. However, considering users' skepticism towards artificial intelligence, this solution provides a manual confirmation function by comparing the left and right images after obtaining the recognition result with the highest similarity.

[0101] In existing technology, users need to manually zoom in on the same area of ​​the vehicle front in both the original prototype image and the image to be compared. Research has revealed that image viewing tools only support zooming in on a single image, and image magnifiers only support zooming in on a specific area of ​​a single image, neither of which meets these requirements. To enhance the user experience, reduce the difficulty of manual comparison and confirmation, and improve work efficiency, this embodiment synchronizes and links each vehicle recognition result with the image to be identified after obtaining it.

[0102] like Figure 4 As shown, the specific steps for synchronous linkage viewing are as follows:

[0103] Obtain the vehicle face area in the image to be identified and the vehicle identification result to be compared;

[0104] Based on the pixel coordinates, calculate the width, height, and center coordinates of the vehicle face area in the image to be identified and the vehicle recognition result to be compared. The width, height, and center coordinates of the image to be identified are: w, h, and (cx, cy), respectively; the width, height, and center coordinates of the vehicle recognition result to be compared are: w', h', and (cx', cy'), respectively.

[0105] scaling the face region in the to-be-identified image and the to-be-compared vehicle identification result to 600*600 pixels, obtaining face scaling ratios rw, rh of the to-be-identified image and face scaling ratios rw', rh' of the to-be-compared vehicle identification result;

[0106] obtaining the coordinates (x, y) of the mouse pointer in the to-be-identified image, and calculating the pre-scaling region center point coordinates of the to-be-compared vehicle identification result:

[0107] x' = (cx' + ((cx - x) * rw) / rw')

[0108] y' = (cy' + ((cy - y) * rh) / rh');

[0109] when the scaling action of the mouse pointer in the to-be-identified image is detected, performing synchronous scaling of the to-be-compared vehicle identification result based on the scaling ratios rw, rh of the to-be-identified image, the pre-scaling region center point coordinates (x', y') of the to-be-compared vehicle identification result, and the scaling ratios rw', rh' of the to-be-compared vehicle identification result.

[0110] It can be understood that the above method supports automatically enlarging the other picture and positioning to the same visual area when one of the pictures is enlarged, which facilitates manual comparison and confirmation of the local features of the face, for example, the left picture is enlarged to the license plate position, and the right picture is automatically enlarged to the license plate position.

[0111] The tool reduces the cost of manual operation, improves the comparison and confirmation efficiency of the artificial intelligence identification result, and helps to quickly generate an identification result report.

[0112] Embodiment 4

[0113] The embodiment provides an AI vision-based vehicle intelligent identification device, which comprises:

[0114] an image acquisition module, configured to acquire a vehicle image without a license plate and take the vehicle image without the license plate as a to-be-identified image;

[0115] a scene selection module, configured to classify the to-be-identified image based on a scene classification model and obtain a scene in which the to-be-identified image is located;

[0116] a feature extraction module, which is internally provided with a plurality of feature extraction models in different scenes; based on a target detection algorithm, a target vehicle image is detected from the to-be-identified image, a face global image is detected and segmented from the target vehicle image, and the face global image is divided into nine face sub-images according to a nine-square grid; vehicle attribute features, face global features and face local features are extracted from the target vehicle image, the face global image and each face sub-image in turn through the feature extraction model in the corresponding scene;

[0117] a first image search module configured to perform image search in a sample image library of a corresponding scene based on the vehicle attribute and the face global feature, and obtain a first search result;

[0118] a region recommendation module configured to select one or more recommended regions from the nine face sub-images of the to-be-identified image based on the nine-square grid and the attention map;

[0119] a second image search module configured to perform image search in the first search result based on the face local feature of the recommended region, and obtain a vehicle identification result.

[0120] The embodiment analyzes the scene of the to-be-identified image, and selects a feature extraction model of the corresponding scene to extract the vehicle attribute and the face feature of the to-be-identified image according to the analysis result, so that the vehicle attribute and the face feature of the to-be-identified image can be accurately extracted. Further, the present application performs similarity measurement in the sample image library of the corresponding scene based on the vehicle attribute and the face feature after extracting the vehicle attribute and the face feature of the to-be-identified image. Since similarity measurement does not need to be performed on the sample image library of all scenes, the amount of computation of similarity measurement can be greatly reduced.

[0121] In the feature extraction process of the embodiment, the vehicle attribute, the face global feature, and the face local feature of the face global image required by the vehicle intelligent identification process are extracted at the same time, so that the extraction time of the vehicle attribute, the face global feature, and the face local feature is reduced, and the extraction efficiency is improved.

[0122] The embodiment adopts a two-step image search method. In the first step, similarity retrieval is performed based on the vehicle attribute and the face global feature, and vehicle pictures with similar vehicle attribute and face global feature are found out to narrow the range of subsequent secondary retrieval. In the second step, secondary retrieval is performed based on the face local feature, and vehicle pictures with unmatched face local region features are filtered, so that the accuracy of the vehicle feature intelligent identification result is improved.

[0123] Embodiment 5

[0124] The embodiment also provides an electronic device, which comprises:

[0125] a memory configured to store executable instructions;

[0126] a processor configured to execute the executable instructions stored in the memory to implement the vehicle intelligent identification method.

[0127] The embodiment also provides a specific implementation of a readable storage medium, which stores instructions. When the instructions are executed by a processor, the steps of the vehicle intelligent identification method of embodiments 1-3 are implemented.

[0128] In the above embodiments, the description of each embodiment is focused on, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0129] Those skilled in the art can appreciate that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0130] In the embodiments provided in the present application, it should be understood that the disclosed system and device can be implemented in other ways. For example, the device embodiments described above are only schematic, and for example, the division of the above modules is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0131] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The above integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0132] The integrated module described above, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above embodiment methods can also be completed by computer programs instructing related hardware, and the above computer programs can be stored in a computer readable storage medium, and the computer programs can implement the steps of each method embodiment when executed by a processor. The above computer programs include computer program codes, and the computer program codes can be in the form of source code, object code, executable files or some intermediate forms.

[0133] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application rather than limit them; although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or some technical features can be replaced by equivalent replacements; without departing from the spirit of the technical solutions of the present application, they should be covered in the technical solution range of the present application claimed.

Claims

1. A vehicle intelligent recognition method based on AI vision, characterized in that: The following steps are involved: Step m1, scene classification Obtain an image of a vehicle without a license plate as an image to be identified, classify the image to be identified based on a scene classification model, and obtain the scene in which the image to be identified is located; Step m2, feature extraction Detect the target vehicle image from the image to be identified based on the target detection algorithm, detect and segment the vehicle face global image from the target vehicle image, and divide the vehicle face global image into nine vehicle face sub-images according to the nine-square grid; Extracting vehicle attribute features, vehicle face global features, and vehicle face local features from the target vehicle image, the vehicle face global image, and each vehicle face sub-image in sequence based on the feature extraction model associated with the scene obtained in step m1; Step m3, first image search Performing an image search in a sample image library associated with the scene obtained in step m1 based on the vehicle attribute features and the global features of the vehicle face to obtain a first search result; Step m4, area recommendation Select one or more recommended regions from the nine car face sub-images of the image to be recognized based on the nine-square grid and the attention map; Step m5, secondary image search An image search is performed in the first search result based on the local features of the vehicle face in the recommended area to obtain a vehicle recognition result.

2. The vehicle intelligent recognition method based on AI vision according to claim 1 is characterized in that: Step m4, based on the nine-square grid and the attention map, selects one or more recommended regions from the nine vehicle face sub-images of the image to be recognized. The specific steps are as follows: The global image of the vehicle face is processed based on the attention map, and the attention map layer data is extracted as the attention feature matrix; Quantize each value of the attention feature matrix, take the median of the attention feature matrix as the threshold, assign all values ​​in the attention feature matrix greater than the threshold to 1, and assign all values ​​in the attention feature matrix less than the threshold to 0, to obtain a quantized feature image M; Divide the feature image M according to the nine-square grid to obtain nine sub-feature images, each of which is denoted as M i , where i=0, 1, ..., 8; Calculate the sum S of all values ​​in each sub-feature image i ; To S i Sort by size, select the top three Ss. i The corresponding sub-feature image and the corresponding car face sub-image in the car face global image are used as the recommended area.

3. The AI ​​vision-based vehicle intelligent recognition method according to claim 2, characterized in that: After obtaining the recommended areas, it is determined whether the local features of the vehicle face in each recommended area are located in the prior significant feature library. If so, the recommended area is retained; otherwise, the recommended area is discarded.

4. The vehicle intelligent recognition method based on AI vision according to claim 2 is characterized in that: Its characteristics are: Before or simultaneously selecting one or more recommended regions from the nine vehicle face sub-images of the image to be recognized based on the nine-square grid and the attention map, determining whether the local features of each vehicle face sub-image are located in the prior salient feature library; if so, using the vehicle face sub-image as the prior recommended region; After selecting one or more recommended regions from the nine vehicle face sub-images of the image to be recognized based on the nine-square grid and the attention map, it is determined whether each recommended region belongs to the prior recommended region. If so, the recommended region is retained; otherwise, the recommended region is discarded.

5. The vehicle intelligent recognition method based on AI vision according to claim 1, characterized in that: The specific steps of step m1 are as follows: m1.1, real-time acquisition of vehicle images, determine whether the vehicle image contains a license plate number, if not, the vehicle image is used as the image to be identified; m1.2, uses the Hue Saturation Value (HSV) color model and the openCV built day / night classifier to classify the image to be identified as day or night, and determines whether the image to be identified is a day image or a night image; m1.3, convert the image to be identified into a grayscale image, calculate the mean and variance of the grayscale image, and perform exposure classification on the image to be identified based on the mean and variance to determine whether the image to be identified is a properly exposed image, an overexposed image, or an underexposed image; m1.4: Based on the recognition results of m1.2 and m1.3, determine whether the scene of the image to be recognized is an overexposed scene during the day, an overexposed scene at night, an underexposed scene during the day, an underexposed scene at night, a scene with normal exposure during the day, or a scene with normal exposure at night.

6. The vehicle intelligent recognition method based on AI vision according to claim 5 is characterized in that: In step m1.1, if it is determined that there is a license plate number in the vehicle image, the license plate number in the vehicle image is identified and compared with the license plate number in the vehicle passing record information reported by the data collection service. If the license plate numbers are consistent, continue to execute steps m1.2, m1.3, and m1.4 to obtain the scene in which the current vehicle image is located, and obtain the vehicle face global image, vehicle attribute features, vehicle face global features, and vehicle face local features of each vehicle face sub-image based on the feature extraction model associated with the scene in which the current vehicle image is located, and store the current vehicle image, vehicle face global image, vehicle attribute features, vehicle face global features, and vehicle face local features in the sample image library of the corresponding scene; if the license plate numbers are inconsistent, the current vehicle image is stored in the suspected wrong license plate sample library.

7. The AI ​​vision-based vehicle intelligent recognition method according to claim 1, characterized in that: After obtaining the vehicle recognition results, each vehicle recognition result is viewed synchronously with the image to be recognized.

8. The vehicle intelligent recognition method based on AI vision according to claim 7 is characterized in that: The specific steps for viewing each vehicle recognition result and the image to be recognized synchronously are as follows: Obtain the vehicle face area in the image to be identified and the vehicle identification result to be compared; Based on the pixel coordinates, calculate the width, height, and center coordinates of the vehicle face area in the image to be identified and the vehicle recognition result to be compared. The width, height, and center coordinates of the image to be identified are: w, h, and (cx, cy), respectively; the width, height, and center coordinates of the vehicle recognition result to be compared are: w', h', and (cx', cy'), respectively. Scale the vehicle face area in the image to be recognized and the vehicle recognition result to be compared to 600*600 pixels, and obtain the vehicle face scaling ratios rw and rh of the image to be recognized and the vehicle face scaling ratios rw' and rh' of the vehicle recognition result to be compared; Get the coordinates (x, y) of the mouse pointer in the image to be identified, and calculate the coordinates of the center point of the pre-zoomed area of ​​the vehicle identification result to be compared: x' = (cx' + ((cx - x) * rw) / rw') y' = (cy' + ((cy - y) * rh) / rh'); When a zooming action of the mouse pointer is detected in the image to be identified, the image of the vehicle to be identified is synchronously zoomed based on the zoom ratios rw and rh of the image to be identified, the coordinates (x', y') of the center point of the pre-zoomed area of ​​the vehicle identification result to be compared, and the zoom ratios rw' and rh'.

9. A vehicle intelligent recognition device based on AI vision, characterized in that: The device comprises: An image acquisition module is used to obtain images of vehicles without license plates and use the images of vehicles without license plates as images to be identified; A scene selection module is used to classify the image to be identified based on the scene classification model and obtain the scene in which the image to be identified is located; The feature extraction module has built-in feature extraction models for multiple different scenarios. It detects the target vehicle image from the image to be identified based on the target detection algorithm, detects and segments the global vehicle face image from the target vehicle image, and divides the global vehicle face image into nine vehicle face sub-images according to the nine-square grid. It then uses the feature extraction models for the corresponding scenarios to extract vehicle attribute features, global vehicle face features, and local vehicle face features from the target vehicle image, the global vehicle face image, and each vehicle face sub-image in turn. The initial image search module is used to search for images in the sample image library of the corresponding scene based on vehicle attributes and global features of the vehicle face to obtain the first search result; A region recommendation module is used to select one or more recommended regions from the nine vehicle face sub-images of the image to be recognized based on the nine-square grid and the attention map; The secondary image search module is used to perform image search in the first search results based on the local features of the vehicle face in the recommended area to obtain the vehicle recognition result.

10. An electronic device, characterized in that: The electronic device comprises: a memory for storing executable instructions; The processor is configured to implement the vehicle intelligent identification method according to any one of claims 1 to 8 when executing the executable instructions stored in the memory.

Images