License plate recognition method and device, electronic equipment and storage medium

Abstract

This application relates to the field of image recognition technology and provides a license plate recognition method, apparatus, electronic device, and storage medium, including: determining whether an obstruction exists in a license plate image to be recognized; if the obstruction exists, extracting features of the obstruction to obtain a first feature; extracting features of the license plate image to obtain a second feature; fusing the first feature with pre-stored standard features to obtain fused features, where the standard features are features of a standard image (a license plate image without obstruction); determining the similarity between the license plate image to be recognized and each standard image containing the obstruction based on the second feature and the fused features; and determining the license plate number in the license plate image based on the determined similarity. This application can improve the recognition accuracy of license plate images with obstructions.

Description

Technical Field

[0001] This application belongs to the field of image recognition technology, and in particular relates to license plate recognition methods, devices, electronic equipment and storage media. Background Technology

[0002] License plates are unique identifiers composed of specific characters arranged according to a certain pattern. With the widespread application of deep learning technology, license plate recognition has become one of the key technologies in the field of intelligent transportation. License plate recognition is a technology that identifies vehicles by recognizing images captured by surveillance cameras, enabling the automatic registration and verification of vehicle "identity".

[0003] Currently, when recognizing license plates that are obscured, the system typically identifies the unobscured portion of the license plate while discarding the obscured portion. This results in some information loss and lower accuracy in license plate recognition. Summary of the Invention

[0004] This application provides a license plate recognition method, apparatus, electronic device, and storage medium, which can improve the recognition accuracy of license plate images with obstructions.

[0005] In a first aspect, embodiments of this application provide a license plate recognition method, including:

[0006] Determine if there are any obstructions in the license plate image to be identified;

[0007] If the license plate image to be identified contains an obstruction, then the features of the obstruction are extracted to obtain a first feature, and the features of the license plate image to be identified are extracted to obtain a second feature;

[0008] The first feature is fused with each of the pre-stored standard features to obtain each fused feature. The standard features are features of a standard image, and the standard image is a license plate image without obstructions.

[0009] Based on the second feature and each of the fused features, the similarity between the license plate image to be identified and each of the standard images containing the occluder is determined respectively;

[0010] The license plate number in the license plate image to be identified is determined based on the determined similarity levels.

[0011] Secondly, embodiments of this application provide a license plate recognition device, including:

[0012] The occlusion confirmation module determines whether there are any obstructions in the license plate image to be identified;

[0013] The feature extraction module is used to extract the features of the occlusion object to obtain a first feature if the license plate image to be identified has an occlusion object, and to extract the features of the license plate image to be identified to obtain a second feature.

[0014] The fusion processing module is used to fuse the first feature with each pre-stored standard feature to obtain each fused feature. The standard feature is a feature of a standard image, and the standard image is a license plate image without obstructions.

[0015] The recognition module is configured to determine the similarity between the license plate image to be recognized and each of the standard images containing the occluder based on the second feature and each of the fused features, and to determine the license plate number in the license plate image to be recognized based on the determined similarity.

[0016] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the license plate recognition method described in the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the license plate recognition method described in the first aspect above.

[0018] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the license plate recognition method described in any of the first aspects above.

[0019] The beneficial effects of the embodiments in this application compared with the prior art are:

[0020] In this embodiment, when there is an occlusion in the license plate image to be identified, the first feature corresponding to the occlusion and the second feature corresponding to the license plate image to be identified are extracted respectively, and the first feature is fused with each standard feature. Therefore, each fused feature is a feature corresponding to the standard image containing the occlusion. This allows the license plate number of the license plate image to be identified to be determined based on the similarity between the license plate image to be identified with the occlusion and the standard image containing the occlusion when identifying the license plate image to be identified. This improves the recognition accuracy of the license plate image to be identified with the occlusion without losing the license plate image information. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0022] Figure 1 This is a schematic flowchart of a license plate recognition method provided in one embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the vehicle posture provided in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of the license plate recognition device provided in the embodiments of this application;

[0025] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0027] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0031] Example 1:

[0032] Figure 1 A schematic flowchart of a license plate recognition method provided by an embodiment of the present invention is shown below:

[0033] Step S101: Determine whether there are any obstructions in the license plate image to be identified.

[0034] Optionally, the aforementioned obstruction can be any object that obscures the license plate number of a vehicle, such as dirt, paint, rust, or multiple license plates. In cases where lighting conditions cause glare or other bright reflections that obscure the license plate number in the captured image, the bright area on the license plate can also be used as the obstruction, meaning the image of the bright area of ​​the license plate in the license plate image is used as the image corresponding to the obstruction.

[0035] Specifically, before performing license plate recognition on a license plate image, it is necessary to pre-determine whether there are any occlusions in the image. This allows for different processing of the license plate image based on its condition. To determine if occlusions exist, a trained classification model can be used to classify the image, thus identifying the presence of occlusions. Alternatively, a trained object detection model can be used to detect the presence of corresponding targets (i.e., occlusions) and output the location information of the detected occlusions if they are present, enabling users to take appropriate action. For example, a detection model can be built based on an SSD (Single Shot MultiBox Detector) network model and trained using a corresponding training set to obtain a detection model for detecting occlusions in the license plate image. Since SSD uses prior boxes of different sizes and extracts feature maps of different sizes for target detection, it can detect targets of different sizes. This ensures that the target detection model built on SSD has good detection performance for occlusions of different sizes, and thus can effectively detect occlusions of different sizes in the license plate image to be identified.

[0036] In this embodiment of the application, since there are obstructions in the license plate image to be identified, the obstructions may block the license plate number in the license plate, making the accuracy of directly identifying the license plate image to be identified not high. Therefore, before identifying the license plate image to be identified, it is determined in advance whether there are obstructions in the license plate image to be identified, so that the license plate image to be identified with obstructions can be processed accordingly in the future.

[0037] Step S102: If the license plate image to be identified has an obstruction, then extract the features of the obstruction to obtain a first feature, and extract the features of the license plate image to be identified to obtain a second feature.

[0038] Specifically, since the presence of occlusions in a license plate image is an abnormal situation that can interfere with license plate recognition and reduce its accuracy, if an occlusion is detected in the license plate image to be recognized, the features of the occlusion are detected and extracted separately to obtain occlusion features (first features) for subsequent processing. Simultaneously, the features of the complete license plate image to be recognized are extracted, i.e., the overall features of the license plate image including the occlusions. Optionally, since there may be more than one occlusion in the license plate image to be recognized, when extracting the features of all occlusions, the features of each occlusion in the license plate image to be recognized can be used as a second feature corresponding to the license plate image, resulting in multiple corresponding second features. Alternatively, the extracted features of each occlusion can be stitched or fused to obtain a single second feature that includes the features of all occlusions in the license plate image to be recognized.

[0039] Optionally, feature extraction of occlusions is performed on the license plate image to be identified, or the license plate itself is used for identification.

[0040] When extracting features from an image, the corresponding convolution process is performed on the license plate image to be identified to obtain the features of the occluders in the license plate image to be identified (first feature) and the overall features of the license plate image to be identified (second feature).

[0041] Step S103: The first feature is fused with each of the pre-stored standard features to obtain each fused feature.

[0042] The aforementioned standard features are features extracted from pre-extracted standard images, which are unobstructed license plate images. To facilitate license plate recognition, standard features corresponding to each standard image are pre-generated.

[0043] This information is then stored for subsequent license plate recognition based on standard features of standard images.

[0044] Specifically, when there are obstructions in the license plate image to be identified, the license plate number may be obscured, making license plate recognition difficult. Furthermore, the stored standard features are for license plates without obstructions.

[0045] The characteristics of the image make license plate recognition accuracy low for license plate images with occlusions. Therefore, in this embodiment, after extracting the first feature of the occlusion, the first feature is fused with various pre-stored standard features to integrate the occlusion feature into the standard license plate image features, resulting in fused features containing the occlusion feature. Since the occlusion feature in the obtained fused features is integrated with the occlusion feature in the second feature (i.e., the features of the license plate image to be recognized), it contains the license plate to be recognized.

[0046] If the features of occlusions in the image are the same occlusion features, then fusing the occlusion features into the standard features can reduce the differences caused by occlusions in the license plate image to be identified, thereby improving the recognition accuracy of the license plate image to be identified.

[0047] Optionally, when fusing the first feature and each of the second features, the first feature can be fused with each standard feature through feature concatenation, feature addition, or other methods. This is existing technology and will not be elaborated here.

[0048] In the five embodiments of this application, since the presence of occlusions in the license plate image to be identified is an abnormal situation, the occlusions reduce the accuracy of license plate recognition. Therefore, before license plate recognition, the extracted occlusion features (first features) are fused into various standard features to reduce the differences caused by occlusions and improve the accuracy of subsequent license plate recognition.

[0049] Step S104: Based on the second feature and each of the fusion features, determine the similarity between the license plate image to be identified and each of the standard images containing the occlusion.

[0050] Specifically, when performing license plate recognition on a license plate image, the second feature is compared with each fused feature. That is, the features of the license plate image to be recognized are compared with the features of each standard image containing the occlusion, thereby obtaining the similarity between the license plate image to be recognized and each standard image containing the occlusion. When comparing the second feature with each fused feature, the similarity between the second feature and each fused feature can be determined based on the distance between them, such as calculating the cosine similarity or Euclidean distance between the second feature and the fused features.

[0051] Optionally, when there are many occlusions in the license plate image to be identified, the proportion of occlusion features in the second feature and each fusion feature is relatively large, and the similarity between the calculated second feature and each fusion feature may be relatively large. A large similarity will affect the recognition of the license plate. Therefore, before determining the similarity between the license plate image to be identified with occlusions and each standard image containing the occlusions based on the second feature and each fusion feature, the proportion of occlusions in the license plate image to be identified is also calculated. If the proportion of occlusions in the license plate image to be identified is greater than or equal to a preset threshold (such as 70%), it is determined that there are too many occlusions in the license plate image to be identified. Manual judgment or other methods are used to identify the license plate image to be identified, thereby reducing unnecessary calculations.

[0052] In this embodiment, since the similarity degree is determined by calculating the similarity between the license plate image to be identified and the standard image containing the occlusion, the difference between the license plate image to be identified and the standard image caused by the occlusion is reduced. The obtained similarity degree is determined based on the difference in the license plate number in the image. Therefore, the similarity degree can better reflect the similarity between the license plate image to be identified and the standard image containing the occlusion, thereby improving the accuracy of license plate recognition when performing license plate recognition based on the similarity degree.

[0053] S105, determine the license plate number in the license plate image to be identified based on the determined similarity levels.

[0054] Optionally, since a higher similarity indicates a smaller difference (i.e., greater similarity) between the second feature and the fused feature, when determining the license plate number in the license plate image to be identified based on various similarity levels, the standard image corresponding to the fused feature with the highest similarity to the license plate image to be identified can be used as the target standard image, and the license plate number in the target standard image can be used as the license plate number in the license plate image to be identified, thereby outputting the corresponding license plate recognition result. If the similarity levels corresponding to multiple fused features are the same as the maximum similarity, that is, multiple fused features correspond to the maximum similarity to the license plate image to be identified, and multiple target standard images are determined, then the license plate numbers in each target standard image and the maximum similarity can be returned to the user as license plate recognition results, and the license plate image to be identified can also be returned to the user, allowing the user to determine the license plate number in the license plate image to be identified based on the license plate image to be identified and each target standard image through manual judgment or other methods. Optionally, when determining the target standard image based on similarity, all standard images corresponding to the fusion features whose similarity meets the user's requirements (e.g., similarity greater than a threshold of 98%) can be used as target standard images, and their corresponding license plate numbers can be returned to the user as license plate recognition results. Optionally, the license plate recognition result may contain only one license plate number from the obtained license plate image to be recognized, or it may include multiple license plate numbers from the obtained license plate image to be recognized and their corresponding images to be recognized, etc. The specific output license plate recognition result can be set by the user and is not limited here.

[0055] In this embodiment, since the similarity level can reflect the similarity between the license plate image to be identified and the standard image containing the occlusion, the license plate image with the occlusion can be identified more accurately based on the similarity level, thereby improving the recognition accuracy of the license plate image with the occlusion.

[0056] In this embodiment, when there is an occlusion in the license plate image to be identified, the features of the occlusion are extracted to obtain the corresponding first feature, and the first feature is fused into the standard features of each pre-stored standard image. Therefore, each fused feature is a standard feature that includes the occlusion feature, which is equivalent to a standard image that includes the occlusion. This reduces the impact of the difference between the license plate image to be identified and the standard image when identifying a license plate image with an occlusion. It improves the accuracy of license plate recognition with an occlusion without losing the image information of the license plate image to be identified.

[0057] In some embodiments, prior to step S101 described above, the method further includes:

[0058] A1. Obtain a vehicle image, which includes the vehicle's exterior shape.

[0059] Specifically, since the acquired images may not necessarily include vehicles, it is necessary to filter the acquired images to obtain vehicle images that include vehicles (i.e., vehicle shapes). Optionally, a pre-trained vehicle detection model can be used to detect vehicles in the acquired images, or a pre-trained classification model can be used to classify the acquired images to obtain vehicle images; there is no limitation here.

[0060] Optionally, the aforementioned image can be an image captured by a camera device or a video frame captured by a camera device. The determination of whether real-time images are needed for license plate detection depends on the actual application scenario. That is, the aforementioned image can be an image or video frame captured in real time or an image or video frame captured in history. For example, when license plate recognition is applied to the penalty of speeding violations, real-time images are usually captured to obtain vehicle images in the real-time images for license plate recognition. In this case, real-time images can be captured by a pre-installed camera, and vehicle images can be obtained from the obtained real-time images.

[0061] Optionally, since the vehicle in the image captured by the camera may be incomplete, that is, only a part of the vehicle may be captured, the image of the captured part of the vehicle is detected. If a license plate number is detected in the image, the image including the part of the vehicle is taken as the vehicle image, and subsequent license plate recognition processing is performed based on the license plate image.

[0062] A2. Based on the above vehicle image, determine the image including the target area to obtain the above license plate image to be identified, wherein the above target area is the area where the license plate number exists.

[0063] Specifically, since license plate recognition only involves the portion of the vehicle image containing the license plate number, directly performing license plate recognition on the vehicle image requires extracting features from the complete vehicle image, increasing unnecessary computation. Furthermore, irrelevant regions may interfere with license plate recognition. Therefore, in this embodiment, the image containing the target region is determined by detecting the region containing the license plate number in the vehicle image. This allows the image containing the target region (i.e., the region containing the license plate number) in the vehicle image to be used as the license plate image to be recognized. This reduces interference from irrelevant regions and the computational load in the license plate recognition process, thereby saving computational resources.

[0064] Optionally, when detecting areas containing license plate numbers in a vehicle image, the vehicle image can be detected using a trained object detection model. This is existing technology and will not be elaborated upon here.

[0065] A3. If the shape of the license plate image to be identified is non-standard, then perform an affine transformation on the license plate image to be identified to obtain a license plate image with a standard shape.

[0066] Specifically, during image acquisition, factors such as the camera's shooting angle and the vehicle's movement affect the vehicle's posture in the resulting image, causing the license plate image to be non-standard (e.g., trapezoidal). Regular license plate images (e.g., standard images) typically have a fixed, standard shape. Therefore, if the obtained license plate image is non-standard, an affine transformation is needed to convert it into a standard shape (e.g., rectangle). Optionally, for computational convenience, when the target detection model outputs the detected area containing the license plate number, it can also output the position information of each corner point of the license plate, allowing for affine transformation of the license plate image based on this corner information.

[0067] The aforementioned affine transformation refers to the process of performing a linear transformation and a translation in a vector space to transform into another vector space. The affine transformation of an image means that an image can achieve various operations such as translation and rotation through a series of set transformations. By performing affine transformation processing on the license plate image to be recognized with a non-standard shape, the license plate image to be recognized can be corrected to obtain the license plate image to be recognized with a standard shape.

[0068] Correspondingly, in step S101 above, when determining whether there is an obstruction in the license plate image to be identified, it is to determine whether there is an obstruction in the license plate image to be identified which has a standard shape.

[0069] In this embodiment, since license plate recognition requires recognition based on images containing license plate numbers, and license plate numbers only occupy a small portion of the vehicle's area, after filtering the collected images to obtain vehicle images containing vehicles, the image of the area containing the license plate number in the vehicle image is obtained as the license plate image to be recognized. This allows for recognition of images containing the license plate number, reducing unnecessary computation and interference from irrelevant areas, thereby improving the efficiency and accuracy of license plate recognition.

[0070] In some embodiments, after step A1 above, the method further includes:

[0071] The vehicle images are inspected to determine the posture of the vehicles in the images.

[0072] If the vehicle posture in the above vehicle image is a preset posture, then the above vehicle image is subjected to posture transformation processing to obtain a first vehicle image and a second vehicle image, wherein the posture of the vehicle in the first vehicle image is the posture corresponding to the side of the vehicle, and the posture of the vehicle in the second vehicle image is the posture corresponding to the front or back of the vehicle.

[0073] Specifically, because some vehicles have license plate numbers in more than one area—for example, dump trucks often also have license plate numbers painted on the side of the vehicle—and the vehicles in the image are in a preset pose (e.g., Figure 2 When the vehicle shown is in a side view at 43 degrees, the vehicle image includes both the side view and the front or back view of the vehicle. Therefore, the vehicle image in the preset view may contain two target areas (the area containing the license plate number). In this case, the vehicle image can be processed by view transformation, that is, the image corresponding to the side view of the vehicle in the view can be generated to obtain the first vehicle image and the image corresponding to the front or back view of the vehicle in the view can be obtained to obtain the second vehicle image.

[0074] Correspondingly, step A2 above, when determining the image including the target region based on the vehicle image to obtain the license plate image to be identified, includes:

[0075] If the target region exists in the first vehicle image, then an image including the target region is determined based on the first vehicle image to obtain the license plate image to be identified corresponding to the first vehicle image.

[0076] If the target region exists in the second vehicle image, then an image including the target region is determined based on the second vehicle image to obtain the license plate image to be identified corresponding to the second vehicle image.

[0077] Specifically, since the vehicle image is transformed to obtain two new vehicle images (first vehicle image and second vehicle image), when determining the target region (i.e. the region containing the license plate number) based on the vehicle image, it is determined whether the corresponding target region exists in each vehicle image based on the first vehicle image and the second vehicle image corresponding to the vehicle image. That is, the first vehicle image is determined to include the target region to obtain the license plate image to be identified corresponding to the first vehicle image, and the second vehicle image is determined to include the target region to obtain the license plate image to be identified corresponding to the second vehicle image.

[0078] In this embodiment, since some vehicles may have license plate numbers in more than one area, and when the vehicle posture in the captured vehicle image is a preset posture, the vehicle in the vehicle image includes both the side and the front or back of the vehicle. Therefore, when the vehicle in the vehicle image is in a preset posture, a first vehicle image with the corresponding side posture and a second vehicle image with the front or back posture are generated based on the vehicle image, thereby increasing the diversity of vehicle images. This allows subsequent recognition based on one or more license plate images to be identified corresponding to the vehicle image, thereby improving the accuracy of license plate recognition.

[0079] In some embodiments, after step S101 described above, the method further includes:

[0080] If there are no obstructions in the license plate image to be identified, then license plate recognition is performed directly on the license plate image to obtain the license plate recognition result.

[0081] Optionally, if no obstruction is detected in the license plate image to be identified, it indicates that there is no abnormality in the license plate image to be identified. In this case, conventional license plate recognition methods can be used to directly identify the license plate image to be identified. After segmenting the license plate image to be identified into characters, each character is identified sequentially through template matching or a trained neural network model, thereby obtaining the license plate number in the license plate image to be identified.

[0082] In this embodiment, when no obstruction is detected in the license plate image to be identified, a conventional license plate recognition method is used to perform license plate recognition, so as to reduce unnecessary calculations and improve license plate recognition efficiency.

[0083] In some embodiments, step A2 above further includes:

[0084] If the first license plate number and the second license plate number are different, then the first target license plate number is taken as the license plate number in the license plate image to be identified, wherein the first target license plate number is the license plate number corresponding to the greater similarity between the first license plate number and the second license plate number.

[0085] The first license plate number is the license plate number obtained after recognizing the license plate image corresponding to the first vehicle image, and the second license plate number is the license plate number obtained after recognizing the license plate image corresponding to the second vehicle image.

[0086] Specifically, since the similarity level indicates the similarity between the license plate image to be identified and the standard image containing the occlusion, the higher the similarity level, the more similar the license plate image to be identified is to the standard image containing the occlusion. Therefore, if the license plate numbers determined by the license plate images to be identified corresponding to two different vehicle images obtained from the vehicle image are different, the license plate number corresponding to the larger similarity level is taken as the first target license plate number, and the first target license plate number is taken as the license plate number in the finally obtained corresponding vehicle image (equivalent to each license plate image to be identified). For example, if the similarity level of the first license plate number A obtained from the license plate image to be identified corresponding to the first vehicle image is 0.97, and the similarity level of the second license plate number B obtained from the license plate image to be identified corresponding to the second vehicle image is 0.99, then the second license plate number B corresponding to the larger similarity level (0.99) is taken as the first target license plate number, that is, the license plate number in each corresponding license plate image to be identified (i.e., the license plate image to be identified corresponding to the first vehicle image and the second vehicle image).

[0087] Optionally, when using the license plate number corresponding to a higher degree of similarity as the license plate number in the corresponding license plate image to be identified, another license plate number corresponding to a lower degree of similarity can be used as an alternative license plate number in the corresponding license plate image to be identified for user reference.

[0088] In this embodiment, since the first license plate number and the second license plate number are obtained from two license plate images to be identified based on the same vehicle image, they should be the same. If the first license plate number and the second license plate number are different, it indicates that the license plate recognition may be abnormal due to occlusion or other reasons. The higher the similarity, the more similar the license plate image to be identified is to the standard image containing occlusion. Therefore, when the first license plate number and the second license plate number are different, the license plate number corresponding to the license plate number with a higher degree of similarity is taken as the license plate number in the license plate image to be identified, so as to enhance the robustness of license plate recognition.

[0089] In some embodiments, if only the first license plate number or only the second license plate number is identified, then step A2 above further includes:

[0090] The third license plate number is determined. The third license plate number is the license plate number obtained after recognizing the license plate image corresponding to the vehicle image that has not undergone attitude transformation processing.

[0091] The second target license plate number is used as the license plate number in the license plate image to be identified, wherein the second target license plate number is the license plate number corresponding to the greater similarity between the first license plate number and the third license plate number, or the second target license plate number is the license plate number corresponding to the greater similarity between the second license plate number and the third license plate number.

[0092] Specifically, if only the first license plate number or the second license plate number is determined based on the first vehicle image and the second vehicle image (i.e., there is no area containing a license plate number in the first or second vehicle image, and therefore no corresponding license plate image and license plate number to be identified), in order to increase image diversity and improve the accuracy of license plate recognition, the image containing the target region is determined based on the vehicle image without pose transformation processing, and the corresponding license plate image to be identified is obtained. Then, it is determined whether there is an occlusion in the license plate image to be identified, and license plate recognition is performed according to the presence of the occlusion (i.e., if there is no occlusion, license plate recognition is performed directly; if there is an occlusion). The process involves extracting features of occluded objects and fusing them with features from the license plate image to obtain a third license plate number. This third license plate number is then compared with the license plate numbers obtained from other license plate images (either the first or second license plate number). If the third license plate number is the same as the first license plate number, or the third license plate number is the same as the second license plate number, it indicates that the license plate number (the license plate number corresponding to the third license plate number) is likely the correct license plate number in the vehicle image, and the accuracy of the license plate recognition result for that vehicle image is high. This identical license plate number is then used as the second target license plate number (i.e., the license plate number of the corresponding vehicle image). If the third license plate number is different from the first license plate number, or if the third license plate number is different from the second license plate number, it indicates that the license plate recognition process may be affected by obstructions, resulting in an anomaly in the license plate recognition result. In this case, the license plate number with a higher degree of similarity between the two license plate numbers (i.e., the license plate number with a higher degree of similarity between the third license plate number and the first license plate number, or the license plate number with a higher degree of similarity between the third license plate number and the second license plate number) is used as the second target license plate number.

[0093] Optionally, if the first license plate number and the second license plate number are identified, but the license plate images corresponding to the first license plate number and the second license plate number have occlusions, and both license plate images determined based on the first vehicle image and the second vehicle image have occlusions, the corresponding third license plate number can also be obtained based on the vehicle image without pose transformation processing. If any two of the first license plate number, the second license plate number and the third license plate number are the same, then the same license plate number is taken as the second target license plate number. If the three license plate numbers are not the same, then the license plate number with the greater similarity is taken as the second target license plate number.

[0094] In this embodiment of the application, when only one license plate image to be identified is obtained from the first vehicle image and the second vehicle image, or when there are occlusions in both license plate images to be identified, the corresponding license plate image to be identified is directly obtained from the corresponding vehicle image, thereby increasing the number of license plate images to be identified and increasing the diversity of images. In this way, the final license plate number of the vehicle image is determined based on the license plate numbers corresponding to the multiple license plate images to be identified, thereby increasing the accuracy of license plate recognition.

[0095] In some embodiments, step S102 includes:

[0096] The above-mentioned license plate image to be identified is segmented to obtain the image of the occlusion.

[0097] Feature extraction is performed on the above-mentioned occlusion image, and the extracted features are then subjected to dimensionality reduction processing to obtain the above-mentioned first feature.

[0098] Feature extraction is performed on the license plate image to be identified, and the extracted features are then subjected to dimensionality reduction processing to obtain the second feature.

[0099] Optionally, since there are occluders in the license plate image to be identified, it is necessary to extract the features of the occluders. Therefore, when detecting occluders in the license plate image to be identified, the pixels corresponding to the detected occluders can be labeled. Then, when extracting the features of the occluders based on the identified license plate image, the license plate image to be identified can be segmented, and the pixels corresponding to the occluders in the license plate image to be identified can be extracted according to the labeled information to obtain the image corresponding to the occluders.

[0100] Specifically, to reduce the computational load and improve the efficiency of license plate recognition, after extracting features from both the occluded object image and the license plate image to be recognized, dimensionality reduction processing is performed on both features to obtain low-dimensional (e.g., two-dimensional) first and second features. For example, the occluded object features are mapped to a two-dimensional space to obtain the two-dimensional first feature, and the features of the license plate image to be recognized are preset to a one-dimensional space to obtain the one-dimensional second feature.

[0101] In this embodiment, since the features obtained are dimensionality-reduced when extracting features from the images of the obstruction and the license plate to be identified, the first and second features obtained are low-dimensional features. This reduces the computational load and storage space required for subsequent license plate recognition. Furthermore, the dimensionality reduction process can remove useless noise, thereby improving the accuracy of subsequent license plate recognition.

[0102] In some embodiments, step S104 includes:

[0103] The second feature and each of the above-mentioned fused features are multiplied by a dot product to obtain the corresponding target distance.

[0104] Based on the distance to each of the aforementioned targets, the similarity between each of the aforementioned standard images containing the aforementioned obstructions and the aforementioned license plate image to be identified is determined.

[0105] Optionally, in order to reduce the computational load of license plate recognition and improve its efficiency, the standard features of the standard images are also subjected to dimensionality reduction processing, and the low-dimensional standard features are pre-stored. Therefore, when fusing the first feature and each of the second features, the computational load can be reduced. At the same time, when calculating the similarity between the license plate image to be recognized and each standard image containing occlusion features, the low-dimensional second features and each fused feature are subjected to dot product processing respectively. The corresponding dot product similarity (i.e., similarity) is determined based on the target distance between the obtained second feature and each fused feature, that is, the similarity between the license plate image to be recognized and each standard image containing occlusion is determined. For example, to reduce the computational and data volume in license plate recognition, each standard feature can be extracted as a one-dimensional standard feature of length M. If P standard features are stored, the storage space is equivalent to storing two-dimensional features of size M×P, which can significantly reduce the storage space occupied. Simultaneously, when extracting features from the license plate image and the occluded object image to be recognized, the features of the occluded object can be extracted as a first feature of size M×N, and the features of the extracted license plate image to be recognized can be extracted as a second feature of length N. Then, when fusing the first feature and each standard feature, they are fused using a dot product. Each fusion feature is obtained as a fusion feature of length N. At this time, all fusion features (P fusion features) can be regarded as a two-dimensional feature of size N×P. When calculating the similarity between the second feature and each fusion feature, the second feature (1×N) and the P fusion features (1×N) are multiplied by a dot product to obtain a vector of length P. Each value in vector P is the similarity between the corresponding standard image containing the occlusion and the license plate image to be identified. The maximum similarity is determined based on the P values ​​of the vector, and the license plate number in the corresponding standard image is determined based on the maximum similarity. This license plate number is then used as the license plate number of the license plate image to be identified.

[0106] In this embodiment of the application, since each feature is extracted into low-dimensional features during the license plate recognition process and calculations are performed based on the low-dimensional features, the amount of computation in the license plate recognition process can be reduced and the license plate recognition efficiency can be improved.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] Example 2:

[0109] Corresponding to the license plate recognition method described in the above embodiments, Figure 3 A structural block diagram of the license plate recognition device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0110] Reference Figure 3 The device includes: an occlusion confirmation module 31, a feature extraction module 32, a fusion processing module 33, a recognition module 34, and a license plate number acquisition module 35.

[0111] The occlusion confirmation module 31 is used to determine whether there is an occlusion in the license plate image to be identified;

[0112] The feature extraction module 32 is used to extract the features of the occlusion to obtain a first feature if the license plate image to be identified has an occlusion, and to extract the features of the license plate image to be identified to obtain a second feature.

[0113] The fusion processing module 33 is used to fuse the first feature with each pre-stored standard feature to obtain each fused feature. The standard features are features of a standard image, and the standard image is a license plate image without obstructions.

[0114] The recognition module 34 is used to determine the similarity between the license plate image to be recognized and each of the standard images containing the occluder based on the second feature and each of the fused features.

[0115] The license plate number acquisition module 35 is used to determine the license plate number in the license plate image to be identified based on the determined similarity levels.

[0116] In this embodiment, when there is an occlusion in the license plate image to be identified, the features of the occlusion are extracted to obtain the corresponding first feature, and the first feature is fused into the standard features of each pre-stored standard image. Therefore, each fused feature is a standard feature that includes the occlusion feature, which is equivalent to a standard image that includes the occlusion. This reduces the impact of the difference between the license plate image to be identified and the standard image when identifying a license plate image with an occlusion. It improves the accuracy of license plate recognition with an occlusion without losing the image information of the license plate image to be identified.

[0117] In some embodiments, the license plate recognition device further includes:

[0118] The vehicle image acquisition module is used to acquire vehicle images, which include the vehicle's external shape.

[0119] The license plate image acquisition module is used to determine an image including a target region based on the vehicle image to obtain the license plate image to be identified, wherein the target region is the region containing the license plate number.

[0120] The image processing module is used to perform affine transformation processing on the license plate image to be identified if the shape of the license plate image to be identified is non-standard, so as to obtain the license plate image to be identified with a standard shape.

[0121] Correspondingly, the aforementioned occlusion confirmation module 31 includes:

[0122] The confirmation unit is used to determine whether there are any obstructions in the license plate image to be identified, which has a standard shape.

[0123] In some embodiments, the license plate recognition device further includes:

[0124] The attitude detection module is used to detect the above vehicle images and determine the attitude of the vehicles in the above vehicle images.

[0125] The attitude transformation module is used to perform attitude transformation processing on the vehicle image if the attitude of the vehicle in the vehicle image is a preset attitude, to obtain a first vehicle image and a second vehicle image, wherein the attitude of the vehicle in the first vehicle image is the attitude corresponding to the side of the vehicle, and the attitude of the vehicle in the second vehicle image is the attitude corresponding to the front or back of the vehicle.

[0126] Correspondingly, the aforementioned license plate image acquisition module includes:

[0127] The first unit is configured to, if the first vehicle image contains the target region, determine an image including the target region based on the first vehicle image, and obtain the license plate image to be identified corresponding to the first vehicle image.

[0128] The second unit is used to determine an image including the target region based on the second vehicle image if the target region exists in the second vehicle image, and to obtain the license plate image to be identified corresponding to the second vehicle image.

[0129] In some embodiments, the license plate recognition device further includes:

[0130] The first confirmation module is used to identify the first target license plate number as the license plate number in the license plate image to be identified, wherein the first target license plate number is the license plate number corresponding to the greater similarity between the first license plate number and the second license plate number.

[0131] In some embodiments, the license plate recognition device further includes:

[0132] The third license plate number acquisition module is used to determine the third license plate number, which is the license plate number obtained after recognizing the license plate image corresponding to the vehicle image that has not undergone attitude transformation processing.

[0133] The second confirmation module is used to identify the license plate number of the license plate image to be recognized as the license plate number of the first license plate number and / or the license plate number of the second license plate number and the license plate number of the third license plate number that has the greater similarity.

[0134] In some embodiments, the feature extraction module 31 includes:

[0135] The segmentation unit is used to segment the license plate image to be identified to obtain the occlusion image.

[0136] The first extraction unit is used to extract features from the image of the occluded object and perform dimensionality reduction processing on the extracted features to obtain the first feature.

[0137] The second extraction unit is used to extract features from the license plate image to be identified and to perform dimensionality reduction processing on the extracted features to obtain the second feature.

[0138] In some embodiments, the above-mentioned fusion processing module includes:

[0139] The distance calculation unit is used to perform dot product processing on the second feature and each of the above-mentioned fused features to obtain the corresponding target distance.

[0140] The similarity calculation unit is used to determine the similarity between each of the standard images containing the occluders and the license plate image to be identified, based on the distance to each of the aforementioned targets.

[0141] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0142] Example 3:

[0143] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 4 of this embodiment includes: at least one processor 40 ( Figure 4 The diagram shows only one processor, a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, which, when executing the computer program 42, performs the steps in any of the above method embodiments.

[0144] The electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This electronic device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that... Figure 4This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0145] The processor 40 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0146] In some embodiments, the memory 41 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. In other embodiments, the memory 41 may be an external storage device of the electronic device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 4. Furthermore, the memory 41 may include both internal and external storage units of the electronic device 4. The memory 41 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments.

[0148] I will not go into details here.

[0149] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0150] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0151] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0153] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0154] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0155] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A license plate recognition method, characterized in that, include: Determine if there are any obstructions in the license plate image to be identified; If the license plate image to be identified contains an obstruction, then the features of the obstruction are extracted to obtain a first feature, and the features of the license plate image to be identified are extracted to obtain a second feature; The first feature is fused with each of the pre-stored standard features to obtain each fused feature. The standard features are features of a standard image, and the standard image is a license plate image without obstructions. Based on the second feature and each of the fused features, the similarity between the license plate image to be identified and each of the standard images containing the occluder is determined respectively; The license plate number in the license plate image to be identified is determined based on the determined similarity levels. Before determining whether the license plate image to be identified has any obstruction, the method further includes: acquiring a vehicle image, wherein the vehicle image is an image including the shape of the vehicle; detecting the vehicle image to determine the posture of the vehicle in the vehicle image; if the posture of the vehicle in the vehicle image is a preset posture, then performing posture transformation processing on the vehicle image to obtain a first vehicle image and a second vehicle image, wherein the posture of the vehicle in the first vehicle image is the posture corresponding to the side of the vehicle, and the posture of the vehicle in the second vehicle image is the posture corresponding to the front or back of the vehicle. If only the first license plate number or only the second license plate number is identified, the method further includes: determining a third license plate number, wherein the third license plate number is a license plate number obtained after recognizing the license plate image to be recognized corresponding to the vehicle image without pose transformation processing; and using a second target license plate number as the license plate number in the license plate image to be recognized, wherein the first license plate number is a license plate number obtained after recognizing the license plate image to be recognized corresponding to the first vehicle image, the second license plate number is a license plate number obtained after recognizing the license plate image to be recognized corresponding to the second vehicle image, and the second target license plate number is the license plate number corresponding to the greater similarity between the first license plate number and the third license plate number, or the second target license plate number is the license plate number corresponding to the greater similarity between the second license plate number and the third license plate number.

2. The license plate recognition method as described in claim 1, characterized in that, The steps of extracting features from the occluder to obtain a first feature, and extracting features from the license plate image to be identified to obtain a second feature, include: The license plate image to be identified is segmented to obtain an image of the occlusion. Feature extraction is performed on the image of the occluder, and the extracted features are then subjected to dimensionality reduction processing to obtain the first feature; Feature extraction is performed on the license plate image to be identified, and the extracted features are then subjected to dimensionality reduction processing to obtain the second feature.

3. The license plate recognition method as described in claim 2, characterized in that, The step of determining the similarity between the license plate image to be identified and each of the standard images containing the occlusion, based on the second feature and each of the fused features, includes: The second feature and each of the fused features are multiplied by a dot product to obtain the corresponding target distance; Based on each target distance, the similarity between each standard image containing the occlusion and the license plate image to be identified is determined.

4. The license plate recognition method according to any one of claims 1 to 3, characterized in that, Before determining whether the license plate image to be identified contains an obstruction, the method further includes: Based on the vehicle image, an image including the target region is determined to obtain the license plate image to be identified, wherein the target region is the region containing the license plate number; If the shape of the license plate image to be identified is non-standard, then an affine transformation is performed on the license plate image to be identified to obtain a license plate image with a standard shape. Determining whether the license plate image to be identified has any obstructions includes: Determine whether the license plate image to be identified, which has a standard shape, contains any obstructions.

5. The license plate recognition method as described in claim 4, characterized in that, The step of determining an image including the target region based on the vehicle image to obtain the license plate image to be identified includes: If the target region exists in the first vehicle image, then an image including the target region is determined based on the first vehicle image to obtain the license plate image to be identified corresponding to the first vehicle image; If the target region exists in the second vehicle image, then an image including the target region is determined based on the second vehicle image to obtain the license plate image to be identified corresponding to the second vehicle image.

6. The license plate recognition method as described in claim 5, characterized in that, If the first license plate number and the second license plate number are different, wherein the first license plate number is the license plate number obtained after recognizing the license plate image to be recognized corresponding to the first vehicle image, and the second license plate number is the license plate number obtained after recognizing the license plate image to be recognized corresponding to the second vehicle image, then the method further includes: The first target license plate number is taken as the license plate number in the license plate image to be identified, wherein the first target license plate number is the license plate number corresponding to the greater similarity between the first license plate number and the second license plate number.

7. A license plate recognition device, characterized in that, include: The occlusion confirmation module is used to determine whether there are any obstructions in the license plate image to be identified; The feature extraction module is used to extract the features of the occlusion object to obtain a first feature if the license plate image to be identified has an occlusion object, and to extract the features of the license plate image to be identified to obtain a second feature. The fusion processing module is used to fuse the first feature with each pre-stored standard feature to obtain each fused feature. The standard feature is a feature of a standard image, and the standard image is a license plate image without obstructions. The recognition module is used to determine the similarity between the license plate image to be recognized and each of the standard images containing the occluder based on the second feature and each of the fused features. The license plate number acquisition module is used to determine the license plate number in the license plate image to be identified based on the determined similarity levels. A vehicle image acquisition module is used to acquire vehicle images, wherein the vehicle image is an image including the vehicle's outline; The attitude detection module is used to detect the vehicle image and determine the attitude of the vehicle in the vehicle image; The attitude transformation module is used to perform attitude transformation processing on the vehicle image if the attitude of the vehicle in the vehicle image is a preset attitude, to obtain a first vehicle image and a second vehicle image, wherein the attitude of the vehicle in the first vehicle image is the attitude corresponding to the side of the vehicle, and the attitude of the vehicle in the second vehicle image is the attitude corresponding to the front or back of the vehicle. If only the first license plate number or only the second license plate number is recognized, the device further includes: The third license plate number acquisition module is used to determine the third license plate number, which is the license plate number obtained after recognizing the license plate image corresponding to the vehicle image that has not undergone attitude transformation processing; The second confirmation module is used to use the second target license plate number as the license plate number in the license plate image to be identified, wherein the first license plate number is the license plate number obtained after identifying the license plate image to be identified corresponding to the first vehicle image, the second license plate number is the license plate number obtained after identifying the license plate image to be identified corresponding to the second vehicle image, and the second target license plate number is the license plate number corresponding to the greater similarity between the first license plate number and the third license plate number, or the second target license plate number is the license plate number corresponding to the greater similarity between the second license plate number and the third license plate number.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images