Methods, devices, electronic equipment, and storage media for recognizing license plate characters

Abstract

This application relates to a method, apparatus, electronic device, and storage medium for recognizing license plate characters. The method includes: acquiring texture features, color features, and gradient direction features of an image of a region to be recognized; wherein the region to be recognized is a segmented region of the license plate area in the acquired image; determining the degradation level of the license plate area based on degradation scores corresponding to the texture features, color features, and gradient direction features; wherein the degradation score characterizes the difference distance between the texture features, color features, and gradient direction features of the license plate area and the corresponding reference features, and the degradation level characterizes the degree of damage and dirtiness of the license plate area; and recognizing characters in the license plate area based on the image of the region to be recognized and the degradation level of the license plate area to obtain character recognition results. Using this method can improve the accuracy of license plate character recognition and expand the application scenarios of license plate character recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for recognizing license plate characters. Background Technology

[0002] As people's living standards improve, the number of cars is constantly increasing, and the number of license plates is also increasing rapidly. Therefore, there are more and more scenarios for license plate detection and recognition.

[0003] Traditional visual license plate recognition solutions generally consist of two steps: license plate localization and character recognition. License plate localization involves image processing of the captured vehicle image, including grayscale conversion, edge detection, and color extraction, to locate the license plate area. Character recognition uses a traditional classifier to categorize the characters in the license plate area, then determines a standard character that matches the categorized characters, and finally identifies the characters on the license plate based on the matching standard character.

[0004] However, due to individual differences between vehicles, some license plates are more damaged and dirty (for example, scrapped license plates are often dirty, broken, or at arbitrary angles (or even upside down), making the characters on the license plate difficult to recognize. This requires manual processing, resulting in low accuracy and application rate of license plate recognition. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for recognizing license plate characters, which can improve the accuracy of character recognition of license plates.

[0006] Firstly, this application provides a method for recognizing license plate characters, including:

[0007] The texture features, color features, and gradient direction features of the image of the region to be identified are obtained; where the image of the region to be identified is a segmented region of the license plate area image;

[0008] The degradation level of the license plate area image is determined based on the degradation scores corresponding to the texture features, color features, and gradient direction features. The degradation score represents the difference distance between the texture features, color features, and gradient direction features and their respective baseline features, while the degradation level represents the degree of damage and dirtiness of the license plate area image.

[0009] Based on the degradation levels of the image of the region to be identified and the image of the license plate region, the characters in the license plate region image are identified, and the character recognition results are obtained.

[0010] Secondly, this application also provides a license plate character recognition device, comprising:

[0011] The acquisition unit is used to obtain the texture features, color features, and gradient direction features of the image of the region to be identified; wherein, the image of the region to be identified is a segmented region of the license plate region image;

[0012] The determination unit is used to determine the degradation level of the license plate area image based on the degradation scores corresponding to the texture features, color features, and gradient direction features respectively; wherein, the degradation score represents the difference distance between the texture features, color features, and gradient direction features and their respective reference features, and the degradation level represents the degree of damage and dirtiness of the license plate area image.

[0013] The recognition unit is used to recognize characters in the license plate area image based on the degradation level of the image of the area to be recognized and the license plate area image, and obtain the character recognition result.

[0014] Thirdly, this application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the license plate character recognition method described above.

[0015] Fourthly, this application also provides a computer-readable storage medium storing program instructions that, when executed by a processor, implement the license plate character recognition method described above.

[0016] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the license plate character recognition method described above.

[0017] The aforementioned methods, devices, electronic equipment, and storage media for recognizing license plate characters can, on the one hand, improve the accuracy and robustness of license plate character recognition by specifically recognizing characters in the license plate region image based on the segmentation image and the degree of degradation of the license plate region image; on the other hand, by judging the degree of damage and dirt in the license plate region based on the difference distance between the texture, color, and gradient direction features of the license plate region image and their corresponding reference features, and then recognizing the license plate characters, the application scenarios for license plate character recognition can be expanded and labor costs can be reduced. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is an application environment diagram of a license plate character recognition method provided in an embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating a method for recognizing license plate characters provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a process for segmenting a license plate area image according to an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of an iterative filtering process for pixel features of a captured image provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of another process for segmenting a license plate area image provided in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of a process for adjusting the geometric transformation of multiple rectangular regions according to an embodiment of this application;

[0025] Figure 7 This is a flowchart illustrating a method for determining the degradation level of a license plate area, as provided in an embodiment of this application.

[0026] Figure 8 This is a schematic diagram of a process for recognizing characters in a license plate area provided in an embodiment of this application;

[0027] Figure 9 This is a schematic diagram of another process for recognizing characters in a license plate area provided in an embodiment of this application;

[0028] Figure 10 This is a flowchart illustrating another method for recognizing license plate characters provided in an embodiment of this application;

[0029] Figure 11 This is a block diagram of a license plate character recognition device provided in an embodiment of this application;

[0030] Figure 12 This is a block diagram of an electronic device provided in an embodiment of this application;

[0031] Figure 13 This is a block diagram of a computer-readable storage medium provided in an embodiment of this application;

[0032] Figure 14 This is a block diagram of a computer program product provided in an embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] The term "and / or" in the embodiments of this application refers to any and all possible combinations including one or more of the associated listed items. It should also be noted that, when used in this specification, "including / comprising" specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or components and / or groups thereof.

[0035] The terms "first," "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0036] Furthermore, although the terms "first," "second," etc., are used repeatedly in this application to describe various operations (or various components, or various applications, or various instructions, or various data), these operations (or components, or applications, or instructions, or data) should not be limited by these terms. These terms are only used to distinguish one operation (or component, or application, or instruction, or data) from another operation (or component, or application, or instruction, or data). For example, a first deterioration level can be called a second deterioration level, and a second deterioration level can be called a first deterioration level; the only difference is the scope they encompass, but they do not depart from the scope of this application. Both the first deterioration level and the second deterioration level represent the degree of damage and dirt in the license plate area, but they are not the same degree of damage and dirt in the same license plate area.

[0037] The license plate character recognition method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, electronic device 102 communicates with server 104 via a communication network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers.

[0038] In some embodiments, reference Figure 1First, server 104 obtains the texture features, color features, and gradient direction features of the image of the region to be identified; wherein, the image of the region to be identified is a segmented region of the license plate region image; then, based on the degradation scores corresponding to the texture features, color features, and gradient direction features, the degradation level of the license plate region image is determined; wherein, the degradation score represents the difference distance between the texture features, color features, and gradient direction features and their respective reference features, and the degradation level represents the degree of damage and dirtiness of the license plate region image; finally, based on the degradation levels of the image of the region to be identified and the license plate region image, the characters in the license plate region image are identified to obtain the character recognition result.

[0039] In some embodiments, the electronic device 102 (such as a mobile terminal or a fixed terminal) can be implemented in various forms. The electronic device 102 can be a mobile terminal such as a mobile phone, smartphone, laptop, portable handheld device, personal digital assistant (PDA), tablet computer (PAD), etc., capable of recognizing characters in a license plate area image based on the degradation level of the image of the area to be recognized and the license plate area image, and obtaining character recognition results. The electronic device 102 can also be a fixed terminal such as an automated teller machine (ATM), access control machine, digital TV, desktop computer, fixed-line computer, etc., capable of recognizing characters in a license plate area image based on the degradation level of the image of the area to be recognized and the license plate area image, and obtaining character recognition results.

[0040] Hereinafter, it is assumed that electronic device 102 is a fixed terminal. However, those skilled in the art will understand that, if there are operations or elements specifically designed for mobile purposes, the construction according to the embodiments disclosed in this application can also be applied to mobile electronic device 102.

[0041] In some embodiments, the image processing component and data processing component running on server 104 may load any of the various additional server applications and / or middleware applications being executed, such as HTTP (Hypertext Transfer Protocol), FTP (File Transfer Protocol), CGI (Common Gateway Interface), RDBMS (Relational Database Management System), etc.

[0042] In some embodiments, server 104 may be implemented using a standalone server or a server cluster consisting of multiple servers. Server 104 may be adapted to run one or more application services or software components that provide the electronic device 102 described in the foregoing disclosure.

[0043] In some embodiments, the application service may include, for example, a service for calculating the difference distance between the texture features, color features, and gradient direction features of the license plate area image and the corresponding reference features, and a service for recognizing characters in the license plate area image based on the degradation level of the image to be recognized and the license plate area image, etc. The software component may include, for example, an app or client with the function of recognizing characters in the license plate area image.

[0044] In some embodiments, server 104 has an APP or client that can recognize characters in license plate area images. Server 104 includes a portal port that provides one-to-one application services to users in the foreground and multiple business systems that perform data processing in the background, so as to extend the application of character recognition function to APP or client, so that users can use and access the character recognition function in license plate area images at any time and any place.

[0045] In some embodiments, the character functions in the license plate area image of the APP or client can be a computer program running in user mode to complete one or more specific tasks. This program can interact with the user and has a visual user interface. The APP or client can include two parts: a graphical user interface (GUI) and an engine, which together provide users with various application services in the form of a user interface, creating a digital customer system.

[0046] In some embodiments, users can input corresponding code data or control parameters into an app or client via an input device to execute application services of a computer program and display application services in the user interface. For example, when it is necessary to determine the difference distance between the texture features, color features, and gradient direction features of a license plate area image and the corresponding reference features, and to recognize characters in the license plate area image based on the degradation level of the image to be recognized and the license plate area image, the user operates through the input device and displays the results through the user interface. Optionally, the input device may be a touch screen input, button input, voice input, or pupil focusing input, etc.

[0047] In some embodiments, the operating system running the app or client may include various versions of Microsoft... Apple and / or Linux operating system, various commercial or similar Operating systems (including but not limited to various GNU / Linux operating systems, Google) OS and / or mobile operating systems, such as Phone OS OS OS operating systems, as well as other online or offline operating systems, are not specifically limited here.

[0048] In some embodiments, such as Figure 2 As shown, a method for recognizing license plate characters is provided, which can be applied to... Figure 1 Taking server 104 as an example, the method includes the following steps:

[0049] Step S11: Obtain the texture features, color features, and gradient direction features of the image of the region to be identified.

[0050] The region to be identified is the segmented region of the license plate area image.

[0051] In some embodiments, the user captures images of the vehicle using a camera device in the electronic device. The electronic device then sends the captured images to a server for license plate recognition and to segment the image containing the license plate area into multiple regions to be recognized.

[0052] In some embodiments, the image of the region to be identified may have been pre-made by a camera device or other device in an electronic device and stored in a third-party organization (e.g., an image database, a cloud storage platform, etc.). When the server responds to receiving the user's instruction to start recognizing license plate characters, the server directly obtains the image of the region to be identified corresponding to the license plate region from the corresponding third-party organization.

[0053] In some embodiments, color features include HSV (Hue, Saturation, Value) features. HSV features are a color space created based on the intuitive characteristics of color, also known as the Hexcone Model. The parameters of the color features are hue (H), saturation (S), and value (V).

[0054] In some embodiments, the gradient direction feature is the gradient direction histogram feature of color and / or texture in the image of the region to be identified.

[0055] Step S12: Determine the degradation level of the license plate area image based on the degradation scores corresponding to the texture features, color features, and gradient direction features.

[0056] Among them, the degradation score represents the difference distance between texture features, color features, and gradient direction features and their respective baseline features.

[0057] In some embodiments, the calculation rule corresponding to the degradation score is (1-x%)×100; where x% is used to characterize the degree of matching between the texture features, color features, and gradient direction features of the license plate area image and their respective baseline features.

[0058] As an example, if the similarity between the texture features of the license plate area image and its corresponding baseline texture features is 80%, the server determines the degradation score corresponding to the texture features of the license plate area image as (1-80%)×100=20 points; if the similarity between the color features of the license plate area image and its corresponding baseline color features is 60%, the server determines the degradation score corresponding to the color features of the license plate area image as (1-60%)×100=40 points; if the similarity between the gradient direction features of the license plate area image and its corresponding baseline gradient direction features is 40%, the server determines the degradation score corresponding to the gradient direction features of the license plate area image as (1-40%)×100=60 points.

[0059] Among them, the degradation level characterizes the degree of damage and dirtiness of the license plate area image.

[0060] In some embodiments, the degradation level is obtained by fusing the degradation score and weight coefficient corresponding to the texture features, the degradation score and weight coefficient corresponding to the color features, and the degradation score and weight coefficient corresponding to the gradient direction features of the license plate area image.

[0061] In some embodiments, the calculation rule for the degradation level is Pi∈(xi), xi=A×a%+B×b%+C×c%.

[0062] Where xi is the fusion score corresponding to texture features, color features and gradient direction features, and each degradation level Pi corresponds to a preset number of score segments. When the fusion score xi belongs to the score segment corresponding to a degradation level Pi, then the fusion score xi corresponds to the degradation level Pi.

[0063] Where A is the degradation score corresponding to the texture features of the license plate area image, and a% is the weight coefficient corresponding to the texture features of the license plate area image; B is the degradation score corresponding to the color features of the license plate area image, and b% is the weight coefficient corresponding to the color features of the license plate area image; C is the degradation score corresponding to the gradient direction features of the license plate area image, and c% is the weight coefficient corresponding to the gradient direction features of the license plate area image.

[0064] As an example, it is known that the score range corresponding to degradation level P1 is [0, 20), the score range corresponding to degradation level P2 is [20, 40), the score range corresponding to degradation level P3 is [40, 60), the score range corresponding to degradation level P4 is [60, 80), and the score range corresponding to degradation level P5 is [80, 100). If the fusion score of the license plate area image is: 20×30% + 40×30% + 80×40% = 50 points, and 50 points belongs to the score range [40, 60), then the degradation level corresponding to the license plate area image is P3.

[0065] Step S13: Based on the degradation levels of the image to be identified and the license plate area image, the characters in the license plate area image are identified to obtain the character recognition results.

[0066] In some embodiments, the server uses CNN-based and / or Transformer-based methods to recognize characters in the license plate area image according to the degradation level of the image, obtaining character recognition results. Specifically, the CNN-based character recognition method uses a character recognition model built on a CNN; the Transformer-based character recognition method uses a character recognition model built on a Transformer.

[0067] In some embodiments, the CNN-based character recognition method and the Transformer-based character recognition method correspond to different degradation levels of the license plate area image. The processes of the CNN-based character recognition method and the Transformer-based character recognition method are different, as are their recognition efficiency and accuracy. Design engineers can adapt their methods based on their respective advantages and disadvantages and the degradation level of the license plate area image.

[0068] For example, if the degradation level of the license plate area image is high, the Transformer character recognition method is used to recognize the characters in the license plate area image of the image to be recognized; if the degradation level of the license plate area image is low, the CNN character recognition method is used to recognize the characters in the license plate area image of the image to be recognized.

[0069] In the above-mentioned license plate character recognition method, on the one hand, the characters in the license plate region image are specifically identified based on the segmentation image of the license plate region and the degree of degradation of the license plate region image, which can improve the accuracy and robustness of license plate character recognition; on the other hand, the degree of damage and dirtiness of the license plate region image is judged based on the difference distance between the texture, color and gradient direction features of the license plate region image and their respective corresponding reference features, and then the license plate characters are recognized, which can expand the application scenarios of license plate character recognition and reduce labor costs.

[0070] Those skilled in the art will understand that the methods disclosed in the above-described specific embodiments can be implemented in more concrete ways. For example, the above-described implementation of the license plate character recognition method is merely an illustrative description.

[0071] For example, the process by which the server determines the degradation level of the license plate area image based on the degradation scores corresponding to the texture features, color features, and gradient direction features is merely one way of combining features. In actual implementation, there may be other ways of dividing features. For example, the degradation scores corresponding to the texture features, color features, and gradient direction features and the degradation level of the license plate area image may be combined or combined into another system, or some features may be ignored or not executed.

[0072] In a more specific implementation, the process of the server recognizing characters in the license plate area image based on the degradation level of the image of the area to be recognized and the license plate area image, and obtaining the character recognition result, may also include the process of using a CNN-based or Transformer-based method for character recognition according to the degradation level of the license plate area image.

[0073] In some embodiments, see Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of segmenting a license plate region image in this application. Before step S11, i.e., before the server obtains the texture features, color features, and gradient direction features of the image of the region to be identified, the method further includes:

[0074] Step S21: Obtain the pixel features of the acquired image.

[0075] In some embodiments, the pixel features are LBP features, that is, the server obtains the LBP features of each pixel in the acquired image.

[0076] Specifically, LBP features refer to the following: For a given pixel in an image, it has 8 connected pixels. Among these 8 connected pixels, pixels with values ​​higher than the given pixel are counted as 1, and those with lower values ​​are counted as 0, resulting in an 8-bit binary code (e.g., 00010011). This binary code is then converted to decimal, resulting in a number between 0 and 255 (e.g., 19). After performing this process once, an LBP image of the same size (ignoring edge pixels) can be obtained. This is the most basic and original LBP feature.

[0077] In some embodiments, the server slides a sliding window of a preset size across the acquired image at preset steps to extract LBP features sequentially.

[0078] Step S22: Iteratively filter the pixel features of the acquired image to filter out the license plate area image.

[0079] In some embodiments, the server inputs the pixel features of the acquired image into multiple cascaded classifiers to classify the pixel features, and then selects the pixel features used to synthesize the license plate area image from the classified pixel features.

[0080] In some embodiments, see Figure 4 , Figure 4 This is a flowchart illustrating an embodiment of iterative filtering of pixel features in a captured image according to this application. In step S22, the server iteratively filters the pixel features of the captured image to filter out the license plate area image, including:

[0081] Step S221: Based on the radial basis function kernel, the pixel features of the acquired image are mapped to multiple hyperplanes for iterative classification to obtain the pixel features of the license plate area image in the classified acquired image.

[0082] In some embodiments, the server inputs the pixel features of the acquired image into multiple cascaded classifiers. Then, the multiple cascaded classifiers map the LBP features onto a hyperplane in a high-dimensional space based on radial basis function kernels. Finally, the LBP features corresponding to the license plate area image and the LBP features corresponding to the background area are classified on the hyperplane.

[0083] Furthermore, multiple cascaded classifiers iteratively classify the LBP features in sequence, and the last classifier outputs the final classified license plate area image and background area image.

[0084] Step S222: Based on the intersection-union ratio (IU) of the pixel features of the classified license plate area image, merge the pixel features of the classified license plate area image to obtain the license plate area image.

[0085] In some embodiments, the server uses the IOU (Intersection over Union) metric to merge pixel features in the classified license plate area image to produce the final license plate area image.

[0086] The license plate area image is a coarse localization image of the license plate area.

[0087] Step S23: Based on the texture features of the license plate area image, the license plate area image is divided into multiple regions to be identified; wherein, the aspect ratio and skew angle of the multiple regions to be identified are the same.

[0088] In some embodiments, see Figure 5 , Figure 5This is a schematic flowchart illustrating another embodiment of segmenting a license plate region image in this application. In step S23, the server segments the license plate region image into multiple regions to be identified based on the texture features of the license plate region image, including:

[0089] Step S231: Based on the texture features of the license plate area image, the license plate area image is divided into multiple initial images.

[0090] Each initial image includes a character image of one character in the license plate area image, and the consistency of the texture features of each character image is greater than a preset threshold.

[0091] In some embodiments, the license plate region image includes multiple characters, each character occupying a region of the image, and the character regions do not intersect, with high similarity in texture features among the character regions. Therefore, the server determines multiple character regions in the license plate region image based on the distribution of texture features in the license plate region image, and then divides these multiple character regions into multiple initial images accordingly.

[0092] Step S232: Perform image pixel processing on multiple initial images to determine multiple corresponding rectangular regions.

[0093] In some embodiments, image pixel processing includes the server sequentially performing Gaussian blurring, grayscale conversion, Sobel operator processing, binarization, and closing operation processing on the multiple initial images to perform contour extraction processing on the multiple initial images, thereby obtaining multiple corresponding bounding rectangular regions; wherein, the closing operation processing includes setting image parameters and thresholds for the initial images. Contour extraction processing includes operations on bounding rectangles based on texture features in the initial images. The rectangular regions are the bounding rectangular regions of characters in the character image.

[0094] Step S233: Perform image geometric transformation on multiple rectangular regions to obtain corresponding multiple regions to be identified images.

[0095] In some embodiments, see Figure 6 , Figure 6 This is a flowchart illustrating an embodiment of image geometric transformation adjustment of multiple rectangular regions in this application. In step S233, the server performs image geometric transformation adjustment on the multiple rectangular regions to obtain corresponding images of multiple regions to be identified, including:

[0096] Step a1: Determine the aspect ratio and skew angle of each rectangular region.

[0097] Step a2: Based on the skew angle of each rectangular region, perform an affine transformation on each rectangular region to obtain each rectangular region with the skew angle rotated to a horizontal angle.

[0098] Step a3: Based on the aspect ratio of each rectangular region after the skew angle is rotated to a horizontal angle, scale and adjust each rectangular region after the skew angle is rotated to a horizontal angle to obtain rectangular regions with the same aspect ratio.

[0099] The server uses rectangular regions with a horizontal skew angle and a preset aspect ratio as corresponding images of multiple regions to be identified.

[0100] In some embodiments, see Figure 7 , Figure 7 This is a flowchart illustrating an embodiment of determining the degradation level of a license plate area in this application. In step S12, where the server determines the degradation level of the license plate area image based on the degradation scores corresponding to texture features, color features, and gradient direction features, it may further include the following methods:

[0101] Step S121: Based on the weight coefficients corresponding to the texture features, color features, and gradient direction features, fuse the degradation scores of the texture features, color features, and gradient direction features to obtain the fused degradation score.

[0102] Step S122: Determine the degradation level of the license plate area image based on the fused degradation score.

[0103] Among them, the degradation level characterizes the degree of damage and dirtiness of the license plate area image.

[0104] In one embodiment, if the server responds to the degradation level of the license plate area image as Level 1, the server identifies the characters in the license plate area image based on the similarity between the characters in the preset character template image and the characters in the image to be identified, and obtains the character recognition result.

[0105] In some embodiments, see Figure 8 , Figure 8 This is a schematic flowchart illustrating an embodiment of character recognition in a license plate region according to this application. In step S13, the server recognizes the characters in the license plate region image based on the similarity between a preset character template image and the characters in the image of the region to be recognized, obtaining a character recognition result, including:

[0106] Step S131: Determine the character region, background region, and the boundary region between the background and the character in the image to be recognized.

[0107] In some embodiments, the server uses a CNN semantic segmentation network to identify character regions, background regions, and the boundary regions between background and characters in each image to be identified.

[0108] Step S132: Segment the character region image in the image to be identified, and perform image geometric transformation on the character region image to obtain the adjusted character region image.

[0109] In some embodiments, the server segments the character region images from the identified character regions, background regions, and the boundary regions between the background and characters. Then, the server performs perspective transformation processing on the segmented character region images to adjust the perspective-transformed images to a bird's-eye view, thereby obtaining the adjusted character region images.

[0110] Step S133: Based on the similarity between the preset character template image and the adjusted character region image, the characters in the license plate region image are identified to obtain the character recognition result.

[0111] In some embodiments, the server uses a CNN-based image recognition network to determine the similarity between a preset character template image and the processed character region images, in order to recognize the characters in each license plate region image and obtain character recognition results.

[0112] In another embodiment, if the server responds to the degradation level of the license plate area image as Level 2, the server identifies the characters in the license plate area image based on the semantic association information between the characters in the license plate area image and the image to be identified, and obtains the character recognition result.

[0113] Among them, the second level represents a higher degree of damage and dirtiness in the license plate area image than the first level.

[0114] In some embodiments, see Figure 9 , Figure 9 This is a schematic flowchart illustrating another embodiment of character recognition in a license plate region according to this application. In step S13, the server recognizes the characters in the license plate region image based on the semantic association information between the characters in the license plate region image and the image to be recognized, obtaining the character recognition result, including:

[0115] Step S134: Perform image geometric transformation on the image of the region to be identified to obtain the adjusted image of the region to be identified.

[0116] In some embodiments, the server sequentially performs perspective transformation processing, bird's-eye view adjustment processing, and image size unification processing on each image of the region to be identified, so as to obtain multiple adjusted images of the regions to be identified.

[0117] Step S135: Perform channel attention enhancement processing and spatial attention enhancement processing on the adjusted image of the region to be recognized to obtain the semantic information and positional information of the characters in the adjusted image of the region to be recognized.

[0118] In some embodiments, the server performs channel attention enhancement processing and spatial attention enhancement processing on the adjusted image of the area to be recognized, which is beneficial to determining the relevance of the expression content between the license plate area and the characters.

[0119] As an example, the Chinese characters on the license plate (i.e., the provincial identifier of the license plate, for example, Jing, Chuan, Gan, Hu, etc. used to represent different provinces) generally appear at the first position of the license plate; the English characters on the license plate (i.e., the municipal identifier of the license plate, for example, A, B, C, D, etc. used to represent different cities in a certain province) generally appear at the second position of the license plate.

[0120] Among them, the Chinese characters and English characters on the license plate are the semantic information of the characters in the image of the area to be recognized obtained by the server; the Chinese characters appearing at the first position of the license plate and the English characters on the license plate appearing at the second position are the position information of the characters in the image of the area to be recognized obtained by the server.

[0121] Step S136: Based on the semantic association information between the semantic information and position information of the characters in the license plate area image and the adjusted image of the area to be recognized, recognize the characters in the license plate area image to obtain a character recognition result.

[0122] In one embodiment, the server inputs the semantic information and position information of the characters in the image of the area to be recognized into a trained string recognition model of Transform to determine the semantic association information between the semantic information and position information of the characters in the license plate area image and the adjusted image of the area to be recognized, so as to predict the true meaning of the characters in the license plate area image and obtain the recognition result of the characters.

[0123] To more clearly illustrate the license plate character recognition method provided by the embodiments of the present disclosure, the following uses a specific embodiment to specifically describe the license plate character recognition method. In one exemplary embodiment, refer to Figure 10 , Figure 10 FIG. is a flowchart of a license plate character recognition method shown according to another exemplary embodiment. The license plate character recognition method is applied to the server 104 and specifically includes the following contents:

[0124] Step S31: Obtain the LBP feature of each pixel point of the acquired image.

[0125] Step S32: Input the LBP features into a plurality of cascaded classifiers in sequence for classifying the license plate area image and the background area image.

[0126] In this process, a single classifier maps LBP features to a hyperplane in a high-dimensional space based on a radial basis function kernel, and then classifies the license plate area image and the background area image on the hyperplane.

[0127] Furthermore, multiple cascaded classifiers iteratively classify the LBP features in sequence, and the last classifier outputs the final classified license plate area image and background area image.

[0128] Specifically, cascaded classifiers refer to two types of classifiers. Boosting is a method that improves algorithm accuracy through multiple learning iterations. It employs a synergistic principle to significantly improve efficiency, transforming weak classifiers into strong ones. AdaBoost is an adaptive boosting algorithm. Its adaptive nature lies in using samples misclassified by the previous classifier to train the next classifier. This iterative algorithm adds a new weak classifier in each round until a predetermined, sufficiently low error rate is achieved. Each training sample is assigned a weight indicating its probability of being selected into the training set by a particular classifier. If a sample has been accurately classified, its probability of being selected in the next training set is reduced; conversely, if a sample is not accurately classified, its weight is increased. In this way, AdaBoost can "focus" on samples that are more difficult to classify (more informative). The basic principle of AdaBoost is to combine multiple weak classifiers (typically single-layer decision trees) in a reasonable way to create a strong classifier.

[0129] Adaboost employs an iterative approach, training only one weak classifier in each iteration. This trained weak classifier is then used in the next iteration. In other words, in the Nth iteration, there are a total of N weak classifiers, of which N-1 are previously trained with their parameters unchanged, and the Nth classifier is trained in this iteration. The relationship between the weak classifiers is that the Nth weak classifier is more likely to correctly classify the data that the previous N-1 weak classifiers have misclassified. The final classification output depends on the combined performance of these N classifiers.

[0130] Step S33: Based on the IOU index, merge the pixels of each license plate area image in the classification to obtain a coarse localization image of the license plate area.

[0131] The server extracts LBP features sequentially by sliding a fixed-size window across the original image with a fixed step size. These features are then input into a cascaded classifier to determine whether the image contains a license plate. After completing the full image interpretation, the server uses the IoU metric to merge the pixels in the license plate region, generating the final coarse localization image of the license plate region.

[0132] Step S34: Extract texture features from the coarse localization image of the license plate area.

[0133] Step S35: Based on texture features, the coarse localization image is segmented into multiple candidate region images.

[0134] Step S36: Gaussian blurring, grayscale conversion, Sobel operator processing, binarization, and closing operation are performed sequentially on the multiple candidate region images to obtain contour processing of the multiple candidate region images, thereby obtaining multiple bounding rectangular regions including texture features.

[0135] Step S37: Determine the aspect ratio and skew angle of each circumscribed rectangular region.

[0136] Step S38: Determine multiple circumscribed rectangular regions that satisfy the minimum aspect ratio.

[0137] Step S39: Rotate the skew angle of each bounding rectangle region to a horizontal angle based on affine transformation, and adjust the aspect ratio of each bounding rectangle region to a uniform size to obtain multiple adjusted bounding rectangle regions.

[0138] In this context, the texture feature of each bounding rectangular region is the texture feature of a single character.

[0139] Specifically, the server extracts the license plate outline from the coarse localization image of the license plate area through texture feature extraction, image morphology processing (i.e., Gaussian blurring, grayscale conversion, Sobel operator processing, and binarization processing in sequence), and sets appropriate threshold parameters (i.e., closing operation processing). The aspect ratio and skew angle of the license plate are obtained by calculating the bounding rectangle of the outline. For candidate regions with excessively large skew angles, affine transformation is used to rotate them to a horizontal position, and the image size of the candidate regions is adjusted to a uniform size. This not only eliminates the interference of background factors in the acquired image, thereby improving the accuracy of subsequent license plate character recognition, but also facilitates the standardization of parameters in subsequent models.

[0140] Step S40: Extract color features, texture features, and histogram features from the adjusted multiple bounding rectangular regions.

[0141] Step S41: Input the color features, texture features and histogram features corresponding to each bounding rectangular region into the preset decision tree to determine the level of dirtiness, so as to obtain the level of dirtiness corresponding to each bounding rectangular region.

[0142] The server uses digital image processing algorithms to extract the color, texture, and histogram features of the license plate surface (i.e., each circumscribed rectangular area), generates corresponding feature vectors, and inputs these feature vectors into a decision function to obtain a description of the degree of dirtiness of the license plate.

[0143] Among them, the color features corresponding to the circumscribed rectangular area include HSV (Hue, Saturation, Value) features. HSV features are a color space created based on the intuitive characteristics of color, also known as the Hexcone Model.

[0144] The parameters of color features are hue (H), saturation (S), and lightness (V).

[0145] First, the decision tree determines the dirtiness score of color features, texture features, and histogram features respectively to obtain the dirtiness score of color features, texture features, and histogram features corresponding to each bounding rectangle region. Then, the decision tree merges the dirtiness scores corresponding to color features, texture features, and histogram features according to the preset weight coefficients of color features, texture features, and histogram features of each bounding rectangle region to obtain the dirtiness level corresponding to each bounding rectangle region.

[0146] The level of dirtiness is divided into five levels: A, B, C, D, and E, with A being the cleanest and E being the dirtiest.

[0147] Furthermore, the server uses either a CNN-based or Transformer-based method to recognize characters based on the level of dirtiness corresponding to each outer rectangular area of ​​the license plate.

[0148] Specifically, when the dirt level corresponding to the circumscribed rectangular area is A or B, a CNN-based character recognition algorithm is used; when the dirt level is C or D, a Transform-based character recognition algorithm is used; and when the dirt level is E, human intervention is required (i.e., Figure 10 Step S42c: Transfer to manual processing.

[0149] The server employs a CNN-based character recognition algorithm, which includes the following steps:

[0150] Step S42a: In response to the dirt level of the circumscribed rectangular region being A or B, the semantic segmentation network based on CNN identifies the character center region, background region, and the boundary region between the background and the character in each adjusted circumscribed rectangular region.

[0151] Step S43a: Based on the character center area, background area, and the boundary area between the background and the character of each adjusted bounding rectangle area, the character center area of ​​each adjusted bounding rectangle area is cut out.

[0152] Step S44a: Perform perspective transformation on the image of the center region of each character after cutting, and then adjust the processed image to a bird's-eye view to obtain the adjusted images of the center regions of multiple characters.

[0153] Step S45a: The CNN-based image recognition network performs character recognition on the adjusted multiple character center region images to obtain the character recognition result corresponding to each character center region image.

[0154] The CNN-based image recognition network performs character recognition on the adjusted multiple character center region images by: the image recognition network identifies the characters in each character center region image based on the similarity between a preset character template image and the adjusted multiple character center region images, and obtains the character recognition result.

[0155] The server employs a Transform-based character recognition algorithm, which includes the following steps:

[0156] Step S42b: In response to the dirt level of the circumscribed rectangular area being C or D, perform perspective transformation, bird's-eye view adjustment, and image size unification processing on each adjusted circumscribed rectangular area in sequence to obtain multiple adjusted rectangular area images.

[0157] Step S43b: Based on the attention mechanism, attention enhancement processing is performed on the adjusted rectangular region images to obtain the semantic and positional information of the characters in the adjusted rectangular region images.

[0158] Step S44b: The string recognition model based on Transform is used to perform string recognition on the adjusted multiple rectangular region images to obtain the character recognition result corresponding to each rectangular region image.

[0159] Among them, the string recognition model based on Transform performs string recognition on multiple rectangular region images after adjustment. The string recognition model predicts characters based on the semantic association information between the license plate region and the semantic and positional information of the characters, and obtains the character recognition result.

[0160] The character prediction process includes: based on the semantic association between the license plate area and the semantic and positional information of the characters, predicting the first and last positions of the Chinese characters most likely to appear in the license plate area, thereby solving to some extent the problem of license plates being unrecognizable due to obstruction or partial loss caused by dirt.

[0161] The above scheme, on the one hand, can improve the accuracy and robustness of license plate character recognition by specifically recognizing characters in the license plate region image based on the segmentation and degradation degree of the license plate region image; on the other hand, it can determine the degree of damage and dirtiness of the license plate region image based on the difference distance between the texture, color, and gradient direction features of the license plate region image and their corresponding reference features, thereby recognizing the license plate characters, which can expand the application scenarios of license plate character recognition and reduce labor costs.

[0162] It should be understood that, although Figures 2-10 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-10 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0163] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0164] Figure 11 This is a block diagram of a license plate character recognition device provided in an embodiment of this application. (Refer to...) Figure 11 The license plate character recognition device 10 includes:

[0165] The acquisition unit 11 is used to acquire the texture features, color features and gradient direction features of the image of the region to be identified; wherein, the image of the region to be identified is a segmented region of the license plate region image.

[0166] The determination unit 12 is used to determine the degradation level of the license plate area image based on the degradation scores corresponding to the texture features, color features, and gradient direction features respectively; wherein, the degradation score represents the difference distance between the texture features, color features, and gradient direction features and their respective reference features, and the degradation level represents the degree of damage and dirtiness of the license plate area image.

[0167] The recognition unit 13 is used to recognize the characters in the license plate area image based on the degradation level of the image of the area to be recognized and the license plate area image, and obtain the character recognition result.

[0168] In some embodiments, the recognition unit 13 is specifically used to: recognize characters in the license plate region image based on the degradation levels of the image of the region to be recognized and the license plate region image to obtain character recognition results.

[0169] Responding to the first level of degradation in the license plate region image, the characters in the license plate region image are identified based on the similarity between the characters in the preset character template image and the characters in the image to be identified, thus obtaining the character recognition result; or,

[0170] In response to the second level of degradation of the license plate area image, the characters in the license plate area image are identified based on the semantic association information between the characters in the license plate area image and the image to be identified, and the character recognition result is obtained.

[0171] Among them, the second level represents a higher degree of damage and dirtiness in the license plate area image than the first level.

[0172] In some embodiments, the recognition unit 13 is specifically used to recognize characters in a license plate region image based on the similarity between characters in a preset character template image and characters in a region image to be recognized, in order to obtain character recognition results:

[0173] Identify the character region, background region, and the boundary region between the background and characters in the image to be recognized;

[0174] The character region image is segmented from the image of the region to be identified, and the character region image is adjusted by image geometric transformation to obtain the adjusted character region image;

[0175] Based on the similarity between the preset character template image and the adjusted character region image, the characters in the license plate region image are identified, and the character recognition results are obtained.

[0176] In some embodiments, the recognition unit 13 is specifically used to: recognize characters in the license plate region image based on semantic association information between characters in the license plate region image and the region image to be recognized, and obtain character recognition results;

[0177] The image of the region to be identified is adjusted by performing geometric transformations to obtain the adjusted image of the region to be identified.

[0178] The adjusted image of the region to be identified is subjected to channel attention enhancement processing and spatial attention enhancement processing to obtain the semantic and positional information of the characters in the adjusted character region image;

[0179] Based on the semantic association information between the semantic and positional information of characters in the license plate region image and the adjusted region image to be recognized, the characters in the license plate region image are recognized to obtain the character recognition results.

[0180] In some embodiments, in determining the degradation level of a license plate region image based on degradation scores corresponding to texture features, color features, and gradient direction features, the determining unit 12 is specifically used for:

[0181] Based on the weight coefficients corresponding to texture features, color features, and gradient direction features, the degradation scores of texture features, color features, and gradient direction features are fused to obtain the fused degradation score.

[0182] The degradation level of the license plate area image is determined based on the fused degradation score.

[0183] Among them, color features include hue features, saturation features, and brightness features, and gradient direction features include gradient direction histogram features.

[0184] In some embodiments, before obtaining the texture features, color features, and gradient direction features of the image of the region to be identified, the license plate character recognition device 10 further includes a filtering unit 14, which is specifically used for:

[0185] Obtain the pixel features of the acquired image;

[0186] The pixel features of the acquired images are iteratively filtered to select the license plate area image;

[0187] Based on the texture features of the license plate area image, the license plate area image is divided into multiple regions to be identified; among them, the aspect ratio and skew angle of the multiple regions to be identified are the same.

[0188] In some embodiments, in dividing the license plate region image into multiple regions to be identified based on the texture features of the license plate region image, the filtering unit 14 is specifically used for:

[0189] Based on the texture features of the license plate region image, the license plate region image is divided into multiple initial images; each initial image includes a character image of one character in the license plate region image, and the consistency of the texture features of each character image is greater than a preset threshold.

[0190] Multiple initial images are processed to determine multiple corresponding rectangular regions; where the rectangular region is the bounding rectangular region of the character in the character image;

[0191] Multiple rectangular regions are subjected to image geometric transformations to adjust the images, resulting in multiple corresponding images of regions to be identified.

[0192] In some embodiments, in adjusting the image geometric transformation of multiple rectangular regions to obtain corresponding multiple regions to be identified, the filtering unit 14 is specifically used for:

[0193] Determine the aspect ratio and skew angle of each rectangular region;

[0194] Based on the skew angle of each rectangular region, an affine transformation is performed on each rectangular region to obtain each rectangular region whose skew angle is rotated to a horizontal angle.

[0195] Based on the aspect ratio of each rectangular region after the skew angle is rotated to a horizontal angle, each rectangular region after the skew angle is rotated to a horizontal angle is scaled and adjusted to obtain rectangular regions with the same aspect ratio.

[0196] In some embodiments, the filtering unit 14 is specifically used for iteratively filtering the pixel features of the acquired image to filter out the license plate area image:

[0197] The pixel features of the acquired image are mapped to multiple hyperplanes based on the radial basis function kernel for iterative classification, thereby obtaining the pixel features of the license plate area image in the classified acquired image.

[0198] Based on the cross-union ratio (CUI) of pixel features in the classified license plate region image, the pixel features of the classified license plate region image are merged to obtain the license plate region image.

[0199] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0200] Figure 12 This is a block diagram of an electronic device 20 provided in an embodiment of this application. For example, the electronic device 20 can be a server. (Refer to...) Figure 12 The electronic device 20 includes a processor 21, which may be a collection of processors, including one or more processors. The electronic device 20 also includes memory resources represented by a memory 22, where computer programs, such as application programs, are stored. The computer programs stored in the memory 22 may include one or more modules, each corresponding to a set of executable instructions. Furthermore, the processor 21 is configured to implement the license plate character recognition method described above when executing the computer program.

[0201] In some embodiments, electronic device 20 is a server, and the computing system within the server can run one or more operating systems, including any operating systems discussed above and any commercially available server operating system. The server can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, database servers, etc. Exemplary database servers include, but are not limited to, commercially available database servers from companies such as IBM.

[0202] In some embodiments, processor 21 typically controls the overall operation of electronic device 20, such as operations associated with display, data processing, data communication, and recording operations. Processor 21 may include one or more processors to execute computer programs to perform all or part of the steps of the methods described above. Furthermore, processor 21 may include one or more modules to facilitate interaction between processor 21 and other components. For example, processor 21 may include a multimedia module to facilitate control of the interaction between the user electronic device and processor 21 using multimedia components.

[0203] In some embodiments, the processor in processor 21 may also be referred to as a CPU (Central Processing Unit). The processor may be an electronic chip with signal processing capabilities. The processor may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, the processor may be implemented using integrated circuit chips.

[0204] In some embodiments, memory 22 is configured to store various types of data to support operation of electronic device 20. Examples of such data include instructions for any application or method operating on electronic device 20, acquired data, messages, images, videos, etc. Memory 22 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, optical disk, or graphene storage.

[0205] In some embodiments, the memory 22 can be a memory module, a TF card, etc., and can store all information in the electronic device 20, including the input raw data, computer programs, intermediate running results, and final running results. It stores and retrieves information according to the location specified by the processor.

[0206] In some embodiments, the memory 22 is essential for the electronic device 20 to have a memory function and ensure normal operation. The memory 22 of the electronic device 20 can be classified according to its purpose as main memory (RAM) and auxiliary memory (external storage), or it can be classified as external memory and internal memory. External storage is typically magnetic media or optical discs, which can store information long-term. RAM refers to the storage components on the motherboard, used to store currently executing data and programs, but it is only used for temporary storage; the data will be lost when the power is turned off.

[0207] In some embodiments, electronic device 20 may further include: a power supply component 23 configured to perform power management of electronic device 20, a wired or wireless network interface 24 configured to connect electronic device 20 to a network, and an input / output (I / O) interface 25. Electronic device 20 may operate an operating system stored in memory 22, such as Windows Server, Mac OSX, Unix, Linux, FreeBSD, or similar.

[0208] In some embodiments, power supply component 23 provides power to various components of electronic device 20. Power supply component 23 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 20.

[0209] In some embodiments, the wired or wireless network interface 24 is configured to facilitate wired or wireless communication between the electronic device 20 and other devices. The electronic device 20 may access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof.

[0210] In some embodiments, the wired or wireless network interface 24 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the wired or wireless network interface 24 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0211] In some embodiments, the input / output (I / O) interface 25 provides an interface between the processor 21 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a power button, and a lock button.

[0212] Figure 13 This is a block diagram of a computer-readable storage medium 30 provided in an embodiment of this application. The computer-readable storage medium 30 stores program instructions 31; wherein, when executed by a processor, the program instructions 31 implement the license plate character recognition method described above.

[0213] If the integrated units of the various functional units in the various embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium 30. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer-readable storage medium 30 includes a number of instructions in a program instruction 31 to cause a computer device (which may be a personal computer, system server, or network device, etc.), an electronic device (e.g., MP3, MP4, etc., or a mobile phone, tablet computer, wearable device, etc., or a desktop computer, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of this application.

[0214] Figure 14 This is a block diagram of a computer program product 40 provided in an embodiment of this application. The computer program product 40 includes a computer program 41, which can be executed by the processor of the electronic device 20 to implement the license plate character recognition method as described above.

[0215] Those skilled in the art will understand that embodiments of this application can provide a method for recognizing license plate characters, a device 10 for recognizing license plate characters, an electronic device 20, a computer-readable storage medium 30, or a computer program product 40. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product 40 implemented on one or more devices including a computer program 41 (including but not limited to disk storage, CD-ROM, optical storage, etc.).

[0216] This application is described with reference to flowchart illustrations and / or block diagrams of a license plate character recognition method, license plate character recognition device 10, electronic device 20, computer-readable storage medium 30, or computer program product 40 according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by the computer program product 40. These computer program products 40 can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that a computer program 41, executed by the processor of the computer or other programmable data processing device, produces a function to implement the process... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0217] These computer program products 40 may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the computer program 41 stored in the computer program product 40 produces an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0218] These computer programs 41 may also be loaded onto a computer or other programmable data processing equipment, causing a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing the computer program 41 executing on the computer or other programmable equipment with the means to implement the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0219] It should be noted that the various methods, apparatuses, electronic devices, computer-readable storage media, computer program products, etc. described above may also include other implementation methods according to the description of the method embodiments. For specific implementation methods, please refer to the description of the relevant method embodiments, which will not be elaborated here.

[0220] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0221] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method of recognizing characters of a license plate, characterized by, The method comprises: obtaining texture features, color features and gradient direction features of a to-be-recognized region image, wherein the to-be-recognized region image is a segmented region of a license plate region image; determining a degradation level of the license plate region image based on degradation scores corresponding to the texture features, the color features and the gradient direction features, wherein the degradation score represents a difference distance between the texture features, the color features and the gradient direction features and respective reference features, and the degradation level represents a degree of damage and dirt of the license plate region image; recognizing characters in the license plate region image based on the degradation level of the to-be-recognized region image and the license plate region image, to obtain a character recognition result.

2. The method of claim 1, wherein, The method of recognizing characters in the license plate region image based on the degradation level of the to-be-recognized region image and the license plate region image to obtain a character recognition result comprises: in response to the degradation level of the license plate region image being a first level, recognizing characters in the license plate region image based on a similarity between a preset character template image and the characters in the to-be-recognized region image, to obtain a character recognition result; or in response to the degradation level of the license plate region image being a second level, recognizing characters in the license plate region image based on semantic association information between the license plate region image and the characters in the to-be-recognized region image, to obtain a character recognition result; wherein the degree of damage and dirt of the license plate region image represented by the second level is higher than that represented by the first level.

3. The method of claim 2, wherein, The method of recognizing characters in the license plate region image based on a similarity between a preset character template image and the characters in the to-be-recognized region image to obtain a character recognition result comprises: determining a character region, a background region and a boundary region between the background and the characters in the to-be-recognized region image; segmenting a character region image from the to-be-recognized region image, and performing image geometric transformation adjustment on the character region image to obtain an adjusted character region image; recognizing characters in the license plate region image based on a similarity between a preset character template image and the adjusted character region image, to obtain a character recognition result.

4. The method of claim 2, wherein, The method of recognizing characters in the license plate region image based on semantic association information between the license plate region image and the characters in the to-be-recognized region image to obtain a character recognition result comprises: performing image geometric transformation adjustment on the to-be-recognized region image to obtain an adjusted to-be-recognized region image; performing channel attention enhancement processing and spatial attention enhancement processing on the adjusted to-be-recognized region image to obtain semantic information and position information of characters in the adjusted to-be-recognized region image; recognizing characters in the license plate region image based on semantic association information between the license plate region image and the semantic information and position information of characters in the adjusted to-be-recognized region image, to obtain a character recognition result.

5. The method of claim 1, wherein, The color features comprise hue features, saturation features and lightness features, and the gradient direction features comprise gradient direction histogram features. determining the degradation level of the license plate region image based on the degradation scores corresponding to the texture feature, the color feature and the gradient direction feature respectively, comprises: fusing the degradation scores of the texture feature, the color feature and the gradient direction feature based on the weight coefficients corresponding to the texture feature, the color feature and the gradient direction feature respectively, to obtain a fused degradation score; determining the degradation level of the license plate region image according to the fused degradation score.

6. The method of claim 1, wherein, Before the texture feature, the color feature and the gradient direction feature of the to-be-recognized region image are obtained, the method further comprises: obtaining pixel features of a collected image; iteratively screening the pixel features of the collected image to screen out the license plate region image; based on the texture feature of the license plate region image, the license plate region image is divided into a plurality of to-be-recognized region images; wherein the aspect ratio and the skew angle of the plurality of to-be-recognized region images are the same.

7. The method of claim 6, wherein, based on the texture feature of the license plate region image, the license plate region image is divided into a plurality of to-be-recognized region images, comprising: based on the texture feature of the license plate region image, the license plate region image is divided into a plurality of initial images; wherein each initial image includes a character image of a character in the license plate region image, and the consistency of the texture feature of each character image is greater than a preset threshold; image pixel processing is performed on the plurality of initial images to determine a plurality of corresponding rectangular regions; wherein the rectangular region is the circumscribed rectangular region of the character in the character image; image geometric transformation adjustment is performed on the plurality of rectangular regions to obtain the plurality of corresponding to-be-recognized region images.

8. The method of claim 7, wherein, the image geometric transformation adjustment on the plurality of rectangular regions to obtain the plurality of corresponding to-be-recognized region images, comprising: determining the aspect ratio and the skew angle of each rectangular region; based on the skew angle of each rectangular region, affine transformation adjustment is performed on each rectangular region to obtain each rectangular region with a horizontal angle after skew angle rotation; based on the aspect ratio of each rectangular region with a horizontal angle after skew angle rotation, scaling adjustment is performed on each rectangular region with a horizontal angle after skew angle rotation to obtain each rectangular region with the same aspect ratio.

9. The method of claim 6, wherein, the iterative screening of the pixel features of the collected image to screen out the license plate region image, comprising: mapping the pixel features of the collected image to a plurality of hyperplanes based on a radial basis function kernel for iterative classification to obtain the pixel features of the license plate region image in the classified collected image; based on the intersection over union index of the pixel features of the classified license plate region image, merging each pixel feature of the classified license plate region image to obtain the license plate region image.

10. An apparatus for recognizing characters of a license plate, characterized by comprising: comprising: an acquisition unit configured to obtain texture features, color features and gradient direction features of a to-be-recognized region image; wherein the to-be-recognized region image is a segmented region of a license plate region image; The determining unit is configured to determine a degradation level of the license plate region image based on degradation scores corresponding to the texture feature, the color feature, and the gradient direction feature respectively, wherein the degradation score represents a difference distance between the texture feature, the color feature, and the gradient direction feature and a corresponding reference feature, and the degradation level represents a damage and dirt degree of the license plate region image. The recognizing unit is configured to recognize characters in the license plate region image based on the to-be-recognized region image and the degradation level of the license plate region image, to obtain a character recognition result.

11. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the license plate character recognition method in any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium having program instructions stored thereon, the program instructions being executable by a processor to perform a method comprising: The program instructions are executed by the processor to implement the license plate character recognition method in any one of claims 1 to 9.

Images