Multi-national license plate recognition system, multi-national license plate recognition method, and storage medium

Abstract

The application discloses a multi-country license plate recognition system and relates to the technical field of license plate recognition.The system comprises a license plate recognition module, and the license plate recognition module comprises a license plate recognition controller, a license plate positioning module, a license plate correction module, a license plate width-height ratio determination module, a license plate background recognition module, a license plate background removal module, a license plate background storage module, a license plate system storage module, a Y-direction character projection module, a Y-direction character segmentation module, an X-direction character projection module, an X-direction character segmentation module, a character recognition module and a license plate generation module.The application can effectively recognize complex overseas license plate information such as double-row license plates and license plates with background patterns and match the country or region of the license plate.

Description

Technical Field

[0001] This invention relates to the field of license plate recognition technology, specifically to a multi-country license plate recognition system and method. Background Technology

[0002] Currently, license plate recognition technology in China is relatively mature. However, the development of recognition systems for license plates from multiple countries has been slower. The key reason is that unlike domestic license plates which have standardized specifications, overseas license plates come in a wide variety of formats. For example, some overseas countries or regions have single-layer and double-layer license plates, and their sizes are not uniform. Some countries and regions also have background patterns on their license plates. China's relatively mature license plate recognition systems cannot adapt to the complex and ever-changing scenarios involving license plates from multiple countries. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-country license plate recognition system and method that can effectively recognize complex overseas license plate information such as double-row plates and plates with background patterns, and match the country or region of the license plate.

[0004] To achieve the above objectives, the present invention provides a multi-national license plate recognition system, including a license plate recognition module; the license plate recognition module includes a license plate recognition controller, a license plate positioning module, a license plate correction module, a license plate aspect ratio determination module, a license plate background recognition module, a license plate background removal module, a license plate background storage module, a license plate format storage module, a Y-axis character projection module, a Y-axis character segmentation module, an X-axis character projection module, an X-axis character segmentation module, a character recognition module, and a license plate generation module.

[0005] The further technical solution includes a data display module connected to the license plate recognition module; the data display module is used to match the corresponding language based on the country or region information contained in the license plate information transmitted by the license plate recognition module, and display the license plate information.

[0006] A further technical solution includes a data management module connected to the license plate recognition module, wherein the data management module is used to modify and set the parameters in the license plate recognition module.

[0007] The further technical solution includes a vehicle detection module, an image acquisition module connected to the vehicle detection module, and an image processing module connected to the image acquisition module and the license plate recognition module.

[0008] This invention also provides a method for recognizing license plates from multiple countries, applicable to the aforementioned multi-country license plate recognition system, comprising the following steps: S1, image acquisition;

[0009] S2, Image Processing;

[0010] S3. License Plate Recognition: Includes the following steps:

[0011] S31. License plate location, obtain license plate sample;

[0012] S32, License Plate Correction: The license plate correction module corrects the tilt of the license plate sample;

[0013] S33, License Plate Width-to-Height Ratio Determination: The license plate width-to-height ratio determination module calculates the width-to-height ratio of the license plate sample;

[0014] S34. Preliminary determination of single or double license plate: The character recognition controller makes a preliminary determination of whether the license plate sample is single or double based on the relationship between the width-to-height ratio of the license plate and the first threshold: if the width-to-height ratio of the license plate is greater than the first threshold, the license plate sample is determined to be single; if the width-to-height ratio is less than or equal to the first threshold, the license plate sample is determined to be single. The character recognition controller filters out the countries or regions that meet the requirements based on whether the license plate sample is single or double, and records them as set one.

[0015] S35. License Plate Background Recognition: Based on the aspect ratio of the license plate, the character recognition controller filters license plate background images stored in the license plate background storage module that meet the aspect ratio requirements, and sends these images to the license plate background recognition module. The license plate background recognition module compares the license plate samples with the corresponding background images one by one. If a matching image is found, the character recognition controller sends it to the license plate background removal module and obtains the country or region corresponding to the background image and its corresponding confidence level.

[0016] Let this be set two; if no matching license plate background image is found, it is determined to be a license plate without a background.

[0017] S36, License Plate Background Removal: The license plate background removal module removes the background from the license plate and sends the background-removed license plate sample to the character recognition controller;

[0018] S37, Character Projection Segmentation:

[0019] S371. If the license plate sample is initially determined to be a single row, then character segmentation is performed according to the single-row character segmentation method: the Y-axis character projection module first projects the license plate sample in the Y direction, the Y-axis character segmentation module removes the projection area smaller than the set value, and performs Y-axis segmentation based on the remaining projection: when the variance of the Y-axis projection area of ​​the characters is less than or equal to the second threshold, character segmentation is performed directly based on the projection boundary line; and

[0020] The characters and their order information are sent to the character recognition module. If the variance is greater than the second threshold, the characters are segmented according to the projection boundary line. The characters with a projection area less than or equal to the third threshold and their order information are sent to the character recognition module. The characters with a projection area greater than or equal to the third threshold are judged as character groups, and the character groups and their order information are sent to the character recognition controller.

[0021] The character recognition controller sends the character set to the X-axis character projection module; the X-axis character projection module projects the character set in the X direction, and the X-axis character segmentation module segments the characters according to the projection boundary line.

[0022] The cut characters and their order are sent to the character recognition module;

[0023] S372. If it is initially determined that the license plate sample is a double row, the X-axis character projection module first projects the license plate sample in the X-axis direction.

[0024] The X-axis character segmentation module segments the characters according to the projection boundary line and sends the two rows of characters and their position information to the character recognition controller. The character recognition controller sends the two rows of characters to the Y-axis character projection module, which then segments the two rows of characters according to the single-row character segmentation method in S3261.

[0025] S38, Character Recognition: The character recognition module recognizes the received characters and sends the character recognition result and the corresponding character order information to the character recognition controller;

[0026] S39. Initial License Plate Generation: The character recognition controller receives the character recognition results and the position of the characters.

[0027] Sequence information is sent to the license plate generation module; the license plate generation module processes the character recognition results and the position of the characters.

[0028] The location and sequence information are integrated to initially generate license plate information;

[0029] S310, Country or Region Determination: The license plate generation module sends the generated license plate information to the character recognition controller;

[0030] The character recognition controller compares the generated license plate information with the license plate formats stored in the license plate format storage module; it then filters out the countries or regions corresponding to the license plate formats that match the information, along with their corresponding credibility levels, denoted as set three; next, it calculates the intersection of the countries or regions in sets one, two, and three; if multiple intersections are found, it selects the country or region with the highest credibility; if no intersection is found, it selects the country or region with the highest credibility from set three; finally, it sends the obtained country or region to the license plate...

[0031] Generate modules;

[0032] S311, Final License Plate Generation: The license plate generation module integrates the received country or region information and the initially generated license plate information to generate the final license plate information.

[0033] The further technical solution is as follows: the third threshold is the average value of the character projection area.

[0034] The further technical solution is as follows: the first threshold, the second threshold, and the third threshold are stored in the data management module.

[0035] The further technical solution is as follows: after the S3 license plate recognition step, there is also an S4 data display step: the license plate recognition module sends the generated license plate information to the data display module, and the data display module matches the corresponding language according to the country or region information contained in the license plate information and displays the license plate information.

[0036] The further technical solution is as follows: S1, the image acquisition step is as follows: when the vehicle detection module detects a vehicle, it sends a signal to the image acquisition module, and the image acquisition module acquires a sample image; S2, the image processing step is as follows: the image processing module processes the acquired sample image to obtain the sample image to be identified; S31, the license plate localization method is as follows: the license plate localization module performs edge detection on the sample image to be identified, and performs binarization processing on the obtained edge information to remove areas that do not conform to the rules, thereby obtaining the localization area of ​​the license plate.

[0037] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by a computer, perform the above-described multi-national license plate recognition method.

[0038] The advantages of this invention compared to the prior art are: this invention can effectively identify complex overseas license plate information such as double rows and background patterns, and match the country or region of the license plate. It also facilitates the addition of new license plate formats, makes it easy to modify parameters, and has high reliability and stability.

[0039] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic block diagram of the entire embodiment 1 of the present invention;

[0042] Figure 2 This is a schematic block diagram of the license plate recognition module in Example 1. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1

[0046] A multi-national license plate recognition system, such as Figure 1-2 As shown, it includes a license plate recognition module; the license plate recognition module includes a license plate recognition controller, a license plate positioning module, a license plate correction module, a license plate aspect ratio determination module, a license plate background recognition module, a license plate background removal module, a license plate background storage module, a license plate format storage module, a Y-axis character projection module, a Y-axis character segmentation module, an X-axis character projection module, an X-axis character segmentation module, a character recognition module, and a license plate generation module.

[0047] The further technical solution includes a data display module connected to the license plate recognition module. The data display module is used to match the corresponding language based on the country or region information contained in the license plate information transmitted by the license plate recognition module, and then display the license plate information.

[0048] The further technical solution includes a data management module connected to the license plate recognition module, which is used to modify and set the parameters in the license plate recognition module.

[0049] Its further technical solution includes a vehicle detection module, an image acquisition module connected to the vehicle detection module, and an image processing module connected to the image acquisition module and the license plate recognition module.

[0050] Example 2

[0051] A method for recognizing license plates from multiple countries, applicable to the aforementioned multi-country license plate recognition system, includes the following steps:

[0052] S1, Image Acquisition;

[0053] S2, Image Processing;

[0054] S3. License Plate Recognition: Includes the following steps:

[0055] S31. License plate location, obtain license plate sample;

[0056] S32, License Plate Correction: The license plate correction module corrects the tilt of the license plate sample;

[0057] S33, License Plate Width-to-Height Ratio Determination: The license plate width-to-height ratio determination module calculates the width-to-height ratio of the license plate sample;

[0058] S34. Preliminary determination of single / double license plate: The character recognition controller makes a preliminary determination based on the relationship between the width-to-height ratio of the license plate and the first threshold.

[0059] Determine whether the license plate sample is single-row or double-row: If the width-to-height ratio of the license plate is greater than the first threshold, the license plate sample is determined to be single-row; if the width-to-height ratio is less than or equal to the first threshold, the license plate sample is determined to be single-row; the character recognition controller filters out the countries or regions that meet the requirements based on whether the license plate sample is single-row or double-row, and records them as set one;

[0060] S35, License Plate Background Recognition: The character recognition controller filters the stored license plate backgrounds based on the aspect ratio of the license plate.

[0061] The block stores license plate background images that meet the aspect ratio requirements and sends them to the license plate background recognition module. The license plate background recognition module compares the license plate sample with the license plate background image one by one. If a matching license plate background image is found, the character recognition controller sends the license plate background image to the license plate background removal module and obtains the country or region corresponding to the license plate background image and the corresponding confidence level, which is denoted as set two. If no matching license plate background image is found, it is determined to be a license plate without background.

[0062] S36. License Plate Background Removal: The license plate background removal module removes the background from the license plate and displays the removed image.

[0063] The card sample is sent to the character recognition controller;

[0064] S37, Character Projection Segmentation:

[0065] S371. If the license plate sample is initially determined to be a single row, then character segmentation is performed according to the single-row character segmentation method: Y direction

[0066] The character projection module first projects the license plate sample in the Y direction. The Y-direction character segmentation module removes projection areas smaller than a set value and performs Y-direction segmentation based on the remaining projection: when the variance of the character's Y-direction projection area is less than or equal to a second threshold, character segmentation is performed directly based on the projection boundary line; and

[0067] The characters and their order information are sent to the character recognition module. If the variance is greater than the second threshold, the characters are segmented according to the projection boundary line. The characters with a projection area less than or equal to the third threshold and their order information are sent to the character recognition module. The characters with a projection area greater than or equal to the third threshold are judged as character groups, and the character groups and their order information are sent to the character recognition controller.

[0068] The character recognition controller sends the character set to the X-axis character projection module; the X-axis character projection module projects the character set in the X direction, and the X-axis character segmentation module segments the characters according to the projection boundary line.

[0069] The cut characters and their order are sent to the character recognition module;

[0070] S372. If it is initially determined that the license plate sample is a double row, the X-direction character projection module first projects the license plate sample in the X direction, the X-direction character segmentation module segments it according to the projection boundary line, and sends the two rows of characters and character position information obtained by segmentation to the character recognition controller; the character recognition controller sends the two rows of characters to the Y-direction character projection module, and segments the two rows of characters according to the single row character segmentation method in S3261 respectively;

[0071] S38, Character Recognition: The character recognition module recognizes the received characters and sends the character recognition result and the corresponding character order information to the character recognition controller;

[0072] S39. Initial license plate generation: The character recognition controller sends the received character recognition results and the position and order information of the characters to the license plate generation module; the license plate generation module integrates the character recognition results and the position and order information of the characters to initially generate license plate information;

[0073] S310, Country or Region Determination: The license plate generation module sends the generated license plate information to the character recognition controller;

[0074] The character recognition controller compares the generated license plate information with the license plate formats stored in the license plate format storage module; it then filters out the countries or regions corresponding to the license plate formats that meet the information and their corresponding credibility, denoted as set three; then it finds the intersection of the countries or regions in sets one, two, and three; if multiple intersections are found, it selects the country or region with the highest credibility; if no intersection is found, it selects the country or region with the highest credibility in set three; and sends the obtained country or region to the license plate generation module.

[0075] S311, Final License Plate Generation: The license plate generation module integrates the received country or region information and the initially generated license plate information to generate the final license plate information.

[0076] The further technical solution is: the third threshold is the average value of the character projection area.

[0077] The further technical solution is as follows: the first threshold, the second threshold, and the third threshold are stored in the data management module.

[0078] Its further technical solution is as follows: S3, after the license plate recognition step, there is also S4, data display: the license plate recognition module sends the generated license plate information to the data display module, the data display module matches the corresponding language according to the country or region information contained in the license plate information, and displays the license plate information.

[0079] The further technical solution is as follows: S1, the image acquisition step is: when the vehicle detection module detects a vehicle, it sends a signal to the image acquisition module, and the image acquisition module acquires a sample image; S2, the image processing step is: the image processing module processes the acquired sample image to obtain the sample image to be identified; S31, the license plate localization method is: the license plate localization module performs edge detection on the sample image to be identified, and performs binarization processing on the obtained edge information to remove areas that do not conform to the rules, thereby obtaining the localization area of ​​the license plate.

[0080] Example 3

[0081] A storage medium storing computer-readable instructions that, when executed by a computer, perform the aforementioned multi-national license plate recognition method.

[0082] The above examples are merely illustrative of the technical content of the present invention to facilitate easier understanding by the reader, but do not imply that the implementation of the present invention is limited to these examples. Any technical extensions or re-creations made based on the present invention are protected by the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for recognizing a multi-national license plate, the method comprising: The method comprises the following steps: ​ S1, image acquisition; S2, image processing; S3, license plate recognition, comprising the following steps: S31, license plate positioning: obtaining a license plate sample; S32, license plate correction: a license plate correction module performs inclination correction on the license plate sample; S33, license plate aspect ratio determination: a license plate aspect ratio determination module calculates the aspect ratio of the license plate sample; S34, single / double row preliminary determination: a character recognition controller preliminarily determines whether the license plate sample is a single row or a double row according to the relationship between the aspect ratio of the license plate sample and a first threshold value: if the aspect ratio of the license plate is greater than the first threshold value, it is determined that the license plate sample is a single row; if the aspect ratio is less than or equal to the first threshold value, it is determined that the license plate sample is a single row; the character recognition controller screens out a country or region that meets the requirement according to whether the license plate sample is a single row or a double row, and records the country or region as set one; S35, license plate background recognition: the character recognition controller screens a license plate background image that meets the aspect ratio requirement from the license plate background storage module according to the aspect ratio of the license plate, and sends the license plate background image to a license plate background recognition module; the license plate background recognition module compares the license plate sample with the license plate background image one by one, and if a matching license plate background image is found, the character recognition controller sends the license plate background image to a license plate background removal module, and obtains the country or region corresponding to the license plate background image and the corresponding confidence, and records the country or region as set two; if no matching license plate background image is found, it is determined that the license plate has no background; S36, license plate background removal: the license plate background removal module removes the background on the license plate, and sends the license plate sample with the background removed to the character recognition controller; S37, character projection segmentation; S38, character recognition: a character recognition module recognizes the received characters, and sends the character recognition result and the corresponding character order information to the character recognition controller; S39, preliminary license plate generation: the character recognition controller sends the received character recognition result and the position and order information of the characters to a license plate generation module; The license plate generation module integrates the character recognition result and the position and order information of the characters, and preliminarily generates license plate information; S310, country or region determination: the license plate generation module sends the generated license plate information to the character recognition controller; the character recognition controller compares the generated license plate information with the license plate standards stored in the license plate standard storage module; screens out the country or region corresponding to the license plate standard that meets the license plate information and the corresponding confidence, and records the country or region as set three; then, the countries or regions in set one, set two and set three are intersected; if multiple intersections are found, the country or region with the highest confidence is taken; if no intersection is found, the country or region with the highest confidence in set three is taken; the obtained country or region is sent to the license plate generation module; S311, final license plate generation: the license plate generation module integrates the received country or region and the preliminarily generated license plate information to generate the final license plate information.

2. The multi-national license plate recognition method of claim 1, wherein: The character projection segmentation is specifically as follows: S371, if the license plate sample is preliminarily determined as single-row, the character segmentation is performed according to the single-row character segmentation method: the Y-direction character projection module first projects the license plate sample in Y direction, the Y-direction character segmentation module removes the projection area smaller than a set value, and performs Y-direction segmentation according to the remaining projection: when the variance of the Y-direction projection area of the character is less than or equal to the second threshold value, the character segmentation is directly performed according to the projection boundary; and the character and the sequence information of the character are sent to the character recognition module; If the variance is greater than the second threshold value, the character segmentation is performed according to the projection boundary, the character whose projection area is less than or equal to the third threshold value and the sequence information of the character are sent to the character recognition module, the character whose projection area is greater than or equal to the third threshold value is determined as a character group, and the character group and the sequence information of the character group are sent to the character recognition controller; the character group is sent to the X-direction character projection module by the character recognition controller; the X-direction character projection module projects the character group in X direction, the X-direction character segmentation module performs character segmentation according to the projection boundary, and the segmented character and the sequence are sent to the character recognition module; S372, if the license plate sample is preliminarily determined as double-row, the X-direction character projection module first projects the license plate sample in X direction, the X-direction character segmentation module performs segmentation according to the projection boundary, and the two rows of characters and the character position information obtained by segmentation are sent to the character recognition controller; the character recognition controller sends the two rows of characters to the Y-direction character projection module, and the two rows of characters are segmented according to the single-row character segmentation method in S3261.

3. The multi-national license plate recognition method of claim 2, wherein: The third threshold value is the average value of the character projection area.

4. The multi-national license plate recognition method of claim 2, wherein: The first threshold value, the second threshold value and the third threshold value are stored in the data management module.

5. The multi-national license plate recognition method of claim 1, wherein: After the S3, the license plate recognition step, S4, data display is further provided: the license plate recognition module sends the generated license plate information to the data display module, the data display module matches the corresponding language according to the country or region information contained in the license plate information, and displays the license plate information.

6. The multi-national license plate recognition method of claim 1, wherein: The S1, the image acquisition step is: when the vehicle detection module detects a vehicle, a signal is sent to the image acquisition module, and the image acquisition module acquires a sample image; the S2, the image processing step is: the image processing module processes the acquired sample image to obtain a to-be-recognized sample image; the S31, the license plate positioning method is: the license plate positioning module performs edge detection on the to-be-recognized sample image, and performs binaryzation processing on the obtained edge information, removes the area not meeting the rules, and obtains the positioning area of the license plate.

7. A multi-national license plate recognition system for implementing the multi-national license plate recognition method of any one of claims 1-6, the system comprising a license plate recognition module; characterized in that: The license plate recognition module includes a license plate recognition controller, a license plate positioning module, a license plate correction module, a license plate width-height ratio determination module, a license plate background recognition module, a license plate background removal module, a license plate background storage module, a license plate standard storage module, a Y-direction character projection module, a Y-direction character segmentation module, an X-direction character projection module, an X-direction character segmentation module, a character recognition module, and a license plate generation module.

8. The multi-national license plate recognition system of claim 7, wherein: The data display module is connected with the license plate recognition module, and is used for matching corresponding language according to the country or region information contained in the license plate information transmitted by the license plate recognition module, and displaying the license plate information.

9. The multi-national license plate recognition system of claim 8, wherein: The data management module is connected with the license plate recognition module, and is used for modifying and setting parameters in the license plate recognition module.

10. The multi-national license plate recognition system of any of claims 7-9, wherein: The vehicle detection module, the image acquisition module connected with the vehicle detection module, and the image processing module connected with the image acquisition module and the license plate recognition module are further included.

Images