License plate recognition method and device capable of configuring license plate rules

Abstract

The invention provides a license plate recognition method capable of configuring license plate rules. The method comprises the following steps: adding a Hanzi class in character classes of a license plate character detection and classification network; a feature extraction network is adopted to perform feature extraction on characters of original Chinese character classes and newly configured Chinese characters, and a common Chinese character feature table and a new Chinese character feature table are established; detecting a license plate area from the image by adopting a license plate positioning network, and obtaining a character string by adopting an improved license plate character detection and classification network; for character strings conforming to license plate rules, calculating the similarity between the Chinese characters of the character strings and the feature vectors of the Chinese characters in the common Chinese character feature table and the new Chinese character feature table, and determining the classification result of the Chinese characters; and outputting a license plate character string classification result. Compared with the prior art, original license plate character detection and recognition precision can be ensured, meanwhile, original characters and other Chinese characters can be effectively distinguished, and universal detection of the Chinese characters can be achieved.

Description

Technical Field

[0001] This invention relates to image processing, smart parking, and license plate recognition, and particularly to a license plate recognition method and apparatus with configurable license plate rules. Background Technology

[0002] Whether in today's various traffic situations, public safety management systems, or in the future fields of driverless cars and autonomous driving, license plate recognition is an indispensable process that brings many conveniences to the healthy and orderly management of cities.

[0003] Existing license plate recognition systems can only recognize license plates according to a few types of license plate rules designated by the state. As the number and types of vehicles increase, new license plate rules will emerge, such as new energy vehicles, trial operation license plates, and local license plates used in special areas. In order to adapt to the new rules, it is often necessary to collect a large number of samples to retrain the model and set rules, and the license plate recognition system needs to be developed again. The development cycle is often long.

[0004] The most significant challenge in redeveloping a license plate recognition system is that the existing model cannot recognize the Chinese characters on the license plates. Given the vast number of Chinese characters—3,500 at the first level and 3,000 at the second level—accurately identifying a single character from thousands requires not only a large number of samples but also substantial network and operational overhead. For resource-constrained embedded chips, simply using general-purpose Chinese character detection and recognition networks often fails to meet storage and processing speed requirements. Furthermore, license plate characters possess characteristics such as low pixel values, complex lighting conditions, varying angles, and dirt, significantly reducing character recognizability and often leading to errors during direct recognition.

[0005] In conclusion, there is an urgent need to propose a license plate recognition method with configurable license plate rules to solve the above problems. Summary of the Invention

[0006] In view of this, the main objective of the present invention is to provide a license plate recognition method with configurable license plate rules that can achieve both accuracy and speed.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a license plate recognition method with configurable license plate rules is provided, the method comprising: The first step is to add a Hanzi class to the character categories of the license plate character detection and classification network to obtain an improved license plate character detection and classification network. The second step involves scaling the character boxes of existing Chinese characters to a fixed width and height, inputting them into the feature extraction network for feature extraction, obtaining N-dimensional feature vectors, recording the corresponding Chinese character + feature vector, and storing them in the "Common Chinese Character Feature Table"; and scaling the character boxes of newly configured Chinese characters to a fixed width and height, inputting them into the feature extraction network for feature extraction, obtaining N-dimensional feature vectors, recording the corresponding Chinese character + feature vector, and storing them in the "New Chinese Character Feature Table". The third step involves inputting an image and using a license plate localization network to detect the license plate region and the four corner points of the license plate outline in the image. The tilted license plate is then corrected into a horizontal rectangle with a fixed width and height to obtain the second license plate region. The second license plate region is then input into an improved license plate character detection and classification network to obtain the category and outline of all characters within the license plate, and stored in order of coordinate size. The fourth step is to determine whether the string conforms to the license plate rules. If it does not conform to the license plate rules, proceed directly to the fifth step. If it conforms to the license plate rules, calculate the similarity between the feature vectors of the Chinese characters in the string and the character boxes of the original Chinese character class and Hanzi class in the "Common Chinese Character Feature Table" and "New Chinese Character Feature Table". Based on the similarity, determine the classification result of the Chinese characters in the string. The fifth step is to output all string classification results for the second license plate area corresponding to the license plate area.

[0008] In the first step, the Hanzi category includes all other Chinese characters except those used in existing license plates. The Chinese characters used in existing license plates include provincial abbreviations and special-purpose characters.

[0009] Wherein, N represents the dimension of the feature vector.

[0010] The existing Chinese character boxes refer to the character boxes used on existing license plates, including: province abbreviation characters and special purpose characters. The newly configured Chinese character boxes refer to the character boxes for manually configured Chinese characters.

[0011] The fixed width and fixed height are the width and height of the input character box required by the feature extraction network, respectively.

[0012] Furthermore, the third step includes: The steps for obtaining the license plate area and four corner points are as follows: input an image, use a license plate localization network to detect the image, and obtain the license plate area and the four corner points of the license plate outline; License plate area correction steps: Define the width and height of the rectangle frame to be transformed for the license plate, and obtain the four corner points of the rectangle frame. Input the four corner points of the actually detected license plate and the four corner points of the rectangle frame into the perspective transformation matrix calculation formula to obtain the perspective transformation relationship. Apply this perspective transformation relationship to correct the tilted license plate into a horizontal license plate, denoted as the second license plate area. Character frame acquisition steps for Hanzi class: Input the second license plate area into the improved license plate character detection and classification network to obtain the categories and outer frames of all characters in the license plate, and store them in the order of coordinate size.

[0013] Among them, the definition of the width and height of the rectangle frame to be transformed for the license plate means defining the width and height of the license plate area as fixed width and fixed height.

[0014] Furthermore, the fourth step includes: License plate rule determination step: Set the license plate rule as [special character][letter][digit][digit][digit][digit][digit]. If the string in the license plate area is [Hanzi class][letter][digit][digit][digit][digit][digit], it conforms to the license plate rule, and then transfer to the similarity calculation step; otherwise, directly jump to the fifth step. Similarity calculation step: Find the position of the corresponding Chinese character [special character] in the "new Chinese character feature table" to obtain the corresponding feature vector A; Extract the character frame at the corresponding position of [Hanzi class] in the second license plate area, scale the character frame of [Hanzi class] to a fixed width and height, input it into the feature extraction network for feature extraction to obtain the N-dimensional feature vector B; Calculate the similarity fscoreB_A between the feature vector B and the feature vector A; Calculate the similarity fscoreB_Xi between the feature vector B and all feature vectors in the "common Chinese character feature table" and the "new Chinese character feature table"; When fscoreB_A > all other fscoreB_Xi and fscoreB_A > Thr_min, it is considered that the classification result of this [Hanzi class] is [special character].

[0015] The [special character] refers to other Chinese characters except those used in the existing license plates. The Thr_min represents the similarity threshold, and the value range of Thr_min is [0.5, 0.6].

[0016] According to another aspect of the present invention, there is provided a license plate recognition device with configurable license plate rules, and the device includes: Improvement module for license plate character detection and classification network, which is used to add a category of Hanzi class to the character categories of the license plate character detection and classification network to obtain an improved license plate character detection and classification network. The module for obtaining feature tables for commonly used Chinese characters and new Chinese characters is used to scale the character boxes of existing Chinese characters to a fixed width and height, input them into the feature extraction network, extract features, obtain N-dimensional feature vectors, record the corresponding Chinese character + feature vector, and store them in the "feature table of commonly used Chinese characters"; and to scale the character boxes of newly configured Chinese characters to a fixed width and height, input them into the feature extraction network, extract features, obtain N-dimensional feature vectors, record the corresponding Chinese character + feature vector, and store them in the "feature table of new Chinese characters". The license plate string acquisition module is used as the input image. It employs a license plate localization network to detect the license plate region and the four corner points of the license plate outline from the image. The tilted license plate is corrected into a horizontal rectangle with a fixed width and height to obtain the second license plate region. The second license plate region is then input into an improved license plate character detection and classification network to obtain the category and outline of all characters within the license plate, and stored in order of coordinate size. The Chinese character classification and determination module is used to determine whether a string conforms to the license plate rules. If it does not conform to the license plate rules, it will directly jump to step five. If it conforms to the license plate rules, it will calculate the similarity between the feature vector of the Chinese characters in the string and the character boxes of the original Chinese character class and Hanzi class in the "Common Chinese Character Feature Table" and "New Chinese Character Feature Table". The classification result of the Chinese characters in the string will be determined based on the similarity. The string classification result output module is used to output all string classification results for the second license plate area corresponding to the license plate area.

[0017] The Hanzi class in the license plate character detection and classification network improvement module includes all other Chinese characters except those used in existing license plates. The Chinese characters used in existing license plates include province abbreviations and special-purpose characters.

[0018] Wherein, N represents the dimension of the feature vector.

[0019] The existing Chinese character boxes refer to the character boxes used on existing license plates, including: province abbreviation characters and special purpose characters. The newly configured Chinese character boxes refer to the character boxes for manually configured Chinese characters.

[0020] The fixed width and fixed height are the width and height of the input character box required by the feature extraction network, respectively.

[0021] Furthermore, the license plate string acquisition module includes: The license plate area and four corner points acquisition module is used to input the image, and the license plate localization network is used to detect the image to acquire the license plate area and the four corner points of the license plate outline; The license plate area correction module is used to define the width and height of the rectangle frame to be transformed for the license plate, obtain the four corner points of the rectangle frame, input the four corner points of the actually detected license plate and the four corner points of the rectangle frame into the perspective transformation matrix calculation formula, so as to obtain the perspective transformation relationship, and apply this perspective transformation relationship to correct the inclined license plate into a horizontal license plate, denoted as the second license plate area; The character frame acquisition module of the Hanzi class is used to input the second license plate area into the improved license plate character detection and classification network, obtain the categories and outer frames of all characters in the license plate, and store them in the order of coordinate size.

[0022] Among them, the definition of the width and height of the rectangle frame to be transformed for the license plate means defining the width and height of the license plate area as fixed width and fixed height.

[0023] Furthermore, the Chinese character classification determination module includes: The license plate rule determination module is used to set the license plate rule as [special][letter][digit][digit][digit][digit][digit]. If the string in the license plate area is [Hanzi class][letter][digit][digit][digit][digit][digit], it conforms to the license plate rule, and then transfers to the similarity calculation module; otherwise, it directly jumps to the string classification result output module; The similarity calculation module is used to find the position of the corresponding Chinese character [special] in the "new Chinese character feature table" and obtain the corresponding feature vector A; extract the character frame at the corresponding position of [Hanzi class] in the second license plate area, scale the character frame of [Hanzi class] to a fixed width and height, input it into the feature extraction network for feature extraction, and obtain the N-dimensional feature vector B; calculate the similarity fscoreB_A between the feature vector B and the feature vector A; calculate the similarity fscoreB_Xi between the feature vector B and all feature vectors in the "common Chinese character feature table" and the "new Chinese character feature table"; when fscoreB_A> all other fscoreB_Xi and fscoreB_A> Thr_min, it is considered that the classification result of this [Hanzi class] is [special].

[0024] The [special] refers to other Chinese characters except those used in the existing license plates. The Thr_min represents the similarity threshold, and the value range of Thr_min is [0.5, 0.6].

[0025] Compared with the existing vehicle status detection technology, a license plate recognition method with configurable license plate rules according to the present invention realizes the detection of Chinese characters by adding a category to the original character detection and classification, can ensure the accuracy of the original license plate character detection and recognition, effectively distinguish the original characters and other Chinese characters, and can realize the general detection of Chinese characters. Brief Description of the Drawings

[0026] Figure 1 A flowchart of a license plate recognition method with configurable license plate rules according to the present invention is shown.

[0027] Figure 2 A frame diagram of a license plate recognition device with configurable license plate rules according to the present invention is shown. Detailed Implementation

[0028] To enable those skilled in the art to further understand the structure, features and other objectives of the present invention, the following detailed description is provided in conjunction with the accompanying preferred embodiments. The preferred embodiments described are only for illustrating the technical solutions of the present invention and are not intended to limit the present invention.

[0029] Figure 1 A flowchart of a license plate recognition method with configurable license plate rules according to the present invention is provided. Figure 1 As shown, a license plate recognition method with configurable license plate rules according to the present invention includes: The first step, S1, is to add a Hanzi class to the character categories of the license plate character detection and classification network to obtain an improved license plate character detection and classification network. In the second step S2, the character boxes of the original Chinese characters are scaled to a fixed width and height, input into the feature extraction network, and N-dimensional feature vectors are obtained. The corresponding Chinese character + feature vector is recorded and stored in the "Common Chinese Character Feature Table". The character boxes of the newly configured Chinese characters are scaled to a fixed width and height, input into the feature extraction network, and N-dimensional feature vectors are obtained. The corresponding Chinese character + feature vector is recorded and stored in the "New Chinese Character Feature Table". In the third step S3, the input image is used, and the license plate localization network is used to detect the license plate area and the four corner points of the license plate outline from the image; the tilted license plate is corrected into a horizontal rectangle with a fixed width and height to obtain the second license plate area; the second license plate area is input into the improved license plate character detection and classification network to obtain the category and outline of all characters in the license plate, and stored in order of coordinate size. Step S4: Determine if the string conforms to the license plate rules. If it does not conform to the license plate rules, proceed directly to step S5. If it conforms to the license plate rules, calculate the similarity between the feature vectors of the Chinese characters in the string and the character boxes of the original Chinese character class and Hanzi class in the "Common Chinese Character Feature Table" and "New Chinese Character Feature Table". Based on the similarity, determine the classification result of the Chinese characters in the string. Step S5: Output all string classification results for the second license plate area corresponding to the license plate area.

[0030] Among them, in the first step S1, the categories of Hanzi characters include all other Chinese characters except those used in existing license plates. The Chinese characters used in existing license plates include province abbreviations and special-purpose Chinese characters. Add a category of Hanzi characters to the character categories of the license plate character detection and classification network to detect and locate the character frames of the newly added Chinese characters. The license plate character detection and classification network is implemented using an existing license plate character classification network.

[0031] Among them, in the second step S2, the feature extraction network can be implemented using an existing feature extraction network. Exemplarily, the feature extraction network uses a ReID network. The N represents the dimension of the feature vector.

[0032] The character frames of the original Chinese characters refer to the character frames of the Chinese characters used in existing license plates, including: province abbreviation Chinese characters and special-purpose Chinese characters. The character frames of the newly configured Chinese characters refer to the character frames of the artificially configured Chinese characters. Exemplarily, if new Chinese characters [A], [B], [C], [D] are added to the license plate later, the character frames of these 4 characters [A], [B], [C], [D] are used as the character frames of the newly configured Chinese characters.

[0033] The fixed width and fixed height are respectively the width and height required by the feature extraction network for the input character frame. Exemplarily, if the feature extraction network requires the width of the input character frame image to be W and the height to be H, then the fixed width and fixed height are W and H respectively.

[0034] Furthermore, the third step S3 includes: License plate area and four corner points acquisition step S31, input an image, use a license plate location network to detect the image, and obtain the license plate area and the four corner points of the license plate outer frame; License plate area correction step S32, define the width and height of the rectangle frame to be transformed for the license plate, and obtain the four corner points of the rectangle frame. Input the four corner points of the actually detected license plate and the four corner points of the rectangle frame into the perspective transformation matrix calculation formula, so as to obtain the perspective transformation relationship. Apply this perspective transformation relationship to correct the tilted license plate into a horizontal license plate, denoted as the second license plate area; Hanzi character frame acquisition step S33, input the second license plate area into the improved license plate character detection and classification network, obtain the categories and outer frames of all characters in the license plate, and store them in the order of coordinate size.

[0035] Among them, the definition of the width and height of the rectangle frame to be transformed for the license plate means defining the width and height of the license plate area as the fixed width and fixed height.

[0036] Furthermore, the fourth step S4 includes: License plate rule determination step S41, set the license plate rule as [Special Character][Letter][Digit][Digit][Digit][Digit][Digit]. If the string in the license plate area is [Chinese Character Category][Letter][Digit][Digit][Digit][Digit][Digit], it conforms to the license plate rule, then transfer to the similarity calculation step S42, otherwise directly jump to the fifth step S5; Similarity calculation step S42, find the position of the corresponding Chinese character [Special Character] in the "New Chinese Character Feature Table" and obtain the corresponding feature vector A; extract the character box at the corresponding position of [Chinese Character Category] in the second license plate area, scale the character box of [Chinese Character Category] to a fixed width and height, input it into the feature extraction network for feature extraction, and obtain the N-dimensional feature vector B; calculate the similarity fscoreB_A between the feature vector B and the feature vector A; calculate the similarity fscoreB_Xi between the feature vector B and all feature vectors in the "Common Chinese Character Feature Table" and the "New Chinese Character Feature Table"; when fscoreB_A > all other fscoreB_Xi and fscoreB_A > Thr_min, it is considered that the classification result of this [Chinese Character Category] is [Special Character].

[0037] The [Special Character] refers to other Chinese characters except those used in existing license plates. The Thr_min represents the similarity threshold, and the value range of Thr_min is [0.5, 0.6].

[0038] The similarity calculation adopts one of the following methods: cosine similarity, Euclidean distance, Manhattan distance, dot product similarity, cosine distance.

[0039] Figure 2 The framework diagram of a license plate recognition device with a configurable license plate rule according to the present invention is given. As Figure 2 shown, a license plate recognition device with a configurable license plate rule according to the present invention includes: License plate character detection and classification network improvement module 1, used to add a category of [Chinese Character Category] to the character categories of the license plate character detection and classification network to obtain an improved license plate character detection and classification network; Common Chinese character and new Chinese character feature table acquisition module 2, used to scale the character boxes of the original Chinese character category to a fixed width and height, input them into the feature extraction network for feature extraction, obtain N-dimensional feature vectors, record the corresponding Chinese characters + feature vectors, and store them in the "Common Chinese Character Feature Table"; scale the character boxes of the newly configured Chinese characters to a fixed width and height, input them into the feature extraction network for feature extraction, obtain N-dimensional feature vectors, record the corresponding Chinese characters + feature vectors, and store them in the "New Chinese Character Feature Table"; The license plate string acquisition module 3 is used to input the image. It uses a license plate localization network to detect the license plate region and the four corner points of the license plate outline from the image. The tilted license plate is corrected into a horizontal rectangle with a fixed width and height to obtain the second license plate region. The second license plate region is input into the improved license plate character detection and classification network to obtain the category and outline of all characters in the license plate, and stored in order of coordinate size. The Chinese character classification and determination module 4 is used to determine whether the string conforms to the license plate rules. If it does not conform to the license plate rules, it will directly jump to the string classification result output module 5. If it conforms to the license plate rules, it will calculate the similarity between the feature vector of the Chinese characters in the string and the character boxes of the original Chinese character class and Hanzi class in the "Common Chinese Character Feature Table" and "New Chinese Character Feature Table". The classification result of the Chinese characters in the string will be determined based on the similarity. The string classification result output module 5 is used to output all string classification results for the second license plate area corresponding to the license plate area.

[0040] The Hanzi class in the license plate character detection and classification network improvement module 1 includes all other Chinese characters except those used in existing license plates. The Chinese characters used in existing license plates include provincial abbreviations and special-purpose characters.

[0041] The feature extraction network in the commonly used Chinese characters and new Chinese characters feature table acquisition module 2 can be implemented using an existing feature extraction network. N represents the dimension of the feature vector.

[0042] The existing Chinese character boxes refer to the character boxes used on existing license plates, including: province abbreviation characters and special purpose characters. The newly configured Chinese character boxes refer to the character boxes for manually configured Chinese characters.

[0043] The fixed width and fixed height are the width and height of the input character box required by the feature extraction network, respectively. For example, if the feature extraction network requires the input character box image to have a width of W and a height of H, then the fixed width and fixed height are W and H, respectively.

[0044] Furthermore, the license plate string acquisition module 3 includes: The license plate area and four corner points acquisition module 31 is used to input the image, use a license plate localization network to detect the image, and acquire the license plate area and the four corner points of the license plate outline; The license plate area correction module 32 is used to define the width and height of the rectangular frame to be converted of the license plate, and obtain the four corner points of the rectangular frame. The four corner points of the license plate and the four corner points of the rectangular frame are actually detected and input into the perspective transformation matrix calculation formula to obtain the perspective transformation relationship. The perspective transformation relationship is applied to correct the tilted license plate to a horizontal license plate, which is recorded as the second license plate area. The Chinese character bounding box acquisition module 33 of the Hanzi type is used to input the second license plate area into the improved license plate character detection and classification network, obtain the categories and outer bounding boxes of all characters in the license plate, and store them in the order of coordinate sizes.

[0045] Wherein, defining the width and height of the rectangular box to be converted of the license plate means defining the width and height of the license plate area as a fixed width and a fixed height.

[0046] Furthermore, the Chinese character classification determination module 4 includes: The license plate rule determination module 41 is used to set the license plate rule as [special][letter][digit][digit][digit][digit][digit]. If the string in the license plate area is [Hanzi type][letter][digit][digit][digit][digit][digit], it conforms to the license plate rule, and then it transfers to the similarity calculation module 42; otherwise, it directly jumps to the string classification result output module 5. The similarity calculation module 42 is used to find the position of the corresponding Chinese character [special] in the "new Chinese character feature table" and obtain the corresponding feature vector A; extract the character bounding box at the corresponding position of [Hanzi type] in the second license plate area, scale the character bounding box of [Hanzi type] to a fixed width and height, input it into the feature extraction network for feature extraction, and obtain the N-dimensional feature vector B; calculate the similarity fscoreB_A between the feature vector B and the feature vector A; calculate the similarities fscoreB_Xi between the feature vector B and all feature vectors in the "common Chinese character feature table" and the "new Chinese character feature table"; when fscoreB_A > all other fscoreB_Xi and fscoreB_A > Thr_min, it is considered that the classification result of this [Hanzi type] is [special].

[0047] The [special] refers to other Chinese characters except those used in the existing license plates. The Thr_min represents the similarity threshold, and the value range of Thr_min is [0.5, 0.6].

[0048] The similarity calculation adopts one of the following methods: cosine similarity, Euclidean distance, Manhattan distance, dot product similarity, cosine distance.

[0049] Compared with existing license plate recognition technologies, the advantages of the configurable license plate rule-based license plate recognition method of this invention are as follows: It separates the detection and classification of Chinese characters, transforming the highly complex classification problem into a feature extraction and comparison problem by leveraging the specific characteristics of the license plate recognition task and rules, thus significantly reducing network complexity and runtime. By adding a category to the existing character detection and classification process, it achieves Chinese character detection, ensuring the accuracy of the original license plate character detection and recognition while effectively distinguishing existing characters from other Chinese characters, and enabling general detection of Chinese characters. The Chinese characters in the newly added rules are extracted and saved once during configuration by the feature extraction network and can be used for subsequent Chinese character comparisons without needing to train a classification network specifically for those characters. By comparing feature similarity among a few specific Chinese characters and combining this with the rule compliance of other characters, the accuracy of Chinese character recognition can be guaranteed, which is much less risky than performing single classification on thousands of Chinese characters.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It should be understood that the present invention is not limited to the implementations described herein. These implementations are described to help those skilled in the art practice the present invention. Any person skilled in the art can easily make further improvements and modifications without departing from the spirit and scope of the present invention. Therefore, the present invention is only limited by the content and scope of the claims, and is intended to cover all alternatives and equivalents included within the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A license plate recognition method with configurable license plate rules, characterized in that, The method includes: The first step is to add a Hanzi class to the character categories of the license plate character detection and classification network to obtain an improved license plate character detection and classification network. The second step involves scaling the character frames of existing Chinese characters to a fixed width and height, inputting them into the feature extraction network for feature extraction, obtaining N-dimensional feature vectors, recording the corresponding Chinese character + feature vector, and storing them in the "Common Chinese Character Feature Table"; and scaling the character frames of newly configured Chinese characters to a fixed width and height, inputting them into the feature extraction network for feature extraction, obtaining N-dimensional feature vectors, recording the corresponding Chinese character + feature vector, and storing them in the "New Chinese Character Feature Table". The third step involves inputting an image and using a license plate localization network to detect the license plate region and the four corner points of the license plate outline in the image. The tilted license plate is then corrected into a horizontal rectangle with a fixed width and height to obtain the second license plate region. The second license plate region is then input into an improved license plate character detection and classification network to obtain the category and outline of all characters within the license plate, and stored in order of coordinate size. The fourth step is to determine whether the string conforms to the license plate rules. If it does not conform to the license plate rules, proceed directly to the fifth step. If it conforms to the license plate rules, calculate the similarity between the feature vectors of the Chinese characters in the string and the character boxes of the original Chinese character class and Hanzi class in the "Common Chinese Character Feature Table" and "New Chinese Character Feature Table". Based on the similarity, determine the classification result of the Chinese characters in the string. The fifth step is to output all string classification results for the second license plate area corresponding to the license plate area; Wherein, the Hanzi class includes all other Chinese characters except those used in existing license plates, and the Chinese characters used in existing license plates include province abbreviations and special-purpose characters; N represents the dimension of the feature vector; the character frame of the original Chinese character class refers to the character frame of the Chinese characters used in existing license plates, including province abbreviations and special-purpose characters; the character frame of the newly configured Chinese characters refers to the character frame of manually configured Chinese characters; the fixed width and fixed height are the width and height of the input character frame required by the feature extraction network, respectively.

2. The method as described in claim 1, characterized in that, The third step includes: The steps for obtaining the license plate area and four corner points are as follows: input an image, use a license plate localization network to detect the image, and obtain the license plate area and the four corner points of the license plate outline; The license plate area correction steps are as follows: Define the width and height of the rectangular frame to be converted of the license plate, and obtain the four corner points of the rectangular frame. Input the four corner points of the license plate and the four corner points of the rectangular frame that were actually detected into the perspective transformation matrix calculation formula to obtain the perspective transformation relationship. Apply the perspective transformation relationship to correct the tilted license plate to a horizontal license plate, which is recorded as the second license plate area. The Hanzi class character frame acquisition step involves inputting the second license plate area into the improved license plate character detection and classification network to obtain the category and outer border of all characters in the license plate, and storing them in order of coordinate size. The definition of the width and height of the license plate to be converted rectangle refers to defining the width and height of the license plate area as a fixed width and a fixed height.

3. The method as described in claim 1, characterized in that, The fourth step includes: License plate rule determination steps. Set the license plate rule as [Special Chinese character][Letter][Digit][Digit][Digit][Digit][Digit]. If the string in the license plate area is [Chinese character type][Letter][Digit][Digit][Digit][Digit][Digit], it conforms to the license plate rule, then transfer to the similarity calculation step; otherwise, directly jump to the fifth step. Similarity calculation step. Find the position of the corresponding Chinese character [Special Chinese character] in the "New Chinese character feature table", and obtain the corresponding feature vector A; extract the character box at the corresponding position of [Chinese character type] in the second license plate area, scale the character box of [Chinese character type] to a fixed width and height, input it into the feature extraction network for feature extraction, and obtain the N-dimensional feature vector B; calculate the similarity fscoreB_A between the feature vector B and the feature vector A; calculate the similarity fscoreB_Xi between the feature vector B and all the feature vectors in the "Common Chinese character feature table" and the "New Chinese character feature table"; when fscoreB_A > all other fscoreB_Xi and fscoreB_A > Thr_min, it is considered that the classification result of this [Chinese character type] is [Special Chinese character]. Among them, the [Special Chinese character] refers to other Chinese characters except those used in the existing license plates; the Thr_min represents the similarity threshold.

4. The method as described in claim 3, characterized in that, The value range of the Thr_min is [0.5, 0.6]; The similarity calculation adopts one of the following methods: cosine similarity, Euclidean distance, Manhattan distance, dot product similarity, cosine distance.

5. A license plate recognition device with configurable license plate rules, characterized in that, The device includes: License plate character detection and classification network improvement module, which is used to add a category of [Chinese character type] to the character categories of the license plate character detection and classification network to obtain an improved license plate character detection and classification network; Common Chinese character and new Chinese character feature table acquisition module, which is used to scale the character boxes of the original Chinese character type to a fixed width and height, input them into the feature extraction network for feature extraction, obtain the N-dimensional feature vectors, record the corresponding Chinese characters + feature vectors, and store them in the "Common Chinese character feature table"; scale the character boxes of the newly configured Chinese characters to a fixed width and height, input them into the feature extraction network for feature extraction, obtain the N-dimensional feature vectors, record the corresponding Chinese characters + feature vectors, and store them in the "New Chinese character feature table"; License plate string acquisition module, which is used to input an image, use the license plate positioning network to detect the license plate area and the 4 corner points of the license plate outer frame from the image; correct the tilted license plate into a horizontal rectangular frame with a fixed width and height to obtain the second license plate area; input the second license plate area into the improved license plate character detection and classification network to obtain the categories and outer frames of all characters in the license plate, and store them in the order of coordinate size; The Chinese character classification and determination module is used to determine whether a string conforms to the license plate rules. If it does not conform to the license plate rules, it directly jumps to step five. If it conforms to the license plate rules, it calculates the similarity between the feature vector of the Chinese characters in the string and the character boxes of the original Chinese character class and Hanzi class in the "Common Chinese Character Feature Table" and "New Chinese Character Feature Table". The classification result of the Chinese characters in the string is determined based on the similarity. The string classification result output module is used to output all string classification results for the second license plate area corresponding to the license plate area; Wherein, the Hanzi class includes all other Chinese characters except those used in existing license plates, and the Chinese characters used in existing license plates include province abbreviations and special-purpose characters; N represents the dimension of the feature vector; the character frame of the original Chinese character class refers to the character frame of the Chinese characters used in existing license plates, including province abbreviations and special-purpose characters; the character frame of the newly configured Chinese characters refers to the character frame of manually configured Chinese characters; the fixed width and fixed height are the width and height of the input character frame required by the feature extraction network, respectively.

6. The apparatus as claimed in claim 5, characterized in that, The license plate string acquisition module includes: The license plate area and four corner points acquisition module is used to input the image, and the license plate localization network is used to detect the image and acquire the license plate area and the four corner points of the license plate outline; The license plate area correction module is used to define the width and height of the rectangular frame to be converted of the license plate, and obtain the four corner points of the rectangular frame. The four corner points of the license plate and the four corner points of the rectangular frame are input into the perspective transformation matrix calculation formula to obtain the perspective transformation relationship. The perspective transformation relationship is applied to correct the tilted license plate to a horizontal license plate, which is recorded as the second license plate area. The Hanzi-class character frame acquisition module is used to input the second license plate area into the improved license plate character detection and classification network, obtain the category and outer border of all characters in the license plate, and store them in order of coordinate size. The definition of the width and height of the license plate to be converted rectangle refers to defining the width and height of the license plate area as a fixed width and a fixed height.

7. The apparatus as claimed in claim 5, characterized in that, The Chinese character classification and determination module includes: The license plate rule determination module is used to set the license plate rule as [Special][Letter][Number][Number][Number][Number][Number]. If the string of the license plate area is [Hanzi type][Letter][Number][Number][Number][Number][Number], it conforms to the license plate rule and then proceeds to the similarity calculation module; otherwise, it directly jumps to the string classification result output module. A similarity calculation module is used to find the position of the corresponding Chinese character [Te] in the "New Chinese Character Feature Table" and obtain the corresponding feature vector A; extract the character box at the corresponding position of [Hanzi class] in the second license plate area, scale the character box of [Hanzi class] to a fixed width and height, input it into the feature extraction network for feature extraction, and obtain the N-dimensional feature vector B; calculate the similarity fscoreB_A between the feature vector B and the feature vector A; calculate the similarity fscoreB_Xi between the feature vector B and all feature vectors in the "Common Chinese Character Feature Table" and the "New Chinese Character Feature Table"; when fscoreB_A > all other fscoreB_Xi and fscoreB_A > Thr_min, it is considered that the classification result of this [Hanzi class] is [Te]; where, the [Te] refers to Chinese characters other than those used in existing license plates; the Thr_min represents the similarity threshold.

8. The apparatus as claimed in claim 7, characterized in that, The value range of the Thr_min is [0.5, 0.6]; The following method is adopted for the similarity calculation: cosine similarity, Euclidean distance, Manhattan distance, dot product similarity, cosine distance.

Images