A license plate recognition method, system and computer readable storage medium

Abstract

The application discloses a license plate recognition method, system and computer readable storage medium, and the license plate recognition method comprises the following steps: S1, extracting vehicle picture features to obtain a license plate feature map; S2, generating a license plate position mask according to the license plate feature map; S3, preprocessing the license plate position mask to obtain a first license plate picture; S4, reconstructing the first license plate picture to obtain a second license plate picture; and S5, recognizing the second license plate picture to output a license plate number. According to the application, the license plate feature map is obtained by extracting the vehicle picture features, the license plate position mask is further generated, the first license plate picture is obtained by preprocessing the license plate position mask, and the second license plate picture is obtained by reconstructing the first license plate picture, so that the influence of the angle deviation generally existing in the collected license plate image is reduced, the definition of the license plate picture is improved, and on this basis, the second license plate picture is finally recognized to output the license plate number, so that the accuracy and robustness of the license plate recognition are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation, and more particularly to license plate recognition methods, systems, and computer-readable storage media. Background Technology

[0002] License plate recognition technology is a core supporting technology for intelligent transportation, security monitoring, and other scenarios, and its recognition performance directly affects the operational reliability of related systems. In actual monitoring scenarios, due to factors such as camera installation angle and equipment performance, the acquired license plate images generally suffer from two major problems: angle deviation and insufficient clarity, which seriously restrict the accuracy and robustness of license plate recognition.

[0003] Currently, mainstream license plate recognition technologies fall into two categories, both of which have significant drawbacks. Traditional computer vision solutions rely on edge detection, which has extremely poor adaptability to angles, and is prone to matching failures when the license plate is tilted or deformed. Clarity optimization relies solely on simple filtering, which cannot effectively repair low-resolution and blurry images. Feature extraction is limited, relying on manually designed shallow features such as edges and colors, making it difficult to capture deep semantic information, and is easily affected by background interference, resulting in poor overall stability.

[0004] General-purpose deep learning solutions employ generic object detection networks without being customized for license plate scenarios. This results in insufficient capture of license plate features under angular deviations, making it difficult to balance localization accuracy and speed. Furthermore, network pooling layers reduce feature map size, exacerbating coordinate mapping errors as the angle increases, further reducing the accuracy of license plate recognition. Existing technologies fail to effectively address these pain points and are ill-suited to the practical needs of complex scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address at least one defect of the related technologies mentioned in the background: due to factors such as camera installation angle and equipment performance, the acquired license plate images generally suffer from angular deviation and insufficient clarity, which seriously restricts the accuracy and robustness of license plate recognition. The present invention provides a license plate recognition method, system and computer-readable storage medium.

[0006] The technical solution adopted by this invention to solve its technical problem is: to construct a license plate recognition method, including the following steps: S1: Extract vehicle image features to obtain license plate feature map; S2: Generate a license plate location mask based on the license plate feature map; S3: Preprocess the license plate location mask to obtain the first license plate image; S4: Reconstruct the first license plate image to obtain the second license plate image; S5: Recognize the second license plate image and output the license plate number.

[0007] In some embodiments, S1 includes: S11: Low-level visual features of vehicle images are extracted by the first layer of the convolutional neural network; S12: After fusing low-level visual features and vehicle images and performing pooling, the resulting data is input into a second convolutional neural network to extract high-level semantic features of the vehicle images; and... S13: Fuse the pooled low-level visual features, pooled vehicle images, and high-level semantic features to obtain a license plate feature map.

[0008] In some embodiments, S2 includes: S21: A preliminary license plate location mask is generated by a shallow sequence network based on the license plate feature map; S22: Input the initial license plate location mask into the first layer transposed convolutional neural network to restore the low-level visual features. Then, concatenate the initial license plate location mask with the low-level visual features and upsample to obtain the initial mask feature map containing the low-level visual features. S23: The initial mask feature map containing low-level visual features is input into the second layer transposed convolutional neural network to recover high-level semantic features. Then, the initial mask feature map containing low-level visual features is concatenated with the high-level semantic features to obtain an accurate mask feature map containing both low-level visual features and high-level semantic features; and... S24: Input the precise mask feature map containing low-level visual features and high-level semantic features into the terminal sequence network for localization inference to generate a precise license plate location mask.

[0009] In some embodiments, S3 includes: extracting the license plate location coordinate information of the license plate location mask, cropping the license plate location mask according to the license plate location coordinate information, and standardizing the cropped license plate location mask to obtain a first license plate image.

[0010] In some embodiments, S4 includes: constructing a license plate training set, training an SRGAN using the license plate training set, and reconstructing a second license plate image from a first license plate image using the SRGAN; wherein the license plate training set includes license plate training images with different angles, lighting conditions, and blur levels. Further, a character edge loss function is added to the SRGAN.

[0011] In some embodiments, S5 includes: using the Tesseract-OCR engine to recognize the second license plate image and output the license plate number.

[0012] In some embodiments, S5 is followed by: S6: Determine whether the output license plate number is valid according to the fixed rules of the license plate; if yes, output the license plate number directly; if no, correct the license plate number before outputting it.

[0013] The present invention also constructs a license plate recognition system, comprising: The extraction module is used to extract features from vehicle images to obtain license plate feature maps; The positioning module is used to generate a license plate location mask based on the license plate feature map; The preprocessing module is used to preprocess the license plate location mask to obtain the first license plate image; The reconstruction module is used to reconstruct the first license plate image to obtain the second license plate image. The recognition module is used to recognize the second license plate image and output the license plate number.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the license plate recognition method as described in any of the above embodiments.

[0015] By implementing this invention, the following beneficial effects are achieved: This invention extracts vehicle image features to obtain license plate feature maps, further generates a license plate location mask, and obtains a first license plate image by preprocessing the license plate location mask and reconstructing the first license plate image to obtain a second license plate image. This reduces the impact of angle deviations commonly found in the collected license plate images and improves the clarity of the license plate images. Based on this, the second license plate image is finally identified and the license plate number is output, thus improving the accuracy and robustness of license plate recognition. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 A flowchart of an embodiment of the license plate recognition method of the present invention is shown; Figure 2 The diagram shows the logical structure of the recognition module and the positioning module in one embodiment of the license plate recognition system of the present invention; Figure 3 The diagram shows the overall logical structure of an embodiment of the license plate recognition system of the present invention. Detailed Implementation

[0017] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] It should be noted that the flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0019] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0020] It should be noted that "at least two" refers to at least two, which can be two, three, or any number. "At least one" can be one, two, or any number.

[0021] like Figure 1 As shown, some embodiments of the present invention disclose a license plate recognition method, including the following steps: S1: Extract vehicle image features to obtain license plate feature map; S2: Generate a license plate location mask based on the license plate feature map; S3: Preprocess the license plate location mask to obtain the first license plate image; S4: Reconstruct the first license plate image to obtain the second license plate image; S5: Recognize the second license plate image and output the license plate number.

[0022] It should be noted that the sources of vehicle images containing license plates include, but are not limited to: real-time vehicle images captured by road surveillance cameras, checkpoint cameras, and electronic police equipment; vehicle entry / exit images captured by capture devices at parking lot entrances / exits, residential area entrances / exits, and factory entrances / exits; vehicle images captured by in-vehicle cameras and dashcams of vehicles in front or around the vehicle; frame images from vehicle video streams captured by security monitoring systems and traffic management systems during normal inspections and recordings; sample images from pre-built vehicle and license plate image datasets; and vehicle images acquired by other image acquisition devices that are legally authorized and comply with relevant laws and regulations on privacy protection and data security.

[0023] A license plate location mask is a feature mask map with the same size as the input vehicle image, used to accurately distinguish between license plate areas and non-license plate areas in an image.

[0024] By implementing this embodiment, the following beneficial effects are achieved: The present invention obtains a license plate feature map by extracting vehicle image features, further generates a license plate location mask, and obtains a first license plate image by preprocessing the license plate location mask and reconstructing the first license plate image to obtain a second license plate image. This reduces the influence of the angle deviation that is common in the collected license plate images and improves the clarity of the license plate image. On this basis, the license plate number is finally output by recognizing the second license plate image, thereby improving the accuracy and robustness of license plate recognition.

[0025] In some embodiments, S1 includes: S11: extracting low-level visual features of the vehicle image from a first-layer convolutional neural network; S12: fusing the low-level visual features and the vehicle image, then pooling the resulting mixture, and inputting it into a second-layer convolutional neural network to extract high-level semantic features of the vehicle image; and S13: fusing the pooled low-level visual features, the pooled vehicle image, and the high-level semantic features to obtain a license plate feature map. Further, each convolutional neural network layer includes at least one convolutional neural network.

[0026] In some embodiments, such as Figure 2 As shown, the first convolutional neural network includes a first convolutional neural network N1 and a second convolutional neural network N2, and the second convolutional neural network includes a third convolutional neural network N3 and a fourth convolutional neural network N4. S1 includes: S11: extracting low-level visual features of the vehicle image from the first convolutional neural network N1 and the second convolutional neural network N2; S12: fusing the low-level visual features and the vehicle image through the blue addition layer, then inputting them into the second pooling layer MP2 for pooling, and then inputting them into the third convolutional neural network N3 and the fourth convolutional neural network N4 respectively to extract high-level semantic features of the vehicle image; and S13: fusing the low-level visual features pooled by the second pooling layer MP2, the vehicle image pooled by the first pooling layer MP1, and the high-level semantic features to obtain a license plate feature map.

[0027] Specifically, convolutional neural networks are the foundation of feature extraction. Convolutional neural networks extract and reduce the dimensionality of image features through sliding calculations of convolutional kernels. Combined with max pooling layers with a stride of 4, the input image of 256×256×3 is reduced to 64×64×3. The core objectives are to compress the feature map size, extract effective features, and improve the network's operating efficiency.

[0028] The first layer of a convolutional neural network extracts low-level visual features, including edges, contours, and textures (these features are less affected by the shooting angle). These are then combined with high-level semantic features, including shape and character arrangement, extracted by a second layer of convolutional neural network, and fused with the original vehicle image. The fused feature set retains both the basic visual attributes and the core semantic attributes of the license plate, accurately capturing its essential features even with angle deviations and avoiding feature loss due to angle issues. The recognition component focuses on layered feature extraction and fusion, providing ample feature support for localization.

[0029] In some embodiments, S2 includes: S21: generating a preliminary license plate location mask from the license plate feature map using a shallow sequence network; S22: inputting the preliminary license plate location mask into a first layer of transposed convolutional neural network to restore low-level visual features, then concatenating the preliminary license plate location mask with the low-level visual features and upsampling to obtain a preliminary mask feature map containing low-level visual features; S23: inputting the preliminary mask feature map containing low-level visual features into a second layer of transposed convolutional neural network to restore high-level semantic features, then concatenating the preliminary mask feature map containing low-level visual features with the high-level semantic features to obtain a precise mask feature map containing both low-level visual features and high-level semantic features; and S24: inputting the precise mask feature map containing both low-level visual features and high-level semantic features into the terminal sequence network for localization inference to generate a precise license plate location mask. Further, each layer of transposed convolutional neural network includes at least one transposed convolutional neural network.

[0030] In some embodiments, such as Figure 2 As shown, the first layer of transposed convolutional neural network includes a first transposed convolutional neural network N6 and a second transposed convolutional neural network N7, and the second layer of transposed convolutional neural network includes a third transposed convolutional neural network N8 and a fourth transposed convolutional neural network N9. S2 includes: S21: A preliminary license plate location mask is generated by a shallow sequence network N5 based on the license plate feature map; S22: The preliminary license plate location mask is input into a first transposed convolutional neural network N6 and a second transposed convolutional neural network N7 to restore low-level visual features, and then the preliminary license plate location mask and the low-level visual features are concatenated in the first concatenation layer concat1 and upsampled in the upsampling layer US1 to obtain a preliminary mask feature map containing low-level visual features; S23: The preliminary mask feature map containing low-level visual features is input into a third transposed convolutional neural network N8 and a fourth transposed convolutional neural network N9 to restore high-level semantic features, and then the preliminary mask feature map containing low-level visual features and the high-level semantic features are concatenated in the second concatenation layer concat2 to obtain a precise mask feature map containing both low-level visual features and high-level semantic features; and S24: The precise mask feature map containing both low-level visual features and high-level semantic features is input into a terminal sequence network N10 for localization inference to generate a precise license plate location mask.

[0031] Specifically, the core function of transposed convolutional neural networks is the opposite of the "dimensionality reduction extraction" of convolutional neural networks. Transposed convolution achieves the dimensionality restoration of feature maps through reverse convolutional calculation logic. The core goal is to restore the compressed small-sized feature maps to the same size as the original input, while restoring the detailed features such as license plate edges and contours lost during convolutional pooling, providing complete feature support for accurate localization.

[0032] After concatenation, the feature maps are merged in terms of the number of channels. The width and height of the final output feature map remain unchanged, but the number of channels is the sum of the number of channels of the input feature maps.

[0033] The upsampling layer accurately upscales the small feature map after transpose convolution and splicing to a size of 256×256×3, consistent with the original input, ensuring that the final output localization result (license plate location mask) matches the size of the original license plate image without the need for additional coordinate mapping transformation, thus avoiding localization errors caused by size mismatch.

[0034] After initially estimating the license plate position using a shallow sequence network, a transposed convolutional neural network is used, combined with a splicing layer and an upsampling layer, to restore and optimize the feature map. Finally, the image is input into an end sequence network, which performs final pixel-level localization inference based on the feature map. By learning the license plate localization rules through the network, the license plate area in the feature map is accurately determined and labeled to generate an accurate license plate position mask. This mask can adaptively correct for license plate contour deformation and positional offset caused by angle deviation. Even if the license plate is tilted or deformed due to the shooting angle, the actual bounding box of the license plate can be accurately defined, providing an accurate positional basis for subsequent cropping and recognition.

[0035] In some embodiments, S3 includes: extracting the license plate location coordinate information of the license plate location mask, cropping the license plate location mask according to the license plate location coordinate information, and standardizing the cropped license plate location mask to obtain a first license plate image.

[0036] It should be noted that since the output of the terminal sequence network is a license plate location mask of the same size as the original input, the output result needs to be parsed to extract the precise location coordinates of the license plate. The image is then cropped based on the coordinates. Further preprocessing such as size standardization and basic noise and interference removal is required to obtain the first license plate image, laying the groundwork for subsequent image reconstruction.

[0037] In some embodiments, S4 includes: constructing a license plate training set, training an SRGAN using the license plate training set, and reconstructing a second license plate image from a first license plate image using the SRGAN; wherein the license plate training set includes license plate training images with different angles, lighting conditions, and blur levels.

[0038] It's important to note that Super-Resolution Generative Adversarial Networks (SRGANs), through their generative adversarial network architecture, restore low-resolution images to high-resolution ones, thereby improving the recognition accuracy of small, blurry characters. This is crucial for improving license plate image quality and achieving accurate license plate character recognition. The input to this step is the previously preprocessed first license plate image, used to improve recognition accuracy. Furthermore, introducing SRGAN before using the Tesseract-OCR engine significantly improves the resolution and clarity of the detected license plate image. The image enhanced by this model helps the subsequent Tesseract-OCR engine more easily and accurately recognize the alphanumeric characters on the license plate.

[0039] In some embodiments, license plate images are categorized by angle, including 0° (front view), 5°-30° (small tilt), and 30°-60° (large tilt), accounting for 40%, 40%, and 20% respectively, covering common license plate tilt angles in surveillance scenarios; by lighting conditions, including license plate images under normal lighting, low light (evening, cloudy), strong light (midday backlight), backlight, and shadow occlusion, each accounting for 20%, adapting to different surveillance imaging environments at different times; and by blur level, including clear, slightly blurred (motion blur, slight noise), moderately blurred, and severely blurred license plate images, each accounting for 25%, simulating insufficient clarity issues caused by differences in surveillance equipment performance and shooting distance. It should be noted that the specific data listed above (angle, composition ratio, etc.) are for illustrative purposes and can be adjusted according to more practical needs. The training set can be sourced from publicly available traffic scene license plate datasets (such as the CCPD license plate dataset and the CALTECH vehicle dataset), where all images have been anonymized to eliminate privacy risks. Alternatively, it can be artificially synthesized data. Based on real license plate samples, images with different angles, blur levels, and lighting conditions are artificially synthesized using image editing techniques. This supplements the data from real-world collections with samples of scarce scenarios (such as large-angle tilt, extreme low light, and severe blur), ensuring the comprehensiveness of the training set. During the training phase, SRGAN can incorporate a large number of license plate samples from different angles, allowing the model to learn the patterns of angle changes and develop generalization capabilities for angle deviations. This makes the reconstructed license plate photos more suitable for subsequent recognition.

[0040] Furthermore, a character edge loss function is incorporated into SRGAN. This function includes at least one of L1 edge loss, L2 edge loss, Sobel gradient edge loss, Canny edge loss, and Dice edge loss. This allows SRGAN to focus on restoring the edge contours and stroke details of license plate characters during super-resolution enhancement, avoiding issues such as character blurring and stroke merging, thus further improving the foundation for subsequent OCR recognition.

[0041] In some embodiments, S5 includes: using the Tesseract-OCR engine to recognize the second license plate image and output the license plate number. Further, S5 is followed by: S6: determining whether the output license plate number is valid according to the fixed rules for license plates; if valid, directly output the license plate number; if not, correct the license plate number before outputting it.

[0042] It should be noted that the Tesseract-OCR engine is an open-source optical character recognition engine that supports multilingual text extraction. It is commonly used for character recognition tasks in scenarios such as license plates and tickets. It can recognize more than 100 languages ​​and supports multiple output formats such as plain text, Hypertext Markup Language (HTML), and Portable Document Format (PDF).

[0043] Specifically, the fixed rules for license plates are that they have a fixed number of characters and a fixed arrangement (e.g., license plates for small cars are "Chinese characters + letters + 5 letters / numbers"). Most existing solutions directly output the OCR recognition results without rule verification, which can easily lead to invalid results due to single-character recognition errors. This invention adds license plate rule verification and correction after license plate recognition, performs a legality judgment on the recognition results (e.g., whether the number of characters is correct and whether the character combination is reasonable), performs adaptive correction for easily confused characters, and removes irrelevant background interference characters to improve the effectiveness of the recognition results.

[0044] like Figure 3 As shown, some embodiments of the present invention also disclose a license plate recognition system, including: The extraction module is used to extract features from vehicle images to obtain license plate feature maps; The positioning module is used to generate a license plate location mask based on the license plate feature map; The preprocessing module is used to preprocess the license plate location mask to obtain the first license plate image; The reconstruction module is used to reconstruct the first license plate image to obtain the second license plate image. The recognition module is used to recognize the second license plate image and output the license plate number.

[0045] It should be noted that the sources of vehicle images containing license plates include, but are not limited to: real-time vehicle images captured by road surveillance cameras, checkpoint cameras, and electronic police equipment; vehicle entry / exit images captured by capture devices at parking lot entrances / exits, residential area entrances / exits, and factory entrances / exits; vehicle images captured by in-vehicle cameras and dashcams of vehicles in front or around the vehicle; frame images from vehicle video streams captured by security monitoring systems and traffic management systems during normal inspections and recordings; sample images from pre-built vehicle and license plate image datasets; and vehicle images acquired by other image acquisition devices that are legally authorized and comply with relevant laws and regulations on privacy protection and data security.

[0046] A license plate location mask is a feature mask map with the same size as the input vehicle image, used to accurately distinguish between license plate areas and non-license plate areas in an image.

[0047] By implementing this embodiment, the following beneficial effects are achieved: The present invention obtains a license plate feature map by extracting vehicle image features, further generates a license plate location mask, and obtains a first license plate image by preprocessing the license plate location mask and reconstructing the first license plate image to obtain a second license plate image. This reduces the influence of the angle deviation that is common in the collected license plate images and improves the clarity of the license plate image. On this basis, the license plate number is finally output by recognizing the second license plate image, thereby improving the accuracy and robustness of license plate recognition.

[0048] In some embodiments, the extraction module includes: a first convolutional neural network layer for extracting low-level visual features; a second convolutional neural network layer for extracting high-level visual features; an additive layer for fusing images and features; and a pooling layer for compressing feature map size and extracting effective features.

[0049] Extracting vehicle image features to obtain a license plate feature map includes: extracting low-level visual features of the vehicle image using a first-layer convolutional neural network; fusing the low-level visual features and the vehicle image, then pooling the resulting mixture, and inputting it into a second-layer convolutional neural network to extract high-level semantic features of the vehicle image; and fusing the pooled low-level visual features, the pooled vehicle image, and the high-level semantic features to obtain the license plate feature map. Furthermore, each convolutional neural network layer includes at least one convolutional neural network.

[0050] In some embodiments, such as Figure 2As shown, the first convolutional neural network includes a first convolutional neural network N1 and a second convolutional neural network N2, and the second convolutional neural network includes a third convolutional neural network N3 and a fourth convolutional neural network N4. Extracting vehicle image features to obtain a license plate feature map includes: extracting low-level visual features of the vehicle image using the first convolutional neural network N1 and the second convolutional neural network N2; fusing the low-level visual features and the vehicle image through a blue additive layer, then inputting them into a second pooling layer MP2 for pooling, and then inputting them into the third convolutional neural network N3 and the fourth convolutional neural network N4 respectively to extract high-level semantic features of the vehicle image; and fusing the low-level visual features pooled by the second pooling layer MP2, the vehicle image pooled by the first pooling layer MP1, and the high-level semantic features to obtain the license plate feature map.

[0051] Specifically, convolutional neural networks are the foundation of feature extraction. Convolutional neural networks extract and reduce the dimensionality of image features through sliding calculations of convolutional kernels. Combined with max pooling layers with a stride of 4, the input image of 256×256×3 is reduced to 64×64×3. The core objectives are to compress the feature map size, extract effective features, and improve the network's operating efficiency.

[0052] The first layer of a convolutional neural network extracts low-level visual features, including edges, contours, and textures (these features are less affected by the shooting angle). These are then combined with high-level semantic features, including shape and character arrangement, extracted by a second layer of convolutional neural network, and fused with the original vehicle image. The fused feature set retains both the basic visual attributes and the core semantic attributes of the license plate, accurately capturing its essential features even with angle deviations and avoiding feature loss due to angle issues. The recognition component focuses on layered feature extraction and fusion, providing ample feature support for localization.

[0053] In some embodiments, the localization module includes: a shallow sequence network for generating a preliminary license plate location mask based on the license plate feature map; a first transposed convolutional neural network for restoring low-level visual features from the preliminary license plate location mask; a second transposed convolutional neural network for restoring high-level semantic features from the preliminary mask feature map containing low-level visual features; a concatenation layer for concatenating the license plate location mask and features in terms of channel dimension; and an upsampling layer for restoring the feature map size to be consistent with the license plate photo.

[0054] Generating a license plate location mask based on a license plate feature map includes: generating a preliminary license plate location mask from the license plate feature map using a shallow sequence network; inputting the preliminary license plate location mask into a first-layer transposed convolutional neural network to reconstruct low-level visual features, then concatenating the preliminary license plate location mask with the low-level visual features and upsampling to obtain a preliminary mask feature map containing low-level visual features; inputting the preliminary mask feature map containing low-level visual features into a second-layer transposed convolutional neural network to reconstruct high-level semantic features, then concatenating the preliminary mask feature map containing low-level visual features with the high-level semantic features to obtain a precise mask feature map containing both low-level visual features and high-level semantic features; and inputting the precise mask feature map containing both low-level visual features and high-level semantic features into a terminal sequence network for localization inference to generate a precise license plate location mask. Further, each layer of the transposed convolutional neural network includes at least one transposed convolutional neural network.

[0055] In some embodiments, such as Figure 2 As shown, the first layer of transposed convolutional neural network includes a first transposed convolutional neural network N6 and a second transposed convolutional neural network N7, and the second layer of transposed convolutional neural network includes a third transposed convolutional neural network N8 and a fourth transposed convolutional neural network N9. Generating a license plate location mask based on a license plate feature map includes: generating a preliminary license plate location mask from the license plate feature map using a shallow sequence network N5; inputting the preliminary license plate location mask into a first transposed convolutional neural network N6 and a second transposed convolutional neural network N7 to restore low-level visual features; then concatenating the preliminary license plate location mask and the low-level visual features in a first concatenation layer concat1 and upsampling in an upsampling layer US1 to obtain a preliminary mask feature map containing low-level visual features; inputting the preliminary mask feature map containing low-level visual features into a third transposed convolutional neural network N8 and a fourth transposed convolutional neural network N9 to restore high-level semantic features; then concatenating the preliminary mask feature map containing low-level visual features and the high-level semantic features in a second concatenation layer concat2 to obtain a precise mask feature map containing both low-level visual features and high-level semantic features; and finally inputting the precise mask feature map containing both low-level visual features and high-level semantic features into a terminal sequence network N10 for localization inference to generate a precise license plate location mask.

[0056] Specifically, the core function of transposed convolutional neural networks is the opposite of the "dimensionality reduction extraction" of convolutional neural networks. Transposed convolution achieves the dimensionality restoration of feature maps through reverse convolutional calculation logic. The core goal is to restore the compressed small-sized feature maps to the same size as the original input, while restoring the detailed features such as license plate edges and contours lost during convolutional pooling, providing complete feature support for accurate localization.

[0057] After concatenation, the feature maps are merged in terms of the number of channels. The width and height of the final output feature map remain unchanged, but the number of channels is the sum of the number of channels of the input feature maps.

[0058] The upsampling layer accurately upscales the small feature map after transpose convolution and splicing to a size of 256×256×3, consistent with the original input, ensuring that the final output localization result (license plate location mask) matches the size of the original license plate image without the need for additional coordinate mapping transformation, thus avoiding localization errors caused by size mismatch.

[0059] After initially estimating the license plate position using a shallow sequence network, a transposed convolutional neural network is used, combined with a splicing layer and an upsampling layer, to restore and optimize the feature map. Finally, the image is input into an end sequence network, which performs final pixel-level localization inference based on the feature map. By learning the license plate localization rules through the network, the license plate area in the feature map is accurately determined and labeled to generate an accurate license plate position mask. This mask can adaptively correct for license plate contour deformation and positional offset caused by angle deviation. Even if the license plate is tilted or deformed due to the shooting angle, the actual bounding box of the license plate can be accurately defined, providing an accurate positional basis for subsequent cropping and recognition.

[0060] In some embodiments, the license plate location mask is preprocessed to obtain a first license plate image, including: extracting the license plate location coordinate information of the license plate location mask, cropping the license plate location mask according to the license plate location coordinate information, and standardizing the cropped license plate location mask to obtain a first license plate image.

[0061] It should be noted that since the output of the terminal sequence network is a license plate location mask of the same size as the original input, the output result needs to be parsed to extract the precise location coordinates of the license plate. The image is then cropped based on the coordinates. Further preprocessing such as size standardization and basic noise and interference removal is required to obtain the first license plate image, laying the groundwork for subsequent image reconstruction.

[0062] In some embodiments, reconstructing the first license plate image to obtain the second license plate image includes: constructing a license plate training set, training an SRGAN using the license plate training set, and reconstructing the first license plate image using the SRGAN to obtain the second license plate image; wherein, the license plate training set includes license plate training images with different angles, lighting, and blur levels.

[0063] It's important to note that Super-Resolution Generative Adversarial Networks (SRGANs), through their generative adversarial network architecture, restore low-resolution images to high-resolution ones, thereby improving the recognition accuracy of small, blurry characters. This is crucial for improving license plate image quality and achieving accurate license plate character recognition. The input to this step is the previously preprocessed first license plate image, used to improve recognition accuracy. Furthermore, introducing SRGAN before using the Tesseract-OCR engine significantly improves the resolution and clarity of the detected license plate image. The image enhanced by this model helps the subsequent Tesseract-OCR engine more easily and accurately recognize the alphanumeric characters on the license plate.

[0064] In some embodiments, license plate images are categorized by angle, including 0° (front view), 5°-30° (small tilt), and 30°-60° (large tilt), accounting for 40%, 40%, and 20% respectively, covering common license plate tilt angles in surveillance scenarios; by lighting conditions, including license plate images under normal lighting, low light (evening, cloudy), strong light (midday backlight), backlight, and shadow occlusion, each accounting for 20%, adapting to different surveillance imaging environments at different times; and by blur level, including clear, slightly blurred (motion blur, slight noise), moderately blurred, and severely blurred license plate images, each accounting for 25%, simulating insufficient clarity issues caused by differences in surveillance equipment performance and shooting distance. It should be noted that the specific data listed above (angle, composition ratio, etc.) are for illustrative purposes and can be adjusted according to more practical needs. The training set can be sourced from publicly available traffic scene license plate datasets (such as the CCPD license plate dataset and the CALTECH vehicle dataset), where all images have been anonymized to eliminate privacy risks. Alternatively, it can be artificially synthesized data. Based on real license plate samples, images with different angles, blur levels, and lighting conditions are artificially synthesized using image editing techniques. This supplements the data from real-world collections with samples of scarce scenarios (such as large-angle tilt, extreme low light, and severe blur), ensuring the comprehensiveness of the training set. During the training phase, SRGAN can incorporate a large number of license plate samples from different angles, allowing the model to learn the patterns of angle changes and develop generalization capabilities for angle deviations. This makes the reconstructed license plate photos more suitable for subsequent recognition.

[0065] Furthermore, a character edge loss function is incorporated into SRGAN. This function includes at least one of L1 edge loss, L2 edge loss, Sobel gradient edge loss, Canny edge loss, and Dice edge loss. This allows SRGAN to focus on restoring the edge contours and stroke details of license plate characters during super-resolution enhancement, avoiding issues such as character blurring and stroke merging, thus further improving the foundation for subsequent OCR recognition.

[0066] In some embodiments, recognizing the second license plate image and outputting the license plate number includes: using the Tesseract-OCR engine to recognize the second license plate image and output the license plate number. Further, the system also includes: a judgment and correction module, used to determine whether the output license plate number is valid according to the fixed rules of license plates after recognizing the second license plate image and outputting the license plate number; if so, the license plate number is directly output; if not, the license plate number is corrected before outputting.

[0067] It should be noted that the Tesseract-OCR engine is an open-source optical character recognition engine that supports multilingual text extraction. It is commonly used for character recognition tasks in scenarios such as license plates and tickets. It can recognize more than 100 languages ​​and supports multiple output formats such as plain text, Hypertext Markup Language (HTML), and Portable Document Format (PDF).

[0068] Specifically, the fixed rules for license plates are that they have a fixed number of characters and a fixed arrangement (e.g., license plates for small cars are "Chinese characters + letters + 5 letters / numbers"). Most existing solutions directly output the OCR recognition results without rule verification, which can easily lead to invalid results due to single-character recognition errors. This invention adds license plate rule verification and correction after license plate recognition, performs a legality judgment on the recognition results (e.g., whether the number of characters is correct and whether the character combination is reasonable), performs adaptive correction for easily confused characters, and removes irrelevant background interference characters to improve the effectiveness of the recognition results.

[0069] Some embodiments of the present invention also disclose a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the license plate recognition method as described in any of the above embodiments.

[0070] By implementing this invention, the following beneficial effects are achieved: 1. Specifically addresses the angle deviation problem, resulting in greater adaptability. Customized for license plate recognition, it captures angle-independent features through multi-feature fusion (fusion of original image, low-level visual features, and high-level semantic features). Combined with a precise location mask achieved through transposed convolution, and preprocessing operations, it can adaptively locate and correct bounding boxes for tilted or deformed license plates. Its generalization ability for different monitoring angles is far superior to existing solutions.

[0071] 2. Layered optimization addresses the clarity issue, resulting in higher recognition accuracy. A combined strategy of SRGAN and Tesseract-OCR is employed. SRGAN, a generative adversarial network specifically designed for super-resolution, can more accurately recover character details and edge features of low-resolution license plates compared to traditional super-resolution algorithms and basic CNN super-resolution. Furthermore, this invention specifically trains SRGAN on license plate characters, laying a high-quality image foundation for OCR recognition and solving the problem of image enhancement and recognition disconnect in existing solutions.

[0072] 3. Coordinated design for localization and recognition leads to higher overall efficiency. During the process of generating the license plate location mask, the input and output image sizes are kept consistent, reducing information loss and computational load in intermediate steps. Simultaneously, during vehicle image feature extraction, the pooling layer effectively reduces the feature map size, improving network inference speed and balancing localization accuracy and operational efficiency.

[0073] 4. Modular design enhances feasibility and optimizability. The system is divided into several major modules: extraction, localization, preprocessing, reconstruction, and recognition. Each module is further subdivided, and the recognition module uses the open-source Tesseract-OCR engine, eliminating the need to develop a character recognition module from scratch and reducing the cost and difficulty of engineering deployment. Furthermore, each module can be optimized independently (e.g., tuning the network parameters of the localization module, and customizing the training of SRGAN for license plate characters), facilitating adaptation to specific monitoring scenarios.

[0074] It is understood that the above embodiments only illustrate some implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above embodiments or technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. That is, the embodiments described "in some embodiments" can be freely combined with any of the preceding and following embodiments. Therefore, all equivalent transformations and modifications made within the scope of the claims of the present invention should be covered by the claims of the present invention.

Claims

1. A license plate recognition method, characterized in that, Includes the following steps: S1: Extract vehicle image features to obtain license plate feature map; S2: Generate a license plate location mask based on the license plate feature map; S3: Preprocess the license plate location mask to obtain the first license plate image; S4: Reconstruct the first license plate image to obtain the second license plate image; S5: Recognize the second license plate image and output the license plate number.

2. The license plate recognition method according to claim 1, characterized in that, S1 includes: S11: Extract low-level visual features of the vehicle image using the first layer of the convolutional neural network; S12: After fusing the low-level visual features and the vehicle image and then pooling them, the mixture is input into the second convolutional neural network to extract the high-level semantic features of the vehicle image. as well as, S13: The pooled low-level visual features, the pooled vehicle image, and the high-level semantic features are fused to obtain the license plate feature map.

3. The license plate recognition method according to claim 2, characterized in that, S2 include: S21: A preliminary license plate location mask is generated by a shallow sequence network based on the license plate feature map; S22: Input the preliminary license plate location mask into the first layer transposed convolutional neural network to restore the low-level visual features, and then concatenate the preliminary license plate location mask with the low-level visual features and upsample to obtain a preliminary mask feature map containing the low-level visual features. S23: Input the preliminary mask feature map containing low-level visual features into the second layer transposed convolutional neural network to reconstruct the high-level semantic features; then concatenate the preliminary mask feature map containing low-level visual features with the high-level semantic features to obtain an accurate mask feature map containing both low-level visual features and high-level semantic features; and, S24: Input the precise mask feature map containing low-level visual features and high-level semantic features into the terminal sequence network for localization inference to generate the precise license plate location mask.

4. The license plate recognition method according to claim 1, characterized in that, S3 includes: Extract the license plate location coordinate information from the license plate location mask, crop the license plate location mask according to the license plate location coordinate information, and standardize the cropped license plate location mask to obtain the first license plate image.

5. The license plate recognition method according to claim 1, characterized in that, S4 includes: A license plate training set is constructed, and an SRGAN is trained using the license plate training set. The second license plate image is obtained by reconstructing the first license plate image through the SRGAN. The license plate training set includes license plate training images with different angles, lighting conditions, and degrees of blur.

6. The license plate recognition method according to claim 5, characterized in that, A character edge loss function is added to the SRGAN.

7. The license plate recognition method according to claim 1, characterized in that, S5 includes: The Tesseract-OCR engine is used to recognize the second license plate image and output the license plate number.

8. The license plate recognition method according to claim 1, characterized in that, S5 also includes: S6: Based on the fixed rules for license plates, determine whether the output license plate number is valid; If so, directly output the license plate number; If not, correct the license plate number before outputting it.

9. A license plate recognition system, characterized in that, include: The extraction module is used to extract features from vehicle images to obtain license plate feature maps; The positioning module is used to generate a license plate location mask based on the license plate feature map; The preprocessing module is used to preprocess the license plate location mask to obtain the first license plate image; The reconstruction module is used to reconstruct the first license plate image to obtain the second license plate image. The recognition module is used to recognize the second license plate image and output the license plate number.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the license plate recognition method as described in any one of claims 1-8.

Images