A method, computing device and storage medium for large-angle license plate image recognition

Abstract

The present invention discloses a method for recognizing large-angle license plate images, comprising: obtaining a training data set and a test data set, wherein the training data set comprises a first number of simulated large-angle motion-blurred license plate images with license plate numbers annotated, and the test data set comprises a second number of real large-angle motion-blurred license plate images; inputting the training data set into a pre-built adaptive license plate recognition network for end-to-end training to obtain a trained adaptive license plate recognition network, and inputting the test data set into the trained adaptive license plate recognition network for model evaluation and optimization to obtain an optimized adaptive license plate recognition network, wherein the adaptive license plate recognition network comprises a feature extraction module, a spatial transformation parameter prediction module, a scale-aware feature adaptation module, a feature adaptive integration module, an upsampling module, and a character recognition module; and inputting a license plate image to be recognized into the optimized adaptive license plate recognition network for license plate recognition to obtain a license plate recognition result.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a large-angle license plate image recognition method, a computing device, and a storage medium. Background Art

[0002] With the development of intelligent transportation technology, license plate recognition (LPR) has become a mature technology used in parking lots, toll booths, and other scenarios, facilitating both travel and vehicle monitoring. Deep learning-based LPR methods have significantly improved both accuracy and efficiency. However, in complex scenarios such as high-angle tilt and blur, LPR accuracy still needs to be improved. Traditional LPR correction methods, such as linear fitting, horizontal projection, and random transformation, can typically only address license plate tilts within 30 degrees.

[0003] The existing wide-angle license plate recognition method establishes a license plate multi-target detection model and a wide-angle license plate recognition model. Vehicle driving video data is framed and preprocessed before being input into the license plate multi-target detection model for license plate detection and target tracking. The wide-angle license plate images are then angle-corrected. The positive-angle license plate images are then input into the wide-angle license plate recognition model for recognition, yielding a license plate recognition result for each license plate target. While this solution can address license plate recognition within a certain angle range, it relies on accurate license plate positioning. The use of multiple model stacking requires manual parameter adjustment, and cumulative errors can degrade model performance. Furthermore, existing vehicle recognition algorithms may fail to recognize or erroneously recognize wide-angle license plate images due to motion blur or low resolution.

[0004] Therefore, in order to improve the accuracy of low-resolution motion-blurred large-angle license plate image recognition, a large-angle license plate image recognition method is needed to solve the problems existing in the above-mentioned prior art. Summary of the Invention

[0005] In view of the above problems, this scheme proposes a large-angle license plate image recognition method, which can reduce the impact of motion blur and low resolution on large-angle license plate image recognition and improve the accuracy of multi-pose large-angle license plate image recognition.

[0006] According to a first aspect of the present invention, a method for wide-angle license plate image recognition is provided, comprising: obtaining a training data set and a test data set, wherein the training data set comprises a first number of simulated wide-angle motion-blurred license plate images with license plate numbers annotated, and the test data set comprises a second number of real wide-angle motion-blurred license plate images; then, inputting the training data set into a pre-built adaptive license plate recognition network for end-to-end training to obtain a trained adaptive license plate recognition network, wherein the adaptive license plate recognition network comprises a feature extraction module, a spatial transformation parameter prediction module, a scale-aware feature adaptation module, a feature adaptive integration module, an upsampling module, and a character recognition module; then, inputting the test data set into the trained adaptive license plate recognition network for model evaluation and optimization to obtain an optimized adaptive license plate recognition network; finally, inputting the license plate image to be recognized into the optimized adaptive license plate recognition network for license plate recognition to obtain a license plate recognition result.

[0007] Through the above scheme, a pre-built adaptive license plate recognition network is adopted, and adaptive correction parameters are combined with deblurring processing and upsampling modules to simultaneously achieve super-resolution and license plate correction and de-motion blur, obtaining a corrected super-resolution deblurred image; through end-to-end training, manual parameter adjustment is reduced, and the degradation of model performance caused by the superposition of multiple models is avoided, which can improve the accuracy and adaptability of large-angle license plate image recognition.

[0008] Optionally, in the step of obtaining a training data set and a test data set, on the one hand, a second number of real large-angle motion blurred images can be collected by an image acquisition device; on the other hand, a first number of true license plate images of different types and colors can be generated according to the license plate number rules, and the resolution of the true license plate image is a first resolution, which is higher than 720p, that is, the true license plate image is a clear image with high resolution; based on the random affine transformation method, the true license plate image is simulated with a large angle offset to obtain a large-angle license plate image; the large-angle license plate image is interpolated, randomly translated, multi-frame superimposed, and random noise processed to obtain a simulated large-angle motion blurred license plate image, and the resolution of the simulated large-angle motion blurred license plate image is a second resolution, which is between 60p and 200p, that is, the simulated large-angle motion blurred license plate image is a low-resolution blurred image.

[0009] Optionally, in the step of inputting the training data set into a pre-built adaptive license plate recognition network for end-to-end training to obtain a trained adaptive license plate recognition network, the training data set can be input into a spatial transformation parameter prediction module to obtain predicted affine transformation matrix parameters, and a first scaling factor is determined based on the target image size and the input image size; the affine transformation matrix parameters and the first scaling factor are used as scale parameters of the scale-aware feature adaptation module to obtain an affine transformation image and a scale transformation image; and a second scaling factor is calculated based on the affine transformation image size and the scale transformation image size.

[0010] The affine transformation matrix parameters, the first scaling factor, and the second scaling factor are input into a feature adaptive integration module to obtain a deblurred image; the deblurred image is input into an upsampling module for correction, upsampling, and spatial filtering to obtain a corrected super-resolution deblurred image; the corrected super-resolution deblurred image is input into a character recognition module for character recognition to obtain a predicted license plate number.

[0011] A first loss function is calculated based on the deblurred image and the input image, a second loss function is calculated based on the rectified super-resolution deblurred image and the input image, and a third loss function is calculated based on the predicted license plate number and the labeled license plate number; the adaptive license plate recognition network is end-to-end iteratively trained based on the first loss function, the second loss function and the third loss function. When the number of iterations reaches a preset number of iterations or the loss value of the loss function is less than a preset threshold, the training is completed to obtain a trained adaptive license plate recognition network.

[0012] Optionally, in the above method, assume that the target image size is (W, H), the input image size is (w, h), and the first scaling factor is (rh, rv), where rh = W / w, rv = H / h.

[0013] Optionally, in the step of performing correction, upsampling, and spatial filtering in the deblurred image input upsampling module to obtain a corrected super-resolution deblurred image, the corrected coordinates (x_new, y_new) are calculated based on the affine transformation matrix to obtain the corrected image:

[0014]

[0015] x_new=a11*x+a12*y+a13

[0016] y_new=a21*x+a22*y+a23

[0017] Among them, (x, y) is the coordinate before correction, (x_new, y_new) is the coordinate after correction, [a11, a12, a13; a21, a22, a23] is the affine transformation matrix;

[0018] The super-resolution corrected coordinates (Hx, Hy) are calculated based on the corrected coordinates and the first scaling factor to obtain the super-resolution corrected image:

[0019] Hx=(x_new+0.5) / rh+0.5

[0020] Hy = (y_new + 0.5) / rv + 0.5, where rh and rv are the first scaling factors;

[0021] The super-resolution rectified image is spatially filtered to obtain a rectified super-resolution deblurred image.

[0022] Optionally, in the above method, the character recognition module includes a convolutional layer, a recurrent layer and a transcription layer, the convolutional layer is used to extract features from the rectified super-resolution deblurred image to obtain a feature map, the recurrent layer is used to predict the feature sequence in the feature map and output the predicted label distribution, and the transcription layer uses CTC loss to convert the predicted label distribution obtained by the recurrent layer into a final license plate number sequence.

[0023] Optionally, in the above method, the upsampling parameters are determined based on the size of the license plate image to be identified and the target image size, and the license plate image to be identified is preprocessed based on the upsampling parameters to obtain a license plate image of the target size; the license plate image of the target size is input into the optimized adaptive license plate recognition network for license plate recognition to obtain a license plate recognition result.

[0024] In a second aspect, the present application provides a computing device that adopts the following technical solution:

[0025] A computing device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the large-angle license plate image recognition method as described in the first aspect.

[0026] In a third aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0027] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the wide-angle license plate image recognition method as described in the first aspect.

[0028] According to the solution of the present invention, by constructing an adaptive license plate recognition network, it is possible to use adaptive correction parameters to deblur, correct, and upsample the license plate image, directly obtaining a corrected super-resolution deblurred image, which significantly improves the multi-pose and large-angle recognition effect of the license plate. Through end-to-end model training and optimization, the performance degradation problem caused by the superposition of multiple models is avoided. This solution fully considers various large-angle offsets such as rotation, tilt, and horizontal angle, and can accurately identify large-angle license plate images with a horizontal angle of less than 70°, a tilt angle of less than 55°, and a rotation angle of ±90° (or all three angles exist at the same time).

[0029] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0031] Figure 1 A schematic flow chart of a method 100 for recognizing a large-angle license plate image according to an embodiment of the present invention is shown;

[0032] Figure 2 A schematic diagram of the structure of an adaptive license plate recognition network according to an embodiment of the present invention is shown;

[0033] Figure 3 FIG. 3 shows a structural diagram of a computing device 300 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0035] In scenarios such as parking lots and toll booths, it is necessary to obtain license plate images for license plate recognition. However, the license plate images captured by image acquisition devices often have a certain angle offset.

[0036] The license plate angle includes the horizontal angle, tilt angle, and rotation angle. Assume that the plane of the license plate is perpendicular to the ground. In the top view of the vehicle, draw a perpendicular line to the line connecting the camera lens and the center point of the license plate. The angle between this perpendicular line and the straight line where the license plate is located is defined as the horizontal angle, and the value range is 0-90°. When the license plate is facing the camera, the horizontal angle is 0°. Assume that the horizontal angle of the license plate is 0°. In the side view, draw a perpendicular line to the line connecting the camera lens and the center point of the license plate. The angle between this perpendicular line and the straight line where the license plate is located is defined as the tilt angle, and the value range is 0-90°. Assume that the horizontal angle and tilt angle of the license plate are both 0°. In the main view, the angle between the ray where the lower edge of the license plate is located and the ray where the ground is located is the rotation angle, and the value range is 0-360°.

[0037] The accuracy of current license plate recognition algorithms is often affected by excessive license plate angles, and they primarily correct for the license plate's horizontal rotation without considering the effects of tilt and rotation. If motion blur is present in large-angle license plate images, the characters processed by existing license plate correction methods are prone to being unrecognizable or incorrectly recognized.

[0038] In order to improve the accuracy of large-angle motion-blurred license plate image recognition, this scheme provides a large-angle license plate image recognition method that fully considers the influence of multiple angles such as rotation angle, tilt angle and horizontal angle, and uses license plate adaptive correction parameters to deblur the image and correct the license plate, significantly improving the recognition effect of license plates in multiple postures and large angles.

[0039] Figure 1 FIG. 1 is a flow chart of a method 100 for recognizing a large-angle license plate image according to an embodiment of the present invention. Figure 1 As shown, the method starts at step S010, obtaining a training data set and a test data set, the training data set includes a first number of simulated large-angle motion blurred license plate images with labeled license plate numbers, and the test data set includes a second number of real large-angle motion blurred license plate images.

[0040] In one embodiment of the present invention, on the one hand, a second number of real large-angle motion blurred images are collected by an image acquisition device: 5,000 real large-angle license plate motion blurred images in actual scenes are collected by an image acquisition device as a test data set, such as the dragging blur effect caused by the relative motion between the camera lens, vehicle, and background, including linear, rotational, scaling and other blur types.

[0041] On the other hand, a first number of true license plate images of different types and colors are generated according to the license plate number rules, and the resolution of the true license plate images is a first resolution, which is higher than 720p, that is, the true license plate images are clear images with high resolution; based on the random affine transformation method, the large-angle offset of the true license plate images is simulated to obtain large-angle license plate images; the large-angle license plate images are interpolated, randomly translated, multi-frame superimposed, and random noise processed to obtain simulated large-angle motion blurred license plate images, and the resolution of the simulated large-angle motion blurred license plate images is a second resolution, which is between 60p and 200p, that is, the simulated large-angle motion blurred license plate images are low-resolution blurred images.

[0042] Specifically, according to the license plate number rules (for example, the license plate character size is 20*20, including: numbers: 0~9; letters: A~Z; provincial and municipal abbreviations: Beijing, Tianjin, Shanxi, Hebei, Inner Mongolia, Liaoning, Jilin, Heilongjiang, Shanghai, Jiangsu, Zhejiang, Anhui, Fujian, Jiangxi, Shandong, Henan, Hubei, Hunan, Guangdong, Guangxi, Hainan, Sichuan, Guizhou, Yunnan, Tibet, Shaanxi, Gansu, Qinghai, Ningxia, Xinjiang, Chongqing), 200,000 high-resolution (resolution greater than or equal to 720p) license plate images of different types (single or double layer) and different colors (blue, yellow, black, white, green) can be generated, and the license plate numbers are marked in the images as the true value images of the license plates.

[0043] Then, a random affine transformation method is used to simulate large-angle offset of the true license plate image to obtain a large-angle clear license plate image, where the affine transformation includes rotation, translation, shearing, scale change, etc.; then, different interpolations are performed on the large-angle clear license plate image, such as two-dimensional linear interpolation and two-dimensional cubic interpolation, to obtain large-angle low-resolution images with different low resolutions (for example, reduced to a resolution of 60-200).

[0044] Finally, the large-angle low-resolution image is randomly translated and multi-frame superposition is performed to simulate motion blur, and random noise is added, such as Gaussian noise, salt and pepper noise, and gamma noise, to obtain a simulated large-angle motion blurred license plate image, which is used as the training dataset for the adaptive license plate recognition network.

[0045] Then, step S020 is executed to input the training data set into the pre-built adaptive license plate recognition network for end-to-end training to obtain a trained adaptive license plate recognition network, wherein the adaptive license plate recognition network includes multiple feature extraction modules, spatial transformation parameter prediction modules, scale-aware feature adaptation modules, feature adaptive integration modules, upsampling modules and character recognition modules.

[0046] Figure 2 FIG. 1 shows a schematic diagram of the structure of an adaptive license plate recognition network according to an embodiment of the present invention. Figure 2As shown in the figure, the adaptive license plate recognition network includes multiple feature extraction modules, spatial transformation parameter prediction modules, scale-aware feature adaptation modules, feature adaptive integration modules, upsampling modules and character recognition modules.

[0047] Among them, the feature extraction module is used to extract feature maps of different scales of the input image; the spatial transformation parameter prediction module is used to predict the parameters of the affine transformation matrix. For example, the input image is first subjected to several convolution or full connection operations followed by a regression layer to output a transformation matrix θ. The transformation matrix θ can represent a transformation, such as the affine transformation θ∈R 2×3 , projection transformation θ∈R 3×3 Different transformations will produce transformation matrices of different sizes; then the grid generator obtains the correspondence between the pixel coordinates before and after the transformation based on the transformation matrix parameters.

[0048] The training data set can be input into the spatial transformation parameter prediction module to obtain the predicted affine transformation matrix parameters, and the first scaling factor can be determined based on the target image size and the input image size. For example, according to the target size (W, H) of the super-resolution image and the size (w, h) of the input image, the first scaling factor (rh, rv) can be calculated, where rh = W / w and rv = H / h.

[0049] The affine transformation matrix parameters and the first scaling factor are used as scale parameters of the scale-aware feature adaptation module to obtain an affine transformed image and a scale transformed image; a second scaling factor is calculated based on the size of the affine transformed image and the size of the scale transformed image. For example, the second scaling factor (Rx, Ry) is calculated based on the size of the affine transformed image (R, S) and the size of the scale transformed image (r, s), where Rx = R / r and Ry = S / s.

[0050] The affine transformation matrix parameters, the first scaling factor, and the second scaling factor are input into the feature adaptive integration module to obtain a deblurred image. The deblurred image is input into the upsampling module for correction, upsampling, and spatial filtering to obtain a corrected super-resolution deblurred image. The corrected coordinates (x_new, y_new) are calculated based on the affine transformation matrix to obtain the corrected image:

[0051]

[0052] x_new=a11*x+a12*y+a13

[0053] y_new=a21*x+a22*y+a23

[0054] Where (x, y) is the coordinate before correction, (x_new, y_new) is the coordinate after correction, [a11, a12, a13; a21, a22, a23] is the affine transformation matrix; based on the corrected coordinates and the first scaling factor, the super-resolution corrected coordinates (Hx, Hy) are calculated to obtain the super-resolution corrected image:

[0055] Hx=(x_new+0.5) / rh+0.5

[0056] Hy = (y_new + 0.5) / rv + 0.5, where rh and rv are first scaling factors; the super-resolution rectified image is spatially filtered to obtain a rectified super-resolution deblurred image. The rectified super-resolution deblurred image is input into a character recognition module for character recognition to obtain a predicted license plate number.

[0057] Among them, the character recognition module includes a convolutional layer, a recurrent layer and a transcription layer. The convolutional layer is used to extract features from the rectified super-resolution deblurred image to obtain a feature map. The recurrent layer is used to predict the feature sequence in the feature map and output the predicted label distribution. The transcription layer uses CTC loss to convert the predicted label distribution obtained by the recurrent layer into the final license plate number sequence.

[0058] A branch can be added after the feature adaptive integration module. This branch uses a 1x1 convolutional layer with 3 channels and a sigmoid activation function to calculate the MSE loss and frequency domain reconstruction loss based on the deblurred image and the input image. This is used as the first loss function.

[0059] The super-resolution rectified image is filtered with a 1x1 filter and normalized to (W, H) with a channel number of 3. After the rectified super-resolution deblurred image is output, the second loss function is calculated based on the rectified super-resolution deblurred image and the input image.

[0060] The CTC loss function is calculated based on the predicted license plate number and the annotated license plate number as the third loss function. The adaptive license plate recognition network is trained end-to-end iteratively based on the first loss function, the second loss function, and the third loss function. When the number of iterations reaches a preset number of iterations (for example, 80 cycles) or the sum of the loss values ​​of the loss function is less than a preset threshold, the training is completed, and a trained adaptive license plate recognition network is obtained.

[0061] Then, step S030 is executed to input the test data set into the trained adaptive license plate recognition network for model evaluation and optimization to obtain an optimized adaptive license plate recognition network.

[0062] After adjusting the parameters of the adaptive license plate recognition network, the test data set can be used to evaluate the model. The parameters of the trained model can be adjusted and optimized based on evaluation indicators such as model recognition accuracy, recall rate, ROC curve, AUC curve, etc. to obtain the optimized adaptive license plate recognition network.

[0063] Finally, step S040 is executed to input the license plate image to be recognized into the optimized adaptive license plate recognition network for license plate recognition to obtain a license plate recognition result.

[0064] The license plate image to be recognized can be a large-angle license plate image with a horizontal angle / rotation angle / tilt angle greater than 45 degrees. The upsampling parameters can be determined based on the size of the license plate image to be recognized and the target image size. The license plate image to be recognized is preprocessed based on the upsampling parameters to obtain a license plate image of the target size. For example, the image to be recognized is processed into a license plate image with a width and height that are multiples of 4; the license plate image to be recognized of the target size is input into the optimized adaptive license plate recognition network for license plate recognition to obtain a license plate recognition result.

[0065] Based on the same technical concept, an embodiment of the present application discloses a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the wide-angle license plate recognition method as described above. Figure 3 FIG. 3 shows a block diagram of a computing device 300 according to an embodiment of the present invention. Figure 3 As shown, in a basic configuration 102, computing device 300 typically includes memory 106 and one or more processors 104. A memory bus 108 may be used for communication between processor 104 and memory 106.

[0066] Depending on the desired configuration, the processor 104 can be any type of processor, including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processing unit (DSP), or any combination thereof. The processor 104 can include one or more levels of cache, such as a level 1 cache 110 and a level 2 cache 112, a processor core 114, and registers 116. An example processor core 114 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 can be used with the processor 104, or in some implementations, the memory controller 118 can be an internal part of the processor 104.

[0067] Depending on the desired configuration, memory 106 can be any type of memory, including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. Physical memory in a computing device generally refers to volatile RAM. Data from a disk must be loaded into physical memory before it can be read by processor 104. Memory 106 may include an operating system 120, one or more applications 122, and program data 124. In some embodiments, application 122 may be arranged so that one or more processors 104 execute instructions on the operating system using program data 124. Operating system 120 may be, for example, Linux, Windows, etc., and includes program instructions for handling basic system services and performing hardware-dependent tasks. Application 122 includes program instructions for implementing various user-desired functions. Application 122 may be, for example, a browser, instant messaging software, software development tools (such as an integrated development environment (IDE), a compiler, etc.), but is not limited thereto. When application 122 is installed in computing device 300, a driver module may be added to operating system 120.

[0068] When computing device 300 is started, processor 104 reads and executes program instructions from operating system 120 from memory 106. Applications 122 run on top of operating system 120, utilizing interfaces provided by operating system 120 and the underlying hardware to implement various user-desired functions. When a user launches application 122, it is loaded into memory 106, and processor 104 reads and executes the program instructions from memory 106.

[0069] Computing device 300 also includes a storage device 132 including a removable storage 136 and a non-removable storage 138 , both of which are connected to storage interface bus 134 .

[0070] The computing device 300 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output devices 142, peripheral interfaces 144, and communication devices 146) to the basic configuration 102 via the bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. These can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which can be configured to facilitate communication with external devices such as input devices (e.g., a keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., a printer, scanner, etc.) via one or more I / O ports 158. Example communication devices 146 may include a network controller 160, which can be arranged to facilitate communication with one or more other computing devices 162 via a network communication link via one or more communication interfaces 164.

[0071] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. As non-limiting examples, communication media can include wired media such as a wired network or a dedicated network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable media used herein may include both storage media and communication media. In the computing device 300 according to the present invention, the application 122 includes instructions for executing the large-angle license plate recognition method 100 of the present invention.

[0072] An embodiment of the present application discloses a computer-readable storage medium, including a computer program that can be loaded by a processor and executed by the wide-angle license plate recognition method 100 as described above.

[0073] Among them, the computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in combination with an instruction execution system, apparatus or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0074] Through the technical solution of the present invention, by constructing an adaptive license plate recognition network, it is possible to use adaptive correction parameters to deblur, correct, and upsample the license plate image, directly obtaining a corrected super-resolution deblurred image, significantly improving the multi-pose and large-angle recognition effect of the license plate. Through end-to-end model training and optimization, the performance degradation problem caused by the superposition of multiple models is avoided. This solution fully considers various large-angle offsets such as rotation, tilt, and horizontal angle, and can accurately identify large-angle license plate images within a horizontal angle of 70°, a tilt angle of 55°, and a rotation angle of ±90° (or all three angles exist at the same time).

[0075] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0076] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims that follow the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0077] Those skilled in the art will appreciate that the modules, units, or components of the devices in the examples disclosed herein may be arranged in the device described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or further divided into multiple submodules.

[0078] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0079] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0080] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Thus, a processor having the necessary instructions for implementing a method or method element forms a device for implementing the method or method element. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0081] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved and are not intended to imply that the objects so described must have a given order in time, space, ranking, or in any other manner.

[0082] Although the present invention has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art, having benefit of the foregoing description, that other embodiments are contemplated within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been selected primarily for readability and didactic purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention. Consequently, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative and not restrictive of the scope of the invention, which is defined by the appended claims.

Claims

1. A large-angle license plate image recognition method, characterized in that: The method comprises: Acquire a training data set and a test data set, wherein the training data set includes a first number of simulated large-angle motion blurred license plate images with license plate numbers annotated, and the test data set includes a second number of real large-angle motion blurred license plate images; Inputting the training data set into a pre-built adaptive license plate recognition network for end-to-end training to obtain a trained adaptive license plate recognition network, wherein the adaptive license plate recognition network includes multiple feature extraction modules, a spatial transformation parameter prediction module, a scale-aware feature adaptation module, a feature adaptive integration module, an upsampling module, and a character recognition module; The step of inputting the training data set into the pre-built adaptive license plate recognition network for end-to-end training to obtain a trained adaptive license plate recognition network includes: Inputting the training data set into the spatial transformation parameter prediction module to obtain predicted affine transformation matrix parameters, and determining a first scaling factor based on the target image size and the input image size; Using the affine transformation matrix parameters and the first scaling factor as scale parameters of a scale and correction feature adaptation module to obtain an affine transformation image and a scale transformation image; Calculating a second scaling factor based on the affine transformed image size and the scale transformed image size; Inputting the affine transformation matrix parameters, the first scaling factor, and the second scaling factor into a scale and correction adaptive de-motion blur feature integration module to obtain a deblurred image; The deblurred image is input into the upsampling module for correction, upsampling and spatial filtering to obtain a rectified super-resolution deblurred image; The rectified super-resolution deblurred image is input into the character recognition module for character recognition to obtain the predicted license plate number; Calculate a first loss function based on the deblurred image and the input image, calculate a second loss function based on the rectified super-resolution deblurred image and the input image, and calculate a third loss function based on the predicted license plate number and the annotated license plate number; Performing end-to-end iterative training on the adaptive license plate recognition network based on the first loss function, the second loss function, and the third loss function. When the number of iterations reaches a preset number of iterations or the sum of the loss values ​​of the loss function is less than a preset threshold, the training is completed to obtain a trained adaptive license plate recognition network; The test data set is input into the trained adaptive license plate recognition network for model evaluation and optimization to obtain the optimized adaptive license plate recognition network; The license plate image to be recognized is input into the optimized adaptive license plate recognition network for license plate recognition to obtain the license plate recognition result.

2. The large-angle license plate image recognition method according to claim 1, characterized in that: The steps of obtaining a training data set and a test data set include: Acquire a second number of real large-angle motion blurred images by an image acquisition device; Generating a first number of true license plate images of different types and colors according to the license plate number rule, wherein the resolution of the true license plate images is a first resolution; Based on a random affine transformation method, a large-angle offset is simulated for the true license plate image to obtain a large-angle license plate image; The large-angle license plate image is interpolated, randomly translated, multi-frame superimposed, and subjected to random noise processing to obtain a simulated large-angle motion blurred license plate image, wherein the resolution of the simulated large-angle motion blurred license plate image is the second resolution.

3. The large-angle license plate image recognition method according to claim 2, characterized in that: The first resolution is higher than 720p, and the second resolution is between 60p and 200p.

4. The large-angle license plate image recognition method according to claim 1, characterized in that: The step of determining the first scaling factor based on the target image size and the input image size comprises: Assume that the target image size is (W, H), the input image size is (w, h), and the first scaling factor is (rh, rv), where rh=W / w, rv=H / h.

5. The large-angle license plate image recognition method according to claim 4, characterized in that: The step of inputting the deblurred image into the upsampling module for correction, upsampling and spatial filtering to obtain a corrected super-resolution deblurred image comprises: Calculate the corrected coordinates (x_new, y_new) based on the affine transformation matrix to obtain the corrected image: (x_new,y_new)= (x,y) x_new=a11*x+a12*y+a13 y_new=a21*x+a22*y+a23 Among them, (x, y) is the coordinate before correction, (x_new, y_new) is the coordinate after correction, [a11, a12, a13; a21, a22, a23] is the affine transformation matrix; The super-resolution corrected coordinates (Hx, Hy) are calculated based on the corrected coordinates and the first scaling factor to obtain the super-resolution corrected image: Hx = (x_new+0.5) / rh +0.5 Hy=(y_new+0.5) / rv+0.5 Where rh, rv are the first scaling factors; The super-resolution rectified image is spatially filtered to obtain a rectified super-resolution deblurred image.

6. The large-angle license plate image recognition method according to claim 1, characterized in that: The character recognition module includes a convolutional layer, a recurrent layer, and a transcription layer. The convolutional layer is used to extract features from the rectified super-resolution deblurred image to obtain a feature map. The recurrent layer is used to predict the feature sequence in the feature map and output the predicted label distribution. The transcription layer uses CTC loss to convert the predicted label distribution obtained by the recurrent layer into the final license plate number sequence.

7. The large-angle license plate image recognition method according to claim 1, characterized in that: The step of inputting the license plate image to be recognized into the optimized adaptive license plate recognition network for license plate recognition to obtain the license plate recognition result comprises: Determining upsampling parameters based on the size of the license plate image to be recognized and the target image size, and preprocessing the license plate image to be recognized based on the upsampling parameters to obtain a license plate image of the target size; The license plate image of the target size is input into the optimized adaptive license plate recognition network for license plate recognition to obtain the license plate recognition result.

8. A computing device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the large-angle license plate image recognition method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The device comprises a computer program stored therein which can be loaded by a processor and execute the wide-angle license plate image recognition method according to any one of claims 1 to 7.

Images