A license plate recognition method, electronic device and readable storage medium

Abstract

The application discloses a license plate recognition method, an electronic device and a readable storage medium. The method comprises the following steps: generating a license plate feature map containing a character layer with a specific layer number for a license plate image, wherein the specific layer number is greater than or equal to two; performing character recognition corresponding to each layer on the license plate feature map respectively to obtain character recognition results corresponding to each layer in the license plate feature map; and obtaining a license plate recognition result of the license plate image based on the character recognition results corresponding to each layer. According to the above scheme, the license plate recognition is prevented from being affected by the accuracy of character region segmentation, and the accuracy of license plate recognition is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a license plate recognition method, electronic device, and readable storage medium. Background Technology

[0002] Artificial intelligence technology is being applied more and more widely in modern transportation, with license plate recognition technology being a typical example. The main purpose of license plate recognition is to identify the characters on a license plate from an image. Currently, there are various different license plate designs; for example, based on the arrangement of the characters, license plates can be categorized into single-layer plates, double-layer plates, etc.

[0003] Current license plate recognition methods typically involve first segmenting the license plate to obtain the region for each character, and then recognizing the characters within that region. The accuracy of license plate recognition in this method is affected by the accuracy of the character region segmentation. Summary of the Invention

[0004] This application provides at least one license plate recognition method, electronic device, and readable storage medium.

[0005] This application provides a license plate recognition method, which includes: generating a license plate feature map containing a specific number of character layers, wherein the specific number of layers is greater than or equal to two; performing character recognition on the license plate feature map for each corresponding layer to obtain the character recognition result corresponding to each layer in the license plate feature map; and obtaining the license plate recognition result of the license plate image based on the character recognition result corresponding to each layer.

[0006] The process of performing character recognition on each layer of the license plate feature map to obtain the character recognition results for each layer of the license plate feature map includes: extracting local features for each layer of the license plate feature map to obtain the first layer features for each layer of the license plate feature map; and decoding the first layer features for each layer to obtain the character recognition results for each layer.

[0007] Specifically, the process of extracting local features from each layer of the license plate feature map to obtain the first layer features of each layer includes: extracting features from each layer of the license plate feature map to obtain the second layer features of each layer; and for each layer, compressing the second layer features of the layer in the height dimension to obtain the first layer features of the layer.

[0008] Before performing character recognition on each layer of the license plate feature map to obtain the character recognition results for each layer, the method further includes: adjusting the adaptive region attention of each layer of the license plate feature map to obtain the adaptive region attention feature map for each layer. The features of the corresponding layer in the adaptive region feature map are relatively enhanced, and the adaptive region feature map is used for character recognition of the corresponding layer.

[0009] Specifically, the adaptive region attention adjustment of each layer of the license plate feature map is performed to obtain the adaptive region attention feature map for each layer. This includes: extracting attention adjustment vectors for each layer based on the license plate feature map, where each attention adjustment vector includes attention adjustment coefficients corresponding to each height of the license plate feature map; and adjusting the feature values ​​contained in the license plate feature map based on the attention adjustment vectors for each layer to obtain the adaptive region attention feature map for each layer.

[0010] The process of extracting attention adjustment vectors for each layer based on the license plate feature map includes: performing global pooling on the feature maps at each height in the license plate feature map to obtain pooled feature vectors, which include pooled feature values ​​corresponding to each height of the license plate feature map; performing convolution on the pooled feature vectors for each corresponding layer to obtain attention adjustment vectors for each layer of the license plate feature map; and adjusting the feature values ​​contained in the license plate feature map based on the attention adjustment vectors of each layer to obtain adaptive region attention feature maps for each layer, including: for each layer, fusing the attention adjustment coefficients corresponding to each height in the layer's attention adjustment vector with the feature values ​​of the corresponding height in the license plate feature map to obtain the layer's adaptive region attention feature map.

[0011] The license plate image contains at least one character layer; generating a license plate feature map containing a specific number of character layers for the license plate image includes: filling the license plate image to obtain a license plate image containing a specific number of character layers; and extracting features from the filled license plate image to obtain the license plate feature map.

[0012] Before extracting features from the license plate image, the process also includes: correcting the license plate image, and / or adjusting the filled license plate image to a preset size.

[0013] The process of filling a license plate image to obtain a license plate image containing a specific number of character layers includes: in response to the license plate image not meeting the preset aspect ratio condition, filling the license plate image in a preset direction using preset parameters to obtain a license plate image containing a specific number of character layers; and generating a license plate feature map containing a specific number of character layers for the license plate image further includes: in response to the license plate image meeting the preset aspect ratio condition, directly extracting features from the license plate image to obtain a license plate feature map.

[0014] The preset aspect ratio condition is that the aspect ratio of the license plate image is a preset aspect ratio; the license plate image is filled in a preset direction using preset parameters to obtain a license plate image containing a specific number of character layers, including: for license plate images with an aspect ratio smaller than the preset aspect ratio, the width direction of the license plate image is filled using preset parameters to obtain a license plate image containing a specific number of character layers; for license plate images with an aspect ratio greater than the preset aspect ratio, the height direction of the license plate image is filled using preset parameters to obtain a license plate image containing a specific number of character layers.

[0015] Among them, the specific number of layers is two layers.

[0016] The license plate recognition result of the license plate image is obtained by concatenating the character recognition results of each layer in the order of each layer.

[0017] The character recognition results of each layer are obtained based on a license plate recognition model, which includes multiple character recognition modules. The method also includes the following steps to train the license plate recognition model: acquiring sample license plate images; generating sample license plate feature maps containing a specific number of character layers for the sample license plate images, wherein the specific number of layers is greater than or equal to two, and the sample license plate images are labeled with the real characters contained in each of the specific number of layers; using multiple character recognition modules to perform character recognition on the corresponding layers of the sample license plate feature maps to obtain the sample character recognition results corresponding to each layer in the sample license plate feature maps; and adjusting the model parameters of the character recognition modules of the corresponding layers based on the differences between the sample character recognition results and the real characters.

[0018] This application also provides an electronic device, including a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement any of the above-described license plate recognition methods.

[0019] This application also provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement any of the above-described license plate recognition methods.

[0020] The above scheme generates a license plate feature map containing a specific number of character layers for the license plate image. The specific number of layers is greater than or equal to two. The character recognition of each layer is directly performed on the entire license plate feature map to obtain the character recognition results of each layer in the license plate feature map. The license plate recognition result of the license plate image is obtained based on the character recognition results of each layer. Compared with the existing technology, it realizes the recognition of license plate characters without relying on the segmentation of character regions, avoids the impact of the accuracy of character region segmentation on the license plate recognition, and improves the accuracy of license plate recognition.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0023] Figure 1 This is a flowchart illustrating an embodiment of the license plate recognition method of this application;

[0024] Figure 2 This is a flowchart illustrating another embodiment of step S120 of this application;

[0025] Figure 3 This is a flowchart illustrating another embodiment of the license plate recognition method of this application;

[0026] Figure 4 This is a flowchart illustrating another embodiment of step S310 of this application;

[0027] Figure 5 This is a schematic diagram of an embodiment of the feature extraction module of this application;

[0028] Figure 6 This is a flowchart illustrating another embodiment of step S320 of this application;

[0029] Figure 7 This is a flowchart illustrating another embodiment of step S320 of this application;

[0030] Figure 8 This is a schematic diagram of an embodiment of the character recognition module of this application;

[0031] Figure 9 This is a schematic diagram of the framework of an embodiment of the license plate recognition model of this application;

[0032] Figure 10 This is a flowchart illustrating another embodiment of the license plate recognition method of this application;

[0033] Figure 11 This is a schematic diagram of the framework of an embodiment of the license plate recognition device of this application;

[0034] Figure 12 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0035] Figure 13 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0036] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0037] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0038] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0039] It is understood that the license plate recognition method in this application can be executed by an electronic device, which can be any device with processing capabilities, such as a mobile phone, tablet computer, etc.

[0040] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the license plate recognition method of this application.

[0041] Specifically, this may include the following steps:

[0042] Step S110: Generate a license plate feature map containing a specific number of character layers for the license plate image.

[0043] The license plate image refers to the image of the license plate area, and the license plate feature map is extracted based on the license plate image. The license plate feature map contains a specific number of character layers, which can be preset according to application needs, and is greater than or equal to two, specifically two or three. Specifically, the characters can include text, numbers, symbols, icons, and other objects that may appear in the license plate.

[0044] It should be noted that a character layer with a specific number of layers means that the characters in the license plate feature image are arranged in a specific number of layers. A character layer can also be simply referred to as a layer. The above arrangement of characters with a specific number of layers does not necessarily mean that each layer contains characters. For example, a license plate feature image may contain two character layers, where the upper layer may not contain any characters, and the lower layer may contain characters.

[0045] It should be noted that the license plate being recognized contains several characters, and license plate recognition is used to determine the characters contained in the license plate based on the license plate image. The license plate recognition method in this application can be implemented by a license plate recognition model, which can recognize the license plate image and obtain the license plate recognition result.

[0046] The specific number of layers in the license plate feature map can be set according to actual needs, and this number of layers can be associated with the number of character layers in the license plate to be identified. The specific number of layers in the license plate feature map can be greater than the number of character layers in the license plate to be identified.

[0047] In some embodiments, the license plate image used for recognition can be a single image or multiple images. When the license plate image comprises multiple images, a corresponding license plate feature map is generated for each image, and each license plate feature map contains the same number of character layers.

[0048] It is understandable that the characters in a license plate can be arranged in at least one character layer, and correspondingly, the license plate image also contains several characters, arranged in at least one character layer. Based on the character arrangement, such as single-layer arrangement or double-layer arrangement, license plates can be classified as single-layer license plates, double-layer license plates, etc.

[0049] In some embodiments, the license plate images used for license plate recognition can have the same character arrangement or may include different character arrangements. For example, the license plates being recognized can all be double-layered, or all be triple-layered, or may include single-layered and double-layered license plates, or may include single-layered, double-layered, and triple-layered license plates. When the license plate image includes images of license plates with one or more character arrangements, all license plate feature maps contain a specific number of character layers.

[0050] In some embodiments, the specific number of layers in the license plate feature map can be consistent with the upper limit of the number of character arrangement layers in the license plate image to be recognized. For example, if the license plate image for license plate recognition is a two-layer license plate, the license plate feature map obtained based on the license plate image will contain two character layers.

[0051] Step S120: Perform character recognition on each layer of the license plate feature map to obtain the character recognition results for each layer of the license plate feature map.

[0052] Among them, character recognition of each layer of the license plate feature map can include character recognition of each layer in a specific number of layers.

[0053] In some embodiments, there are two layers, referred to as the upper layer and the lower layer. Character recognition is performed on the license plate feature map corresponding to the upper layer and the lower layer, respectively.

[0054] Among them, character recognition of a certain layer of the license plate feature map is used to identify the characters contained in that layer of the license plate feature map, and the character recognition result can represent the character recognition result corresponding to that layer.

[0055] In a specific application scenario, character recognition is performed on the upper layer and the lower layer of the license plate feature image, respectively, to identify the characters contained in the upper layer or the lower layer of the license plate feature image, and to obtain the character recognition results of the upper layer and the lower layer.

[0056] In some embodiments, the license plate image contains an image with an arrangement different from the specified number of layers. For example, the license plate images to be identified include single-layer license plate images and double-layer license plate images, where the specified number of layers is two. Generating a license plate feature map may also include padding the license plate image to obtain a license plate image containing a specific number of character layers, so that after feature extraction on the padded license plate image, a license plate feature map representing characters arranged in a specific number of layers can be obtained.

[0057] In one implementation scenario, for license plate images that already contain a specific number of character layers, no padding is required, and feature extraction can be performed directly.

[0058] In a specific application scenario, with a certain number of two license plate layers, at least part of the license plate image is a single-layer image. The double-layer license plate image does not need to be filled, while the single-layer license plate image is filled to arrange the characters in a double-layer format.

[0059] In a specific application scenario, the license plate images that need to be recognized include double-layered and triple-layered license plate images, with a specific number of layers being three. For double-layered license plates, the image is padded so that the characters are arranged in a three-layered pattern.

[0060] Furthermore, different license plate arrangements have different aspect ratios; therefore, license plate images with different aspect ratios can be distinguished. Further, aspect ratio conditions can be preset to differentiate license plate images with different arrangements. These preset aspect ratio conditions can be adjusted according to the needs of the actual application; for example, a preset aspect ratio range or a preset aspect ratio value can be set as the preset aspect ratio condition. Specifically, filling the license plate image can include filling the license plate image in response to the license plate image not meeting the preset aspect ratio condition. Of course, the conditions for filling are not limited to setting conditions related to aspect ratio. And, in response to the license plate image meeting the preset aspect ratio condition, feature extraction can be directly performed on the license plate image to obtain a license plate feature map.

[0061] Furthermore, the number of character layers after padding can be consistent with the upper limit of the number of character layers in the license plate image to be recognized. Then, a license plate feature map is extracted based on the padded license plate image, and the number of character layers in the license plate feature map is consistent with the upper limit. The padding method can also be adjusted according to the needs of the actual application.

[0062] In one implementation scenario, license plate images that do not meet the preset aspect ratio condition can be filled in a preset direction using preset parameters to obtain a license plate image containing a specific number of character layers. Specifically, the preset aspect ratio condition is that the aspect ratio of the license plate image is a preset aspect ratio. For license plate images with an aspect ratio smaller than the preset aspect ratio, the preset parameters are used to fill in the width direction of the license plate image. For license plate images with an aspect ratio greater than the preset aspect ratio, the preset parameters are used to fill in the height direction of the license plate image to obtain a license plate image containing a specific number of character layers.

[0063] In some embodiments, when filling a license plate image, preset parameters can be used. After character recognition of each layer in the filled license plate, the filled content can be removed to obtain the characters contained in the original license plate. The preset parameters can be 0, or they can be parameters different from the original parameters in the license plate image, so that the filled content can be distinguished from the original parameters in subsequent processing, such as character recognition. The specific setting can be determined according to the actual application needs. Furthermore, the preset parameters can be 0, so the filled license plate image shows characters arranged in a specific number of layers. In this case, the result of character recognition of the filled portion is empty, which also reduces the computational load.

[0064] In a specific application scenario, the number of layers is two, and the preset aspect ratio is [missing information]. The aspect ratio of the license plate image is expressed as like You can fill the top of the license plate image with 0s, with a fill height of [value missing]. Width is W T ;if You can fill the right side of the license plate image with 0s, with a fill height of H. T Width is

[0065] The above method, by filling the image, can adjust and unify the arrangement of characters in the image to accommodate the recognition of license plates with different character arrangements, thus enabling the simultaneous recognition of license plates with different character arrangements. Taking the recognition of single-layer and double-layer license plates as an example, when the aspect ratio of the image is greater than the preset aspect ratio of the input image, appropriate filling is performed in the height direction of the image, which is helpful in judging the case where there are no valid characters in the upper layer of the single-layer license plate. When the aspect ratio of the image is less than the preset aspect ratio of the input image, appropriate filling in the width direction of the image has virtually no impact on character recognition.

[0066] Step S130: Based on the character recognition results corresponding to each layer, obtain the license plate recognition result of the license plate image.

[0067] Based on the character recognition results corresponding to each layer in the license plate feature map, the license plate recognition result of the license plate image can be obtained. The license plate recognition result represents the characters contained in the license plate.

[0068] In a specific application scenario, character recognition is performed on the corresponding layers of the license plate feature map. If the character recognition result of the upper layer is "ABC" and the character recognition result of the lower layer is "DEF", then the license plate recognition result can be obtained.

[0069] In some embodiments, the license plate recognition model may include several branches, the number of branches may be consistent with a specific number of layers, and each branch is used to perform license plate recognition on a corresponding layer in the license plate feature map, thereby realizing character recognition on each corresponding layer of the license plate feature map.

[0070] In a specific application scenario, the license plate recognition model includes two branches: the first branch is used to perform license plate recognition on the upper layer of the license plate feature map, and the second branch is used to perform license plate recognition on the lower layer of the license plate feature map.

[0071] The above scheme generates a license plate feature map containing a specific number of character layers for the license plate image. The specific number of layers is greater than or equal to two. The character recognition of each layer is directly performed on the entire license plate feature map to obtain the character recognition results of each layer in the license plate feature map. The license plate recognition result of the license plate image is obtained based on the character recognition results of each layer. Compared with the existing technology, it realizes the recognition of license plate characters without relying on the segmentation of character regions, avoids the impact of the accuracy of character region segmentation on the license plate recognition, and improves the accuracy of license plate recognition.

[0072] Furthermore, character recognition is adopted for each layer. Each layer of character recognition only focuses on the features of that layer in the overall license plate feature map. This can achieve character recognition of the corresponding layer without segmenting the license plate image into a single-layer character image.

[0073] Please see Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of step S120 of this application. Specifically, step S120 includes:

[0074] Step S221: Extract local features from each layer of the license plate feature map to obtain the first layer features corresponding to each layer of the license plate feature map.

[0075] This involves extracting local features from the license plate feature map for each corresponding layer, including extracting local features from each layer within a specific number of layers. Local feature extraction is used to extract the first-layer features of the corresponding local layer from the license plate feature map.

[0076] In one implementation scenario, the license plate feature map contains two character layers. Local features of the upper layer and the lower layer are extracted from the license plate feature map to obtain the first-layer features of the upper layer and the first-layer features of the lower layer, respectively. The width of the first-layer features corresponding to different layers can be different.

[0077] Furthermore, feature extraction can be performed on the license plate feature map for each corresponding layer to obtain the second layer features for each layer. For each layer, the second layer features corresponding to that layer are compressed along the height dimension to obtain the first layer features for that layer. The feature compression along the height dimension can be achieved by averaging feature values ​​of the same width but different heights.

[0078] In a specific application scenario, the features of the input upper-level character recognition module are denoted as... The features of the input lower-level character recognition module are denoted as: After obtaining the upper-layer local feature extraction and the lower-layer local feature extraction respectively, the following results are obtained: Wherein, num represents the number of character categories, and in this embodiment, num = 78; w1 and w2 represent the width of the output features of the upper and lower layer character recognition modules, respectively, and in this embodiment, w1 = 8 and w2 = 18.

[0079] Step S222: Decode the characters of the first-layer features corresponding to each layer to obtain the character recognition results corresponding to each layer.

[0080] Character decoding can employ methods such as the CTC algorithm (Connectionist temporal classification), and the specific method can be selected based on the actual application requirements.

[0081] Please see Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the license plate recognition method of this application.

[0082] Specifically, the method may include:

[0083] Step S310: Generate a license plate feature map containing a specific number of character layers for the license plate image.

[0084] The relevant description of step S310 can be found in the description of step S110 in the foregoing embodiments.

[0085] Please see Figure 4 , Figure 4 This is a flowchart illustrating another embodiment of step S310 of this application. Specifically, step S310 may include the following steps:

[0086] Step S411: Fill in the license plate image.

[0087] The license plate image contains only the license plate area. Filling the license plate image yields a license plate image with a specific number of character layers. Furthermore, in response to a license plate image not meeting a preset aspect ratio, the method involves filling the license plate image in a preset direction using preset parameters to obtain a license plate image with a specific number of character layers.

[0088] In some embodiments, generating a license plate feature map containing a specific number of character layers for a license plate image also includes directly extracting features from the license plate image in response to the license plate image meeting a preset aspect ratio condition to obtain the license plate feature map.

[0089] In some embodiments, the license plate image can be cropped from an original image containing the license plate, which can be acquired by an acquisition device. The acquisition device can capture the original image directly facing the license plate or from the side, therefore the license plate image may have some distortion. Therefore, before feature extraction, the license plate image can undergo preprocessing steps other than padding. Specifically, this can include correcting the license plate image to ensure that the corrected image is free of distortion and can be used for subsequent license plate recognition, thus improving the accuracy of license plate recognition.

[0090] In a specific application scenario, license plate image correction can be achieved based on the original corner coordinates of the license plate in the original image. Of course, the specific correction method can be set according to the actual application. Taking correction based on the original corner coordinates as an example, the lengths of the four sides of the license plate are calculated using the coordinates of the four corner points, and the longest side among the top and bottom sides is taken as the corrected license plate width W. T The longest side value among the left and right sides is used as the corrected license plate height H. T We obtain the four corner points of the license plate after perspective transformation. Let the coordinates of the top left corner be (0,0), then the coordinates of the top right, bottom right, and bottom left corners are (W... T ,0)(W T H T (0,H) T The perspective transformation matrix is ​​obtained by using the original corner coordinates and the transformed corner coordinates. The perspective transformation matrix is ​​then used to correct the license plate image, resulting in the corrected license plate image.

[0091] In some embodiments, the license plate image may be cropped from an original image containing the license plate. Specifically, license plate detection can be performed on the original image, and the license plate image can be cropped from the original image based on the license plate detection results.

[0092] In one implementation scenario, a license plate detection model is used to detect the original image and obtain the license plate detection result, including the corner coordinates of the license plate. The license plate image is then extracted from the original image using the corner coordinates.

[0093] In a specific application scenario, the license plate detection model can adopt the YOLOv5S model. The license plate detection result includes the category, center coordinates, width and height, and confidence score of the detection box, as well as the coordinates of the four detection corner points of the license plate. Further, the output dimension of the detection head is increased by increasing the number of convolutional kernels in the last convolutional layer, changing the output from (cls, x, y, w, h, conf) to (cls, x, y, w, h, conf, x1, y1, x2, y2, x3, y3, x4, y4), where (x1, y1), (x2, y2), (x3, y3), and (x4, y4) represent the detection corner point coordinates of the top-left, top-right, bottom-right, and bottom-left corner points, respectively.

[0094] In this process, the coordinates of the detected corner points are already obtained during license plate detection. However, since the original image has been transformed during detection, the detected corner point coordinates cannot directly represent the coordinates of the license plate corner points in the original image. Therefore, the actual original corner point coordinates can be obtained by transformation based on the detected corner point coordinates. Specifically, the following formula can be used to convert the detected corner point coordinates into the actual corner point coordinates:

[0095] k xi =a w x i +c x s

[0096] k yi =a h y i +c y s

[0097] Where, x i y i These represent the x and y coordinates of the detected corner point, respectively. w a h These represent the width and height of the detection frame during license plate detection, respectively. x c y The x and y coordinates of the top-left corner of the grid where the license plate is located are respectively. The grid where the center point of the detection box is located can be taken as the grid where the license plate is located. s represents the downsampling step size of the corresponding output layer, and k xi k yi These represent the x and y coordinates of the original corner points after conversion. These original corner point coordinates can represent the license plate region in the original image, used for cropping the license plate image and for correcting the license plate image.

[0098] In some embodiments, the license plate image may undergo preprocessing other than padding, specifically including size adjustment. Specifically, the license plate recognition model has a preset size for the input image, and the license plate image is adjusted to the preset size through size adjustment.

[0099] It should be noted that the above steps can be performed as correction, filling, and resizing to ensure the image quality of the license plate image.

[0100] Step S412: Extract features from the filled license plate image to obtain a license plate feature map.

[0101] Specifically, the license plate recognition model may include a feature extraction module, which is shared by all branches. The license plate image is input into the feature extraction module to obtain the license plate feature map, and then input into different branches respectively.

[0102] In one implementation scenario, the feature extraction module consists of convolutional layers, pooling layers, and small basic blocks. The kernel size, stride, and padding values ​​of the convolutional and pooling layers are designed according to the size of the input image and can be flexibly adjusted according to the actual situation in practical applications.

[0103] In a specific application scenario, H0 and W0 represent the height and width of the license plate image, respectively, which are 64 and 144. The height, width, and number of channels of the license plate feature map are represented by H, W, and C, respectively.

[0104] Please see Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the feature extraction module of this application.

[0105] In this embodiment, Figure 5 Figure (a) shows a schematic diagram of the feature extraction module. Figure 5 Figure (b) shows a schematic diagram of the structure of the small basic block. The size of the feature map does not change after passing through the small basic block.

[0106] The feature extraction module mainly consists of convolutional layers, pooling layers, and Small Basic Blocks. The kernel size, stride, and padding values ​​used in the convolutional and pooling layers are designed based on the size of the input image and can be flexibly adjusted according to the specific application requirements.

[0107] In this embodiment, the input image The feature map is obtained after the feature extraction module. Where H0 and W0 are the height and width of the input image, respectively, which are 64 and 144 in this embodiment.

[0108] Step S320: Adjust the adaptive region attention of each layer of the license plate feature map to obtain the adaptive region attention feature map for each layer.

[0109] Specifically, the adjustment of the adaptive region attention for a certain layer is used to adjust the license plate feature map. In other words, the attention of features at different heights in the license plate feature map is adjusted so that the features of that layer are enhanced accordingly for character recognition at that layer.

[0110] In one implementation scenario, adjusting the adaptive region attention for each layer includes adjusting the adaptive region attention for each layer within a specific number of layers. For example, if there are two layers, the adaptive region attention for the license plate feature map is adjusted for the upper layer and the lower layer, resulting in adaptive region attention feature maps for the upper and lower layers. Specifically, in the upper layer's adaptive region attention feature map, the features of the upper layer are relatively enhanced, and in the lower layer's adaptive region attention feature map, the features of the lower layer are relatively enhanced.

[0111] In some embodiments, relative enhancement may manifest as the feature of this layer being enhanced, or the feature of other layers being weakened, or the feature of this layer being enhanced and the feature of other layers being weakened.

[0112] The above scheme adjusts the model's focus on features at different height ranges on the feature map, enabling it to pay more attention to character features in specific regions (corresponding layers) and reduce attention to character features in irrelevant regions during character recognition, thus facilitating more accurate character recognition at each layer.

[0113] Furthermore, adaptive attention can adapt to various layer width ratios. For example, adaptive attention adjustment can be accurately applied to situations where the height ratio of the upper and lower layers of characters on a large truck license plate is 1:2, as well as situations where the height ratio of the upper and lower layers of characters on a motorcycle license plate is 1:1, thus improving the accuracy of character recognition.

[0114] Please see Figure 6 , Figure 6 This is a flowchart illustrating another embodiment of step S320 of this application. Specifically, step S320 may include:

[0115] Step S621: Extract attention adjustment vectors for each layer based on the license plate feature map.

[0116] Specifically, attention adjustment vectors are extracted for each layer based on the license plate feature map. These attention adjustment vectors are used to adjust the attention of the corresponding layer in an adaptive manner.

[0117] Specifically, attention adjustment vectors for each layer can be extracted based on the license plate feature map, and then different adaptive region attention feature maps can be obtained by using the attention adjustment vectors for each layer.

[0118] In some embodiments, the feature map extracted based on the license plate feature map can also be a attention adjustment feature map, which can be used to adaptively adjust the regional attention of the corresponding layer.

[0119] In some embodiments, each branch of the license plate recognition model includes an adaptive region attention module. The adaptive region attention modules of different branches are used to extract the attention adjustment vector of the layer corresponding to the branch, and to adjust the license plate feature map using the attention adjustment vector.

[0120] Furthermore, extracting the attention adjustment vector may include: performing global pooling on the feature maps at each height in the license plate feature map to obtain pooled feature vectors, where the pooled feature vectors may include the pooled feature values ​​corresponding to each height of the license plate feature map. Then, performing convolution processing on the pooled feature vectors for each layer to obtain attention adjustment vectors for each layer. These attention adjustment vectors may include attention adjustment coefficients corresponding to each height of the license plate feature map.

[0121] Step S622: Adjust the feature values ​​contained in the license plate feature map based on the attention adjustment vector of each layer to obtain the adaptive region attention feature map of each layer.

[0122] Specifically, the attention adjustment vector includes attention adjustment coefficients corresponding to each height of the license plate feature map. For each layer, the attention adjustment coefficients corresponding to each height in the attention adjustment vector of that layer are fused with the feature map of the corresponding height in the license plate feature map to obtain the adaptive region attention feature map of that layer. The fusion method can be multiplication, addition, etc.

[0123] Please see Figure 7 , Figure 7 This is a flowchart illustrating another embodiment of step S320 of this application.

[0124] In this embodiment, Let H represent the license plate feature map, where H represents the height, W represents the width, and C represents the number of channels. Channel swapping is performed on the license plate feature map, exchanging the height dimension with the channel dimension to obtain the swapped license plate feature map. This facilitates operations on feature maps at the same height. Then, global average pooling is used to compress the feature maps at each height into pooled feature values, resulting in a pooled feature vector. For each layer, the attention adjustment vector of the pooling feature vector is performed using the corresponding one-dimensional convolutional layer Conv1d. Then, a Sigmoid layer is used to normalize the feature values ​​at each height to obtain the attention adjustment vector for each layer. Multiplying the feature map F1 by the attention adjustment vector S2 yields the license plate feature map after adjusting the feature strength in the height dimension. Finally, an adaptive region attention feature map is obtained by restoring the height and channel dimensions to their original order through a channel swapping operation. The kernel size, stride, and padding size used in Conv1d can be set according to the height of the actual feature map; larger feature maps can use larger kernels.

[0125] The above method adjusts the network's focus on features at different height ranges on the feature map by generating adjustment vectors in the height dimension, so that different branches of the model can focus on character features at different layers, thereby improving the accuracy of character recognition.

[0126] Step S330: Perform character recognition on the adaptive region attention feature maps of each layer to obtain the character recognition results for each layer.

[0127] The relevant descriptions of steps S330-S340 can be found in the relevant content of the foregoing embodiments.

[0128] Specifically, for each layer, the corresponding adaptive region attention feature map is used to perform character recognition for that layer, and the character recognition result for that layer is obtained.

[0129] In some embodiments, each branch of the license plate recognition model includes a character recognition module, which is used to perform character recognition on the license plate feature map. Specifically, it is used to perform character recognition on the adaptive region attention feature map.

[0130] Please see Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the character recognition module of this application.

[0131] Different branches of the license plate recognition model are used for character recognition at different layers. An adaptive region-of-interest feature map is input into the character recognition module, which outputs layer features for decoding to obtain the character recognition result. The layer feature sizes output by the character recognition modules of different branches vary. Specifically, the kernel size, stride, and padding values ​​of the convolutional and pooling layers in the character recognition module are designed based on the size of the input image and can be flexibly adjusted according to actual needs in practical applications. The layer feature sizes output by each character recognition module can be set according to the specific application.

[0132] In this embodiment, Figure 8Figure (a) shows a schematic diagram of the upper-level character recognition module. Figure 8 Figure (b) shows a schematic diagram of the structure of the last two convolutional layers of the lower-level character recognition module. Figure 8 The last two convolutional layers in (a) are replaced with the two convolutional layers in (b) to obtain the lower-level character recognition module.

[0133] Here, Mean(dim=2) represents the mean of the feature map along the height dimension, that is, the mean of feature values ​​at different heights for the same width. In this embodiment, the features input to the upper-layer character recognition module are denoted as... The features of the input lower-level character recognition module are denoted as: The features obtained after passing through the upper-level character recognition module and the lower-level character recognition module are as follows: Where num represents the total number of character categories that may appear in a license plate; in this embodiment, num = 78. For example, before averaging, the feature size can be num × 4 × w1. Averaging calculates the average of feature values ​​with the same width but different heights, thus changing the feature size of the height dimension from 4 to 1. w1 and w2 represent the width of the output features of the upper and lower character recognition modules, respectively; in this embodiment, w1 = 8 and w2 = 18.

[0134] Please see Figure 9 , Figure 9 This is a schematic diagram of the framework of an embodiment of the license plate recognition model of this application.

[0135] In this embodiment, a two-layer model is used as an example. The license plate recognition model 90 includes a feature extraction module 91 shared by two branches, an upper-layer adaptive region attention module 92, a lower-layer adaptive region attention module 93, an upper-layer character recognition module 94, and a lower-layer character recognition module 95. The input image is processed by feature extraction, and then two branches are output, which output the upper-layer character and the lower-layer character respectively.

[0136] Step S340: Based on the character recognition results corresponding to each layer, obtain the license plate recognition result of the license plate image.

[0137] Specifically, the character recognition results corresponding to each layer can be concatenated in the order of the layers to obtain the license plate recognition result. In some embodiments, the character recognition results corresponding to some layers can be empty.

[0138] Please see Figure 10 , Figure 10 This is a flowchart illustrating another embodiment of the license plate recognition method of this application. In this embodiment, the character recognition result is obtained based on the license plate recognition model, and the method further includes the following steps to train the license plate recognition module:

[0139] Step S1010: Obtain sample license plate images.

[0140] In some embodiments, the sample license plate images may include several types, and the characters in the sample license plate images of different types are arranged in different layers.

[0141] The sample license plate images are labeled with the actual characters contained in each layer when arranged in a specific number of layers. The specific number of layers can be predetermined according to actual needs, such as two layers, three layers, etc. Furthermore, the specific number of layers can be associated with the upper limit of the number of character arrangement layers in the sample license plate image, for example, equal to the upper limit of the number of character arrangement layers in the sample license plate image.

[0142] In some embodiments, the sample license plate image includes a license plate image with characters arranged in a specific number of layers.

[0143] In some embodiments, the method may further include padding the sample license plate image to obtain a license plate image containing a specific number of character layers, as detailed in the foregoing embodiments. The sample license plate image is labeled with the actual characters contained in each of the specific number of layers. Specifically, the sample license plate image is labeled with the actual characters contained in each of the specific number of layers, and these characters serve as the actual labels for the branches corresponding to each layer.

[0144] In a specific application scenario, license plate characters are arranged in single-layer and double-layer configurations. For single-layer license plates, the upper label is a blank symbol "-", and the lower label is the corresponding character of the license plate. For double-layer license plates, the upper label is the corresponding character of the upper layer, and the lower label is the corresponding character of the lower layer.

[0145] In some embodiments, the license plate image may also be preprocessed, specifically including at least one of correcting the license plate image through perspective transformation and adjusting its size.

[0146] The sample license plate images can be cropped from images containing license plates. The cropping of sample license plate images can be obtained based on the detection results of the license plate detection model.

[0147] Before obtaining sample license plate images, the license plate detection model can be trained to extract sample license plate images from images containing license plates.

[0148] The method may also include a training step for the license plate detection model: acquiring sample original images, wherein the sample original images are labeled with the true location information of the license plates; using the license plate detection model to detect the license plates in the sample original images to obtain sample detection results; and adjusting the license plate detection model based on the difference between the sample detection results and the true location information.

[0149] The true location information can include the center coordinates, width and height, and coordinates of the four corner points of the license plate. The sample detection results can include the category, center coordinates, width and height, and confidence score of the detection box, as well as the coordinates of the four corner points. The category, center coordinates, width and height, and confidence score of the detection box can be optimized using a detection loss function, while the corner coordinates can be optimized using a WingLoss loss function.

[0150] In a specific application scenario, the license plate detection model is the YOLOv5S model, and the category, position, and confidence of the license plate frame are optimized using the same loss function as YOLOv5S.

[0151] The trained license plate detection model can be used to detect the license plate detection box and the corresponding coordinates of the four corner points from an image containing a license plate, in order to extract a sample license plate image. For related descriptions, please refer to the relevant content in the foregoing embodiments. The image containing the license plate can also be the original sample image. In this case, the original sample image, in addition to being labeled with the actual location information of the license plate, can also be labeled with the characters contained in the license plate. These characters can serve as the true labels for the sample license plate image.

[0152] Step S1020: Generate a sample license plate feature map containing a specific number of character layers for the sample license plate image.

[0153] The sample license plate feature map contains a specific number of character layers.

[0154] Step S1030: Use multiple character recognition modules to perform character recognition on each layer of the sample license plate feature map to obtain the sample character recognition results for each layer of the sample license plate feature map.

[0155] The relevant descriptions of steps S1020-S1030 can be found in the relevant content of the foregoing embodiments.

[0156] Step S1040: Adjust the model parameters of the character recognition module of the corresponding layer by using the difference between the sample character recognition results and the real characters in each layer.

[0157] Specifically, the model parameters of the character recognition module for each layer are adjusted by using the differences between the sample character recognition results and the real characters.

[0158] By setting different layers of characters as real labels for the character recognition modules of different branches, the character recognition modules of different branches can learn character features of different layers from the same feature map. After training, they can recognize characters of different layers from the same feature map, realizing direct character recognition of each layer of the license plate feature map as a whole, and obtaining the character recognition results of each layer in the license plate feature map. Based on the character recognition results of each layer, the license plate recognition result of the license plate image is obtained. Compared with the existing technology, it realizes the recognition of license plate characters without relying on character region segmentation, avoids the impact of the accuracy of character region segmentation on license plate recognition, and improves the accuracy of license plate recognition.

[0159] In a specific application scenario, the license plate recognition model includes a shared feature extraction module across all branches and separate character recognition modules for each branch. During training, both the upper and lower branches of the model use CTC Loss to calculate the training loss and optimize the model. The loss of each branch applies to the corresponding branch and the shared feature extraction module.

[0160] In some embodiments, each branch of the license plate recognition module further includes an adaptive region attention module. After obtaining the sample license plate feature map, the adaptive region attention of each layer in the sample license plate feature map can be adjusted to obtain the sample adaptive region attention map for each layer. The features of the corresponding layer in the sample adaptive region attention map are relatively enhanced. Furthermore, the sample adaptive region attention map for each layer is used for character recognition at that layer. During training, both the upper and lower branches of the model use CTC Loss to calculate the training loss and optimize the model. Related descriptions can be found in the foregoing embodiments and will not be repeated here.

[0161] Furthermore, the adaptive region attention module enables the model to adaptively adjust its attention to character features at different layers based on the input image, allowing each branch to focus on the features of its corresponding layer, thereby improving the accuracy of character recognition at the corresponding layer by each branch.

[0162] Furthermore, this adaptive attention adjustment is more reasonable than directly segmenting the feature map by the number of layers. The adaptive attention adjustment can adjust the attention according to the feature map itself and can adapt to various different layer width ratios. Taking single-layer license plates and double-layer license plates as examples, the adaptive attention adjustment can adjust the attention according to the feature map itself. It can be accurately used for situations such as the height ratio of the upper and lower characters of a large truck license plate being 1:2, as well as the height ratio of the upper and lower characters of a motorcycle license plate being 1:1, thus improving the accuracy of character recognition.

[0163] Please see Figure 11 , Figure 11 This is a schematic diagram of the framework of an embodiment of the license plate recognition device of this application.

[0164] In this embodiment, the license plate recognition device 110 includes a generation module 111, a character recognition module 112, and a license plate recognition module 113. The generation module 111 is used to generate a license plate feature map containing a specific number of character layers for the license plate image, wherein the specific number of layers is greater than or equal to two. The character recognition module 112 is used to perform character recognition on the license plate feature map for each corresponding layer to obtain the character recognition result corresponding to each layer in the license plate feature map. The license plate recognition module 113 is used to obtain the license plate recognition result of the license plate image based on the character recognition result corresponding to each layer.

[0165] The character recognition module 112 includes a local feature extraction submodule and a character decoding submodule. The local feature extraction submodule is used to extract local features corresponding to each layer of the license plate feature map to obtain the first layer features corresponding to each layer in the license plate feature map. The character decoding submodule is used to decode the first layer features corresponding to each layer to obtain the character recognition results corresponding to each layer.

[0166] The local feature extraction submodule includes a layer feature extraction unit and a compression unit. The layer feature extraction unit is used to extract features from each layer of the license plate feature map to obtain the second layer features corresponding to each layer. The compression unit is used to compress the second layer features corresponding to each layer in the height dimension to obtain the first layer features corresponding to each layer.

[0167] The license plate recognition device 110 also includes an attention adjustment module, which is used to adjust the adaptive region attention of each layer of the license plate feature map before performing character recognition on each layer of the license plate feature map to obtain the character recognition results of each layer of the license plate feature map. The adaptive region attention feature map is obtained for each layer, wherein the features of the corresponding layer in the adaptive region feature map are relatively enhanced, and the adaptive region feature map is used for character recognition of the corresponding layer.

[0168] The attention adjustment module includes an extraction submodule and an adjustment submodule. The extraction submodule is used to extract the attention adjustment vectors for each layer based on the license plate feature map. The attention adjustment vectors include the attention adjustment coefficients corresponding to each height of the license plate feature map. The adjustment submodule is used to adjust the feature values ​​contained in the license plate feature map based on the attention adjustment vectors of each layer to obtain the adaptive region attention feature map of each layer.

[0169] The extraction submodule includes a pooling unit and a convolution unit. The pooling unit performs global pooling on the feature maps at each height in the license plate feature map to obtain pooled feature vectors, which include pooled feature values ​​corresponding to each height of the license plate feature map. The convolution unit performs convolution on the pooled feature vectors for each corresponding layer to obtain attention adjustment vectors for each layer of the license plate feature map. The adjustment submodule adjusts the feature values ​​contained in the license plate feature map based on the attention adjustment vectors of each layer to obtain adaptive region attention feature maps for each layer. Specifically, for each layer, the attention adjustment coefficients corresponding to each height in the layer's attention adjustment vector are fused with the feature values ​​of the corresponding heights in the license plate feature map to obtain the layer's adaptive region attention feature map.

[0170] The generation module 111 includes a filling submodule and a feature extraction submodule. The filling submodule is used to fill the license plate image to obtain a license plate image containing a specific number of character layers. The feature extraction submodule is used to extract features from the filled license plate image to obtain a license plate feature map.

[0171] The filling submodule is specifically used to respond to license plate images that do not meet a preset aspect ratio condition by filling the license plate image in a preset direction using preset parameters to obtain a license plate image containing a specific number of character layers. The preset aspect ratio condition is that the aspect ratio of the license plate image is a preset aspect ratio. Filling the license plate image in a preset direction using preset parameters to obtain a license plate image containing a specific number of character layers specifically includes: for license plate images with an aspect ratio smaller than the preset aspect ratio, filling the width direction of the license plate image using preset parameters to obtain a license plate image containing a specific number of character layers; for license plate images with an aspect ratio greater than the preset aspect ratio, filling the height direction of the license plate image using preset parameters to obtain a license plate image containing a specific number of character layers.

[0172] The generation module 111 is also used to directly extract features from the license plate image in response to the license plate image meeting the preset aspect ratio condition, so as to obtain the license plate feature map.

[0173] The generation module 111 is also used to correct the license plate image; and / or adjust the filled license plate image to a preset size.

[0174] The license plate recognition module 113 is used to obtain the license plate recognition result of the license plate image based on the character recognition results corresponding to each layer. Specifically, it includes: splicing the character recognition results corresponding to each layer in the order of each layer to obtain the license plate recognition result.

[0175] The character recognition results at each layer are obtained based on a license plate recognition model, which includes multiple character recognition modules. The license plate recognition device 110 also includes a training module, which is used to acquire sample license plate images; generate sample license plate feature maps containing a specific number of character layers for the sample license plate images, where the specific number of layers is greater than or equal to two; and annotate the sample license plate images with the real characters contained in each of the specific number of layers; use the multiple character recognition modules to perform character recognition on the corresponding layers of the sample license plate feature maps to obtain the sample character recognition results for each layer in the sample license plate feature maps; and adjust the model parameters of the character recognition modules for the corresponding layers based on the differences between the sample character recognition results and the real characters.

[0176] Please see Figure 12 , Figure 12 This is a schematic diagram of the framework of an embodiment of the electronic device of this application.

[0177] In this embodiment, the electronic device 120 includes a memory 121 and a processor 122 coupled to each other. The processor 122 is used to execute program instructions stored in the memory 121 to implement the steps in any of the above-described license plate recognition method embodiments. In a specific implementation scenario, the electronic device 120 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 120 may also include mobile devices such as laptops and tablets, which are not limited here.

[0178] Specifically, processor 122 controls itself and memory 121 to implement the steps in any of the above-described license plate recognition method embodiments. Processor 122 can also be referred to as a CPU (Central Processing Unit). Processor 122 may be an integrated circuit chip with signal processing capabilities. Processor 122 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 122 can be implemented using integrated circuit chips.

[0179] The above scheme generates a license plate feature map containing a specific number of character layers for the license plate image. The specific number of layers is greater than or equal to two. The character recognition of each layer is directly performed on the entire license plate feature map to obtain the character recognition results of each layer in the license plate feature map. The license plate recognition result of the license plate image is obtained based on the character recognition results of each layer. Compared with the existing technology, it realizes the recognition of license plate characters without relying on the segmentation of character regions, avoids the impact of the accuracy of character region segmentation on the license plate recognition, and improves the accuracy of license plate recognition.

[0180] Please see Figure 13 , Figure 13 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application.

[0181] In this embodiment, the computer-readable storage medium 130 stores program instructions 131 that can be executed by a processor. The program instructions 131 are used to implement the steps in any of the above-described license plate recognition method embodiments.

[0182] The above scheme generates a license plate feature map containing a specific number of character layers for the license plate image. The specific number of layers is greater than or equal to two. The character recognition of each layer is directly performed on the entire license plate feature map to obtain the character recognition results of each layer in the license plate feature map. The license plate recognition result of the license plate image is obtained based on the character recognition results of each layer. Compared with the existing technology, it realizes the recognition of license plate characters without relying on the segmentation of character regions, avoids the impact of the accuracy of character region segmentation on the license plate recognition, and improves the accuracy of license plate recognition.

[0183] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0184] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0186] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0187] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A license plate recognition method, characterized in that, The method includes: Generate a license plate feature map containing a specific number of character layers for the license plate image, wherein the specific number of layers is greater than or equal to two; The character recognition of each layer in the license plate feature map is performed to obtain the character recognition results of each layer in the license plate feature map. Based on the character recognition results corresponding to each layer, the license plate recognition result of the license plate image is obtained; Before performing character recognition on each layer of the license plate feature image to obtain the character recognition results for each layer of the license plate feature image, the method further includes: The adaptive region attention of each layer of the license plate feature map is adjusted to obtain the adaptive region attention feature map corresponding to each layer. The features of the corresponding layer in the adaptive region attention feature map are relatively enhanced. The adaptive region attention feature map is used for the character recognition of the corresponding layer. The adjustment of the adaptive region attention refers to adjusting the attention of features at different heights in the license plate feature map of each layer. The step of performing character recognition on each layer of the license plate feature map to obtain the character recognition results for each layer of the license plate feature map includes: Local feature extraction is performed on each corresponding layer of the license plate feature map to obtain the first layer features corresponding to each layer of the license plate feature map. The first-layer features corresponding to each layer are decoded to obtain the character recognition results corresponding to each layer. The step of extracting local features corresponding to each layer of the license plate feature map to obtain the first layer features corresponding to each layer of the license plate feature map includes: The features of each corresponding layer are extracted from the license plate feature map to obtain the second layer features corresponding to each layer. For each layer, the second-layer features corresponding to that layer are compressed in the height dimension to obtain the first-layer features corresponding to that layer.

2. The method according to claim 1, characterized in that, The step of adjusting the adaptive region attention of each layer of the license plate feature map to obtain the adaptive region attention feature map for each layer includes: Based on the license plate feature map, attention adjustment vectors corresponding to each layer are extracted, and the attention adjustment vectors include attention adjustment coefficients corresponding to each height of the license plate feature map. Based on the attention adjustment vectors of each layer, the feature values ​​contained in the license plate feature map are adjusted to obtain the adaptive region attention feature maps of each layer.

3. The method according to claim 2, characterized in that, The process of extracting attention adjustment vectors for each layer based on the license plate feature map includes: Global pooling is performed on the feature maps at each height of the license plate feature map to obtain pooled feature vectors, which include pooled feature values ​​corresponding to each height of the license plate feature map. The pooled feature vectors are subjected to convolution processing at each corresponding layer to obtain the attention adjustment vectors at each layer of the license plate feature map; The step of adjusting the feature values ​​contained in the license plate feature map based on the attention adjustment vector of each layer to obtain the adaptive region attention feature map of each layer includes: For each layer, the attention adjustment coefficient corresponding to each height in the attention adjustment vector of the layer is fused with the feature value of the corresponding height in the license plate feature map to obtain the adaptive region attention feature map of the layer.

4. The method according to claim 1, characterized in that, The license plate image contains at least one character layer; generating a license plate feature map containing a specific number of character layers for the license plate image includes: The license plate image is filled to obtain a license plate image containing a specific number of character layers; Feature extraction is performed on the filled license plate image to obtain the license plate feature map.

5. The method according to claim 4, characterized in that, Before performing feature extraction on the license plate image, the process also includes: The license plate image is corrected, and / or the filled license plate image is adjusted to a preset size; and / or, The process of filling the license plate image to obtain a license plate image containing a specific number of character layers includes: In response to the license plate image not meeting the preset aspect ratio condition, the license plate image is filled in a preset direction using preset parameters to obtain the license plate image containing a specific number of character layers; and The method of generating a license plate feature map containing a specific number of character layers for a license plate image further includes: in response to the license plate image satisfying a preset aspect ratio condition, directly extracting features from the license plate image to obtain the license plate feature map.

6. The method according to claim 5, characterized in that, The preset aspect ratio condition is that the aspect ratio of the license plate image is a preset aspect ratio; the step of filling the license plate image in a preset direction using preset parameters to obtain the license plate image containing a specific number of character layers includes: For license plate images with an aspect ratio smaller than the preset aspect ratio, the preset parameters are used to fill the width direction of the license plate image to obtain a license plate image containing a specific number of character layers; For a license plate image with an aspect ratio greater than the preset aspect ratio, the preset parameters are used to fill the height direction of the license plate image to obtain a license plate image containing a specific number of character layers.

7. The method according to claim 1, characterized in that, The specific number of layers is two layers; And / or, The license plate recognition result obtained from the license plate image based on the character recognition results corresponding to each layer includes: The license plate recognition result is obtained by concatenating the character recognition results corresponding to each layer in the order of each layer.

8. The method according to claim 1, characterized in that, The character recognition results of each layer are obtained based on a license plate recognition model, which includes multiple character recognition modules. The method further includes the following steps to train the license plate recognition model: Obtain sample license plate images; Generate a sample license plate feature map containing a specific number of character layers for the sample license plate image, wherein the specific number of layers is greater than or equal to two, and the sample license plate image is labeled with the real characters contained in each of the specific number of layers; The multiple character recognition modules are used to perform character recognition on each layer of the sample license plate feature map to obtain the sample character recognition results for each layer of the sample license plate feature map. By utilizing the differences between the sample character recognition results and the real characters corresponding to each layer, the model parameters of the character recognition module of the corresponding layer are adjusted.

9. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory to implement the license plate recognition method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the license plate recognition method according to any one of claims 1 to 8.

Images