License Plate Cooperative Recognition Method and System Based on Semantic Feature Enhancement

Abstract

The present invention provides a license plate collaborative recognition method and system based on semantic feature enhancement, including: using a line number processing module to process standard license plates to obtain standard and non-standard license plate pairs; using a Resnet feature extraction network to perform feature extraction to obtain license plate visual features; using a sequence modeling network to discover and obtain license plate semantic features; using an encoder to perform single license plate recognition and license plate collaborative recognition respectively; according to the license plate semantic features, performing single license plate recognition operations to obtain single license plate recognition results, calculating losses and training optimizations for the single license plate recognition results to obtain applicable single license plate recognition results; performing collaborative recognition sequence modeling, and accordingly performing license plate collaborative recognition operations to obtain license plate collaborative recognition results; calculating losses and training optimizations for the license plate collaborative recognition results to obtain applicable license plate collaborative recognition results. The present invention solves the technical problems of low accuracy of license plate recognition results and poor recognition effects in specific scenarios.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a license plate collaborative recognition method and system based on semantic feature enhancement. Background Art

[0002] License plate recognition is an important task in the fields of computer vision and intelligent transportation management, and can be applied in multiple aspects such as vehicle management and traffic safety. Nowadays, the recognition methods for standard license plates have been relatively mature and widely applied. In addition, in order to address problems such as occlusion or pollution that may occur to the standard license plates of large vehicles, the traffic management department stipulates that non-standard license plates need to be sprayed on the rear of large vehicles, and their sizes are often several times that of standard license plates.

[0003] "Towards end-to-end license plate detection and recognition: A large dataset and baseline" proposed a large-scale standard license plate dataset and an end-to-end baseline method, which promoted the development of the standard license plate recognition field. Another example is the existing invention patent application document "A license plate recognition method and system in a complex scene" with publication number CN113378972A. The existing method includes: obtaining a license plate dataset as a training sample and preprocessing the obtained license plate dataset; inputting the license plate dataset into a recognition model for processing to obtain the recognition result of the recognition model; the license plate dataset includes a number of license plate pictures; inputting the recognition result into an evaluation model to obtain an evaluation result; wherein, the recognition model sequentially includes a downsampling module, a serialized feature extraction module, and an image enhancement module; the downsampling module is used to downsample the input license plate dataset and extract underlying features; the serialized feature extraction module is used to perform serialized feature extraction on the extracted underlying features; the image enhancement module is used to perform image enhancement on the extracted underlying features. However, for large vehicles, problems such as low resolution, pollution, and occlusion seriously affect the recognition effect of their standard license plates in the foregoing prior art.

[0004] "Disentangled Generation Network for Enlarged License Plate Recognition and A Unified Dataset" first recognized the research value of non-standard license plate recognition tasks, proposed a non-standard license plate dataset, and proposed a disentangled generation method to simulate and generate realistic non-standard license plate images to assist model training, achieving good results in non-standard license plate recognition. Another example is the existing invention patent application document "Multi-Type License Plate Recognition Method and System Based on Single-Character Attention" with publication number CN113554030A. The existing method includes the following steps: Step S1, collect license plate images through a camera, convert them into RGB data, and perform numerical normalization operations; Step S2, use a multi-scale feature extraction backbone network to perform multi-scale feature extraction and fusion on the license plate image to obtain a global feature map; Step S3, perform downsampling and upsampling based on single-character attention on the global feature map obtained in Step S2 to obtain a multi-channel mask image, and then perform single-character feature soft segmentation on the global feature based on the mask image to obtain independent single-character feature vectors; Step S4, use multiple single-character classifiers to perform single-character prediction classification on the multiple single-character feature vectors obtained in Step S3 and splice the classification results into a string in order and output; Step S5, perform database comparison according to the recognition results to achieve multi-type license plate recognition; The downsampling and upsampling based on single-character attention refer to: through multiple symmetrically arranged convolutional layers and transposed convolutional layers, mutual skip cascading is used to achieve downsampling and upsampling, and the independent attention regions of all single characters are predicted in parallel, realizing the decoupling between the attentions of each character, that is, the attention prediction of the latter character does not depend on the position information of the previous character. However, due to the lack of strict restrictions on the size, font, and layout of spray-painted non-standard license plates, and the background being more complex than standard license plates, problems such as wear will also occur during long-term use. The non-standard license plate recognition task is more challenging than standard license plate recognition.

[0005] In summary, in the existing work, the recognition of the two types of license plates is not regarded as an overall task, but as two independent problems, ignoring the fact that the two license plates of the same vehicle express the same license plate number, resulting in the technical problems of relatively low accuracy of license plate recognition results and poor recognition effects in specific scenarios in the existing technology. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: how to solve the technical problems of relatively low accuracy of license plate recognition results and poor recognition effects in specific scenarios in the existing technology.

[0007] The present invention solves the above technical problems by adopting the following technical solutions: The license plate collaborative recognition method based on semantic feature enhancement includes:

[0008] S1. Design a collaborative recognition network CRNet, including: a line number processing module, a Resnet feature extraction network, a sequence modeling network, and a decoder;

[0009] S2. Use the line number processing module to process the standard license plate to obtain single-line text data, so as to obtain standard and non-standard license plate pairs;

[0010] S3. Use the Resnet feature extraction network to extract features from the original feature maps in the standard and non-standard license plate pairs to obtain license plate visual features;

[0011] S4. Send the license plate visual features into the sequence modeling network to explore the semantic information in the license plate visual features, and accordingly obtain license plate semantic features;

[0012] S5. According to the license plate semantic features, use the encoder to perform single license plate recognition operations and license plate collaborative recognition operations respectively;

[0013] S6. According to the license plate semantic features, perform single license plate recognition operations to obtain single license plate recognition results, calculate the loss and perform training optimization on the single license plate recognition results to obtain applicable single license plate recognition results;

[0014] S7. According to the license plate semantic features, perform collaborative recognition sequence modeling, and accordingly perform license plate collaborative recognition operations to obtain license plate collaborative recognition results;

[0015] S8. Calculate the loss and perform training optimization on the license plate collaborative recognition results to obtain applicable license plate collaborative recognition results.

[0016] In the existing work, the recognition of the two types of license plates is not regarded as an overall task, but as two independent problems, ignoring the fact that the two license plates of the same vehicle express the same license plate number. The present invention designs a collaborative recognition network CRNet, which includes three parts: a line number processing module, feature extraction, and a decoder block. Using the collaborative recognition network CRNet for license plate collaborative recognition can simultaneously utilize the information provided by the standard license plate and the enlarged license plate of the vehicle, so as to more accurately recognize the license plate number.

[0017] In a more specific technical solution, step S2 includes:

[0018] S21. Collect the image of the standard license plate;

[0019] S22. Use the line number processing module to determine whether the standard license plate is a single-line license plate;

[0020] S23. If not, perform cropping, scaling, and recombination operations on the image of the standard license plate to process and obtain single-line text data;

[0021] S24. If so, directly output the image of the standard license plate.

[0022] In a more specific technical solution, step S3 includes:

[0023] S31. Send the standard and non-standard license plate pairs into the Resnet feature extraction network to obtain a scaled-down feature map;

[0024] S32. Through average pooling operation, compress the height of the scaled-down feature map to a preset dimension to obtain visual features (V standard , V enlarged ), where the subscript standard represents the features extracted from the standard license plate picture, and the subscript enlarged represents the features extracted from the non-standard license plate picture.

[0025] In a more specific technical solution, in step S32, compress the height of the scaled-down feature map to 1 dimension to obtain visual features:

[0026] (V standard , V enlarged )

[0027] where the subscript standard represents the features extracted from the standard license plate picture, and the subscript enlarged represents the features extracted from the non-standard license plate picture.

[0028] In a more specific technical solution, step S4 includes:

[0029] S41. Send the license plate visual features (V standard , V enlarged ) into the sequence modeling network biLSTM;

[0030] S42. Use the sequence modeling network biLSTM to capture the long-range dependence relationships in the license plate visual features, aggregate the features corresponding to each character, and explore semantic information to generate semantic features (S standard , S enlarged ).

[0031] The present invention uses biLSTM multiple times to explore, enhance, and fuse semantic information from visual features because this network can establish long-span connections at the feature level, integrate the fragmented local character features in the visual features, and form feature representations of each complete character. The present invention fully explores and fuses the semantic information in the two license plate features through two-step progressive semantic enhancement, achieving the effect of more accurately identifying the license plate number by combining the two license plate information.

[0032] In a more specific technical solution, in step S5, the license plate semantic features are sent to no less than two decoders, and each decoder includes: a semantic enhancement branch and an auxiliary learning branch. Step S5 includes:

[0033] S51. Using the auxiliary learning branch, identify the valid information in the image of the single license plate;

[0034] S52. Using the semantic enhancement branch, establish a character feature correspondence relationship among license plate semantic features of different types, screen and fuse the valid information of the license plate semantic features in the standard and non-standard license plate pairs for additional attention decoding.

[0035] The decoder module adopted by the present invention includes a semantic enhancement branch and an auxiliary learning branch. Among them, the auxiliary learning branch identifies the visible characters on each license plate picture, ensuring that the extracted features fully contain the character information on the license plate.

[0036] The present invention uses the semantic enhancement branch to screen and fuse the valid parts of the two license plate features for additional attention decoding, and selects the interesting parts from the enhanced features with rich semantic information, achieving the purpose of identifying by combining two types of license plates, thereby improving the accuracy of the license plate recognition result.

[0037] In a more specific technical solution, step S7 includes:

[0038] S71. Concatenate the license plate semantic features (S standard , S enlarged ) and the license plate visual features (V standard , V enlarged ) of the standard license plate and the non-standard license plate respectively to obtain semantic concatenation features and visual concatenation features (C standard , C enlarged );

[0039] S72. Send the semantic concatenation features and the visual concatenation features (C standard , C enlarged ) into the attention decoder for recognition;

[0040] S73. Send the semantic concatenation features into the second sequence modeling network biLSTM to enhance and fuse the semantic information of the standard license plate and the non-standard license plate to obtain the fused semantic feature S collaborate ;

[0041] S74. Concatenate the fused semantic feature S collaborate and the visual features (V standard , V enlarged ) to obtain the hybrid feature C collaborate , and send it into the attention decoder for prediction to obtain the license plate collaborative recognition result Y collaborate。

[0042] The present invention applies this network to the process of fusing the features of two types of license plates. Based on this network, it can establish the connection of the similar semantic information part of the features of the two types of license plates according to the common characters, character distributions, etc. of the two types of license plates, and finally integrate them into a semantic feature sequence representing the complete license plate number. The present invention can effectively combine the features of two license plate images for more accurate recognition.

[0043] In a more specific technical solution, in step S74, the attention decoder is used to perform autoregressive prediction on the visual splicing features (C standard , C enlarged ) of the standard license plate and the non-standard license plate respectively to obtain the prediction results (Y standard , Y enlarged ) of the standard license plate and the non-standard license plate.

[0044] In a more specific technical solution, the network structure adopted by the attention decoder includes: RNN.

[0045] In a more specific technical solution, the license plate collaborative recognition system based on semantic feature enhancement includes:

[0046] The recognition network design module is used to design the collaborative recognition network CRNet, including: the line number processing module, the Resnet feature extraction network, the sequence modeling network, and the decoder;

[0047] The line number processing module is used to process the standard license plate to obtain single-line text data to obtain the standard and non-standard license plate pairs;

[0048] The Resnet feature extraction network is used to extract features from the original feature maps in the standard and non-standard license plate pairs to obtain the license plate visual features. The feature extraction module is connected to the line number processing module;

[0049] The license plate semantic feature discovery module is used to send the license plate visual features into the sequence modeling network to discover the semantic information in the license plate visual features, and accordingly obtain the license plate semantic features. The license plate semantic feature discovery module is connected to the feature extraction module;

[0050] The recognition branch module is used to perform single license plate recognition operations and license plate collaborative recognition operations respectively using the encoder according to the license plate semantic features. The recognition branch module is connected to the license plate semantic feature discovery module;

[0051] The single license plate recognition module is used to perform single license plate recognition operations according to the license plate semantic features to obtain the single license plate recognition results, calculate the loss and perform training optimization on the single license plate recognition results to obtain the applicable single license plate recognition results. The single license plate recognition module is connected to the recognition branch module;

[0052] The license plate collaborative recognition module is used to perform collaborative recognition sequence modeling based on the semantic features of license plates, and accordingly perform license plate collaborative recognition operations to obtain license plate collaborative recognition results. The license plate collaborative recognition module is connected to the recognition branch module;

[0053] The collaborative recognition result optimization module is used to calculate losses and perform training optimization on the license plate collaborative recognition results to obtain applicable license plate collaborative recognition results. The collaborative recognition result optimization module is connected to the license plate collaborative recognition module.

[0054] The present invention has the following advantages compared with the prior art: In the existing work, the recognition of two types of license plates is not regarded as an overall task, but as two independent problems, ignoring the fact that the two license plates of the same vehicle represent the same license plate number. The present invention designs a collaborative recognition network CRNet, which includes a line number processing module, a feature extraction module, and a decoder block. By using the collaborative recognition network CRNet for license plate collaborative recognition, the information provided by the standard license plate and the enlarged license plate of the vehicle can be utilized simultaneously, so as to more accurately recognize the license plate number.

[0055] The present invention repeatedly uses biLSTM to discover, enhance, and fuse semantic information from visual features, because this network can establish long-span connections at the feature level, integrate the fragmented local character features in the visual features, and form feature representations of each complete character. The present invention fully explores and fuses the semantic information in the two license plate features through two-step progressive semantic enhancement, achieving the effect of combining the two license plate information for more accurate license plate number recognition.

[0056] The decoder module adopted by the present invention includes a semantic enhancement branch and an auxiliary learning branch. Among them, the auxiliary learning branch recognizes the visible characters on each license plate image, ensuring that the extracted features fully contain the character information on the license plate.

[0057] The present invention uses the semantic enhancement branch to screen and fuse the effective parts of the two license plate features, perform additional attention decoding, and select the interested parts from the enhanced features with rich semantic information, achieving the purpose of combining the two license plates for recognition, thereby improving the accuracy of the license plate recognition results.

[0058] The present invention applies this network to the process of fusing the two license plate features. It is also based on the fact that this network can establish connections for the similar semantic information parts of the two license plate features according to information such as the common characters and character distributions of the two license plates, and finally integrate them into a semantic feature sequence representing the complete license plate number. The present invention can effectively combine the features of the two license plate images for more accurate recognition.

[0059] The present invention solves the technical problems of low accuracy of license plate recognition results and poor recognition effects in specific scenarios existing in the prior art. Description of the Drawings

[0060] Figure 1 It is a schematic diagram of the basic steps of the license plate collaborative recognition method based on semantic feature enhancement in Embodiment 1 of the present invention;

[0061] Figure 2a It is a structural diagram of the network model training stage of the license plate collaborative recognition method based on semantic feature enhancement in Embodiment 1 of the present invention;

[0062] Figure 2b It is a structural diagram of the network model testing stage of the license plate collaborative recognition method based on semantic feature enhancement in Embodiment 1 of the present invention;

[0063] Figure 3 It is a schematic diagram of the specific steps of the license plate collaborative recognition in Embodiment 1 of the present invention;

[0064] Figure 4 It is a schematic diagram of the basic steps of the test operation in Embodiment 1 of the present invention;

[0065] Figure 5 It is a comparison diagram of the experimental results of this method and other methods in Embodiment 2 of the present invention. Detailed Embodiments

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0067] Embodiment 1

[0068] As Figure 1 shown, the license plate collaborative recognition method based on semantic feature enhancement provided by the present invention includes the following basic steps:

[0069] Step S1, using a line number processing module to process a standard license plate to obtain single-text-line data;

[0070] In this embodiment, the standard license plate image is sent into the line number processing module. This module determines whether the standard license plate is single-line or double-line, and crops, scales, and reorganizes the double-line standard license plate image according to preset rules, processes it into the style of a single text line and outputs it. This module does not perform any processing on the standard license plate image that is originally single-line and directly outputs it.

[0071] Step S2: Use the Resnet feature extraction network to extract features;

[0072] In this embodiment, the paired standard license plates and non-standard license plates processed in the foregoing step S1 are sent into the Resnet feature extraction network. After the network extracts features, the height of the feature map is reduced to 1 / 16 of the original, and the width is reduced to 1 / 4 of the original. Finally, the height of the feature map is compressed to one dimension through average pooling to obtain visual features (V standard ,V enlarged ), where the subscript standard indicates the features extracted from the standard license plate image, and the subscript enlarged indicates the features extracted from the non-standard license plate image.

[0073] Step S3: Sequence modeling;

[0074] In this embodiment, the visual features (V standard ,V enlarged ) are sent into the biLSTM sequence modeling network. The sequence modeling network discovers the semantic information in the visual features to obtain semantic features (S standard ,S enlarged ).

[0075] Step S4: Single license plate recognition;

[0076] Step S5: Calculate the loss and perform training optimization on the single license plate recognition result;

[0077] In this embodiment, using the following formula logic, for the recognition results Y sstandard 、Y enlarged , and the collaborative recognition result Y collaborate , respectively and the true values Y standard 、Y enlarged 、Y collaborate calculate the CE loss and sum them up. Take the sum result as the recognition loss L rec :

[0078]

[0079] In the formula, α controls the weight of the collaborative recognition loss, and here α = 1.0 is taken.

[0080] Step S6: Collaborative recognition sequence modeling;

[0081] Step S7: License plate collaborative recognition;

[0082] The license plate sends the semantic features (S standard ,S enlarged ) into the decoder module, including: 5 cascaded decoders, and each decoder contains two branches, which respectively complete the tasks of single license plate recognition and collaborative recognition.

[0083] In this embodiment, the decoder module includes, but is not limited to, a semantic enhancement branch and an auxiliary learning branch. The auxiliary learning branch identifies the characters in a single license plate image to ensure that the features of a single license plate image contain accurate semantic information. The semantic enhancement branch filters and fuses the effective parts of the two types of license plate features, performs additional attention decoding, and obtains the recognition result from the enhanced semantic information features.

[0084] As Figure 2a and Figure 2b shown, in this embodiment, standard license plate images and non-standard license plate images are respectively sent into the backbone network to extract features, obtaining visual features V standard 、V enlarged . Among them, before the standard license plate image is sent into the backbone network, it will first pass through a single / double-line processing module, which will distinguish whether the structure of the standard license plate is single-line or double-line, and crop, deform, and reconstruct the double-line standard license plate image into a single-line standard license plate image. The visual features are sent into 5 consecutive decoding blocks. In this embodiment, each decoding block includes, but is not limited to, a single license plate recognition block, a collaborative license plate recognition block, and a biLSTM layer.

[0085] In this embodiment, after the visual features are processed by biLSTM to extract semantic information, semantic features S standard 、S enlarged are obtained. Then the visual features and semantic features will be sent into two parallel modules. In the single license plate recognition module, the visual features and semantic features extracted from the same license plate image are concatenated in the feature dimension to obtain C standard 、C enlarged , and then sent into the attention decoder to recognize the recognition results Y standard 、Y enlarged of a single license plate. In the collaborative recognition module, the semantic features from the two types of license plates are first concatenated in the feature dimension, and after being filtered and fused by an additional biLSTM layer for their semantic information, they are concatenated with the visual features of the two types of license plates to obtain C collaborate and sent into the attention decoder to obtain the collaborative recognition result Y collaborate . During the test process, the single license plate recognition module is not used, and the final recognition result is obtained only through one branch of the collaborative recognition module.

[0086] As Figure 3 shown, in this embodiment, step S7 of license plate collaborative recognition includes the following specific steps:

[0087] Step S71: Concatenate the semantic features (S standard , S enlarged ) and visual features (V standard,V enlarged ) are respectively spliced to obtain the features (C standard ,C enlarged ), and are sent to an attention-based decoder for recognition;

[0088] In this embodiment, the attention-based decoder used is an RNN-like structure, which makes predictions in an autoregressive manner. For each time step t, the previously inferred character y t-1 is input, combined with the current hidden state h t . The decoder searches for the region of interest corresponding to the next character in the feature sequence and outputs the next character prediction y t , and updates the hidden state h t+1 . The attention-based decoder makes predictions on the features (C standard ,C enlarged ) respectively. After the prediction is completed, two prediction results (Y standard ,Y enlarged ) are output, which are the results of independent recognition of the visible characters on the two types of license plates respectively.

[0089] Step S72: Splice the semantic features (S standard ,S enlarged ) and send them into an additional biLSTM sequence modeling network to enhance and fuse the semantic information from the two types of license plates, obtaining the fused semantic feature S collaborate ;

[0090] Step S73: Splice S collaborate and the visual features (V standard ,V enlarged ) to obtain the hybrid feature C collaborate , and send it into the attention-based decoder with the same structure as in the previous step S71 for prediction, obtaining the collaborative recognition result Y collaborate .

[0091] S8: Loss calculation and training optimization.

[0092] As Figure 4 shown, the license plate collaborative recognition method based on semantic feature enhancement provided by the present invention includes the following basic test steps in the test phase:

[0093] S1’: Perform row number processing operations using the row number processing module;

[0094] S2’: Feature extraction;

[0095] S3’: First sequence modeling;

[0096] S4’: Second sequence modeling;

[0097] S5’: License plate collaborative recognition;

[0098] S6', Recognition result output.

[0099] In this embodiment, the testing process is basically the same as the training process, except that in the testing process, the collaborative recognition result is only output through the aforementioned step S52, and the aforementioned step S51 is removed. This step is only used for auxiliary training, and the testing process does not require the recognition of a single license plate.

[0100] Embodiment 2

[0101] As Figure 5 shown, the present invention has achieved an improvement in the recognition rate for the above 4 typical scenario text recognition methods based on independent recognition of standard license plates and enlarged license plates, and for directly splicing features. This fully demonstrates that our method has better performance in the license plate recognition task compared to treating the recognition of the two types of license plates as independent tasks, or independently recognizing the two types of license plates and then using some post-processing methods to combine the prediction results.

[0102] In summary, in the existing work, the recognition of the two types of license plates is not regarded as an overall task, but as two independent problems, ignoring the fact that the two types of license plates of the same vehicle express the same license plate number. The present invention designs a collaborative recognition network CRNet, which includes a line number processing module, feature extraction, and a decoder block. By using the collaborative recognition network CRNet for license plate collaborative recognition, the information provided by the standard license plate and the enlarged license plate of the vehicle can be utilized simultaneously, thereby more accurately recognizing the license plate number.

[0103] The present invention repeatedly uses biLSTM to explore, enhance, and fuse semantic information from visual features because this network can establish long-span connections at the feature level, integrating the fragmented local character features in the visual features to form a feature representation of each complete character. The present invention fully explores and fuses the semantic information in the two types of license plate features through two-step progressive semantic enhancement, achieving the effect of combining the information of the two types of license plates for more accurate recognition of the license plate number.

[0104] The decoder module adopted by the present invention includes a semantic enhancement branch and an auxiliary learning branch. Among them, the auxiliary learning branch guarantees that the extracted features fully contain the character information on the license plate by recognizing the visible characters on each license plate picture.

[0105] The present invention uses the semantic enhancement branch to screen and fuse the effective parts of the two types of license plate features, perform additional attention decoding, and select the interesting parts from the enhanced features with rich semantic information, achieving the purpose of combining the two types of license plates for recognition, thereby improving the accuracy of the license plate recognition result.

[0106] The present invention applies this network to the process of fusing the features of two types of license plates. Based on this network, it can establish the connection of the similar semantic information part of the features of the two types of license plates according to the common characters, character distributions, etc. of the two types of license plates, and finally integrate them into a semantic feature sequence representing the complete license plate number. The present invention can effectively combine the features of two license plate images for more accurate recognition.

[0107] The present invention solves the technical problems existing in the prior art, such as the low accuracy of license plate recognition results and the poor recognition effect in specific scenarios.

[0108] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A license plate collaborative recognition method based on semantic feature enhancement, characterized in that The method includes: S1. Design a collaborative recognition network CRNet, including: a line number processing module, a Resnet feature extraction network, a sequence modeling network, and a decoder; S2. Use the line number processing module to process the standard license plate to obtain single-line text data, so as to obtain standard and non-standard license plate pairs; S3. Use the Resnet feature extraction network to extract features from the original feature maps in the standard and non-standard license plate pairs to obtain license plate visual features; S4. Send the license plate visual features into the sequence modeling network to discover the semantic information in the license plate visual features, and accordingly obtain license plate semantic features; S5. According to the license plate semantic features, use the decoder to perform single license plate recognition operations and license plate collaborative recognition operations respectively; S6. According to the license plate semantic features, perform the single license plate recognition operation to obtain a single license plate recognition result, calculate the loss and perform training optimization on the single license plate recognition result to obtain an applicable single license plate recognition result; S7. According to the license plate semantic features, perform collaborative recognition sequence modeling, and accordingly perform the license plate collaborative recognition operation to obtain a license plate collaborative recognition result; S8. Perform the loss calculation and the training optimization on the license plate collaborative recognition result to obtain an applicable license plate collaborative recognition result.

2. The license plate collaborative recognition method based on semantic feature enhancement according to claim 1, wherein The step S2 includes: S21. Collect an image of the standard license plate; S22. Use the line number processing module to determine whether the standard license plate is a single-line license plate; S23. If not, perform cropping, scaling, and recombination operations on the image of the standard license plate to process and obtain the single-line text data; S24. If so, directly output the image of the standard license plate.

3. The license plate collaborative recognition method based on semantic feature enhancement according to claim 1, wherein The step S3 includes: S31. Send the standard and non-standard license plate pairs into the Resnet feature extraction network to obtain a scaled feature map; S32. Compress the height of the downscaled feature map to a preset dimension through average pooling operation, so as to obtain the visual feature (V standard , V enlarged ), where the subscript standard represents the feature extracted from the standard license plate image, and the subscript enlarged represents the feature extracted from the non-standard license plate image.

4. The license plate collaborative recognition method based on semantic feature enhancement according to claim 3, wherein In the step S32, compress the height of the scaled feature map to 1 dimension to obtain the visual feature: (V standard ,V enlarged ) Where the subscript standard represents the feature extracted from the standard license plate picture, and the subscript enlarged represents the feature extracted from the non-standard license plate picture.

5. The license plate collaborative recognition method based on semantic feature enhancement according to claim 1, wherein The step S4 includes: S41. Feed the license plate visual features (V standard , V enlarged ) into the sequence modeling network biLSTM; S42. Use the sequence modeling network biLSTM to capture the long-range dependencies in the license plate visual features, aggregate the features corresponding to each character, discover the semantic information, and generate the semantic features based on this (S standard , S enlarged ).

6. The license plate collaborative recognition method based on semantic feature enhancement according to claim 1, wherein In the step S5, send the license plate semantic features into no less than 2 preset decoders. Each preset decoder includes: a semantic enhancement branch and an auxiliary learning branch. The step S5 includes: S51. Use the auxiliary learning branch to identify the valid information in the image of the single license plate; S52. Use the semantic enhancement branch to establish a character feature correspondence relationship among different types of license plate semantic features, screen and fuse the valid information of the license plate semantic features in the standard and non-standard license plate pairs to perform additional attention decoding.

7. The license plate collaborative recognition method based on semantic feature enhancement according to claim 1, wherein The step S7 includes: S71. Concatenate the license plate semantic features (S standard , S enlarged ) and the license plate visual features (V standard , V enlarged ) of the standard license plate and the non-standard license plate respectively to obtain semantic concatenation features and visual concatenation features (C standard , C enlarged ); S72. Feed the semantic splicing feature and the visual splicing feature (C standard , C enlarged ) into the attention decoder for recognition; S73. Feed the semantic splicing feature into the second sequence modeling network biLSTM to enhance and fuse the semantic information of the standard license plate and the non-standard license plate, and obtain the fused semantic feature S collaborate ; S74. Concatenate the fused semantic feature S collaborate and the visual feature (V standard , V enlarged ) to obtain the hybrid feature C collaborate , and send it to the attention decoder for prediction to obtain the license plate collaborative recognition result Y collaborate .

8. The license plate collaborative recognition method based on semantic feature enhancement according to claim 7, characterized in that In the step S74, the attention decoder is used to perform autoregressive prediction on the visual splicing features (C standard , C enlarged ) of the standard license plate and the non-standard license plate respectively to obtain the prediction results (Y standard , Y enlarged ) of the standard license plate and the non-standard license plate.

9. The license plate collaborative recognition method based on semantic feature enhancement according to claim 7, characterized in that, The network structure adopted by the attention decoder includes: RNN.

10. A license plate collaborative recognition system based on semantic feature enhancement, characterized in that, The system includes: A recognition network design module for designing a collaborative recognition network CRNet, including: a line number processing module, a Resnet feature extraction network, a sequence modeling network, and a decoder; The line number processing module is used to process standard license plates to obtain single text line data and acquire standard and non-standard license plate pairs; The Resnet feature extraction network is used to extract features from the original feature maps in the standard and non-standard license plate pairs to obtain license plate visual features. The Resnet feature extraction network is connected to the line number processing module; The license plate semantic feature discovery module is used to send the license plate visual features into the sequence modeling network to discover the semantic information in the license plate visual features, thereby obtaining license plate semantic features. The license plate semantic feature discovery module is connected to the Resnet feature extraction network; The recognition branch module is used to perform single license plate recognition operations and license plate collaborative recognition operations respectively using a decoder based on the license plate semantic features. The recognition branch module is connected to the license plate semantic feature discovery module; The single license plate recognition module is used to perform the single license plate recognition operation based on the license plate semantic features to obtain a single license plate recognition result, calculate the loss and optimize the training for the single license plate recognition result to obtain an applicable single license plate recognition result. The single license plate recognition module is connected to the recognition branch module; The license plate collaborative recognition module is used to perform collaborative recognition sequence modeling based on the license plate semantic features, thereby performing the license plate collaborative recognition operation to obtain a license plate collaborative recognition result. The license plate collaborative recognition module is connected to the recognition branch module; The collaborative recognition result optimization module is used to perform the loss calculation and training optimization on the license plate collaborative recognition result to obtain an applicable license plate collaborative recognition result. The collaborative recognition result optimization module is connected to the license plate collaborative recognition module.

Images