High-value license plate character labeling and auditing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of image recognition, and discloses a high-value license plate character labeling and auditing method and device, equipment and a storage medium, and the method comprises the steps: inputting a vehicle video stream into a vehicle detection model, and determining a plurality of license plate pictures corresponding to the same vehicle; respectively inputting the plurality of license plate pictures into a lightweight character recognition model, and determining and judging whether the license plate pictures are consistent with first recognition results corresponding to the license plates or not; in response to inconsistency of the first recognition result, inputting the plurality of license plate pictures as high-value license plate pictures into a plurality of character recognition models, and determining a second recognition result corresponding to the high-value license plate pictures; and performing auditing based on the second identification result, and determining a license plate character labeling result of the vehicle. According to the scheme of the invention, the recognition of the high-value license plate is completed through the character recognition model, on the basis of completing the recognition, the participation of artificial factors is reduced, the license plate character labeling efficiency and labeling yield are improved, and the accuracy of character labeling is ensured through an auditing mode.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a method, device, equipment, and storage medium for annotating and auditing high-value license plate characters. Background Art

[0002] License plate character recognition is widely used in the field of intelligent transportation. The model training of license plate character recognition requires a large amount of annotated data. The current license plate annotation scheme is to first record videos and then annotate all license plates in the pictures of the recorded videos. In this process, annotators need to annotate license plate characters one by one. In addition, clear large-size license plates have little improvement on the accuracy of the model during the incremental training process of the model. The annotation of license plates that the model has difficulty in correctly recognizing characters is more valuable. License plates that the model has difficulty in correctly recognizing characters are herein referred to as high-value license plates.

[0003] In the related art, since the model has difficulty in recognizing high-value license plates, usually annotators intervene as human factors. The annotators annotate license plate characters and determine high-value license plates according to the annotation results. However, the proficiency of each annotator will affect the annotation efficiency, accuracy, and annotation output of license plate characters. Therefore, due to the influence of human factors, it is difficult to have a unified standard for the recognition and auditing of high-value license plates, and it is impossible to ensure the accuracy of the annotation results of high-value license plate characters. Summary of the Invention

[0004] In view of this, the present invention provides a method, device, equipment, and storage medium for annotating and auditing high-value license plate characters to solve the technical problem of how to recognize and audit high-value license plates.

[0005] In a first aspect, the present invention provides a method for annotating and auditing high-value license plate characters. The method includes: inputting a vehicle video stream into a vehicle detection model to determine multiple license plate pictures corresponding to the same vehicle; respectively inputting the multiple license plate pictures into a lightweight character recognition model to determine first recognition results respectively corresponding to the multiple license plate pictures, and determining whether the first recognition results corresponding to the license plates are consistent; in response to the first recognition results being inconsistent, respectively inputting the multiple license plate pictures as high-value license plate pictures into multiple character recognition models to determine second recognition results corresponding to the high-value license plate pictures; and performing auditing based on the second recognition results to determine the license plate character annotation result of the vehicle.

[0006] In combination with the first aspect, in a possible implementation manner of the first aspect, performing auditing based on each second recognition result to determine the license plate character annotation result of the vehicle includes: in response to each second recognition result being consistent, using the second recognition result as an automatic annotation result; and auditing the automatic annotation result to determine the license plate character annotation result of the vehicle.

[0007] Combined with the first aspect, in a possible implementation manner of the first aspect, based on each second recognition result, the license plate character annotation result of the vehicle is determined, including: in response to the inconsistency of each second recognition result, selecting the second recognition result with a high confidence level as the automatic annotation result; auditing the automatic annotation result to determine the license plate character annotation result of the vehicle.

[0008] Combined with the first aspect, in a possible implementation manner of the first aspect, auditing the automatic annotation result to determine the license plate character annotation result of the vehicle, including: based on the automatic annotation result, determining the hierarchical information, color information, character length information, separator information, and first character information of the license plate; in response to the hierarchical information, color information, character length information, separator information, and first character information all conforming to the corresponding audit rules, taking the automatic annotation result as the license plate character annotation result of the vehicle.

[0009] Combined with the first aspect, in a possible implementation manner of the first aspect, the training process of the vehicle detection model includes: obtaining the image to be detected and dividing the image to be detected into multiple blocks; based on the attention mechanism, splicing each block to form multiple spliced images; dividing each spliced image into multiple parts at a preset ratio, and using residuals and re-parameters to complete the splicing of the multiple parts to form a feature image; adjusting the parameters of the vehicle detection model based on the feature image until the model converges.

[0010] Combined with the first aspect, in a possible implementation manner of the first aspect, based on the attention mechanism, splicing each block to form multiple spliced images, including: performing self-attention calculation on each block to determine the calculation result corresponding to each block; adjusting each calculation result to the size of the corresponding block; based on the position of each block in the image to be detected, replacing the adjusted calculation result with the corresponding block to form a spliced image corresponding to the block; summarizing the spliced images corresponding to each block to form multiple spliced images.

[0011] Combined with the first aspect, in a possible implementation manner of the first aspect, dividing each spliced image into multiple parts at a preset ratio, and using residuals and re-parameters to complete the splicing of the multiple parts to form a feature image, including: dividing each spliced image at a preset ratio to determine the first part, the second part, and the third part; performing residual calculation on the second part to determine the residual calculation result; performing re-parameter calculation on the third part to determine the re-parameter calculation result; splicing the first part, the residual calculation result, and the re-parameter calculation result to form a feature image.

[0012] In a second aspect, the present invention provides a device for labeling and auditing high-value license plate characters. The device includes: a picture determination module, configured to input a vehicle video stream into a vehicle detection model to determine a plurality of license plate pictures corresponding to the same vehicle; a judgment module, configured to input the plurality of license plate pictures into a lightweight character recognition model respectively to determine first recognition results corresponding to the plurality of license plate pictures respectively, and judge whether the first recognition results corresponding to the license plates are consistent; a recognition determination module, configured to, in response to the inconsistency of the first recognition results, input the plurality of license plate pictures as high-value license plate pictures into a plurality of character recognition models respectively to determine second recognition results corresponding to the high-value license plate pictures; and a labeling determination module, configured to perform auditing based on the second recognition results to determine a license plate character labeling result of the vehicle.

[0013] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the method for labeling and auditing high-value license plate characters according to the first aspect or any corresponding embodiment thereof.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored. The computer instructions are used to cause a computer to execute the method for labeling and auditing high-value license plate characters according to the first aspect or any corresponding embodiment thereof.

[0015] The technical solution of the present invention has the following advantages: The method, device, equipment, and storage medium for labeling and auditing high-value license plate characters provided by the present invention input a video stream into a vehicle detection model to determine a plurality of license plate pictures corresponding to a vehicle, and complete the recognition of high-value license plates according to the license plate pictures and a lightweight character recognition model. On the basis of completing the recognition of high-value license plates, the second recognition results recognized by a trained character recognition model are audited to complete the character labeling in the high-value license plates. In this process, the vehicle pictures are recognized by a trained vehicle detection model, and then the license plate pictures are determined. The lightweight character recognition model is used to complete the recognition of high-value license plates, so that high-value license plates can be accurately recognized. On the basis of completing the recognition of high-value license plates, a plurality of trained character recognition models are used to complete the recognition of characters in the high-value license plates. Thus, by auditing the recognition results, the license plate character labeling result of the vehicle is determined, and the participation of human factors is reduced in both the recognition and auditing processes of high-value license plates. This method determines how to use the model to complete the recognition of high-value license plates and the auditing mechanism of the recognition results, ensuring the accuracy of the license plate character labeling result of the vehicle. And because the recognition and auditing of high-value license plates have a unified standard, this process can be completed using the model, further improving the license plate character labeling efficiency and labeling output. Brief Description of the Drawings

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 is a schematic flowchart of a method for labeling and auditing high-value license plate characters provided according to an embodiment of the present invention; Figure 2 is a schematic flowchart of image feature extraction provided according to an embodiment of the present invention; Figure 3 is a structural block diagram of a device for labeling and auditing high-value license plate characters provided according to an embodiment of the present invention; Figure 4 is a schematic hardware structure diagram of a computer device according to an embodiment of the present invention. Specific Embodiments

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0019] According to an embodiment of the present invention, an embodiment of a method for labeling and auditing high-value license plate characters is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0020] This embodiment provides a method for labeling and auditing high-value license plate characters. As Figure 1 shown, the method includes the following steps: S101. Input the vehicle video stream into the vehicle detection model to determine multiple license plate pictures corresponding to the same vehicle.

[0021] Specifically, the vehicle video stream can be a pre-recorded intersection traffic video stream containing moving vehicles, a traffic video stream collected by roadside sensing devices or other devices, or other video streams.

[0022] Specifically, during the training process of the vehicle detection model, the image to be detected is divided into multiple blocks, and the attention mechanism is used to splice each block to form multiple spliced images. On this basis, each spliced image is divided into multiple parts at a preset ratio, and through residuals and re-parameters, the splicing is completed to form a feature image. Therefore, after the vehicle video stream is framed and input into the vehicle detection model, multiple license plate images corresponding to the same vehicle can be obtained. It should be understood that by dividing the image to be detected into multiple blocks and forming multiple spliced images, the vehicle detection model pays more attention to the local feature information of the foreground and has a smaller computational amount compared to calculating the attention for the entire image; and by dividing each spliced image into multiple parts at a preset ratio and using the process of residuals and re-parameters to form a feature image, the model can calculate residuals and re-parameters in parallel, enhancing the feature extraction efficiency and further enhancing the model detection accuracy of the vehicle detection model. For example, vehicle a is included in the first frame to the third frame of the video. After the first frame to the third frame of the video are input into the vehicle detection model, license plate images corresponding to vehicle a are obtained.

[0023] S102. Input the multiple license plate images into the lightweight character recognition model respectively, determine the first recognition results corresponding to the multiple license plate images respectively, and determine whether the first recognition results corresponding to the license plates are consistent.

[0024] Specifically, inputting the multiple license plate images into the lightweight character recognition model respectively and determining the first recognition results corresponding to the multiple license plate images respectively means inputting the recognized license plate images into the lightweight character recognition model respectively to determine the first recognition results corresponding to each license plate image. Taking the first frame to the third frame of the video as an example again, the license plate image corresponding to the first frame of the video is denoted as a1, the license plate image corresponding to the second frame of the video is denoted as a2, and the license plate image corresponding to the third frame of the video is denoted as a3. The first recognition result corresponding to a1 can be X132C, the first recognition result corresponding to a2 can be X132C, and the first recognition result corresponding to a3 can be X123C. Among them, the lightweight character recognition model refers to a character recognition model with a fast response speed, low deployment cost, and less resource occupation compared to the following character recognition models. In this embodiment, no specific limitation is made on the specific lightweight character recognition model to be selected.

[0025] Specifically, determining whether the first recognition results corresponding to license plates are consistent means determining whether the recognized characters and their orders in each first recognition result are the same. Thus, when the first recognition results are inconsistent, the identification of high-value license plate images is completed. Similarly, taking the first recognition result corresponding to a1 as X132C, the first recognition result corresponding to a2 as X132C, and the first recognition result corresponding to a3 as X123C as an example, the recognition results of a1 and a2 are the same, the recognition results of a1, a2 and a3 are different, and the license plate images corresponding to a1, a2, and a3 are the license plate images corresponding to vehicle a. Therefore, the license plate image corresponding to vehicle a is used as a high-value image and is used to determine the second recognition result subsequently.

[0026] In the embodiment of the present disclosure, during the process of screening high-value license plates using a lightweight character recognition model, due to the above characteristics of the lightweight character model, the screening process of high-value license plates can have a faster response speed, reduce the occupation of system resources during the license plate screening process, and reduce the data calculation amount for subsequent recognition and further review of high-value license plates. At the same time, using the trained lightweight character recognition model does not require considering the training process of the character recognition model, that is, license plate annotation is not required during the screening process of high-value license plates, avoiding the intervention of annotators as a human factor, thereby further improving the efficiency of screening high-value license plates, enabling high-value license plates to be accurately recognized, and providing a data basis for subsequent further recognition of high-value license plates.

[0027] S103. In response to the inconsistency of the first recognition results, input multiple license plate images as high-value license plate images into multiple character recognition models respectively, and determine the second recognition results corresponding to the high-value license plate images.

[0028] Specifically, responding to the inconsistency of each first recognition result means that if the character recognition results of multiple frames of license plates are inconsistent, it is considered that the license plate is a high-value license plate.

[0029] Specifically, inputting the license plate images corresponding to the recognition results into multiple trained character recognition models respectively and determining the second recognition results corresponding to the high-value license plates means inputting the license plate images corresponding to the inconsistent recognition results into multiple different character recognition models with high complexity and high accuracy respectively, and determining the second recognition results corresponding to the high-value license plates. After using the lightweight model to complete the recognition of high-value license plates, for the license plate images corresponding to the high-value license plates, a character recognition model with higher recognition accuracy is used to determine the corresponding recognition results, so that high-value license plates can be recognized more accurately, that is, the second recognition results corresponding to the high-value license plate images, thereby providing a data basis for subsequent review of license plate characters. S104. Perform a review based on the second recognition results to determine the license plate character annotation result of the vehicle.

[0030] Specifically, the license plate character annotation result of the vehicle is determined based on each second recognition result, which means that the automatic annotation result is determined according to whether the second recognition results are consistent, and the automatic annotation result is audited according to the preset audit rules to determine the license plate character annotation result of the vehicle.

[0031] A method, device, equipment, and storage medium for annotating and auditing high-value license plate characters provided by the present invention. This method inputs a video stream into a vehicle detection model to determine multiple license plate pictures corresponding to the vehicle, and completes the recognition of high-value license plates according to the license plate pictures and a lightweight character recognition model. On the basis of completing the recognition of high-value license plates, the second recognition results recognized by the trained character recognition model are audited to complete the character annotation in the high-value license plates. In this process, the vehicle pictures are recognized through the trained vehicle detection model, and then the license plate pictures are determined. The lightweight character recognition model is used to complete the recognition of high-value license plates, so that high-value license plates can be accurately recognized. On the basis of completing the recognition of high-value license plates, multiple trained character recognition models are used to complete the recognition of characters in the high-value license plates. Therefore, by auditing the recognition results, the license plate character annotation result of the vehicle is determined, and the participation of human factors is reduced in both the recognition and auditing processes of high-value license plates. This method determines how to use the model to complete the recognition of high-value license plates and the auditing mechanism of the recognition results to ensure the accuracy of the license plate character annotation result of the vehicle. And because there is a unified standard for the recognition and auditing of high-value license plates, this process can be completed using the model, further improving the license plate character annotation efficiency and output.

[0032] In an alternative embodiment, determining the license plate character annotation result of the vehicle based on each second recognition result includes: In response to the consistency of each second recognition result, taking the second recognition result as the automatic annotation result; auditing the automatic annotation result to determine the license plate character annotation result of the vehicle.

[0033] Specifically, in response to the consistency of each second recognition result, taking the second recognition result as the automatic annotation result means that the second recognition results of multiple trained character recognition models for the vehicle pictures corresponding to the recognition results are the same. On this basis, without further judging which second recognition result is more accurate, any second recognition result can be directly selected as the automatic annotation result.

[0034] In an alternative embodiment, determining the license plate character annotation result of the vehicle based on each second recognition result includes: In response to the inconsistency of each second recognition result, selecting the second recognition result with a high confidence level as the automatic annotation result; auditing the automatic annotation result to determine the license plate character annotation result of the vehicle.

[0035] Specifically, in response to the inconsistency of each second recognition result, selecting the second recognition result with a high confidence as the automatic annotation result means that the second recognition results of multiple trained character recognition models for the vehicle pictures corresponding to the recognition results are different. On this basis, according to the confidence of each second recognition result, it is judged which second recognition result is more accurate, and the second recognition result with a high confidence is selected as the automatic annotation result.

[0036] In an alternative implementation, auditing the automatic annotation result to determine the license plate character annotation result of the vehicle includes: Based on the automatic annotation result, determining the hierarchical information, color information, character length information, separator information, and first character information of the license plate; in response to the hierarchical information, color information, character length information, separator information, and first character information all conforming to the corresponding audit rules, using the automatic annotation result as the license plate character annotation result of the vehicle.

[0037] Specifically, the hierarchical information of the license plate includes single-layer and double-layer. The color information includes: yellow, green, blue, or other colors. The character length information refers to the length of the characters in the corresponding annotation result, such as 7 digits, 8 digits, or other numbers of digits. The separator information refers to a specific character, such as "_", and the separator information is used to distinguish each level of the license plate when the hierarchical information of the license plate is not single-layer. The first character information refers to the characters in the corresponding annotation result.

[0038] Specifically, in response to the hierarchical information, color information, character length information, separator information, and first character information all conforming to the preset audit rules, using the automatic annotation result as the license plate character annotation result of the vehicle means judging the hierarchical information, color information, character length information, separator information, and first character information based on the preset audit rules. When the above information all conforms to the preset audit rules, it passes the audit, that is, using the automatic annotation result as the license plate character annotation of the vehicle. When some information does not conform to the preset audit rules, the audit is rejected and waiting for modification before continuing the audit.

[0039] For example, after the automatic annotation result is input, based on the preset audit rules, judge the number of layers of the license plate being labeled. If it is single-layer, then judge whether the license plate color label is green. If the license plate color label is green, then judge whether the length of the license plate character label is 8 digits. If it is not 8 digits, then reject it for the annotator to modify; if the length of the green license plate character label is 8 digits, then further judge whether only the first character of the license plate character label is a Chinese character. If not, then reject it for the annotator to modify; if the single-layer green license plate label meets the conditions that the license plate character length is 8 digits and only the first character is a Chinese character, then the single-layer green license plate annotation result passes the audit.

[0040] If the single-layer license plate color label is blue or yellow, it should meet the requirements that the license plate character length is 7 digits and only the first character is a Chinese character. Therefore, when the system automatically determines single-layer blue or yellow, it checks whether the license plate character length is 7 digits. If not, it is rejected for the annotator to modify; if the single-layer is blue or yellow and the license plate character length is 7 digits, it further checks whether only the first character is a Chinese character. If not, it is rejected for the annotator to modify; if so, the single-layer blue or yellow license plate annotation result passes the review.

[0041] If the license plate annotation label is double-layer, it checks whether the double-layer license plate character label contains the agreed upper and lower double-layer character separators. If it does not contain the double-layer character separator, it is rejected for the annotator to modify; if it contains the double-layer character separator, it checks whether the label color is yellow. The double-layer yellow license plate label should meet the requirements that the license plate character length is 7 digits and only the first character is a Chinese character. If the double-layer license plate color label is yellow, it checks whether the double-layer yellow license plate character label length is 7 digits. If not, it is rejected for the annotator to modify; if so, it checks whether only the first character of the character label is a Chinese character. If so, it passes the review; otherwise, it is rejected for the annotator to modify.

[0042] And so on, non-green, non-blue, non-yellow license plates in single-layer license plate labels and non-yellow license plates in double-layer license plates are also reviewed based on the preset review rules. Through this review method, some common annotation errors are initially filtered out, significantly reducing the review workload and saving labor costs for the enterprise.

[0043] In an alternative implementation, the training process of the vehicle detection model includes: Obtain the image to be detected and divide the image to be detected into multiple blocks; based on the attention mechanism, splice each block to form multiple spliced images; divide each spliced image into multiple parts at a preset ratio, and use residuals and re-parameters to complete the splicing of multiple parts to form a feature image; adjust the parameters of the vehicle detection model based on the feature image until the model converges.

[0044] Specifically, dividing the image to be detected into multiple blocks means dividing the image to be detected into N*N blocks (patches).

[0045] Specifically, based on the attention mechanism, splicing each block to form multiple spliced images means dividing the image to be detected into multiple blocks, performing self-attention calculation on each block, and splicing the calculation results corresponding to each block according to the position of each block in the image to be detected, so that the neural network pays more attention to the local feature information of the foreground during feature fusion. And because the self-attention calculation is performed on each divided block and the calculated result replaces the block at the original position, the computational complexity is smaller than that of performing self-attention calculation on the entire image, ensuring the accuracy and recall rate of model calculation.

[0046] Specifically, dividing each spliced image into multiple parts at a preset ratio and using residuals and re-parameters to complete the splicing of multiple parts to form a feature image means dividing each spliced image into multiple parts at a preset ratio. Among them, residual calculation is performed on the second part, re-parameter calculation is performed on the third part, and the remaining first part is retained. Then, the residual calculation result, re-parameter calculation result and the first part are spliced together, so that the neural network enables the same image to have the characteristics of a residual network and re-parameterization characteristics during image feature extraction, improving the convergence speed of the model training process. And because the same image has re-parameterization characteristics, the model decomposes the process of feature extraction and information integration into multiple branches, and these multi-branch structures will be replaced by a single-branch structure during inference, further reducing the memory capacity requirements of the model deployment for hardware devices, reducing enterprise costs, and at the same time improving the model detection accuracy.

[0047] Specifically, predicting the vehicle image based on the feature image means using the feature image as the prediction basis of the vehicle detection model until the model converges. Generally, at this time, the loss function basically no longer decreases or the model reaches the maximum number of iterations, and it is considered that the vehicle detection model is trained, so as to obtain a trained vehicle detection model.

[0048] In an alternative embodiment, based on the attention mechanism, splicing each block to form multiple spliced images includes: Performing self-attention calculation on each block to determine the calculation result corresponding to each block; adjusting each calculation result to the size of the corresponding block; based on the position of each block in the image to be detected, replacing the adjusted calculation result with the corresponding block to form a spliced image corresponding to the block; summarizing the spliced images corresponding to each block to form multiple spliced images.

[0049] Specifically, performing self-attention calculation on each block to determine the calculation result corresponding to each block means flattening each block for self-attention calculation to determine the calculation result corresponding to each block.

[0050] Specifically, adjusting each calculation result to the size of the corresponding block means reshaping the calculated result to the size of the original corresponding block.

[0051] Specifically, based on the position of each block in the image to be detected, replacing the calculation result after size adjustment with the corresponding block to form a stitched image corresponding to the block means that according to the position of each block in the image to be detected, replacing the calculation result after size adjustment with the corresponding block to form a stitched image corresponding to the block. For example, if the image to be detected contains blocks a, b, and c, the position of block a is denoted as (x, y), and the self-attention result of block a after being adjusted to the size of block a is denoted as a`, then the stitched image corresponding to block a contains a`, b, and c, where the position of a` is (x, y).

[0052] Specifically, summarizing the stitched images corresponding to each block to form multiple stitched images means summarizing the stitched images for each block to form multiple stitched images. It should be understood that each stitched image has the self-attention calculation result corresponding to the block and other blocks at other positions in the image to be detected that have not undergone self-attention calculation, so that the neural network pays more attention to the local feature information of the foreground during feature fusion. And since the self-attention calculation is performed on each divided block and the calculation result replaces the block at the original position, the computational complexity is smaller than that of performing self-attention calculation on the entire image, ensuring the accuracy and recall rate of the model calculation.

[0053] In an alternative embodiment, each stitched image is divided into multiple parts in a preset ratio, and the stitching of the multiple parts is completed using residuals and re-parameters to form a feature image, including: Dividing each stitched image in a preset ratio to determine a first part, a second part, and a third part; performing residual calculation on the second part to determine the residual calculation result; performing re-parameter calculation on the third part to determine the re-parameter calculation result; stitching the first part, the residual calculation result, and the re-parameter calculation result to form a feature image.

[0054] Specifically, as Figure 2 shown, dividing each stitched image in a preset ratio to determine a first part, a second part, and a third part means splitting each stitched image in a preset ratio to form a first part, a second part, and a third part, where the preset ratio includes: 4:3:3, 2:3:5, or other ratios, and usually 4:3:3 is adopted.

[0055] Specifically, performing residual calculation on the second part to determine the residual calculation result means performing residual connection calculation on the split second part, and using the input and the convolution result added together to determine the residual calculation result, where the residual connection calculation includes two convolutional layers weight layer.

[0056] Specifically, reparameterization calculation is performed on the third part. Determining the reparameterization calculation result means performing reparameterization calculation on the split third part, and using 3x3 convolution and 1x1 convolution to determine the reparameterization calculation result.

[0057] Specifically, as Figure 2 shown, the first part, the residual calculation result, and the reparameterization calculation result are concatenated to form a feature image. And because the concatenated feature image endows the same image with the characteristics of a residual network and reparameterization, the convergence speed of the model training process is improved. And because the same image has the characteristics of reparameterization, the model decomposes the process of feature extraction and information integration into multiple branches, and these multi-branch structures will be replaced by a single-branch structure during inference, further reducing the memory capacity requirements of the model deployment for hardware devices, reducing enterprise costs, and at the same time improving the model detection accuracy.

[0058] In this embodiment, a high-value license plate character annotation and review device is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0059] This embodiment provides a high-value license plate character annotation and review device, as Figure 3 shown, including: A picture determination module 201, configured to input a vehicle video stream into a vehicle detection model to determine multiple license plate pictures corresponding to the same vehicle. The specific process can refer to the relevant description of step S101 in the above-mentioned embodiment, and will not be repeated here.

[0060] A judgment module 202, configured to input multiple license plate pictures into a lightweight character recognition model respectively, determine first recognition results corresponding to the multiple license plate pictures respectively, and judge whether the first recognition results corresponding to the license plates are consistent. The specific process can refer to the relevant description of step S102 in the above-mentioned embodiment, and will not be repeated here.

[0061] An identification determination module 203, configured to, in response to the inconsistency of the first recognition results, input multiple license plate pictures as high-value license plate pictures into multiple character recognition models respectively, and determine second recognition results corresponding to the high-value license plate pictures. The specific process can refer to the relevant description of step S103 in the above-mentioned embodiment, and will not be repeated here.

[0062] The annotation determination module 204 is configured to perform an audit based on the second recognition result and determine the license plate character annotation result of the vehicle. For the specific process, reference can be made to the relevant description of step S104 in the foregoing embodiments, which will not be elaborated herein.

[0063] In this embodiment, the high-value license plate character annotation and auditing device is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0064] An embodiment of the present invention further provides a computer device having the above Figure 3 shown high-value license plate character annotation and auditing device.

[0065] Please refer to Figure 4 , Figure 4 is a schematic structural diagram of a computer device provided by an optional embodiment of the present invention. As shown in Figure 4 , the computer device includes: one or more processors 301, a memory 302, and an interface for connecting each component, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 4 In

[0066] Processor 301 can be a central processor, a network processor, or a combination thereof. Among them, processor 301 can further include a hardware chip. The above hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0067] Among them, the memory 302 stores instructions executable by at least one processor 301, so that the at least one processor 301 executes the method shown in the above embodiments.

[0068] The memory 302 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 302 may optionally include a memory remotely provided with respect to the processor 301, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.

[0069] The memory 302 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive; the memory 302 may further include a combination of the above types of memory. The computer device further includes a communication interface 303 for the computer device to communicate with other devices or communication networks.

[0070] The embodiments of the present invention also provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or non-transitory machine-readable storage medium and downloaded through a network and to be stored in a local storage medium, so that the methods described herein can be stored in such software processed on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0071] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for marking and reviewing high-value license plate characters, characterized in that: The method comprises: Input the vehicle video stream into the vehicle detection model to determine multiple license plate images corresponding to the same vehicle; Input the plurality of license plate images into the lightweight character recognition model respectively, determine the first recognition results corresponding to the plurality of license plate images respectively, and judge whether the first recognition results corresponding to the license plate are consistent; In response to the first recognition results being inconsistent, inputting the plurality of license plate images as high-value license plate images into a plurality of character recognition models respectively, and determining a second recognition result corresponding to the high-value license plate images; An audit is performed based on the second recognition result to determine the vehicle license plate character marking result.

2. The method according to claim 1, characterized in that The review based on the second recognition result to determine the vehicle license plate character marking result includes: In response to the second recognition results being consistent, taking the second recognition results as automatic labeling results; The automatic labeling result is reviewed to determine the vehicle license plate character labeling result.

3. The method according to claim 1, characterized in that The review based on the second recognition result to determine the vehicle license plate character marking result includes: In response to the second recognition results being inconsistent, selecting a second recognition result with a high confidence level as an automatic labeling result; The automatic labeling result is reviewed to determine the vehicle license plate character labeling result.

4. The method according to claim 2 or 3, characterized in that: The step of reviewing the automatic labeling result to determine the vehicle license plate character labeling result includes: Based on the automatic labeling result, determining the level information, color information, character length information, separator information and first character information of the license plate; In response to the level information, the color information, the character length information, the separator information and the first character information all complying with corresponding review rules, the automatic labeling result is used as the license plate character labeling result of the vehicle.

5. The method according to claim 1, characterized in that The training process of the vehicle detection model includes: Obtain a picture to be detected, and divide the picture to be detected into multiple blocks; Based on the attention mechanism, each of the blocks is spliced ​​to form multiple spliced ​​images; Dividing each of the stitched images into a plurality of parts according to a preset ratio, and stitching the plurality of parts together using residuals and re-parameters to form a feature image; Parameters of the vehicle detection model are adjusted based on the feature image until the model converges.

6. The method according to claim 5, characterized in that Based on the attention mechanism, each of the blocks is spliced ​​to form multiple spliced ​​images, including: Performing self-attention calculation on each of the blocks to determine a calculation result corresponding to each of the blocks; Adjusting each of the calculation results to a size corresponding to the block; Based on the position of each block in the image to be detected, replacing the corresponding block with the resized calculation result to form a spliced ​​image corresponding to the block; The stitched images corresponding to each of the blocks are aggregated to form a plurality of stitched images.

7. The method according to claim 5, characterized in that The step of dividing each of the stitched images into a plurality of parts according to a preset ratio, and stitching the plurality of parts together using residuals and re-parameters to form a feature image comprises: Dividing each of the stitched images according to a preset ratio to determine a first part, a second part, and a third part; Perform residual calculation on the second part to determine the residual calculation result; Performing re-parameter calculation on the third part to determine the re-parameter calculation result; The first part, the residual calculation result, and the re-parameter calculation result are spliced ​​to form a feature image.

8. A device for marking and verifying high-value license plate characters, characterized in that: The device comprises: An image determination module is used to input a vehicle video stream into a vehicle detection model to determine multiple license plate images corresponding to the same vehicle; A judgment module, used to input the plurality of license plate images into the lightweight character recognition model respectively, determine the first recognition results corresponding to the plurality of license plate images respectively, and judge whether the first recognition results corresponding to the license plate are consistent; an identification and determination module, configured to, in response to the first identification result being inconsistent, input the plurality of license plate images as high-value license plate images into a plurality of character recognition models respectively, and determine a second identification result corresponding to the high-value license plate image; The labeling determination module is used to review and determine the vehicle license plate character labeling result based on the second recognition result.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for marking and reviewing high-value license plate characters as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for marking and reviewing high-value license plate characters according to any one of claims 1 to 7.

Images