License plate recognition method and device, and storage medium

Abstract

The application relates to a license plate recognition method and device and a storage medium, and belongs to the technical field of computers.The method comprises the following steps: obtaining a target image; inputting the target image into a pre-trained character detection model to obtain character information in the target image; the first sample data used during training of the character detection model is less than the first sample data used during training of a license plate recognition model; and the positions of the characters are sorted according to position information to obtain license plate information in the target image.The method can solve the problem that a license plate recognition model needs to be trained using a large amount of sample data, and the accuracy of a license plate recognition model trained under the condition that the collected sample data is less is low.The first sample data required during training is reduced because the first sample data used during training of the character detection model is less than the first sample data used during training of the license plate recognition model, and the use range of the license plate recognition method is expanded.

Description

[0001] The application relates to a license plate recognition method, device and storage medium, and belongs to the technical field of computers.

[0002] With the continuous development of license plate recognition technology and the rapid improvement of computer operation capacity, the license plate recognition technology is widely applied to scenarios such as automatic driving, auxiliary driving, parking lot management and the like. License plate recognition refers to a technology of extracting and displaying the text in the license plate region in a vehicle image.

[0003] A traditional license plate recognition method comprises the following steps: an initial network model is trained based on a deep learning manner using sample data to obtain a license plate recognition model. In the process of license plate recognition, a target image collected is input into the license plate recognition model to obtain license plate information corresponding to the target image data.

[0004] However, the license plate recognition model needs to be trained using a large amount of sample data, and in the case that the collected sample data is less, the license plate recognition model obtained by training has a low accuracy in recognizing license plates.

[0005] The application provides a license plate recognition method, device and storage medium, and can solve the problem that a license plate recognition model needs to be trained using a large amount of sample data, and in the case that the collected sample data is less, the license plate recognition model obtained by training has a low accuracy in recognizing license plates. The application provides the following technical solutions:

[0006] In a first aspect, a license plate recognition method is provided, and the method comprises the following steps:

[0007] Obtaining a target image;

[0008] Inputting the target image into a pre-trained character detection model to obtain character information in the target image, the character information comprising a character category of each character and position information of the character, and the first sample data used in training of the character detection model being less than the first sample data used in training of a license plate recognition model;

[0009] Positionally sorting each character according to the position information to obtain license plate information in the target image, the license plate information comprising the character category.

[0010] Optionally, the training process of the character detection model comprises the following steps:

[0011] Training a pre-created initial network model using a first training set to obtain an intermediate model, the first training set comprising a second sample vehicle image and a character category label of a license plate region in the second sample vehicle image; ​​​

[0012] The intermediate model is trained using a second training set to obtain the character detection model; the second training set includes the second sample vehicle image, the character category label corresponding to the second sample vehicle image, the first sample vehicle image, and the character category label of the license plate area in the first sample vehicle image;

[0013] The character features in the first sample vehicle image are different from the character features in the second sample vehicle image.

[0014] Optionally, in the second sample data set, the number of images of the first sample vehicle images is greater than the number of images of the second sample vehicle images.

[0015] Optionally, the character detection model is established based on a network model without a priori box anchor.

[0016] Optionally, the character detection model includes a backbone network, a connection layer and a prediction layer, and the backbone network is connected to the prediction layer through the connection layer;

[0017] The backbone network includes a plurality of pruned residual blocks and a downsampling layer located after the residual blocks;

[0018] The prediction layer is used to locate the characters in the feature map output by the backbone network and classify the characters to obtain the character information.

[0019] Optionally, the character information further includes a confidence level corresponding to each position information;

[0020] The step of sorting the characters according to the position information to obtain the license plate information in the target image includes:

[0021] For each character, determining, based on the position information, a degree of overlap between the detection frame of the character and the detection frame of other characters;

[0022] In the case where the degree of overlap is greater than the overlap threshold, determining a character with the smallest confidence from the other characters and the character to obtain a candidate character;

[0023] After all characters are traversed, the candidate characters are deleted;

[0024] The deleted characters are sorted according to the position information to obtain the license plate information of the target image.

[0025] Optionally, the step of deleting the candidate characters after traversing all characters is completed includes:

[0026] determining whether the number of characters matches a preset number;

[0027] In a case where the number of the characters does not match the preset number, the candidate characters are deleted after all the characters are traversed.

[0028] Optionally, after the determination of whether the number of the characters matches the preset number, the method further includes:

[0029] In a case where the number of the characters matches the preset number, the characters are positionally sorted according to the position information to obtain the license plate information in the target image.

[0030] In a second aspect, an electronic device is provided, and the device includes a processor and a memory; the memory stores a program, and the program is loaded and executed by the processor to implement the license plate recognition method provided in the first aspect.

[0031] In a second aspect, a computer-readable storage medium is provided, and the storage medium stores a program, and the program is executed by a processor to implement the license plate recognition method provided in the first aspect.

[0032] The application has at least the following beneficial effects: the target image is obtained; the target image is input into a pre-trained character detection model to obtain character information in the target image, the character information including a character category of each character and position information of the character, and the first sample data used in training of the character detection model is less than the first sample data used in training of the license plate recognition model; the characters are positionally sorted according to the position information to obtain license plate information in the target image, and the license plate information includes the character category; the problem that a large amount of sample data is required for training of the license plate recognition model, and the license plate recognition model trained under the condition of less sample data has a low accuracy in license plate recognition can be solved; since the first sample data used in training of the character detection model is less than the first sample data used in training of the license plate recognition model, the first sample data required in training is reduced, and the use range of the license plate recognition method is expanded.

[0033] In addition, the initial network model is trained using the first training set to obtain an intermediate model; the intermediate model is fine-tuned using the second training set again on the basis of the intermediate model, and the intermediate model focuses on learning of character features in the first sample vehicle image in the fine-tuning process, which can improve the sensitivity of the character detection model in recognizing the character features in the first sample vehicle image.

[0034] In addition, by fusing the second sample vehicle image, the character category label corresponding to the second sample vehicle image, the first sample vehicle image, and the character category label corresponding to the first sample vehicle image in the second training set, the sensitivity of the character detection model in identifying the character features in the first sample vehicle image can be improved, and the effect of the oscillation of the verification set detection accuracy can be reduced, thereby improving the robustness of the model.

[0035] In addition, since the character detection model is established based on a network model without a priori box anchor, there is no need to pre-set the anchor size. Therefore, the problem of poor character detection accuracy caused by poor anchor size setting can be avoided; it can improve both the efficiency and accuracy of license plate recognition.

[0036] In addition, since the backbone network of the character detection model is obtained by pruning the backbone network of the traditional network model, and the connection relationship between the residual block and the downsampling layer is not changed during the pruning process, the difficulty of training the character detection model can be reduced while ensuring the detection accuracy of the character detection model.

[0037] In addition, since candidate characters are determined based on the degree of overlap between characters and the confidence level corresponding to the characters, and candidate characters are deleted after all characters are traversed, repeated detection caused by detection errors in the character detection network can be avoided, and the accuracy of license plate recognition can be improved.

[0038] In addition, since the position of each character is directly sorted according to the position information to obtain the license plate information when the number of characters matches the preset number, the deletion of correctly detected characters can be avoided, and the accuracy of license plate recognition can be improved.

[0039] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement it in accordance with the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0040] Figure 1 This is a flowchart of a license plate recognition method provided by one embodiment of the present application;

[0041] Figure 2 This is a structural diagram of a character detection model and an initial network model provided by an embodiment of the present application;

[0042] Figure 3 This is a flowchart of a character detection model training process provided by one embodiment of the present application;

[0043] Figure 4is a block diagram of a license plate recognition device provided by one embodiment of the present application;

[0044] Figure 5 A block diagram of an electronic device provided by one embodiment of the present application. [Specific implementation method]

[0045] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0046] First, several terms involved in the embodiments of the present application are introduced.

[0047] License plate recognition model: Identify the license plate information in the target image, including the type of characters and the order of each character.

[0048] Character Detection Model: Detects the type of characters and the location of the characters in the target image.

[0049] Anchor: A pre-set frame of different sizes and aspect ratios on an image. When predicting an object in an image, the location of the object in the image is determined based on the anchor.

[0050] Intersection over Union (IOU): The ratio between the intersection and union of two regions, ranging from 0 to 1. The higher the overlap, the closer the two regions are.

[0051] Optionally, this application uses the license plate recognition method provided in each embodiment as an example for explanation in an electronic device, where the electronic device is a terminal or a server. The terminal can be a mobile phone, computer, tablet computer, scanner, electronic eye, surveillance camera, etc. This embodiment does not limit the type of electronic device.

[0052] Figure 1 This is a flowchart of a license plate recognition method provided by an embodiment of the present application, which includes at least the following steps:

[0053] Step 101: Acquire a target image.

[0054] Optionally, the target image refers to an image to be subjected to license plate recognition. The target image may be a frame of image in a video stream, or may be a single captured image. This embodiment does not limit the source of the target image.

[0055] Optionally, the target image contains a license plate or does not contain a license plate, and there are one target image or at least two target images. This embodiment does not limit the image content in the target image and the number of target images.

[0056] Step 102: Input the target image into a pre-trained character detection model to obtain character information in the target image.

[0057] The character information includes the character category and position information of each character.

[0058] In one example, the character categories include 26 English letter categories and 10 numeric categories. In other embodiments, the character categories may also include Chinese characters, such as the abbreviations of municipalities or provincial capitals, such as "Su," "Ji," or "Hei," or Greek letters, such as "α" and "β." This embodiment does not limit the method of dividing character categories.

[0059] The position information of a character refers to the position information of the character's detection frame. In one example, the position information is the coordinates of the center position of the character detection frame. Alternatively, the position information is the coordinates of the left vertex of the detection frame. This embodiment does not limit the implementation method of the position information.

[0060] Optionally, the character information further includes the confidence level corresponding to each piece of position information.

[0061] The confidence level is used to indicate the probability of a character existing in a character detection frame and the accuracy of the position of the character detection frame.

[0062] Since the character detection model detects the category and position information of each character in the target image without identifying the arrangement order of all characters in the target image, the training difficulty of the character detection model is lower than that of the license plate recognition model, and the first sample data used in the training of the character detection model is less than the first sample data used in the training of the license plate recognition model.

[0063] Optionally, the first sample data includes: a first sample vehicle image, and a character category label of a license plate area in the first sample vehicle image.

[0064] In one example, the character category label is a character annotation box, which is used to indicate the position of the character and the category of the character in the first sample vehicle image.

[0065] Optionally, in order to further reduce the amount of first sample data used in the character detection model training process, a transfer learning training method is used to train the character detection model. Specifically, the character detection model training process includes the following steps:

[0066] Step 1: Use the first training set to train the pre-created initial network model to obtain an intermediate model.

[0067] The first training set includes the second sample vehicle image and the character category label of the license plate area in the second sample vehicle image.

[0068] The character features in the first sample vehicle image are different from the character features in the second sample vehicle image. That is, in this embodiment, the second sample vehicle image is first used to train an intermediate model, and then the first sample vehicle image is used to transfer learning from the intermediate model, thereby learning the character features in the second sample vehicle image.

[0069] Optionally, the character features include the character shape, the character arrangement, and / or the proportion of the character in all sample images.

[0070] Assuming that the character detection model is used to identify license plate information of vehicles at a target geographic location, the first sample vehicle image is a vehicle image captured at the target geographic location, and the second sample vehicle image is a vehicle image captured at an arbitrary geographic location.

[0071] The model structure of the character detection model is the same as that of the initial network model. Traditional license plate recognition methods are based on network models with anchors. However, due to the pre-set anchor size, network models with anchors are not conducive to detecting small objects. Therefore, traditional license plate recognition methods typically use the license plate recognition model to identify the location of the license plate in the target image, and then use other methods to identify the characters at the license plate location.

[0072] Based on this, in one example, a character detection model is established based on a network model without a priori box Anchor. Since the character detection model is obtained by training the initial network model, the model structure of the character detection model is the same as the model structure of the initial network model and the same as the model structure of the intermediate model below. Accordingly, the initial network model and the intermediate model are established based on a network model without a priori box Anchor. Since the network model without Anchor does not require the pre-setting of the Anchor size, the problem of poor character detection accuracy due to poor Anchor size setting can be avoided; it can improve both the efficiency and accuracy of license plate recognition.

[0073] Optionally, the network model without a priori box anchor may be a YOLOX network model, or may be an FCOS network model, or may be a CenterNet network model. This embodiment does not limit the type of the network model without anchor.

[0074] In this embodiment, the network model without a priori anchor box is taken as the YOLOX network model as an example for explanation. Figure 2A model structure of the character detection model and the initial network model is shown, which includes a backbone network 21, a connection layer 22, and a prediction layer 23, the backbone network 21 being connected to the prediction layer 23 through the connection layer 22. The backbone network 21 is configured to perform feature extraction on an input image to obtain a feature map. The prediction layer 23 is configured to locate characters in the feature map output by the backbone network 21 and classify the characters to obtain character information.

[0075] Optionally, the backbone network of the character detection model includes five down-sampling layers, and each down-sampling layer has a down-sampling step of 2. That is, the down-sampling step of the entire backbone network of the character detection model is 32.

[0076] Since the number of hidden layers in the traditional backbone network is large, and these hidden layers do not increase the accuracy of character detection, but rather reduce the speed of character detection. Based on this, in the embodiment, the traditional backbone network is pruned to reduce the training difficulty and the required model parameters of the character detection model. Specifically, the traditional network model is pruned, including: pruning the backbone network of the YOLOX network model to obtain the backbone network of the character detection model.

[0077] In one example, the backbone network of the character detection model is obtained by deleting a preset number of residual blocks in front of each down-sampling layer of the backbone network of the traditional network model. At this time, the backbone network of the character detection model includes a plurality of residual blocks after pruning and down-sampling layers located after the residual blocks.

[0078] Each down-sampling layer corresponds to a same or different preset number of pruned residual blocks.

[0079] For example: the traditional network model is the YOLOX network model, the backbone network Darknet53 of the YOLOX network model includes five down-sampling layers, and the number of residual blocks in front of each down-sampling layer is 1, 2, 8, 8, and 4, respectively. In the pruning process, the number of residual blocks deleted in front of each down-sampling layer is 0, 1, 3, 4, and 3, respectively. The number of residual blocks in front of each down-sampling layer of the backbone network of the character detection model is 1, 1, 5, 4, and 1, respectively. At this time, the structure of the backbone network of the character detection model is as shown in reference Figure 2 .

[0080] In another example, the width of the backbone network of the traditional network model is reduced to obtain the backbone network of the character detection model.

[0081] Optionally, the width of the backbone network of the traditional network model is reduced to obtain the backbone network of the character detection model, including: reducing the number of convolution kernels in the convolution layer of the backbone network of the traditional network model to obtain the backbone network of the character detection model.

[0082] For example, the width of the backbone network of the traditional network model is reduced to half of its original width to obtain the backbone network of the character detection model.

[0083] Since the character information includes the character category and position information of each character, the loss function used when training the initial network model includes a first loss function and a second loss function, wherein the first loss function is used to minimize the difference in character category, and the second loss function is used to minimize the difference in position information. The first loss function can be a classification loss function, and the second loss function can be a regression loss function.

[0084] Step 2: Use the second data set to train the intermediate model to obtain a character recognition model.

[0085] The second training set includes a second sample vehicle image, a character category label corresponding to the second sample vehicle image, the first sample vehicle image, and a character category label of a license plate area in the first sample vehicle image.

[0086] Since the second sample data set includes the first sample vehicle image and the character category label corresponding to the first sample vehicle image, the character detection network trained using the second sample data set can recognize character information in the target image.

[0087] Since the intermediate model was trained using the first sample dataset, training with the second sample vehicle images has already enabled the intermediate model to detect characters. Therefore, it is only necessary to fine-tune the character detection model's ability to recognize character features in the first sample vehicle images based on the intermediate model. In this case, the number of first sample vehicle images used is smaller than the number used when training the character detection model based on the initial network model, allowing for training the character detection network with a small sample size.

[0088] Since the number of first sample vehicle images is very small, in order to improve the accuracy of the intermediate model, the first sample vehicle images may be augmented after being collected. Specifically, the augmentation of the first sample vehicle images includes offline augmentation and / or online augmentation.

[0089] Among them, online enhancement refers to transforming sample data to expand sample data during network training; offline enhancement refers to expanding sample data first and using the expanded sample data to train the network.

[0090] Optionally, the method of offline enhancement of the first sample vehicle image may be to reorganize the RGB channels of the first sample vehicle image and / or randomly scale the license plate area in the first sample vehicle image, or to crop the license plate area in the first sample vehicle image. This embodiment does not limit the method of offline enhancement of the first sample vehicle image.

[0091] In one example, offline enhancement of the first sample vehicle image includes: randomly recombining RGB channels of the first sample vehicle image to obtain a recombined sample image; and randomly scaling and cropping the license plate area in the recombined sample image to obtain an expanded first sample vehicle image.

[0092] For example, after randomly recombining the RGB channels of the first sample vehicle image, the number of recombined sample images is twice the number of the first sample vehicle images; after randomly scaling and cropping the license plate area of ​​the recombined sample image, the number of expanded first sample vehicle images is five times the number of the recombined sample images. Finally, the number of expanded first sample vehicle images is ten times the number of the first sample vehicle images.

[0093] In other model training methods, the electronic device can also use the first training set and the first sample vehicle image, and the character category label of the license plate area in the first sample vehicle image to train the initial network model to obtain a character detection model. However, at this time, the proportion of the first sample vehicle image in all the training data is relatively small, and the character detection model may not be sensitive to the extraction of character features in the first sample vehicle image. In this embodiment, the initial network model is first trained with the first training set to obtain an intermediate model; then, based on the intermediate model, the second training set is used again to fine-tune the intermediate model. During the fine-tuning process, the intermediate model focuses on learning the character features in the first sample vehicle image, which can improve the sensitivity of the character detection model in identifying the character features in the first sample vehicle image.

[0094] In addition, in other model training methods, only the first sample vehicle image and the character category label of the license plate area in the first sample vehicle image can be used to train the intermediate model. In this case, the verification set detection accuracy fluctuates and the model lacks robustness. In this embodiment, by fusing the second sample vehicle image, the character category label corresponding to the second sample vehicle image, the first sample vehicle image, and the character category label corresponding to the first sample vehicle image in the second training set, the sensitivity of the character detection model in identifying the character features in the first sample vehicle image can be improved, and the effect of the fluctuation of the verification set detection accuracy can be reduced, thereby improving the robustness of the model.

[0095] To improve the accuracy of the character detection model, after the second sample vehicle image is collected, the second sample vehicle image may be expanded. The method for expanding the second sample vehicle image is similar to the method for expanding the first sample vehicle image, and will not be described in detail in this embodiment.

[0096] Optionally, to improve the license plate detection network's detection accuracy for character information in a target image, the number of first sample vehicle images in the second sample dataset is greater than the number of second sample vehicle images. In other words, the proportion of first sample vehicle images in the second sample dataset is greater than the proportion of second sample vehicle images in the second sample dataset.

[0097] To more clearly understand the character detection model training process provided by this embodiment, refer to Figure 3 The following example illustrates the training process. In this example, the first sample vehicle image is a vehicle image from country B, and the second sample vehicle image is a vehicle image from country A. The process includes at least the following steps:

[0098] Step 31: Acquire a first number of vehicle images of country A.

[0099] The first number is greater than or equal to the minimum number of training sample images required for the initial network model to achieve a preset detection accuracy.

[0100] In step 32, the pre-created initial network model is trained using the vehicle images of country A to obtain an intermediate model.

[0101] Step 33: Acquire license plate images of a second number B of countries.

[0102] The second number is smaller than the minimum number of training sample images required for the initial network model to achieve a preset detection accuracy.

[0103] In step 34, the license plate images of country B and the license plate images of country A are used to perform transfer learning on the intermediate model to obtain a character detection model.

[0104] Since the character detection model can be obtained by using a small amount of images from country B to perform transfer learning on the intermediate model, the license plate recognition method provided in this application is also applicable to countries with very few license plates, which can expand the scope of application of the license plate recognition method.

[0105] Generally speaking, the minimum number of training sample images required for the initial network model to achieve a preset detection accuracy is in the tens of thousands or millions, that is, the first number is in the tens of thousands, hundreds of thousands or millions, and the second number can be in the hundreds. At this time, the second number is extremely small compared to the first number.

[0106] Step 103 , sorting the characters according to the position information to obtain the license plate information in the target image.

[0107] Among them, the license plate information includes character categories.

[0108] The character detection network may have detection errors, for example: repeatedly detecting a character may result in two detection boxes corresponding to the same character.

[0109] Based on this, each character is sorted according to the position information to obtain the license plate information in the target image, including: for each character, the overlap between the detection frame of other characters and the detection frame of the character is determined based on the position information; when the overlap is greater than the overlap threshold, the character with the smallest confidence is determined from other characters and characters to obtain the candidate character; after all characters are traversed, the candidate character is deleted; and the deleted characters are sorted according to the position information to obtain the license plate information of the target image.

[0110] The overlap is calculated as the ratio of the area of ​​the overlapping portion of the two detection frames to the area of ​​the combined portion of the two detection frames.

[0111] Optionally, the preset overlap threshold is a preset value, for example, the preset overlap threshold is 80%, 90%, etc. This embodiment does not limit the value of the preset overlap threshold.

[0112] Optionally, after all characters are traversed, the candidate characters are deleted, including: determining whether the number of characters matches a preset number. If the number of characters does not match the preset number, the candidate characters are deleted after all characters are traversed. If the number of characters matches the preset number, the characters are sorted according to the position information to obtain the license plate information in the target image.

[0113] The preset number is determined according to the number of characters in the license plate in the target image.

[0114] In one example, the preset number is the same as the number of characters in the license plate in the target image.

[0115] If the number of characters matches the preset number, deleting candidate characters may result in the number of characters being less than the preset number, i.e., correctly detected characters being deleted. This can lead to the problem of mistakenly deleting license plate characters. Therefore, in this embodiment, if the number of characters matches the preset number, the step of deleting candidate characters is not required, thus avoiding the problem of mistakenly deleting license plate characters and improving license plate recognition accuracy.

[0116] To sum up, the license plate recognition method provided in this embodiment obtains a target image; inputs the target image into a pre-trained character detection model to obtain character information in the target image, the character information includes the character category and position information of each character, and the first sample data used in character detection model training is less than the first sample data used in license plate recognition model training; sorts each character according to the position information to obtain license plate information in the target image, the license plate information includes the character category; this can solve the problem that the license plate recognition model needs to be trained using a large amount of sample data, and when the collected sample data is small, the trained license plate recognition model has a low accuracy rate in recognizing license plates; since the first sample data used in character detection model training is less than the first sample data used in license plate recognition model training, the first sample data required for training is reduced, and the scope of use of the license plate recognition method can be expanded.

[0117] In addition, the first training set is used to train the initial network model to obtain an intermediate model; then, based on the intermediate model, the second training set is used to fine-tune the intermediate model again. During the fine-tuning process, the intermediate model focuses on learning the character features in the first sample vehicle image, which can improve the sensitivity of the character detection model in recognizing the character features in the first sample vehicle image.

[0118] In addition, by fusing the second sample vehicle image, the character category label corresponding to the second sample vehicle image, the first sample vehicle image, and the character category label corresponding to the first sample vehicle image in the second training set, the sensitivity of the character detection model in identifying the character features in the first sample vehicle image can be improved, and the effect of the oscillation of the verification set detection accuracy can be reduced, thereby improving the robustness of the model.

[0119] In addition, since the character detection model is established based on a network model without a priori box anchor, there is no need to pre-set the anchor size. Therefore, the problem of poor character detection accuracy caused by poor anchor size setting can be avoided; it can improve both the efficiency and accuracy of license plate recognition.

[0120] In addition, since the backbone network of the character detection model is obtained by pruning the backbone network of the traditional network model, and the connection relationship between the residual block and the downsampling layer is not changed during the pruning process, the difficulty of training the character detection model can be reduced while ensuring the detection accuracy of the character detection model.

[0121] In addition, since candidate characters are determined based on the degree of overlap between characters and the confidence level corresponding to the characters, and candidate characters are deleted after all characters are traversed, repeated detection caused by detection errors in the character detection network can be avoided, and the accuracy of license plate recognition can be improved.

[0122] In addition, since the position of each character is directly sorted according to the position information to obtain the license plate information when the number of characters matches the preset number, the deletion of correctly detected characters can be avoided, and the accuracy of license plate recognition can be improved.

[0123] Figure 4 4 is a block diagram of a license plate recognition device provided by an embodiment of the present application. The device includes at least the following modules: an image acquisition module 410, a character detection module 420, and a position sorting module 430.

[0124] An image acquisition module 410 is used to acquire a target image;

[0125] a character detection module 420 configured to input the target image into a pre-trained character detection model to obtain character information in the target image, wherein the character information includes the character category and position information of each character, wherein the first sample data used in training the character detection model is less than the first sample data used in training the license plate recognition model;

[0126] The position sorting module 430 sorts the characters according to the position information to obtain the license plate information in the target image, where the license plate information includes the character category.

[0127] For relevant details, please refer to the above method embodiment.

[0128] It should be noted that the license plate recognition device provided in the above embodiments is merely an example of the division of the above functional modules when performing license plate recognition. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the license plate recognition device can be divided into different functional modules to complete all or part of the functions described above. In addition, the license plate recognition device provided in the above embodiments and the license plate recognition method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0129] Figure 5 A block diagram of an electronic device provided in one embodiment of the present application includes at least a processor 501 and a memory 502 .

[0130] The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0131] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction, which is executed by the processor 501 to implement the license plate recognition method provided in the method embodiment of the present application.

[0132] In some embodiments, the electronic device may optionally include a peripheral device interface and at least one peripheral device. The processor 501, memory 502, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Illustratively, the peripheral devices include, but are not limited to, a radio frequency circuit, a touchscreen display, an audio circuit, and a power supply.

[0133] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0134] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored. The program is loaded and executed by a processor to implement the license plate recognition method of the above method embodiment.

[0135] Optionally, the application further provides a computer product, comprising a computer readable storage medium, wherein a program is stored in the computer readable storage medium, and the program is loaded and executed by a processor to implement the license plate recognition method of the above method embodiment.

[0136] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the description.

[0137] The above embodiments only express several implementation manners of the application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which are all within the protection scope of the application. Therefore, the patent protection scope of the application should be subject to the appended claims.

Claims

1. A license plate recognition method, characterized in that: The method comprises: Acquire the target image; Inputting the target image into a pre-trained character detection model to obtain character information in the target image, wherein the character information includes a character category of each character and position information of the character, wherein the first sample data used in training the character detection model is less than the first sample data used in training the license plate recognition model; The training process of the character detection model includes: Using a first training set to train a pre-created initial network model to obtain an intermediate model; the first training set includes a second sample vehicle image and a character category label of a license plate region in the second sample vehicle image; The intermediate model is trained using a second training set to obtain the character detection model; the second training set includes the second sample vehicle image, the character category label corresponding to the second sample vehicle image, the first sample vehicle image, and the character category label of the license plate area in the first sample vehicle image; wherein the character features in the first sample vehicle image are different from the character features in the second sample vehicle image; and the number of images of the first sample vehicle image in the second training set is greater than the number of images of the second sample vehicle image; When the character detection model is used to identify license plate information of a vehicle at a target geographic location, the first sample vehicle image represents a vehicle image captured at the target geographic location, and the second sample vehicle image represents a vehicle image captured at any geographic location; The characters are sorted according to the position information to obtain the license plate information in the target image, where the license plate information includes the character category.

2. The method according to claim 1, characterized in that The character detection model is established based on a network model without a priori box anchor.

3. The method according to claim 1, characterized in that The character detection model includes a backbone network, a connection layer and a prediction layer, wherein the backbone network is connected to the prediction layer through the connection layer; The backbone network includes a plurality of pruned residual blocks and a downsampling layer located after the residual blocks; The prediction layer is used to locate the characters in the feature map output by the backbone network and classify the characters to obtain the character information.

4. The method according to claim 1, wherein The character information also includes the confidence level corresponding to each position information; The step of sorting the characters according to the position information to obtain the license plate information in the target image includes: For each character, determining, based on the position information, a degree of overlap between the detection frame of the character and the detection frame of other characters; In the case where the degree of overlap is greater than the overlap threshold, determining a character with the smallest confidence from the other characters and the character to obtain a candidate character; After all characters are traversed, the candidate characters are deleted; The deleted characters are sorted according to the position information to obtain the license plate information of the target image.

5. The method according to claim 4, characterized in that Deleting the candidate characters after all characters have been traversed includes: determining whether the number of characters matches a preset number; In the case that the number of the characters does not match the preset number, the candidate characters are deleted after all characters are traversed.

6. The method according to claim 5, characterized in that After determining whether the number of characters matches the preset number, the method further includes: When the number of the characters matches the preset number, the characters are sorted according to the position information to obtain the license plate information in the target image.

7. An electronic device, characterized in that: The device includes a processor and a memory; the memory stores a program, and the program is loaded and executed by the processor to implement the license plate recognition method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The storage medium stores a program, which, when executed by a processor, is used to implement the license plate recognition method according to any one of claims 1 to 6.

Images