License plate number recognition method and related equipment

Abstract

The present application discloses a method for recognizing a license plate number and related equipment thereof, the method comprising: obtaining an image to be processed; recognizing the license plate number in the image to be processed by a preset supervised branch sub-model in a preset license plate number recognition model to obtain a first license plate number recognition result; the preset license plate number recognition model comprises a preset contrast learning branch sub-model, the sub-model is trained by training negative samples and training positive samples, the training negative samples are derived from images corresponding to incorrectly predicted license plate numbers, the training positive samples are images corresponding to correctly predicted license plate numbers, the preset supervised branch sub-model extracts target features having a first feature distance from negative sample features and a second feature distance from positive sample features at different feature levels when predicting the image to be processed, the negative sample features are features having a counter-effect, the positive sample features are features having a positive effect, and the first feature distance is greater than the second feature distance; the present application improves the recognition accuracy of license plates.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a license plate number recognition method and related equipment. Background Art

[0002] With the continuous development of modern manufacturing and technology, more and more motor vehicles appear in society. Based on this, the automated management of motor vehicles is becoming more and more important. Among them, the recognition of the license plate number as an important identification of the vehicle is the top priority.

[0003] In current technologies, license plate number recognition technology based on deep learning models has achieved a high recognition rate in scenarios such as mobile traffic and entrances and exits. However, in low-quality license plate number recognition scenarios such as poor imaging, since license plate number recognition technology based on deep learning models does not effectively utilize low-quality samples, there is a problem of low license plate number recognition accuracy in low imaging scenarios. Summary of the invention

[0004] The main purpose of this application is to provide a license plate number recognition method and related equipment, aiming to solve the technical problem of low license plate number recognition accuracy in low imaging scenarios.

[0005] To achieve the above purpose, the present application proposes a license plate number recognition method, the method comprising:

[0006] Get the image to be processed;

[0007] Based on the image to be processed, the license plate number in the image to be processed is recognized by a preset supervised branch sub-model in the preset license plate number recognition model to obtain a first license plate number recognition result; wherein the preset license plate number recognition model also includes a preset contrast learning branch sub-model, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, the training negative samples are from the images corresponding to the license plate numbers previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, the training positive samples are images of the correctly predicted license plate numbers corresponding to the training negative samples, and the preset supervised branch sub-model extracts target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels during the process of predicting the image to be processed, the negative sample features are features that have a reverse effect on the correctly predicted license plate numbers learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples, the positive sample features are features that have a positive effect on the correctly predicted license plate numbers learned, and the first feature distance is greater than the second feature distance.

[0008] In one embodiment, before the step of recognizing the license plate number in the image to be processed by a preset license plate number recognition model based on the image to be processed, the method further comprises:

[0009] Get a sample image of a license plate number with a preset label;

[0010] Recognize the license plate number in the license plate number sample image by using the first training model, and obtain a second license plate number recognition result;

[0011] Select a first target license plate number that is the same as a preset label from the second license plate number recognition result, and use the license plate image corresponding to the first target license plate number as a training positive sample; select a second target license plate number that is different from the preset label from the second license plate number recognition result, and use the second target license plate number as the predicted wrong license plate number;

[0012] Based on the license plate image corresponding to the wrongly predicted license plate number, determining the training negative samples of the preset contrast learning branch sub-model, wherein the training negative samples also include license plate images corresponding to other license plate numbers;

[0013] Based on the training negative samples and the training positive samples, the preset contrast learning branch sub-model and the preset supervised branch sub-model are synchronously trained, and the trained preset contrast learning branch sub-model and the preset supervised branch sub-model are used as the preset license plate number recognition model.

[0014] In one embodiment, the step of determining the training negative sample of the preset contrastive learning branch sub-model based on the license plate image corresponding to the wrongly predicted license plate number includes any one of the following:

[0015] Using the license plate image corresponding to the wrongly predicted license plate number as a training negative sample for the preset contrast learning branch sub-model;

[0016] The wrongly predicted license plate number is partially simulated or simulated, and the license plate image corresponding to the partially simulated or simulated license plate number is used as a training negative sample for the preset contrast learning branch sub-model.

[0017] In one embodiment, the loss function of the preset license plate number recognition model is a preset overall loss function, and the preset overall loss function includes a cross entropy loss function corresponding to a preset supervised branch sub-model and a contrastive learning branch loss function corresponding to a preset contrastive learning branch sub-model;

[0018] The step of synchronously training to obtain the preset contrastive learning branch sub-model and the preset supervised branch sub-model based on the training negative samples and the training positive samples comprises:

[0019] The first training model is used to identify the license plate number of the corresponding training positive sample to obtain a third license plate number recognition result, and a first loss result is determined based on the third license plate number recognition result and the preset label; the first training model and the second training model are used to extract features of different levels for the training negative sample and / or the corresponding training positive sample, and the features extracted by different branch models at the same level are compared to obtain a multi-level feature comparison loss result;

[0020] Substitute the first loss result into the cross entropy loss function in the preset overall loss function, and substitute the multi-level feature comparison loss result into the contrast learning branch loss function in the preset overall loss function, and determine whether the preset overall loss function is the minimum. If it is determined that the preset overall loss function is not the minimum, return to the step of identifying the license plate number of the corresponding training positive sample through the first training model to obtain the third license plate number recognition result, until the number of training times reaches the preset number of times or the preset overall loss function is minimized, so as to train the preset contrast learning branch sub-model and the preset supervised branch sub-model.

[0021] In one embodiment, before the step of substituting the multi-level feature comparison loss result into the contrastive learning branch loss function in the preset overall loss function, the step includes:

[0022] Receiving the image quality of the license plate number sample image sent by a preset quality scoring system;

[0023] According to the image quality, a contrastive learning branch loss function in a preset overall loss function is determined, wherein the contrastive learning branch loss function is a multi-level feature fusion loss function that is negatively correlated with the quality score.

[0024] In one embodiment, the step of substituting the first loss result into the cross entropy loss function in the preset overall loss function includes:

[0025] Determine the overall loss function for:

[0026] ;in, is the cross entropy loss function; is the loss function for contrastive learning branch;

[0027] Said ;

[0028] in, It is the comparative loss result of the same level features extracted by different branch sub-models based on training positive samples and / or training negative samples. is the corresponding weight, Rate the quality.

[0029] In one embodiment, the different levels of feature contrast loss results include global feature contrast loss results, character feature contrast loss results, and character contrast loss results;

[0030] The step of performing feature extraction at different levels on the training negative samples and / or the corresponding training positive samples through the first training model and the second training model, and comparing the features extracted by different branch models at the same level to obtain a multi-level feature comparison loss result includes:

[0031] Through the first training model, each training positive sample Extract global features, character features and characters to obtain first extracted features; and use the second training model to train each positive sample. Another training positive sample that constitutes a positive sample pair Extracting global features, character features, and characters to obtain second extracted features, wherein different license plate images corresponding to the same license plate number constitute positive sample pairs;

[0032] Through the second training model, for each training positive sample Another training negative sample yi constituting a negative sample pair is subjected to global feature, character feature and character extraction to obtain a third extracted feature, wherein different license plate images corresponding to different license plate numbers constitute a negative sample pair with each other;

[0033] According to the first extracted feature, the second extracted feature and the third extracted feature, a multi-level feature comparison loss result is determined as:

[0034] ;

[0035]

[0036]

[0037] in, is the global feature contrast loss result, is the character feature comparison loss result, is the character comparison loss result; It means to obtain the expectation corresponding to the global feature, Indicates the expectation corresponding to the character feature, E S Indicates the expectation of obtaining the corresponding characters. is the distance function defined to characterize the similarities at different feature levels. ,in, is the first extracted feature extracted by the preset supervised learning sub-model branch, = , is the second extracted feature or the third extracted feature extracted by the preset contrast learning branch sub-model, = or = ;

[0038] The step of substituting the multi-level feature comparison loss result into the contrastive learning branch loss function in the preset overall loss function includes:

[0039] Will , , Substitute into the In, confirm ;

[0040] in, is the weight corresponding to the global feature contrast loss result, where is the weight corresponding to the character feature comparison loss result, where is the weight corresponding to the character contrast loss result.

[0041] In addition, to achieve the above purpose, the present application also proposes a license plate number recognition device, the device comprising:

[0042] An acquisition module, used for acquiring an image to be processed;

[0043] A recognition module is used to recognize the license plate number in the image to be processed based on the image to be processed through a preset supervised branch sub-model in a preset license plate number recognition model to obtain a first license plate number recognition result; wherein the preset license plate number recognition model also includes a preset contrast learning branch sub-model, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, the training negative samples are derived from the images corresponding to the license plate numbers previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, the training positive samples are images of the correctly predicted license plate numbers corresponding to the training negative samples, the preset supervised branch sub-model extracts target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels during the process of predicting the image to be processed, the negative sample features are features that have a reverse effect on the correctly predicted license plate numbers learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples, the positive sample features are features that have a positive effect on the correctly predicted license plate numbers learned, and the first feature distance is greater than the second feature distance.

[0044] In one embodiment, the license plate number recognition device is further used to implement:

[0045] Get a sample image of a license plate number with a preset label;

[0046] Recognize the license plate number in the license plate number sample image by using the first training model, and obtain a second license plate number recognition result;

[0047] Select a first target license plate number that is the same as a preset label from the second license plate number recognition result, and use the license plate image corresponding to the first target license plate number as a training positive sample; select a second target license plate number that is different from the preset label from the second license plate number recognition result, and use the second target license plate number as the predicted wrong license plate number;

[0048] Based on the license plate image corresponding to the wrongly predicted license plate number, determining the training negative samples of the preset contrast learning branch sub-model, wherein the training negative samples also include license plate images corresponding to other license plate numbers;

[0049] Based on the training negative samples and the training positive samples, the preset contrast learning branch sub-model and the preset supervised branch sub-model are synchronously trained, and the trained preset contrast learning branch sub-model and the preset supervised branch sub-model are used as the preset license plate number recognition model.

[0050] In one embodiment, the license plate number recognition device is further used to implement any one of the following:

[0051] Using the license plate image corresponding to the wrongly predicted license plate number as a training negative sample for the preset contrast learning branch sub-model;

[0052] The wrongly predicted license plate number is partially simulated or simulated, and the license plate image corresponding to the partially simulated or simulated license plate number is used as a training negative sample for the preset contrast learning branch sub-model.

[0053] In one embodiment, the loss function of the preset license plate number recognition model is a preset overall loss function, and the preset overall loss function includes a cross entropy loss function corresponding to a preset supervised branch sub-model and a contrastive learning branch loss function corresponding to a preset contrastive learning branch sub-model;

[0054] The vehicle license plate number recognition device is also used to achieve:

[0055] The first training model is used to identify the license plate number of the corresponding training positive sample to obtain a third license plate number recognition result, and a first loss result is determined based on the third license plate number recognition result and the preset label; the first training model and the second training model are used to extract features of different levels for the training negative sample and / or the corresponding training positive sample, and the features extracted by different branch models at the same level are compared to obtain a multi-level feature comparison loss result;

[0056] Substitute the first loss result into the cross entropy loss function in the preset overall loss function, and substitute the multi-level feature comparison loss result into the contrast learning branch loss function in the preset overall loss function, and determine whether the preset overall loss function is the minimum. If it is determined that the preset overall loss function is not the minimum, return to the step of identifying the license plate number of the corresponding training positive sample through the first training model to obtain the third license plate number recognition result, until the number of training times reaches the preset number of times or the preset overall loss function is minimized, so as to train the preset contrast learning branch sub-model and the preset supervised branch sub-model.

[0057] In one embodiment, the license plate number recognition device is further used to implement:

[0058] Receiving the image quality of the license plate number sample image sent by a preset quality scoring system;

[0059] According to the image quality, a contrastive learning branch loss function in a preset overall loss function is determined, wherein the contrastive learning branch loss function is a multi-level feature fusion loss function that is negatively correlated with the quality score.

[0060] In one embodiment, the license plate number recognition device is further used to implement:

[0061] Determine the overall loss function for:

[0062] ;in, is the cross entropy loss function; is the loss function for contrastive learning branch;

[0063] Said ;

[0064] in, It is the comparative loss result of the same level features extracted by different branch sub-models based on training positive samples and / or training negative samples. is the corresponding weight, Rate the quality.

[0065] In one embodiment, the different levels of feature contrast loss results include global feature contrast loss results, character feature contrast loss results, and character contrast loss results;

[0066] The vehicle license plate number recognition device is also used to achieve:

[0067] Through the first training model, each training positive sample Extract global features, character features and characters to obtain first extracted features; and use the second training model to train each positive sample. Another training positive sample that constitutes a positive sample pair Extracting global features, character features, and characters to obtain second extracted features, wherein different license plate images corresponding to the same license plate number constitute positive sample pairs;

[0068] Through the second training model, for each training positive sample Another training negative sample yi constituting a negative sample pair is subjected to global feature, character feature and character extraction to obtain a third extracted feature, wherein different license plate images corresponding to different license plate numbers constitute a negative sample pair with each other;

[0069] According to the first extracted feature, the second extracted feature and the third extracted feature, a multi-level feature comparison loss result is determined as:

[0070] ;

[0071]

[0072]

[0073] in, is the global feature contrast loss result, is the character feature comparison loss result, is the character comparison loss result; It means to obtain the expectation corresponding to the global feature, It represents the expectation corresponding to the character feature, E S Indicates the expectation of obtaining the corresponding characters. is the distance function defined to characterize the similarities at different feature levels. ,in, is the first extracted feature extracted by the preset supervised learning sub-model branch, = , is the second extracted feature or the third extracted feature extracted by the preset contrast learning branch sub-model, = or = ;

[0074] The step of substituting the multi-level feature comparison loss result into the contrastive learning branch loss function in the preset overall loss function includes:

[0075] Will , , Substitute into the In, confirm ;

[0076] in, is the weight corresponding to the global feature contrast loss result, where is the weight corresponding to the character feature comparison loss result, where is the weight corresponding to the character comparison loss result.

[0077] In addition, to achieve the above-mentioned purpose, the present application also proposes a license plate number recognition device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the license plate number recognition method described above.

[0078] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the license plate number recognition method described above are implemented.

[0079] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the license plate number recognition method described above are implemented.

[0080] The above technical solution proposed in this application has the following technical effects:

[0081] Compared with the related art, in which the license plate number recognition technology based on the deep learning model does not effectively utilize low-quality samples, resulting in low license plate number recognition accuracy in low imaging scenarios, the present application obtains an image to be processed; based on the image to be processed, the license plate number in the image to be processed is recognized by a preset supervised branch sub-model in a preset license plate number recognition model, to obtain a first license plate number recognition result; wherein the preset license plate number recognition model also includes a preset contrast learning branch sub-model, and the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, and the training negative samples are derived from the first training model corresponding to the preset supervised branch sub-model. Predict an image corresponding to a license plate number, the training positive sample is an image of the correctly predicted license plate number corresponding to the training negative sample, the preset supervised branch sub-model extracts target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels during the process of predicting the processed image, the negative sample features are features that have a reverse effect on the correct prediction of the license plate number learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples, the positive sample features are features that have a positive effect on the correct prediction of the license plate number, and the first feature distance is greater than the second feature distance.It can be understood that in the present application, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, and the training negative samples are derived from the images corresponding to the license plate numbers that were previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model (the training positive samples are the correctly predicted license plate numbers corresponding to the training negative samples). In low imaging scenarios, there are generally many incorrect predictions. That is, in the present application, the training negative samples used for training the contrast learning branch sub-model are generated based on low-quality samples in low imaging scenarios. That is, the contrast learning branch sub-model of the present application selectively selects high-value negative samples in low imaging scenarios. Therefore, based on the high-value training negative samples, the second training model corresponding to the preset contrast learning branch sub-model can accurately learn the correct predictions from the training negative samples and the training positive samples. The features of measuring the license plate number have a reverse effect, and the features that have a positive effect on correctly predicting the license plate number can be accurately learned. In the process of predicting the image to be processed, the preset supervised branch sub-model extracts target features that have a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels, and the first feature distance is greater than the second feature distance. Therefore, the preset supervised branch sub-model described in the present application can shorten the feature distance between the image to be processed and the training positive sample, and increase the feature distance between the image to be processed and the training negative sample during the prediction process. That is, the preset supervised branch sub-model can predict the results that are closer to the features of the positive sample, that is, closer to the real results, even in low imaging scenarios during the prediction process. Therefore, the recognition accuracy of the license plate number can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0083] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0084] Figure 1 A first flow chart of the first embodiment of the vehicle license plate number recognition method of the present application;

[0085] Figure 2 A second flow chart of the second embodiment of the vehicle license plate number recognition method of the present application;

[0086] Figure 3 This is a schematic diagram of the third process involved in the third embodiment of the license plate number recognition method of this application;

[0087] Figure 4 A schematic diagram of a first scenario provided in Example 1 of the vehicle license plate number recognition method of the present application;

[0088] Figure 5 A schematic diagram of a second scenario provided for the second embodiment of the vehicle license plate number recognition method of the present application;

[0089] Figure 6 A schematic diagram of the device structure of the hardware operating environment involved in the license plate number recognition method in the embodiment of the present application;

[0090] Figure 7 This is a schematic diagram of the module structure of the license plate number recognition device according to an embodiment of the present application.

[0091] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0092] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0093] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0094] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a license plate number recognition device, etc. The following takes the license plate number recognition device as an example to illustrate this embodiment and the following embodiments.

[0095] Based on this, the present application embodiment provides a license plate number recognition method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the license plate number recognition method of the present application.

[0096] In this embodiment, the license plate number recognition method includes steps S10 to S20:

[0097] Step S10, obtaining an image to be processed;

[0098] It should be noted that the image to be processed can be an image input into the license plate number recognition device through any means. For example, the image to be processed can be an image directly transmitted to the license plate number recognition device by a camera such as a street camera, or the image to be processed can be an image uploaded to the license plate number recognition device by a client such as a user's mobile phone terminal, or the image to be processed can be an image uploaded to the license plate number recognition device by parking lot monitoring, or the image to be processed can be an image uploaded to the license plate number recognition device by a cloud platform, etc., without specific limitation.

[0099] Furthermore, the image to be processed may be any type of image. For example, the image to be processed may be an image captured by a camera, or the image to be processed may be an image obtained by screenshot.

[0100] Furthermore, the image to be processed may be an image in various formats, such as a gpg format, or a BMP format, etc., which is not specifically limited.

[0101] It should be emphasized that the image to be processed may not include the license plate number of any vehicle, or may include the license plate number of only one vehicle, or may include the license plate numbers of more than one vehicle.

[0102] It should be emphasized that the license plate number of the vehicle in the image to be processed can be clear or blurred, such as Figure 4 The license plate number of the vehicle in the image to be processed is blurred. If the license plate number of the vehicle in the image to be processed is clear, the corresponding image quality is high quality. If the license plate number of the vehicle in the image to be processed is blurred, the corresponding image quality is low quality.

[0103] After the license plate number recognition device obtains the image to be processed, it can process the image to be processed. For example, after the license plate number recognition device receives an image of a vehicle traveling on the emergency lane of a highway uploaded by the user, it can identify and process the image to be processed (including the vehicle number of the vehicle traveling on the emergency lane). Or, for example, after the license plate number recognition device receives an overloaded vehicle photographed by a camera, it can identify and process the image to be processed (including the vehicle number of the overloaded vehicle).

[0104] Step S20, based on the image to be processed, the license plate number in the image to be processed is recognized by a preset supervised branch sub-model in the preset license plate number recognition model to obtain a first license plate number recognition result; wherein the preset license plate number recognition model also includes a preset contrast learning branch sub-model, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, the training negative samples are from the images corresponding to the license plate numbers previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, the training positive samples are images of the correctly predicted license plate numbers corresponding to the training negative samples, the preset supervised branch sub-model extracts target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels during the process of predicting the image to be processed, the negative sample features are features that have a reverse effect on the correctly predicted license plate numbers learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples, the positive sample features are features that have a positive effect on the correctly predicted license plate numbers learned, and the first feature distance is greater than the second feature distance.

[0105] In this embodiment, the network structure of the preset license plate number recognition model is as follows: Figure 5 As shown, the preset license plate number recognition model is composed of a preset supervised branch sub-model and a comparative learning branch sub-model, wherein the basic model structures of the preset supervised branch sub-model and the comparative learning branch sub-model can be exactly the same, for example, the first training model corresponding to the supervised branch model and the second training model corresponding to the comparative learning branch sub-model can both be license plate recognition baseline models.

[0106] In this embodiment, it should be noted that the training process corresponding to the preset supervised branch sub-model and the contrastive learning branch sub-model can be performed synchronously. And the contrastive learning branch sub-model is specially designed (special design of training samples and special design of loss function), which enables the corresponding preset supervised branch sub-model to learn different levels of difference features and consistent features, and then, accurate recognition results can be obtained by performing recognition processing on the processed image through the preset supervised branch sub-model.

[0107] Among them, the training process for the supervised branch model can be:

[0108] Specifically, a sample image of a license plate number with a preset label is first obtained. In the process of obtaining the sample image of the license plate number, for vehicles with difficult-to-distinguish license plate numbers, the license plate numbers are found through the appearance of the corresponding vehicles (the found license plate numbers are used as preset labels to ensure that the labels are noise-free). After the corresponding license plate numbers are found, a license plate rendering tool is used to generate an image corresponding to the standard font template license plate number as a training positive sample. Then, the first training model is trained based on the training positive sample to finally obtain a model with a preset supervised branch.

[0109] The training process for the contrastive learning branch sub-model can be:

[0110] First, obtain the license plate number that is incorrectly predicted by the first training model corresponding to the preset supervised learning branch sub-model, use the license plate rendering tool, and generate an image corresponding to the standard font template license plate number based on the incorrectly predicted license plate number as a training negative sample for the comparative learning branch sub-model; that is, in this embodiment, the training negative sample used to train the comparative learning branch sub-model is derived from the image corresponding to the license plate number that was previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, and therefore, the training negative sample is a negative sample that conforms to the actual situation, rather than a simulated or emulated negative sample that deviates from the actual situation, that is, the training negative sample in this embodiment is a relatively high-quality negative sample. Since the training negative sample is a relatively high-quality negative sample, the model trained by the relatively high-quality training negative sample can more accurately learn the features that have an inverse effect on the correct prediction of the license plate number. Furthermore, in the present application, the training positive sample of the comparative learning branch sub-model is an image of the correctly predicted license plate number corresponding to the training negative sample, and therefore, the training positive sample is also a corresponding high Quality positive samples, therefore, the model obtained by training based on the high-quality positive samples can more accurately learn the features that have a positive effect on correctly predicting the license plate number, that is, the preset supervised learning branch sub-model in this embodiment is based on the contrast learning branch sub-model (specifically by jointly training a loss function to minimize the corresponding loss function). In the process of predicting the image to be processed, target features with a corresponding first feature distance to the negative sample features and a corresponding second feature distance to the positive sample features can be extracted at different feature levels, and the first feature distance is greater than the second feature distance (that is, far away from the negative sample features and close to the positive sample features). That is, the preset supervised branch sub-model in the present application can shorten the feature distance between the image to be processed and the training positive sample, and shorten the feature distance between the image to be processed and the training negative sample during the prediction process, that is, the preset supervised branch sub-model can predict the result closer to the feature of the positive sample, that is, closer to the real result, even in the low imaging scenario during the prediction process, thereby improving the recognition accuracy of the license plate number.

[0111] Specifically, before the step of recognizing the license plate number in the image to be processed by a preset license plate number recognition model based on the image to be processed, the following steps S01-S05 are included:

[0112] Step S01, obtaining a sample image of a license plate number with a preset label;

[0113] Step S02, recognizing the license plate number in the license plate number sample image by using the first training model, and obtaining a second license plate number recognition result;

[0114] Step S03, selecting a first target license plate number that is the same as a preset label from the second license plate number recognition result, and using the license plate image corresponding to the first target license plate number as a training positive sample, selecting a second target license plate number that is different from the preset label from the second license plate number recognition result, and using the second target license plate number as the predicted wrong license plate number;

[0115] Step S04, determining the training negative samples of the preset contrast learning branch sub-model based on the license plate image corresponding to the wrongly predicted license plate number, wherein the training negative samples also include license plate images corresponding to other license plate numbers;

[0116] Step S05, based on the training negative samples and the training positive samples, synchronously train to obtain the preset contrast learning branch sub-model and the preset supervised branch sub-model, and use the trained preset contrast learning branch sub-model and the preset supervised branch sub-model as the preset license plate number recognition model.

[0117] In this embodiment, the process of how to obtain training negative samples and training positive samples is specifically explained. The biggest difference between the present technology and the present technology is that the training negative samples are not only derived from the images of the license plate numbers that were previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, but the training negative samples also include license plate images corresponding to other license plate numbers. It can be understood that the proportion of license plate images corresponding to other license plate numbers in the training negative samples is less than the first preset proportion of 3%. Therefore, the training negative samples maintain the characteristics consistent with the actual situation on the basis of broadening the sample dimension. Therefore, the preset comparative learning branch sub-model can be accurately obtained, and then, the preset supervised branch sub-model can accurately learn the corresponding positive sample features and negative sample features, thereby improving the prediction accuracy of the preset supervised branch sub-model.

[0118] Specifically, for example, in this embodiment, the training negative samples are not only from the license plate numbers that were previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, such as Figure 4As shown, it not only includes the images corresponding to the license plate numbers Sichuan QX**50, Zhejiang AH**K1, Hunan DH**50, and Sichuan QH**80 (the images of the wrong license plate numbers predicted before), but also includes the license plate images corresponding to other license plate numbers. For example, a license plate number Su AX**6H is not in the training positive sample, but the license plate image of the license plate number Su AX**6H can be used as a training negative sample.

[0119] In this embodiment, the step of determining the training negative sample of the preset contrast learning branch sub-model based on the license plate image corresponding to the wrongly predicted license plate number includes any one of the following:

[0120] Using the license plate image corresponding to the wrongly predicted license plate number as a training negative sample for the preset contrast learning branch sub-model;

[0121] The wrongly predicted license plate number is partially simulated or simulated, and the license plate image corresponding to the partially simulated or simulated license plate number is used as a training negative sample for the preset contrast learning branch sub-model.

[0122] In this embodiment, only the license plate image corresponding to the wrongly predicted license plate number can be used as the training negative sample of the preset contrast learning branch sub-model, or the wrongly predicted license plate number can be partially simulated or simulated, and the license plate image corresponding to the partially simulated or simulated license plate number can be used as the training negative sample of the preset contrast learning branch sub-model. In order to ensure the high quality of the training negative samples, the proportion of the license plate image corresponding to the partially simulated or simulated license plate number does not exceed the preset second preset proportion, such as 5%, wherein the first preset proportion can be equal to the preset second preset proportion, or the first preset proportion can be less than the preset second preset proportion.

[0123] Overall, the positive and negative sample selection scheme proposed in this embodiment enables the comparative learning branch sub-model to utilize the information of the supervised learning branch sub-model to selectively select relatively high-value negative samples and learn the characteristics of relatively high-value negative samples, which in turn promotes the prediction of the supervised learning branch sub-model, thereby helping the model solve the problem of poor license plate recognition performance in low-quality scenarios.

[0124] That is, it can be understood that in the present application, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, and the training negative samples are derived from the images corresponding to the license plate numbers that were previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model (the training positive samples are the correctly predicted license plate numbers corresponding to the training negative samples), and in low imaging scenarios, there are generally many incorrect predictions, that is, in the present application, the training negative samples used for training the contrast learning branch sub-model are targetedly generated based on low-quality samples in low imaging scenarios, that is, the contrast learning branch sub-model of the present application selectively selects high-value negative samples in low imaging scenarios, and therefore, based on the high-value training negative samples, the second training model corresponding to the preset contrast learning branch sub-model can accurately learn the positive samples from the training negative samples and the training positive samples. The features that have a negative effect on correctly predicting the license plate number can be accurately learned, and the features that have a positive effect on correctly predicting the license plate number can be accurately learned. In the process of predicting the image to be processed, the preset supervised branch sub-model extracts target features that have a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels, and the first feature distance is greater than the second feature distance. Therefore, the preset supervised branch sub-model in the present application can shorten the feature distance between the image to be processed and the training positive sample, and increase the feature distance between the image to be processed and the training negative sample during the prediction process. That is, the preset supervised branch sub-model can predict the results that are closer to the features of the positive sample, that is, closer to the real results, even in low imaging scenarios during the prediction process. Therefore, the recognition accuracy of the license plate number can be improved.

[0125] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above introduction, and will not be repeated in the following. Figure 2 , the loss function of the preset license plate number recognition model is a preset overall loss function, and the preset overall loss function includes a cross entropy loss function corresponding to a preset supervised branch sub-model and a contrastive learning branch loss function corresponding to a preset contrastive learning branch sub-model;

[0126] The step of synchronously training to obtain the preset contrastive learning branch sub-model and the preset supervised branch sub-model based on the training negative samples and the training positive samples comprises:

[0127] Step A1, using the first training model to identify the license plate number of the corresponding training positive sample, obtain a third license plate number recognition result, and determine a first loss result based on the third license plate number recognition result and the preset label; using the first training model and the second training model, extracting features at different levels for the training negative sample and / or the corresponding training positive sample, and comparing the features extracted by different branch models at the same level to obtain a multi-level feature comparison loss result;

[0128] In this embodiment, the process of how to train the corresponding preset contrast learning branch sub-model and the preset supervised branch sub-model is specifically explained, and how to specially design the corresponding loss function is explained so that the preset supervised branch sub-model can extract target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels during the process of predicting the processed image.

[0129] Specifically, in each round of training, the license plate number of the corresponding training positive sample is recognized through the first training model to obtain a third license plate number recognition result, and the first loss result is determined based on the third license plate number recognition result and the preset label, that is, the predicted result (the third license plate number recognition result) is compared with the actual result (preset label) to determine the corresponding first loss value, that is, the first loss result.

[0130] In synchronization with the process of obtaining the first loss result, in this embodiment, the first training model and the second training model are also used to perform feature extraction at different levels such as the global feature level, the character feature level, and the character level on the training negative samples and / or the corresponding training positive samples, and the features extracted by different branch models at the same level are compared to obtain a multi-level feature comparison loss result.

[0131] Specifically, the multi-level feature comparison loss result includes a global feature comparison loss result, a character feature comparison loss result and a character comparison loss result; wherein the global feature comparison loss result is to compare the global features obtained by different model branches, and then obtain the global feature comparison loss result, the character feature comparison loss result is to compare the character features obtained by different model branches, and then obtain the character feature comparison loss result, and the character comparison loss result is to compare the characters obtained by different model branches, and then obtain the character comparison loss result.

[0132] The step of performing feature extraction at different levels on the training negative samples and / or the corresponding training positive samples through the first training model and the second training model, and comparing the features extracted by different branch models at the same level to obtain a multi-level feature comparison loss result includes:

[0133] Through the first training model, each training positive sample Extract global features, character features and characters to obtain first extracted features; and use the second training model to train each positive sample. Another training positive sample that constitutes a positive sample pair Extracting global features, character features, and characters to obtain second extracted features, wherein different license plate images corresponding to the same license plate number constitute positive sample pairs;

[0134] Through the second training model, for each training positive sample Another training negative sample yi constituting a negative sample pair is subjected to global feature, character feature and character extraction to obtain a third extracted feature, wherein different license plate images corresponding to different license plate numbers constitute a negative sample pair with each other;

[0135] According to the first extracted feature, the second extracted feature and the third extracted feature, a multi-level feature comparison loss result is determined as:

[0136] ;

[0137]

[0138]

[0139] in, is the global feature contrast loss result, is the character feature comparison loss result, is the character comparison loss result; It means to obtain the expectation corresponding to the global feature, Indicates the expectation corresponding to the character feature, E S Indicates the expectation of obtaining the corresponding characters. is the distance function defined to characterize the similarities at different feature levels. ,in, is the first extracted feature extracted by the preset supervised learning sub-model branch, = , is the second extracted feature or the third extracted feature extracted by the preset contrast learning branch sub-model, = or = Specifically, if the = for ,but is the global feature in the second extracted feature, if for ,but is the character feature in the second feature extraction, if for ,but is a character in the second extracted feature. Specifically, if the = for ,but is the global feature in the third extracted feature, if for ,but is the character feature in the third feature extraction, if for ,but is the character in the third extracted feature.

[0140] Step A2, substitute the first loss result into the cross entropy loss function in the preset overall loss function, and substitute the multi-level feature comparison loss result into the comparative learning branch loss function in the preset overall loss function, and determine whether the preset overall loss function is the minimum. If it is determined that the preset overall loss function is not the minimum, return to the step of identifying the license plate number of the corresponding training positive sample through the first training model to obtain the third license plate number recognition result, until the number of training times reaches the preset number of times or the preset overall loss function is minimized, so as to train the preset comparative learning branch sub-model and the preset supervised branch sub-model.

[0141] In this embodiment, before the step of substituting the first loss result into the cross entropy loss function in the preset overall loss function, the step includes:

[0142] Determine the overall loss function for:

[0143] ;in, is the cross entropy loss function; is the loss function for contrastive learning branch;

[0144] Said ;

[0145] in, It is the comparative loss result of the same level features extracted by different branch sub-models based on training positive samples and / or training negative samples. is the corresponding weight, Rate the quality.

[0146] In this embodiment, the overall loss function includes a cross entropy loss function of a preset supervised sub-model and a contrastive learning branch loss function of a preset contrastive learning branch sub-model, wherein the contrastive learning branch loss function is not determined, but is associated with the quality score of the corresponding training sample. If the quality score of the training sample is low, the proportion of low-quality samples is increased accordingly. This is because the quality score is low, then As increases, the proportion of the preset contrastive learning branch sub-model in the overall loss function increases, and the impact on the response result is greater. In addition, the contrastive learning branch loss function is also association, the association, is a distance function defined to characterize similarities at different feature levels. And the feature association extracted by the preset supervised sub-model (refer to the above formula), that is, based on this, the corresponding preset supervised sub-model can extract target features with a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels.

[0147] It can be understood that the two sub-models are trained synchronously, and the purpose of training is to minimize the overall loss function.

[0148] The specific training process may be: iterative training, and adjusting the model parameters of the overall model according to the training results of each round, until the number of training times reaches a preset number of times or the preset overall loss function is minimized, so as to train and obtain the preset contrast learning branch sub-model and the preset supervised branch sub-model. It should be noted that in the adjustment process, it can be understood that the output result of the preset contrast learning branch sub-model will affect the adjustment of the model parameters of the corresponding preset supervised branch sub-model.

[0149] In this embodiment, the first training model is used to identify the license plate number of the corresponding training positive sample to obtain a third license plate number recognition result, and the first loss result is determined based on the third license plate number recognition result and the preset label; the first training model and the second training model are used to extract features at different levels for the training negative sample and / or the corresponding training positive sample, and the features extracted by different branch models at the same level are compared to obtain a multi-level feature comparison loss result; the first loss result is substituted into the cross entropy loss function in the preset overall loss function, and the multi-level feature comparison loss result is substituted into the contrast learning branch loss function in the preset overall loss function, and it is determined whether the preset overall loss function is the smallest. If it is determined that the preset overall loss function is not the smallest, the step of identifying the license plate number of the corresponding training positive sample through the first training model to obtain the third license plate number recognition result is returned until the number of training times reaches the preset number of times or the preset overall loss function is minimized, so as to train the preset contrast learning branch sub-model and the preset supervised branch sub-model. In this embodiment, since the purpose of training the two sub-models is to minimize the overall loss function, it provides a basis for the preset supervised branch sub-model to extract target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels in the process of predicting the processed image.

[0150] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 3 , before the step of substituting the multi-level feature comparison loss result into the contrast learning branch loss function in the preset overall loss function, it includes:

[0151] Step B1, receiving the image quality of the license plate number sample image sent by a preset quality scoring system;

[0152] Step B2: determining a contrastive learning branch loss function in a preset overall loss function according to the image quality, wherein the contrastive learning branch loss function is a multi-level feature fusion loss function that is negatively correlated with the quality score.

[0153] In this embodiment, it can be understood that the contrast learning branch loss function in the preset overall loss function is determined by receiving the image quality of the license plate number sample image sent by the preset quality scoring system (which has been calculated by the preset quality scoring system), and the image quality of the license plate number sample image is different, and the corresponding preset overall loss function can be changed. Although the image quality has an impact on the contrast learning branch loss function, as the training continues, the value of the overall contrast learning branch loss function tends to become smaller and smaller until convergence. That is, the difference between this embodiment and the current technology is that: since the image quality of this application is changing, the more times the training is performed, the smaller the value of the contrast learning branch loss function tends to be, but there may be fluctuations. Therefore, it is necessary to judge that multiple continuous values ​​of the preset overall loss function have all decreased or converged before the preset overall loss function can be judged to converge, so as to accurately judge whether the loss function converges, so as to accurately train the overall model, and thus improve the accuracy of the image to be processed.

[0154] This application also provides a license plate number recognition device, please refer to Figure 7 , the device comprises:

[0155] An acquisition module 10 is used to acquire an image to be processed;

[0156] The recognition module 20 is used to recognize the license plate number in the image to be processed based on the image to be processed through a preset supervised branch sub-model in the preset license plate number recognition model to obtain a first license plate number recognition result; wherein the preset license plate number recognition model also includes a preset contrast learning branch sub-model, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, the training negative samples are from the images corresponding to the license plate numbers previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, the training positive samples are images of the correctly predicted license plate numbers corresponding to the training negative samples, and the preset supervised branch sub-model extracts target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features at different feature levels during the process of predicting the image to be processed, the negative sample features are features that have a reverse effect on the correctly predicted license plate numbers learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples, the positive sample features are features that have a positive effect on the correctly predicted license plate numbers learned, and the first feature distance is greater than the second feature distance.

[0157] In one embodiment, the license plate number recognition device is further used to implement:

[0158] Get a sample image of a license plate number with a preset label;

[0159] Recognize the license plate number in the license plate number sample image by using the first training model, and obtain a second license plate number recognition result;

[0160] Select a first target license plate number that is the same as a preset label from the second license plate number recognition result, and use the license plate image corresponding to the first target license plate number as a training positive sample; select a second target license plate number that is different from the preset label from the second license plate number recognition result, and use the second target license plate number as the predicted wrong license plate number;

[0161] Based on the license plate image corresponding to the wrongly predicted license plate number, determining the training negative samples of the preset contrast learning branch sub-model, wherein the training negative samples also include license plate images corresponding to other license plate numbers;

[0162] Based on the training negative samples and the training positive samples, the preset contrast learning branch sub-model and the preset supervised branch sub-model are synchronously trained, and the trained preset contrast learning branch sub-model and the preset supervised branch sub-model are used as the preset license plate number recognition model.

[0163] In one embodiment, the license plate number recognition device is further used to implement any one of the following:

[0164] Using the license plate image corresponding to the wrongly predicted license plate number as a training negative sample for the preset contrast learning branch sub-model;

[0165] The wrongly predicted license plate number is partially simulated or simulated, and the license plate image corresponding to the partially simulated or simulated license plate number is used as a training negative sample for the preset contrast learning branch sub-model.

[0166] In one embodiment, the loss function of the preset license plate number recognition model is a preset overall loss function, and the preset overall loss function includes a cross entropy loss function corresponding to a preset supervised branch sub-model and a contrastive learning branch loss function corresponding to a preset contrastive learning branch sub-model;

[0167] The vehicle license plate number recognition device is also used to achieve:

[0168] The first training model is used to identify the license plate number of the corresponding training positive sample to obtain a third license plate number recognition result, and a first loss result is determined based on the third license plate number recognition result and the preset label; the first training model and the second training model are used to extract features of different levels for the training negative sample and / or the corresponding training positive sample, and the features extracted by different branch models at the same level are compared to obtain a multi-level feature comparison loss result;

[0169] Substitute the first loss result into the cross entropy loss function in the preset overall loss function, and substitute the multi-level feature comparison loss result into the contrast learning branch loss function in the preset overall loss function, and determine whether the preset overall loss function is the minimum. If it is determined that the preset overall loss function is not the minimum, return to the step of identifying the license plate number of the corresponding training positive sample through the first training model to obtain the third license plate number recognition result, until the number of training times reaches the preset number of times or the preset overall loss function is minimized, so as to train the preset contrast learning branch sub-model and the preset supervised branch sub-model.

[0170] In one embodiment, the license plate number recognition device is further used to implement:

[0171] Receiving the image quality of the license plate number sample image sent by a preset quality scoring system;

[0172] According to the image quality, a contrastive learning branch loss function in a preset overall loss function is determined, wherein the contrastive learning branch loss function is a multi-level feature fusion loss function that is negatively correlated with the quality score.

[0173] In one embodiment, the license plate number recognition device is further used to implement:

[0174] Determine the overall loss function for:

[0175] ;in, is the cross entropy loss function; is the loss function for contrastive learning branch;

[0176] Said ;

[0177] in, It is the comparative loss result of the same level features extracted by different branch sub-models based on training positive samples and / or training negative samples. is the corresponding weight, Rate the quality.

[0178] In one embodiment, the different levels of feature contrast loss results include global feature contrast loss results, character feature contrast loss results, and character contrast loss results;

[0179] The vehicle license plate number recognition device is also used to achieve:

[0180] Through the first training model, each training positive sample Extract global features, character features and characters to obtain first extracted features; and use the second training model to train each positive sample. Another training positive sample that constitutes a positive sample pair Extracting global features, character features, and characters to obtain second extracted features, wherein different license plate images corresponding to the same license plate number constitute positive sample pairs;

[0181] Through the second training model, for each training positive sample Another training negative sample yi constituting a negative sample pair is subjected to global feature, character feature and character extraction to obtain a third extracted feature, wherein different license plate images corresponding to different license plate numbers constitute a negative sample pair with each other;

[0182] According to the first extracted feature, the second extracted feature and the third extracted feature, a multi-level feature comparison loss result is determined as:

[0183] ;

[0184]

[0185]

[0186] in, is the global feature contrast loss result, is the character feature comparison loss result, is the character comparison loss result; It means to obtain the expectation corresponding to the global feature, Indicates the expectation corresponding to the character feature, E S Indicates the expectation of obtaining the corresponding characters. is the distance function defined to characterize the similarities at different feature levels. ,in, is the first extracted feature extracted by the preset supervised learning sub-model branch, = , is the second extracted feature or the third extracted feature extracted by the preset contrast learning branch sub-model, = or = ;

[0187] The step of substituting the multi-level feature comparison loss result into the contrastive learning branch loss function in the preset overall loss function includes:

[0188] Will , , Substitute into the In, confirm ;

[0189] in, is the weight corresponding to the global feature contrast loss result, where is the weight corresponding to the character feature comparison loss result, where is the weight corresponding to the character contrast loss result.

[0190] The license plate number recognition device provided by the present application adopts the license plate number recognition method in the above embodiment, which can solve the technical problem of license plate number recognition. Compared with the prior art, the beneficial effects of the license plate number recognition device provided by the present application are the same as the beneficial effects of the license plate number recognition method provided by the above embodiment, and other technical features in the license plate number recognition device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0191] The present application provides a license plate number recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the license plate number recognition method in the above-mentioned embodiment one.

[0192] Reference below Figure 6 , which shows a schematic diagram of the structure of a license plate number recognition device suitable for implementing the embodiment of the present application. The license plate number recognition device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The license plate number recognition device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0193] like Figure 6As shown, the license plate number recognition device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the license plate number recognition device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the license plate recognition device to communicate with other devices wirelessly or by wire to exchange data. Although the license plate recognition device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0194] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0195] The license plate number recognition device provided by the present application adopts the license plate number recognition method in the above embodiment, which can solve the technical problem of license plate number recognition. Compared with the prior art, the beneficial effects of the license plate number recognition device provided by the present application are the same as the beneficial effects of the license plate number recognition method provided by the above embodiment, and other technical features in the license plate number recognition device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0196] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0197] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0198] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the license plate number recognition method in the above-mentioned embodiment.

[0199] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0200] The computer-readable storage medium may be included in the license plate number recognition device; or may exist independently without being assembled into the license plate number recognition device.

[0201] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the license plate number recognition device, the license plate number recognition device executes the license plate number recognition method.

[0202] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the client computer, partially on the client computer, as a separate software package, partially on the client computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the client computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0203] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0204] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0205] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned license plate number recognition method, and can solve the technical problem of license plate number recognition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the license plate number recognition method provided in the above-mentioned embodiment, and will not be repeated here.

[0206] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned license plate number recognition method when executed by a processor.

[0207] The computer program product provided by this application can solve the technical problem of license plate number recognition. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the license plate number recognition method provided by the above embodiment, which will not be repeated here.

[0208] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A vehicle license plate recognition method, characterized in that: The method comprises: Get the image to be processed; Based on the image to be processed, the license plate number in the image to be processed is identified by a preset supervised branch sub-model in the preset license plate number recognition model to obtain a first license plate number recognition result; wherein, the preset license plate number recognition model also includes a preset contrast learning branch sub-model, and the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, and the training negative samples are derived from images corresponding to license plate numbers that were previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, and the training positive samples are images of correctly predicted license plate numbers corresponding to the training negative samples, and the preset supervised branch sub-model is used in the image to be processed. In the process of prediction, target features having a corresponding first feature distance with the negative sample features and a corresponding second feature distance with the positive sample features are extracted at different feature levels, the negative sample features are features learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples that have an inverse effect on the correct prediction of the license plate number, the positive sample features are features learned that have a positive effect on the correct prediction of the license plate number, the first feature distance is greater than the second feature distance, wherein, before the step of recognizing the license plate number in the image to be processed by the preset license plate number recognition model based on the image to be processed, it includes: Get a sample image of a license plate number with a preset label; Recognize the license plate number in the license plate number sample image by using the first training model, and obtain a second license plate number recognition result; Select a first target license plate number that is the same as a preset label from the second license plate number recognition result, and use the license plate image corresponding to the first target license plate number as a training positive sample; select a second target license plate number that is different from the preset label from the second license plate number recognition result, and use the second target license plate number as the predicted wrong license plate number; Based on the license plate image corresponding to the wrongly predicted license plate number, determining the training negative samples of the preset contrast learning branch sub-model, wherein the training negative samples also include license plate images corresponding to other license plate numbers; Based on the training negative samples and the training positive samples, the preset contrast learning branch sub-model and the preset supervised branch sub-model are synchronously trained, and the trained preset contrast learning branch sub-model and the preset supervised branch sub-model are used as the preset license plate number recognition model.

2. The method according to claim 1, characterized in that The step of determining the training negative sample of the preset contrast learning branch sub-model based on the license plate image corresponding to the wrongly predicted license plate number includes any one of the following: Using the license plate image corresponding to the wrongly predicted license plate number as a training negative sample for the preset contrast learning branch sub-model; The wrongly predicted license plate number is partially simulated or simulated, and the license plate image corresponding to the partially simulated or simulated license plate number is used as a training negative sample for the preset contrast learning branch sub-model.

3. The method according to claim 1, characterized in that The loss function of the preset license plate number recognition model is a preset overall loss function, and the preset overall loss function includes a cross entropy loss function corresponding to a preset supervised branch sub-model and a contrastive learning branch loss function corresponding to a preset contrastive learning branch sub-model; The step of synchronously training to obtain the preset contrastive learning branch sub-model and the preset supervised branch sub-model based on the training negative samples and the training positive samples comprises: The first training model is used to identify the license plate number of the corresponding training positive sample to obtain a third license plate number recognition result, and a first loss result is determined based on the third license plate number recognition result and the preset label; the first training model and the second training model are used to extract features of different levels for the training negative sample and / or the corresponding training positive sample, and the features extracted by different branch models at the same level are compared to obtain a multi-level feature comparison loss result; Substitute the first loss result into the cross entropy loss function in the preset overall loss function, and substitute the multi-level feature comparison loss result into the contrast learning branch loss function in the preset overall loss function, and determine whether the preset overall loss function is the minimum. If it is determined that the preset overall loss function is not the minimum, return to the step of identifying the license plate number of the corresponding training positive sample through the first training model to obtain the third license plate number recognition result, until the number of training times reaches the preset number of times or the preset overall loss function is minimized, so as to train the preset contrast learning branch sub-model and the preset supervised branch sub-model.

4. The method according to claim 3, characterized in that Before the step of substituting the multi-level feature comparison loss result into the contrastive learning branch loss function in the preset overall loss function, the method includes: Receiving the image quality of the license plate number sample image sent by a preset quality scoring system; According to the image quality, a contrastive learning branch loss function in a preset overall loss function is determined, wherein the contrastive learning branch loss function is a multi-level feature fusion loss function that is negatively correlated with the quality score.

5. The method according to claim 4, characterized in that The step of substituting the first loss result into the cross entropy loss function in the preset overall loss function includes: Determine the overall loss function for: ;in, is the cross entropy loss function; is the loss function for contrastive learning branch; Said ; in, It is the comparative loss result of the same level features extracted by different branch sub-models based on training positive samples and / or training negative samples. is the corresponding weight, Rate the quality.

6. The method according to claim 5, characterized in that The different levels of feature contrast loss results include global feature contrast loss results, character feature contrast loss results, and character contrast loss results; The step of performing feature extraction at different levels on the training negative samples and / or the corresponding training positive samples through the first training model and the second training model, and comparing the features extracted by different branch models at the same level to obtain a multi-level feature comparison loss result includes: Through the first training model, each training positive sample Extract global features, character features and characters to obtain first extracted features; and use the second training model to train each positive sample. Another training positive sample that constitutes a positive sample pair Extracting global features, character features, and characters to obtain second extracted features, wherein different license plate images corresponding to the same license plate number constitute positive sample pairs; Through the second training model, for each training positive sample Another training negative sample yi constituting a negative sample pair is subjected to global feature, character feature and character extraction to obtain a third extracted feature, wherein different license plate images corresponding to different license plate numbers constitute a negative sample pair with each other; According to the first extracted feature, the second extracted feature and the third extracted feature, a multi-level feature comparison loss result is determined as: ； ； ； in, is the global feature contrast loss result, is the character feature comparison loss result, is the character comparison loss result; It means to obtain the expectation corresponding to the global feature, It represents the expectation corresponding to the character feature, E S Indicates the expectation of obtaining the corresponding character is the distance function defined to characterize the similarities at different feature levels. ,in, is the first extracted feature extracted by the preset supervised learning sub-model branch, = , is the second extracted feature or the third extracted feature extracted by the preset contrast learning branch sub-model, = or = ; The step of substituting the multi-level feature comparison loss result into the contrastive learning branch loss function in the preset overall loss function includes: Will , , Substitute into the In, confirm ; in, is the weight corresponding to the global feature contrast loss result, where is the weight corresponding to the character feature comparison loss result, where is the weight corresponding to the character comparison loss result.

7. A vehicle license plate recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed; A recognition module, for recognizing the license plate number in the image to be processed based on the image to be processed through a preset supervised branch sub-model in a preset license plate number recognition model to obtain a first license plate number recognition result; wherein the preset license plate number recognition model also includes a preset contrast learning branch sub-model, the preset contrast learning branch sub-model is obtained by training with training negative samples and training positive samples, the training negative samples are from images corresponding to license plates previously incorrectly predicted by the first training model corresponding to the preset supervised branch sub-model, the training positive samples are images of license plates correctly predicted corresponding to the training negative samples, the preset supervised branch sub-model extracts target features having a corresponding first feature distance with negative sample features and a corresponding second feature distance with positive sample features at different feature levels during prediction of the image to be processed, the negative sample features are features that have a reverse effect on the correct prediction of the license plate number learned by the second training model corresponding to the preset contrast learning branch sub-model from the training negative samples and the training positive samples, the positive sample features are features that have a positive effect on the correct prediction of the license plate number learned, and the first feature distance is greater than the second feature distance; The vehicle license plate number recognition device is used to achieve: Get a sample image of a license plate number with a preset label; Recognize the license plate number in the license plate number sample image by using the first training model, and obtain a second license plate number recognition result; Select a first target license plate number that is the same as a preset label from the second license plate number recognition result, and use the license plate image corresponding to the first target license plate number as a training positive sample; select a second target license plate number that is different from the preset label from the second license plate number recognition result, and use the second target license plate number as the predicted wrong license plate number; Based on the license plate image corresponding to the wrongly predicted license plate number, determining the training negative samples of the preset contrast learning branch sub-model, wherein the training negative samples also include license plate images corresponding to other license plate numbers; Based on the training negative samples and the training positive samples, the preset contrast learning branch sub-model and the preset supervised branch sub-model are synchronously trained, and the trained preset contrast learning branch sub-model and the preset supervised branch sub-model are used as the preset license plate number recognition model.

8. A vehicle license plate recognition device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the vehicle license plate number recognition method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the license plate number recognition method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the license plate number recognition method according to any one of claims 1 to 6 are implemented.

Images