License plate number recognition method, recognition model training method and device

Abstract

The embodiment of the invention provides a license plate number recognition method and device and a recognition model training method and device, and relates to the technical field of intelligent transportation. Extracting a first to-be-processed area where the license plate number of the target vehicle is located and a second to-be-processed area where the amplified number is located from the to-be-processed image; performing license plate number identification on the first to-be-processed area to obtain a first to-be-processed license plate number of the target vehicle; performing feature fusion on the first to-be-processed area, the second to-be-processed area and the first to-be-processed license plate number by using a trained target recognition model to obtain a target fusion feature; and obtaining the target license plate number of the target vehicle based on the target fusion feature, so that the license plate number of the vehicle can be accurately identified based on the target fusion feature fused by the first to-be-processed region, the second to-be-processed region and the first to-be-processed license plate number feature even if the amplified number is defective or fuzzy, namely, the license plate number identification accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a license plate number recognition method and a license plate number recognition model training method and device. BACKGROUND

[0002] In the field of intelligent transportation, a large-scale license plate number (which can be referred to as a large number) is sprayed on the tail or body of a large vehicle, which facilitates the traffic management department to identify the license plate number of the large vehicle and to ensure road traffic safety.

[0003] In related technologies, an image of a vehicle is obtained, a recognition model for a large number is used to recognize a large number of a vehicle in the image, and a license plate number of the vehicle is obtained. However, when the large number is damaged or blurred, the large number of the vehicle cannot be correctly recognized, and the accuracy of license plate number recognition is reduced. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a license plate number recognition method and a license plate number recognition model training method and device to improve the accuracy of license plate number recognition. The specific technical solutions are as follows:

[0005] In a first aspect, to achieve the above purpose, the embodiments of the present application provide a license plate number recognition method, which comprises: obtaining a to-be-processed image of a target vehicle; extracting a first to-be-processed region where a license plate number of the target vehicle is located and a second to-be-processed region where a large number is located from the to-be-processed image; performing license plate number recognition on the first to-be-processed region to obtain a first to-be-processed license plate number of the target vehicle; inputting the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle into a target recognition model trained, performing feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model, and obtaining target fusion features; obtaining a target license plate number of the target vehicle based on the target fusion features; wherein the target recognition model is trained based on a first sample region where a license plate number of a sample vehicle in a sample image is located, a second sample region where a large number is located, a first predicted license plate number obtained by performing license plate number recognition on the first sample region, and a sample license plate number of the sample vehicle.

[0006] Optionally, the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle are input into a trained target recognition model, feature fusion of the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number is performed by the target recognition model, and target fusion features are obtained; based on the target fusion features, a target license plate number of the target vehicle is obtained, including: inputting the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, performing feature fusion of the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model, and obtaining target fusion features; based on the target fusion features, a second to-be-processed license plate number of the target vehicle, a first correction label and a second correction label corresponding to the first to-be-processed license plate number are obtained; wherein the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, an incorrect character indicated by the second correction label in the first to-be-processed license plate number is determined; a target character at the same position in the second to-be-processed license plate number is determined according to the position of the incorrect character in the first to-be-processed license plate number; the target character is used to replace the incorrect character in the first to-be-processed license plate number, and a target license plate number of the target vehicle is obtained; when the first correction label indicates that the first to-be-processed license plate number is correct, the first to-be-processed license plate number is determined as the target license plate number of the target vehicle.

[0007] Optionally, the trained target recognition model includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier.

[0008] The first to be processed region, the second to be processed region, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed region are extracted by the first feature extractor, and first to be processed image features are obtained; image features of the second to be processed region are extracted by the second feature extractor, and second to be processed image features are obtained; the first to be processed license plate number is encoded by the encoder, and target semantic features of the first to be processed license plate number are obtained; the first to be processed image features, the second to be processed image features, and the target semantic features are fused by the feature fusioner, and target fusion features are obtained.

[0009] The target fusion features are input into the first classifier, and the second to be processed license plate number of the target vehicle is output; the target fusion features are input into the second classifier, and the first correction label is output; the target fusion features are input into the third classifier, and the second correction label is output.

[0010] Optionally, the first feature extractor comprises a first feature extraction module and a first encoding module; and the second feature extractor comprises a second feature extraction module and a second encoding module.

[0011] The first to be processed region, the second to be processed region, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed region are extracted by the first feature extractor, and first to be processed image features are obtained; image features of the second to be processed region are extracted by the second feature extractor, and second to be processed image features are obtained; the first to be processed license plate number is encoded by the encoder, and target semantic features of the first to be processed license plate number are obtained; the first to be processed image features, the second to be processed image features, and the target semantic features are fused by the feature fusioner, and target fusion features are obtained.

[0012] The first to be processed region, the second to be processed region, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed region are extracted by the first feature extractor, and first to be processed image features are obtained; image features of the second to be processed region are extracted by the second feature extractor, and second to be processed image features are obtained; the first to be processed license plate number is encoded by the encoder, and target semantic features of the first to be processed license plate number are obtained; the first to be processed image features, the second to be processed image features, and the target semantic features are fused by the feature fusioner, and target fusion features are obtained.

[0013] Optionally, the trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner and a first classifier.

[0014] The first processing area, the second processing area, and the first processing license plate number of the target vehicle are input into the trained target recognition model, the first processing area, the second processing area, and the first processing license plate number are subjected to feature fusion by the target recognition model, and target fusion features are obtained; based on the target fusion features, a target license plate number of the target vehicle is obtained, comprising: inputting the first processing area, the second processing area, and the first processing license plate number into the trained target recognition model, extracting image features of the first processing area by the first feature extractor to obtain first processing image features; extracting image features of the second processing area by the second feature extractor to obtain second processing image features; encoding the first processing license plate number by the encoder to obtain target semantic features of the first processing license plate number; performing feature fusion on the first processing image features, the second processing image features, and the target semantic features by the feature fusioner to obtain target fusion features; and inputting the target fusion features into the first classifier to obtain an output target license plate number of the target vehicle.

[0015] In a second aspect, to achieve the above object, the embodiments of the present application provide a recognition model training method, which comprises: obtaining a sample image containing a sample vehicle and a sample license plate number of the sample vehicle; extracting a first sample area where the sample license plate number is located and a second sample area where the enlarged number is located from the sample image; performing license plate number recognition on the first sample area to obtain a first predicted license plate number of the sample vehicle; determining a first sample label and a second sample label corresponding to the first predicted license plate number based on the sample license plate number and the first predicted license plate number of the sample vehicle; wherein the first sample label indicates whether the first predicted license plate number is correct; and the second sample label indicates whether each character in the first predicted license plate number is correct; using the first sample area, the second sample area, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to train a target recognition model to be trained to obtain a trained target recognition model.

[0016] Optionally, the target recognition model to be trained comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier and a third classifier;

[0017] The first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label are used to train the target recognition model to be trained to obtain a trained target recognition model, including: based on the first sample region, the second sample region and the sample license plate number, adjusting parameters of a first feature extractor, a second feature extractor, a feature fusioner and a first classifier of the target recognition model to be trained until a first end condition is met, to obtain a first-stage trained target recognition model; based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, adjusting parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier of the first-stage trained target recognition model until a second end condition is met, to obtain the trained target recognition model.

[0018] Optionally, the adjusting parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region and the sample license plate number until the first end condition is met to obtain the first-stage trained target recognition model includes: adjusting parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region and the sample license plate number until a preset iteration number is reached to obtain a first-stage first-sub-stage trained target recognition model; adjusting parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the first-sub-stage trained target recognition model based on the first sample region, the second sample region and the sample license plate number until the first end condition is met to obtain the first-stage trained target recognition model.

[0019] Optionally, the parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained are adjusted based on the first sample region, the second sample region and the sample license plate number until a preset iteration number is reached, to obtain the target recognition model trained in the first stage in the first sub-stage, including: using the first feature extractor of the target recognition model to be trained to extract the image features of the first sample region to obtain first sample image features; using the second feature extractor of the target recognition model to be trained to extract the image features of the second sample region to obtain second sample image features; using the feature fusioner of the target recognition model to be trained to perform feature fusion on the first sample image features, the second sample image features and the preset first semantic features to obtain first fusion features; inputting the first fusion features into the first classifier of the target recognition model to be trained to obtain the second predicted license plate number of the sample vehicle as output; calculating a first loss value based on the second predicted license plate number and the sample license plate number; adjusting the parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first loss value until the preset iteration number is reached, to obtain the target recognition model trained in the first stage in the first sub-stage.

[0020] Optionally, the parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage are adjusted based on the first sample region, the second sample region and the sample license plate number until the first end condition is met, to obtain the target recognition model trained in the first stage, including: using the first feature extractor of the target recognition model trained in the first sub-stage to extract the image features of the first sample region to obtain third sample image features; using the second feature extractor of the target recognition model trained in the first sub-stage to extract the image features of the second sample region to obtain fourth sample image features; using the feature fusioner of the target recognition model trained in the first sub-stage to perform fusion on the third sample image features, the fourth sample image features and the preset first semantic features to obtain second fusion features; inputting the second fusion features into the first classifier of the target recognition model trained in the first sub-stage to obtain the third predicted license plate number of the sample vehicle as output; calculating a second loss value based on the third predicted license plate number and the sample license plate number; adjusting the parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage based on the second loss value until the first end condition is met, to obtain the target recognition model trained in the first stage in the second sub-stage as the target recognition model trained in the first stage.

[0021] Optionally, the parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier of the target recognition model trained in the first stage are adjusted based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, the first sample label and the second sample label until the second end condition is met, and a trained target recognition model is obtained, comprising: using the first feature extractor of the target recognition model trained in the first stage to extract image features of the first sample region to obtain fifth sample image features; using the second feature extractor of the target recognition model trained in the first stage to extract image features of the second sample region to obtain sixth sample image features; using the encoder of the target recognition model trained in the first stage to encode the first predicted license plate number to obtain second semantic features of the first predicted license plate number; using the feature fusioner of the target recognition model trained in the first stage to fuse the fifth sample image features, the sixth sample image features and the second semantic features to obtain third fusion features; inputting the third fusion features into the first classifier of the target recognition model trained in the first stage to obtain the fourth predicted license plate number of the sample vehicle as output; inputting the third fusion features into the second classifier of the target recognition model trained in the first stage to obtain the first predicted label of the first predicted license plate number of the sample vehicle as output; inputting the third fusion features into the third classifier of the target recognition model trained in the first stage to obtain the second predicted label of the first predicted license plate number of the sample vehicle as output; adjusting the parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier of the target recognition model trained in the first stage based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label until the second end condition is met, and obtaining a trained target recognition model.

[0022] Optionally, the parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier in the target recognition model trained in the first stage are adjusted based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, the second predicted label and the second sample label until the second end condition is met, and a trained target recognition model is obtained, comprising: calculating a third loss value based on the fourth predicted license plate number and the sample license plate number; adjusting the parameters of the first classifier of the target recognition model trained in the first stage based on the third loss value; calculating a fourth loss value based on the first predicted label and the first sample label; adjusting the parameters of the second classifier of the target recognition model trained in the first stage based on the fourth loss value; calculating a fifth loss value based on the second predicted label and the second sample label; adjusting the parameters of the third classifier of the target recognition model trained in the first stage based on the fifth loss value; calculating a target loss value based on the third loss value, the fourth loss value and the fifth loss value; adjusting the parameters of the first feature extractor, the second feature extractor, the encoder and the feature fusioner of the target recognition model trained in the first stage based on the target loss value until the second end condition is met, and a trained target recognition model is obtained.

[0023] Optionally, before the target recognition model to be trained is trained based on the first sample area, the second sample area, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain a trained target recognition model, the method further comprises: obtaining a trained first recognition model and a trained second recognition model; wherein the first recognition model is used to identify a license plate number; the second recognition model is used to identify an enlarged number; the first recognition model comprises a first feature extractor and a classifier; the second recognition model comprises a second feature extractor and a classifier; determining a target recognition model to be trained based on the trained first recognition model and the trained second recognition model; wherein the first feature extractor in the target recognition model to be trained is the same as the first feature extractor in the first recognition model; the second feature extractor in the target recognition model to be trained is the same as the second feature extractor in the second recognition model.

[0024] Optionally, after the target recognition model to be trained is trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the target recognition model trained, the method further comprises: using a first feature extractor in the target recognition model trained to replace a first feature extractor in the first recognition model to obtain an updated first recognition model; and using a second feature extractor in the target recognition model trained to replace a second feature extractor in the second recognition model to obtain an updated second recognition model.

[0025] Optionally, the first sample region where the license plate number of the sample vehicle is located and the second sample region where the enlarged number is located are extracted from the sample image, comprising: extracting a first image region where the license plate number of the sample vehicle is located from the sample image, and adjusting the first image region to obtain the first sample region; wherein the adjustment of the first image region comprises at least one of the following: truncation processing, shielding processing; extracting a second image region where the enlarged number of the sample vehicle is located from the sample image, and adjusting the second image region to obtain the second sample region; wherein the adjustment of the second image region comprises at least one of the following: brightness adjustment, angle adjustment, color adjustment, and cropping processing.

[0026] In a third aspect, to achieve the above object, an embodiment of the present application provides a license plate number recognition device, the device comprising: an image acquisition module configured to acquire a to-be-processed image of a target vehicle; an image extraction module configured to extract, from the to-be-processed image, a first to-be-processed region where a license plate number of the target vehicle is located and a second to-be-processed region where an enlarged number is located; a first to-be-processed license plate number determination module configured to perform license plate number recognition on the first to-be-processed region to obtain a first to-be-processed license plate number of the target vehicle; a target license plate number determination module configured to input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle into a target recognition model trained, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number through the target recognition model trained, and obtain a target fusion feature; and determine a target license plate number of the target vehicle based on the target fusion feature; wherein the target recognition model is trained based on a first sample region where a license plate number of a sample vehicle in a sample image is located, a second sample region where an enlarged number is located, a first predicted license plate number obtained by performing license plate number recognition on the first sample region, and a sample license plate number of the sample vehicle.

[0027] Optionally, the target license plate number determination module is specifically configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by using the target recognition model, and obtain target fusion features; based on the target fusion features, obtain an output second to-be-processed license plate number of the target vehicle, a first correction label and a second correction label corresponding to the first to-be-processed license plate number; the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, determine an error character indicated by the second correction label in the first to-be-processed license plate number; determine a target character at the same position in the second to-be-processed license plate number according to the position of the error character in the first to-be-processed license plate number; replace the error character in the first to-be-processed license plate number with the target character to obtain a target license plate number of the target vehicle; and when the first correction label indicates that the first to-be-processed license plate number is correct, determine that the first to-be-processed license plate number is the target license plate number of the target vehicle.

[0028] Optionally, the trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier.

[0029] The target license plate number determination module is specifically configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, extract image features of the first to-be-processed region by using the first feature extractor to obtain first to-be-processed image features; extract image features of the second to-be-processed region by using the second feature extractor to obtain second to-be-processed image features; encode the first to-be-processed license plate number by using the encoder to obtain target semantic features of the first to-be-processed license plate number; and perform feature fusion on the first to-be-processed image features, the second to-be-processed image features, and the target semantic features by using the feature fusioner to obtain target fusion features.

[0030] The target license plate number determination module is specifically configured to: input the target fusion features into the first classifier to obtain an output second to-be-processed license plate number of the target vehicle; input the target fusion features into the second classifier to obtain an output first correction label; and input the target fusion features into the third classifier to obtain an output second correction label.

[0031] Optionally, the first feature extractor comprises a first feature extraction module and a first encoding module; and the second feature extractor comprises a second feature extraction module and a second encoding module.

[0032] The target license plate number determination module is specifically configured to: perform deep feature extraction on the first to-be-processed region by the first feature extraction module to obtain image features of the first to-be-processed region; and encode the extracted image features by the first encoding module to obtain first to-be-processed image features.

[0033] The target license plate number determination module is specifically configured to: perform deep feature extraction on the second to-be-processed region by the second feature extraction module to obtain image features of the second to-be-processed region; and encode the extracted image features by the second encoding module to obtain second to-be-processed image features.

[0034] Optionally, the trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, and a first classifier.

[0035] The target license plate number determination module is specifically configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into the trained target recognition model; extract image features of the first to-be-processed region by the first feature extractor to obtain first to-be-processed image features; extract image features of the second to-be-processed region by the second feature extractor to obtain second to-be-processed image features; encode the first to-be-processed license plate number by the encoder to obtain target semantic features of the first to-be-processed license plate number; perform feature fusion on the first to-be-processed image features, the second to-be-processed image features, and the target semantic features by the feature fusioner to obtain target fusion features; and input the target fusion features into the first classifier to obtain an output target license plate number of the target vehicle.

[0036] In a fourth aspect, to achieve the above object, an embodiment of the present application provides a device for training a recognition model, the device comprising: an image acquisition module configured to acquire a sample image containing a sample vehicle and a sample license plate number of the sample vehicle; an image extraction module configured to extract, from the sample image, a first sample region in which the sample license plate number of the sample vehicle is located and a second sample region in which the enlarged number is located; a first predicted license plate number determination module configured to perform license plate number recognition on the first sample region to obtain a first predicted license plate number of the sample vehicle; a sample label determination module configured to determine a first sample label and a second sample label corresponding to the first predicted license plate number based on the sample license plate number and the first predicted license plate number of the sample vehicle; wherein the first sample label indicates whether the first predicted license plate number is correct, and the second sample label indicates whether each character in the first predicted license plate number is correct; and a training module configured to train a target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, to obtain a trained target recognition model.

[0037] Optionally, the target recognition model to be trained comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier.

[0038] The training module is specifically configured to: adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region, and the sample license plate number, until a first end condition is met, to obtain a target recognition model trained in a first stage; and adjust parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier, and the third classifier of the target recognition model trained in the first stage based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, until a second end condition is met, to obtain the trained target recognition model.

[0039] Optionally, the training module is specifically configured to: adjust parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region and the sample license plate number until a preset iteration number is reached, to obtain the target recognition model trained in the first stage in the first sub-stage; adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage based on the first sample region, the second sample region and the sample license plate number until a first end condition is met, to obtain the target recognition model trained in the first stage.

[0040] Optionally, the training module is specifically configured to: extract image features of the first sample region using the first feature extractor of the target recognition model to be trained, to obtain first sample image features; extract image features of the second sample region using the second feature extractor of the target recognition model to be trained, to obtain second sample image features; perform feature fusion on the first sample image features, the second sample image features and a preset first semantic feature using the feature fusioner of the target recognition model to be trained, to obtain first fusion features; input the first fusion features into the first classifier of the target recognition model to be trained, to obtain an output second predicted license plate number of the sample vehicle; calculate a first loss value based on the second predicted license plate number and the sample license plate number; adjust parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first loss value until a preset iteration number is reached, to obtain the target recognition model trained in the first stage in the first sub-stage.

[0041] Optionally, the training module is specifically configured to: extract image features of the first sample region using the first feature extractor of the target recognition model trained in the first sub-stage to obtain third sample image features; extract image features of the second sample region using the second feature extractor of the target recognition model trained in the first sub-stage to obtain fourth sample image features; fuse the third sample image features, the fourth sample image features, and a preset first semantic feature using the feature fusioner of the target recognition model trained in the first sub-stage to obtain second fusion features; input the second fusion features into the first classifier of the target recognition model trained in the first sub-stage to obtain an output third predicted license plate number of the sample vehicle; calculate a second loss value based on the third predicted license plate number and the sample license plate number; and adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model trained in the first sub-stage based on the second loss value until a first end condition is met, so as to obtain a target recognition model trained in the second sub-stage of the first stage as the target recognition model trained in the first stage.

[0042] Optionally, the training module is specifically configured to: extract image features of the first sample region using the first feature extractor of the target recognition model trained in the first stage to obtain fifth sample image features; extract image features of the second sample region using the second feature extractor of the target recognition model trained in the first stage to obtain sixth sample image features; encode the first predicted license plate number using the encoder of the target recognition model trained in the first stage to obtain second semantic features of the first predicted license plate number; fuse the fifth sample image features, the sixth sample image features, and the second semantic features using the feature fusioner of the target recognition model trained in the first stage to obtain third fusion features; input the third fusion features into the first classifier of the target recognition model trained in the first stage to obtain an output fourth predicted license plate number of the sample vehicle; input the third fusion features into the second classifier of the target recognition model trained in the first stage to obtain an output first predicted label of the first predicted license plate number of the sample vehicle; input the third fusion features into the third classifier of the target recognition model trained in the first stage to obtain an output second predicted label of the first predicted license plate number of the sample vehicle; and adjust parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier, and the third classifier of the target recognition model trained in the first stage based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label until a second end condition is met, so as to obtain the target recognition model trained.

[0043] Optionally, the training module is specifically configured to: calculate a third loss value based on the fourth predicted license plate number and the sample license plate number; adjust parameters of the first classifier of the target recognition model trained in the first stage based on the third loss value; calculate a fourth loss value based on the first predicted label and the first sample label; adjust parameters of the second classifier of the target recognition model trained in the first stage based on the fourth loss value; calculate a fifth loss value based on the second predicted label and the second sample label; adjust parameters of the third classifier of the target recognition model trained in the first stage based on the fifth loss value; calculate a target loss value based on the third loss value, the fourth loss value, and the fifth loss value; adjust parameters of the first feature extractor, the second feature extractor, the encoder, and the feature fusioner of the target recognition model trained in the first stage based on the target loss value until a second end condition is met, and obtain the trained target recognition model.

[0044] Optionally, the apparatus further comprises a model obtaining module configured to, before the training module performs training on the target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the trained target recognition model, perform obtaining a first recognition model and a second recognition model trained; wherein the first recognition model is configured to recognize a license plate number; the second recognition model is configured to recognize a magnification number; the first recognition model comprises a first feature extractor and a classifier; the second recognition model comprises a second feature extractor and a classifier.

[0045] A model construction module is configured to determine a target recognition model to be trained based on the trained first recognition model and the trained second recognition model; wherein the first feature extractor in the target recognition model to be trained is the same as the first feature extractor in the first recognition model; and the second feature extractor in the target recognition model to be trained is the same as the second feature extractor in the second recognition model.

[0046] Optionally, the apparatus further comprises a first recognition model updating module configured to, after the training module performs training on the target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the trained target recognition model, perform replacing the first feature extractor in the first recognition model with the first feature extractor in the trained target recognition model to obtain an updated first recognition model.

[0047] The second identification model updating module is configured to replace the second feature extractor in the second identification model with a second feature extractor in the trained target identification model to obtain an updated second identification model.

[0048] Optionally, the image extraction module is specifically configured to: extract a first image region where the license plate number of the sample vehicle is located from the sample image, adjust the first image region to obtain a first sample region, and the adjustment manner of the first image region includes at least one of the following: truncation processing, shielding processing; extract a second image region where the enlarged number of the sample vehicle is located from the sample image, adjust the second image region to obtain a second sample region, and the adjustment manner of the second image region includes at least one of the following: brightness adjustment, angle adjustment, color adjustment, and cropping processing.

[0049] Embodiments of the present application further provide an electronic device, comprising: a memory for storing a computer program;

[0050] The processor is configured to execute the program stored in the memory to implement the license plate number recognition method of any one of the first aspect or the identification model training method of any one of the second aspect.

[0051] Embodiments of the present application further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the license plate number recognition method of any one of the first aspect or the identification model training method of any one of the second aspect.

[0052] Embodiments of the present application further provide a computer program product containing instructions, which, when executed on a computer, cause the computer to perform the license plate number recognition method of any one of the first aspect or the identification model training method of any one of the second aspect.

[0053] Embodiments of the present application have the following beneficial effects:

[0054] The technical scheme provided in the embodiments of the present application comprises: obtaining a to-be-processed image of a target vehicle; extracting a first to-be-processed region where a license plate number of the target vehicle is located and a second to-be-processed region where an enlarged number is located from the to-be-processed image; performing license plate number recognition on the first to-be-processed region to obtain a first to-be-processed license plate number of the target vehicle; performing feature fusion on the first to-be-processed region, the second to-be-processed region and the first to-be-processed license plate number using a target recognition model trained to obtain target fusion features; and obtaining a target license plate number of the target vehicle based on the target fusion features. Even if the enlarged number is damaged or blurred, the target fusion features based on the feature fusion of the first to-be-processed region, the second to-be-processed region and the first to-be-processed license plate number can accurately identify the license plate number of the vehicle, that is, the accuracy of license plate number recognition can be improved.

[0055] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0057] Figure 1 The flowchart of the first license plate number recognition method provided in the embodiments of the present application;

[0058] Figure 2 The flowchart of the second license plate number recognition method provided in the embodiments of the present application;

[0059] Figure 3 The flowchart of the third license plate number recognition method provided in the embodiments of the present application;

[0060] Figure 4 The working principle diagram of an encoder provided in the embodiments of the present application;

[0061] Figure 5 The flowchart of the first recognition model training method provided in the embodiments of the present application;

[0062] Figure 6 The flowchart of the second recognition model training method provided in the embodiments of the present application;

[0063] Figure 7 The principle diagram of a recognition model training method provided in the embodiments of the present application;

[0064] Figure 8 The structural diagram of a license plate number recognition device provided in the embodiments of the present application;

[0065] Figure 9 A structural diagram of a recognition model training device provided by an embodiment of the present application is shown in the figure.

[0066] Figure 10 A structural diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application belong to the scope of protection of the present application.

[0068] In order to solve the problem of low accuracy of license plate number recognition in the related art, an embodiment of the present application provides a license plate number recognition method, which is applied to an electronic device. The electronic device obtains a to-be-processed image of a target vehicle. According to the license plate number recognition method provided by the embodiment of the present application, the first to-be-processed region where the license plate number of the target vehicle is located and the second to-be-processed region where the magnifying number is located in the to-be-processed image are combined to determine the target license plate number of the target vehicle.

[0069] In an application scenario, when the target vehicle enters a target place, safety verification is performed based on the target license plate number of the target vehicle to determine whether the target vehicle is a vehicle of a person in the target place. If the target vehicle is a vehicle of a person in the target place, it is determined that the target vehicle passes the safety verification, and the target vehicle is allowed to enter the target place. If the target vehicle is not a vehicle of a person in the target place, it is determined that the target vehicle fails to pass the safety verification, and the target vehicle is prohibited from entering the target place, so as to ensure the safety of the target place.

[0070] Referring to Figure 1 , Figure 1 A flowchart of a license plate number recognition method provided by an embodiment of the present application is shown in the figure. The method includes the following steps.

[0071] S101: Obtain a to-be-processed image of a target vehicle.

[0072] S102: Extract, from the to-be-processed image, a first to-be-processed region where a license plate number of the target vehicle is located and a second to-be-processed region where a magnifying number is located.

[0073] S103: Perform license plate number recognition on the first to-be-processed region to obtain a first to-be-processed license plate number of the target vehicle.

[0074] S104: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle into the trained target recognition model, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number through the target recognition model, and obtain target fusion features; and based on the target fusion features, obtain the target license plate number of the target vehicle.

[0075] The target recognition model is trained based on the first sample region where the sample license plate number of the sample vehicle in the sample image is located, the second sample region where the magnified number is located, the first predicted license plate number obtained by performing license plate number recognition on the first sample region, and the sample license plate number.

[0076] Based on the license plate number recognition method provided in the embodiments of the present application, the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are fused using the trained target recognition model to obtain target fusion features; and based on the target fusion features, the target license plate number of the target vehicle is obtained. Even if the magnified number is missing or blurred, the target fusion features obtained by fusing the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number can accurately identify the license plate number of the vehicle, that is, the accuracy of license plate number recognition can be improved.

[0077] For steps S101 and S102, the to-be-processed image is an image of the target vehicle captured by an image acquisition device. The image acquisition device can be a camera arranged in a scene such as a toll gate, a parking lot, and a road. The to-be-processed image contains the license plate number and the magnified number of the target vehicle, and then the electronic device extracts the first to-be-processed region where the license plate number of the target vehicle is located and the second to-be-processed region where the magnified number is located from the to-be-processed image.

[0078] In an implementation manner, the electronic device extracts the first to-be-processed region and the second to-be-processed region in the to-be-processed image in the following manner: input the to-be-processed image into the trained first detection model to obtain the output first to-be-processed region where the license plate number of the target vehicle is located. Input the to-be-processed image into the trained second detection model to obtain the output second to-be-processed region where the magnified number of the target vehicle is located.

[0079] The first detection model and the second detection model are models with target detection functions. The first detection model and the second detection model can use the same model structure but are trained using different training samples. For example, the first detection model and the second detection model both use a YOLO (You Only Lock Once, a deep learning neural network model) structure. Alternatively, the first detection model and the second detection model both use a Fast R-CNN (Faster Region Convolutional Neural Network) structure.

[0080] The first detection model and the second detection model can also use different model structures and are trained using different training samples. For example, the first detection model uses a YOLO structure. The second detection model uses a Fast R-CNN structure.

[0081] The first detection model is trained in the following manner: a sample image and a first position of a sample license plate number of a sample vehicle in the sample image are obtained. The sample image is input into the first detection model to be trained to obtain a predicted position of the sample license plate number of the sample vehicle. Based on the first position and the predicted position of the sample license plate number of the sample vehicle, the parameters of the first detection model are adjusted to obtain the trained first detection model.

[0082] The second detection model is trained in the following manner: a sample image and a second position of a sample license plate number of a sample vehicle in the sample image are obtained. The sample image is input into the second detection model to be trained to obtain a predicted position of the sample license plate number of the sample vehicle. Based on the second position and the predicted position of the sample license plate number of the sample vehicle, the parameters of the second detection model are adjusted to obtain the trained second detection model.

[0083] For step S103, license plate number recognition is performed on the first to-be-processed region to obtain the license plate number of the target vehicle (i.e., the first to-be-processed license plate number). For example, the first to-be-processed region is input into the trained first recognition model to obtain the first to-be-processed license plate number of the target vehicle. The first recognition model is a network model having a classification function. For example, the first recognition model can be a CNN (Convolutional Neural Network) model.

[0084] The first recognition model includes a feature extractor (i.e., a first feature extractor) and a classifier, and the first recognition model is trained in the following manner: a sample image and a sample license plate number of a sample vehicle in the sample image are obtained. The sample image is input into the first recognition model to be trained to obtain a predicted license plate number (which can be referred to as a fifth predicted license plate number) of the sample vehicle. Based on the sample license plate number and the fifth predicted license plate number of the sample vehicle, the parameters of the feature extractor and the classifier of the first recognition model are adjusted to obtain the trained first recognition model.

[0085] For step S104, the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle are input to the trained target recognition model. The target recognition model is trained based on the first sample region in which the sample license plate number of the sample vehicle in the sample image is located, the second sample region in which the magnified number is located, the first predicted license plate number obtained by performing license plate number recognition on the first sample region, and the sample license plate number of the sample vehicle. The training method of the target recognition model is described in the subsequent embodiments.

[0086] Based on the output of the target recognition model, different ways can be used to obtain the target license plate number of the target vehicle.

[0087] Method 1: Based on Figure 1 , referring to Figure 2 , step S104 can include the following steps:

[0088] S1041: The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input to the trained target recognition model. The target recognition model is trained based on the first sample region in which the sample license plate number of the sample vehicle in the sample image is located, the second sample region in which the magnified number is located, the first predicted license plate number obtained by performing license plate number recognition on the first sample region, and the sample license plate number of the sample vehicle. The training method of the target recognition model is described in the subsequent embodiments.

[0089] Among them, the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct.

[0090] S1042: When the first correction label indicates that the first to-be-processed license plate number is incorrect, the second correction label indicates the incorrect character in the first to-be-processed license plate number. According to the position of the incorrect character in the first to-be-processed license plate number, the target character in the same position in the second to-be-processed license plate number is determined. The target character is used to replace the incorrect character in the first to-be-processed license plate number to obtain the target license plate number of the target vehicle.

[0091] S1043: When the first correction label indicates that the first to-be-processed license plate number is correct, the first to-be-processed license plate number is determined as the target license plate number of the target vehicle.

[0092] The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input to the trained target recognition model to obtain the second to-be-processed license plate number of the target vehicle, the first correction label and the second correction label corresponding to the first to-be-processed license plate number.

[0093] The first correction label indicates whether the first to-be-processed license plate number is correct. For example, the first correction label is 0, indicating that the first to-be-processed license plate number is incorrect; and the first correction label is 1, indicating that the first to-be-processed license plate number is correct.

[0094] The second correction label indicates whether each character in the first to-be-processed license plate number is correct. The second correction label is an array containing multiple elements. One element indicates whether the character at the corresponding position in the first to-be-processed license plate number is correct. For example, element 1 indicates that the character at the corresponding position is incorrect, and element 0 indicates that the character at the corresponding position is correct. For example, the second correction label 0000010 indicates that the sixth character in the first to-be-processed license plate number is incorrect and needs to be corrected.

[0095] If the first correction label indicates that the first to-be-processed license plate number is incorrect, the first to-be-processed license plate number needs to be corrected. The second to-be-processed license plate number is determined based on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number. The accuracy of the second to-be-processed license plate number is higher, and the second to-be-processed license plate number can be used to correct the first to-be-processed license plate number.

[0096] Correspondingly, the error character indicated by the second correction label in the first to-be-processed license plate number is determined. For example, the position of a specified element in the second correction label is determined, the character at the same position in the first to-be-processed license plate number is determined, the error character in the first to-be-processed license plate number is obtained, and the error character is the character that needs to be corrected. According to the position of the error character in the first to-be-processed license plate number, the target character at the same position in the second to-be-processed license plate number is determined. The target character is used to replace the error character in the first to-be-processed license plate number to obtain the target license plate number of the target vehicle.

[0097] For example, the specified element is 1, and the second correction label 0000010 indicates that the sixth character in the first to-be-processed license plate number is an error character. The sixth character in the second to-be-processed license plate number is determined as the target character, and the sixth character in the second to-be-processed license plate number is used to replace the sixth character in the first to-be-processed license plate number to obtain the target license plate number.

[0098] If the first correction label indicates that the first to-be-processed license plate number is correct, the first to-be-processed license plate number does not need to be corrected, and the first to-be-processed license plate number is directly determined as the target license plate number of the target vehicle.

[0099] Based on the above processing, the magnified image (i.e., the second to-be-processed region), the license plate image (i.e., the first to-be-processed region), and the license plate prediction result (i.e., the first to-be-processed license plate number) are used to realize fusion recognition based on a deep learning network (i.e., a target recognition model), thereby improving the accuracy of the license plate number device. Moreover, the target recognition model outputs the recognition results of the three sub-tasks (i.e., three classifiers), i.e., the second to-be-processed license plate number, the first correction label, and the second correction label, so that the recognition results of the three sub-tasks constrain each other, and the final license plate recognition result (i.e., the target license plate number) is obtained, thereby further improving the accuracy of license plate number recognition.

[0100] In some embodiments, the trained target recognition model includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier.

[0101] Correspondingly, the step of determining the target fusion feature in step S1041 includes the following steps: inputting the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into the trained target recognition model, extracting the image features of the first to-be-processed region through the first feature extractor to obtain the first to-be-processed image features; extracting the image features of the second to-be-processed region through the second feature extractor to obtain the second to-be-processed image features; encoding the first to-be-processed license plate number through the encoder to obtain the target semantic features of the first to-be-processed license plate number; and fusing the first to-be-processed image features, the second to-be-processed image features, and the target semantic features through the feature fusioner to obtain the target fusion feature.

[0102] The step of determining the second to-be-processed license plate number of the target vehicle, the first correction label, and the second correction label corresponding to the first to-be-processed license plate number in step S1041 includes the following steps: inputting the target fusion feature into the first classifier to obtain the output second to-be-processed license plate number of the target vehicle; inputting the target fusion feature into the second classifier to obtain the output first correction label; and inputting the target fusion feature into the third classifier to obtain the output second correction label.

[0103] The first feature extractor includes a first feature extraction module and a first encoding module. The first feature extraction module is configured to perform deep feature extraction. For example, the first feature extraction module is a deep convolution module, or the first feature extraction module is a ViTs (Vision Transformers). The first encoding module is configured to map the image features extracted by the first feature extractor to a specified dimension. The first to-be-processed region is subjected to deep feature extraction through the first feature extraction module to obtain the image features of the first to-be-processed region, and the extracted image features are encoded through the first encoding module to obtain the first to-be-processed image features.

[0104] The second feature extractor comprises a second feature extraction module and a second encoding module. The first feature extractor and the second feature extractor are of the same structure, or the first feature extractor and the second feature extractor are of different structures. The second feature extraction module is configured to perform deep feature extraction. For example, the second feature extraction module is a deep convolution module, or the second feature extraction module is a ViTs. The second encoding module is configured to map the image features extracted by the second feature extractor to a specified dimension. The second feature extraction module is configured to perform deep feature extraction on the second to-be-processed region to obtain image features of the second to-be-processed region, and the second encoding module is configured to encode the extracted image features to obtain the second to-be-processed image features.

[0105] The encoder is configured to implement position encoding. For example, the encoder can be a word 2 vector model for mapping the first to-be-processed license plate number to a feature vector. The encoder is configured to encode the first to-be-processed license plate number to obtain the target semantic feature of the first to-be-processed license plate number.

[0106] The feature fusioner comprises a splicing module and a feature extraction module. The splicing module is configured to splice the first to-be-processed image features, the second to-be-processed image features and the target semantic feature, and the feature extraction module is configured to extract features from the splicing result to obtain target fusion features. The target fusion features are features with strong correlation between semantics and images.

[0107] The first classifier, the second classifier and the third classifier are all configured to implement a classification function. The first classifier, the second classifier and the third classifier can use the same network structure, or can use different network structures.

[0108] The first classifier comprises a convolution layer, a full connection layer and a normalization layer. The target fusion features are input into the convolution layer, and are sequentially processed by the convolution layer, the full connection layer and the normalization layer to obtain the second to-be-processed license plate number of the target vehicle.

[0109] The second classifier comprises a convolution layer, a full connection layer and a normalization layer. The target fusion features are input into the convolution layer, and are sequentially processed by the convolution layer, the full connection layer and the normalization layer to obtain the first correction label.

[0110] The third classifier comprises a convolution layer, a full connection layer and a normalization layer. The target fusion features are input into the convolution layer, and are sequentially processed by the convolution layer, the full connection layer and the normalization layer to obtain the second correction label.

[0111] Reference is made to Figure 3, the license plate image is the first to-be-processed region. The enlarged image is the second to-be-processed region. The license plate prediction result is the first to-be-processed license plate number. The license plate image is input into the first feature extractor. The first feature extractor includes a Backbone (i.e., a first feature extraction module) and an Encoder (i.e., a first encoding module). The Backbone performs deep feature extraction on the license plate image to obtain an image feature of a dimension of BxCxHxW. The Encoder maps the extracted image feature to a dimension of BxCxT to obtain a first to-be-processed image feature.

[0112] The enlarged image is input into the second feature extractor. The second feature extractor includes a Backbone (i.e., a second feature extraction module) and an Encoder (i.e., a second encoding module). The Backbone performs deep feature extraction on the enlarged image to obtain an image feature of a dimension of BxCxHxW. The Encoder maps the extracted image feature to a dimension of BxCxT to obtain a second to-be-processed image feature. The license plate prediction result is input into an Embedding (i.e., an encoder) to encode the license plate prediction result to obtain a target semantic feature of the license plate prediction result. The target semantic feature is of a dimension of BxCxT.

[0113] The first to-be-processed image feature, the second to-be-processed image feature, and the target semantic feature are spliced in a Fusion-features (i.e., a splicing module) to obtain a splicing result of a dimension of BxCx3xT. A TransformerEncoder (i.e., a feature extraction module) is used to extract features of the splicing result to obtain a target fusion feature of a dimension of BxC’xT.

[0114] The target fusion feature is input into a Classifier0 (i.e., a first classifier) to obtain an output fusion recognition result (i.e., a second to-be-processed license plate number). The target fusion feature is input into a Classifier1 (i.e., a third classifier) to obtain an output license plate position that needs to be corrected (i.e., a second correction label). The target fusion feature is input into a Classifier2 (i.e., a second classifier) to obtain an output of whether the license plate prediction is correct (i.e., a first correction label).

[0115] In some embodiments, Figure 3 The working principle of the Encoder is as shown in Figure 4 The image feature of a dimension of BxCx(HW) is processed by a CNN to output an image feature of a dimension of BxTx(HW). The tensor product of the image feature of a dimension of BxCxHxW and the image feature of a dimension of BxTx(HW) output by the CNN is calculated to obtain an image feature of a dimension of BxCxT.

[0116] If the image feature of the BxCx(HW) dimension is output by the first feature extraction module, the image feature of the BxCxT dimension is the first to-be-processed image feature. If the image feature of the BxCx(HW) dimension is output by the second feature extraction module, the image feature of the BxCxT dimension is the second to-be-processed image feature.

[0117] Based on the above processing, the first to-be-processed image feature of the first to-be-processed region, the second to-be-processed image feature of the second to-be-processed region, and the target semantic feature of the first to-be-processed license plate number are fused at the feature level to obtain a target fusion feature. Using the target fusion feature for license plate number recognition can improve the accuracy of license plate number recognition.

[0118] Method 2: The trained target recognition model includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, and a first classifier. Correspondingly, step S104 can include the following steps:

[0119] The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input into the trained target recognition model. The first feature extractor extracts the image feature of the first to-be-processed region to obtain the first to-be-processed image feature. The second feature extractor extracts the image feature of the second to-be-processed region to obtain the second to-be-processed image feature. The encoder encodes the first to-be-processed license plate number to obtain the target semantic feature of the first to-be-processed license plate number. The feature fusioner fuses the first to-be-processed image feature, the second to-be-processed image feature, and the target semantic feature to obtain the target fusion feature. The target fusion feature is input into the first classifier to obtain the output target license plate number of the target vehicle.

[0120] The implementation of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, and the first classifier is described in the foregoing embodiments. After obtaining the target fusion feature, the target fusion feature is input into the first classifier to obtain the license plate number output by the first classifier (i.e., the second to-be-processed license plate number). Since the second classifier and the third classifier are not set, the license plate number output by the first classifier does not need to be used to correct the first to-be-processed license plate number, and the license plate number output by the first classifier is directly used as the target license plate number of the target vehicle.

[0121] The second to-be-processed license plate number is determined in combination with the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number. The accuracy of the second to-be-processed license plate number is high. Determining the second to-be-processed license plate number as the target license plate number of the target vehicle can improve the accuracy of the license plate number device. Moreover, the target recognition model does not set the second classifier and the third classifier, which can reduce the processing time of the target recognition model and improve the efficiency of license plate number recognition.

[0122] The identification model training method provided in the embodiments of the present application is introduced as follows. According to the identification model training method provided in the embodiments of the present application, a trained target identification model is obtained. The trained target identification model is used to determine the target license plate number in the foregoing embodiments.

[0123] Referring to Figure 5 , Figure 5 A flowchart of an identification model training method provided in the embodiments of the present application is shown in FIG. 5. The method includes the following steps.

[0124] S501: Obtain a sample image containing a sample vehicle and a sample license plate number of the sample vehicle.

[0125] S502: Extract, from the sample image, a first sample region in which the sample license plate number of the sample vehicle is located and a second sample region in which the magnification number is located.

[0126] S503: Perform license plate number recognition on the first sample region to obtain a first predicted license plate number of the sample vehicle.

[0127] S504: Determine a first sample label and a second sample label corresponding to the first predicted license plate number based on the sample license plate number of the sample vehicle and the first predicted license plate number.

[0128] The first sample label indicates whether the first predicted license plate number is correct, and the second sample label indicates whether each character in the first predicted license plate number is correct.

[0129] S505: Train a target identification model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain a trained target identification model.

[0130] According to the identification model training method provided in the embodiments of the present application, a target identification model is trained. Using the trained target identification model, in combination with the second to-be-processed region in which the magnification number is located, the first to-be-processed region in which the license plate number is located, and the first to-be-processed license plate number recognized, the target license plate number of the target vehicle is determined. Even if the magnification number is missing or blurred, in combination with the first to-be-processed region in which the license plate number is located and the first to-be-processed license plate number recognized, the license plate number of the vehicle can be accurately recognized, thereby improving the accuracy of license plate number recognition.

[0131] For steps S501 and S502, the sample image is from a public data set. The sample license plate number is the real license plate number of the sample vehicle labeled.

[0132] The sample image includes a first sample region where the license plate number of the sample vehicle is located, and a second sample region where the license plate number is enlarged. In an implementation, the first sample region where the license plate number of the sample vehicle is located, and the second sample region where the license plate number is enlarged are directly extracted from the sample image. The manner of extracting the first sample region and the second sample region from the sample image is similar to the manner of extracting the first to-be-processed region and the second to-be-processed region from the to-be-processed image, and reference is made to the related description in the foregoing embodiments.

[0133] In another implementation, in actual application scenarios, images in various complex scenarios are captured. For example, a camera arranged at a lens or on a road is affected by weather, illumination, and other factors, so that the captured image is not clear. For another example, a camera arranged on a pole is tilted due to external force, so that the vehicle and the license plate in the captured image can also be tilted.

[0134] To simulate the complex scenarios of actual image capturing, so that the trained target recognition model is adapted to actual application scenarios, and the robustness of the target recognition model is improved, after the image region where the license plate number of the sample vehicle is located (i.e., the first image region) is extracted from the sample image, the first image region is adjusted to obtain the first sample region. For example, the first image region is subjected to truncation processing, shielding processing, and the like.

[0135] After the image region where the license plate number of the sample vehicle is located (i.e., the second image region) is extracted from the sample image, the second image region is adjusted to obtain the second sample region. For example, the second image region is subjected to brightness adjustment, angle adjustment, color adjustment, cropping processing, and the like.

[0136] Based on the above processing, the complex scenarios of actual image capturing are simulated, the fusion capability of the target recognition model is improved, and the robustness of the target recognition model is further improved.

[0137] For steps S503 and S504, the first sample region is input to the trained first recognition model to obtain the first predicted license plate number of the sample vehicle. The first recognition model is described with reference to the related description in the foregoing embodiments.

[0138] The sample license plate number of the sample vehicle and the first predicted license plate number are compared to generate a first sample label and a second sample label corresponding to the first predicted license plate number. The first sample label indicates whether the first predicted license plate number is correct. For example, 1 indicates that the first predicted license plate number is incorrect, and 0 indicates that the first predicted license plate number is correct. If the sample license plate number and the first predicted license plate number are completely identical, the first sample label is determined to be 1. If the sample license plate number and the first predicted license plate number are not completely identical, the first sample label is determined to be 0.

[0139] The second sample label represents whether each character in the first predicted license plate number is correct. The second sample label is an array containing multiple elements. One element represents whether the character at the corresponding position in the first predicted license plate number is correct. For example, element 1 indicates that the character at the corresponding position is incorrect, and element 0 indicates that the character at the corresponding position is correct.

[0140] Each character in the first predicted license plate number is compared with the character at the same position in the sample license plate number. If the two characters are consistent, the element at the corresponding position in the second sample label is determined to be 0. If the two characters are inconsistent, the element at the corresponding position in the second sample label is determined to be 1, and the second sample label is obtained.

[0141] For step S505, the training sample of the target recognition model is obtained through the above processing. The first sample region, the second sample region, and the first predicted license plate number in the training sample are used as input data of the target recognition model. The sample license plate number, the first sample label, and the second sample label in the training sample are used as labels corresponding to the input data. The model parameters of the target recognition model to be trained are adjusted, and the trained target recognition model is obtained.

[0142] In some embodiments, due to the actual application scenario, it is impossible to guarantee the correctness of the output of the license plate detection frame, the license plate recognition result, the enlarged number detection frame, and the enlarged number recognition result. In order to improve the robustness of the target recognition model, data augmentation is performed on the training sample, so that the target recognition model sees more complex inputs, and the trained target recognition model can adapt to the actual application scenario.

[0143] For example, a background image that does not contain a license plate number and / or an enlarged number is obtained as a negative sample to improve the anti-interference ability of the target recognition model. In addition, some special license plate numbers are not common, and the training sample containing this type of license plate number is less. For example, the license plate number of the number combination. For example, the number in the license plate number is 111, 222, 333, etc. Increasing the number of training samples containing this type of license plate number can achieve balanced sampling of the enlarged number and the license plate number frequency, prevent the target recognition model from being affected by the distribution of the training sample, and further improve the robustness of the target recognition model.

[0144] Different training methods are used based on the structure of the target recognition model.

[0145] Method one: the electronic device determines the target recognition model to be trained in the following manner:

[0146] obtaining a first recognition model and a second recognition model which are trained; the first recognition model is used to recognize license plate numbers; the second recognition model is used to recognize magnification numbers; the first recognition model comprises a first feature extractor and a classifier; the second recognition model comprises a second feature extractor and a classifier; determining a target recognition model to be trained based on the first recognition model and the second recognition model which are trained; the first feature extractor in the target recognition model to be trained is the same as the first feature extractor in the first recognition model; the second feature extractor in the target recognition model to be trained is the same as the second feature extractor in the second recognition model.

[0147] The first recognition model is a trained model used to recognize license plate numbers. The first recognition model comprises a first feature extractor and a classifier. The training method of the first recognition model is described in the foregoing embodiments.

[0148] The second recognition model is a trained model used to recognize magnification numbers. The second recognition model comprises a second feature extractor and a classifier. The second recognition model is trained in the following manner: obtaining a sample image and a sample magnification number of a sample vehicle in the sample image. The sample image is input into the second recognition model to be trained, and a predicted magnification number of the sample vehicle is obtained. Based on the sample magnification number and the predicted magnification number of the sample vehicle, the parameters of the second feature extractor and the classifier of the second recognition model are adjusted, and the trained second recognition model is obtained.

[0149] The first recognition model has been trained, and the first feature extractor in the first recognition model can accurately extract image features of the first sample region. Similarly, the second recognition model has been trained, and the second feature extractor in the second recognition model can accurately extract image features of the second sample region.

[0150] Correspondingly, the target recognition model to be trained is constructed based on the first feature extractor of the first recognition model and the second feature extractor of the second recognition model.

[0151] The first feature extractor in the first recognition model is used as the first feature extractor in the target recognition model for extracting features of an image region where a license plate number is located; the second feature extractor in the second recognition model is used as the second feature extractor in the target recognition model for extracting features of an image region where a magnification number is located. Furthermore, an encoder for extracting features of a license plate prediction result (i.e. a license plate number output by the first recognition model), a feature fusioner for performing feature fusion, and a first classifier for outputting a fusion prediction result (i.e. a license plate number), a second classifier for outputting whether the license plate prediction result is accurate, and a third classifier for outputting whether each character in the license plate prediction result is accurate are set, and the target recognition model to be trained is obtained.

[0152] The target recognition model to be trained comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier and a third classifier. Figure 5 On the basis of Figure 6 , step S505 can comprise the following steps:

[0153] S5051: based on the first sample region, the second sample region and the sample license plate number, adjusting parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained until a first end condition is met, to obtain a target recognition model trained in a first stage.

[0154] S5052: based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, adjusting parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier of the target recognition model trained in the first stage until a second end condition is met, to obtain a target recognition model trained.

[0155] The first predicted license plate number output by the first recognition model has high accuracy, the first feature extractor in the first recognition model can accurately extract image features of the first sample region, and the second feature extractor in the second recognition model can accurately extract image features of the second sample region. In order to avoid the trained target device model excessively relying on the license plate prediction result (i.e. the first predicted license plate number) and falling into a local optimum, the target recognition model to be trained needs to be trained in two stages.

[0156] First stage training:

[0157] Since the first feature extractor comes from the pre-trained first recognition model and the second feature extractor also comes from the pre-trained second recognition model, the parameters of the first feature extractor and the second feature extractor are obtained by pre-training, and the parameters of the remaining parts (such as the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier) are randomly initialized.

[0158] In order to prevent the randomly initialized parameters from distorting the parameters of the first feature extractor and the second feature extractor, i.e. affecting the feature extraction effect of the first feature extractor and the second feature extractor, as shown in Figure 7 , the encoder is removed, i.e. the first predicted license plate number is not taken as the input of the target recognition model and does not participate in the first stage training, and only the first sample region and the second sample region are retained as the input of the target recognition model and participate in the first stage training.

[0159] Since the first predicted license plate number does not participate in the first stage training, the second classifier (i.e., Classifier2) and the third classifier (i.e., Classifier1) are not trained in the first stage, that is, based on the first sample region, the second sample region and the sample license plate number, the parameters of the first feature extractor (i.e., net (network) 0), the second feature extractor (i.e., net1), the feature fusioner (including Fusion-features and Transformer Encoder) and the first classifier (i.e., Classifier0) of the target recognition model to be trained are adjusted until the first end condition is met, and the target recognition model trained in the first stage is obtained.

[0160] In some embodiments, the warmup training method can be used for the first stage training, and the first stage training can be divided into two sub-stage training. Correspondingly, step S5041 can include the following steps: based on the first sample region, the second sample region and the sample license plate number, the parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained are adjusted until a preset iteration number is reached, and the target recognition model trained in the first stage is obtained. The first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the sub-stage are adjusted based on the first sample region, the second sample region and the sample license plate number, until the first end condition is met, and the target recognition model trained in the first stage is obtained.

[0161] In the first sub-stage training, since the performance of the first recognition model is more stable and the accuracy of the first feature extractor is higher, the parameters of the first feature extractor are frozen first, and the second feature extractor, the feature fusioner and the first classifier are trained with a lower learning rate, that is, based on the first sample region, the second sample region and the sample license plate number, the parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained are adjusted according to a lower learning rate until a preset iteration number is reached, and the target recognition model trained in the first sub-stage is obtained. The preset iteration number can be set by the technician according to the demand. For example, the preset iteration number is 1000.

[0162] In some embodiments, the first sub-stage training is performed in the following manner: using a first feature extractor of the target recognition model to be trained, image features of the first sample region are extracted to obtain first sample image features. Using a second feature extractor of the target recognition model to be trained, image features of the second sample region are extracted to obtain second sample image features. Using a feature fusioner of the target recognition model to be trained, the first sample image features and the second sample image features and a preset first semantic feature are spliced, and feature extraction is performed on the spliced result to obtain first fusion features.

[0163] Since the training stage removes the encoder, in order to ensure the dimension of the feature fusion performed by the feature fusioner, the preset first semantic feature is used for feature fusion with the first sample image features and the second sample image features. The preset first semantic feature is set by a technician according to requirements, for example, the elements in the preset first semantic feature are all set to 0.

[0164] Further, the first fusion features are input into a first classifier of the target recognition model to be trained to obtain an output second predicted license plate number of the sample vehicle. Based on the second predicted license plate number and the sample license plate number, a first loss value is calculated, the first loss value representing a difference between the second predicted license plate number and the sample license plate number. Based on the first loss value, parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained are adjusted, so that the difference between the second predicted license plate number and the sample license plate number gradually decreases until a preset iteration number is reached, and a target recognition model of the first sub-stage training is obtained.

[0165] In the second sub-stage training, the parameters of the first feature extractor are released, the learning rate is increased, and the first feature extractor, the second feature extractor, the feature fusioner and the first classifier are continuously trained, that is, based on the first sample region, the second sample region and the sample license plate number, the parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model of the first sub-stage training are adjusted at a higher learning rate until a first end condition is met, and a target recognition model of the first stage training is obtained.

[0166] In some embodiments, the target recognition model of the first sub-stage training is trained in the following manner to obtain a target recognition model of the second sub-stage training as a target recognition model of the first stage training:

[0167] The first feature extractor of the target recognition model trained by the first sub-stage is used to extract image features of the first sample region to obtain third sample image features. The second feature extractor of the target recognition model trained by the first sub-stage is used to extract image features of the second sample region to obtain fourth sample image features. The feature fusioner of the target recognition model trained by the first sub-stage is used to fuse the third sample image features, the fourth sample image features and the preset first semantic features to obtain second fusion features.

[0168] The second fusion features are input into the first classifier of the target recognition model trained by the first sub-stage to obtain a third predicted license plate number of the sample vehicle. Based on the third predicted license plate number and the sample license plate number, a second loss value is calculated. Based on the second loss value, parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained by the first sub-stage are adjusted until a first end condition is met, and a target recognition model trained by a second sub-stage is obtained as a target recognition model trained by a first stage. The second end condition is set by a technician according to requirements.

[0169] Second stage training:

[0170] The image feature fusion of the target recognition model trained by the first stage has a preliminary performance, and then the text feature fusion is continued for the second stage training. An encoder of the license plate prediction result is added to the target recognition model trained by the first stage to extract features of the license plate prediction result (i.e., the first predicted license plate number) to obtain second semantic features.

[0171] In some embodiments, step S5042 can include the following steps: using the first feature extractor of the target recognition model trained by the first stage to extract image features of the first sample region to obtain fifth sample image features, and using the second feature extractor of the target recognition model trained by the first stage to extract image features of the second sample region to obtain sixth sample image features. The first feature extractor and the second feature extractor are trained by the first stage and can accurately extract image features.

[0172] The encoder of the target recognition model trained by the first stage is used to encode the first predicted license plate number to obtain second semantic features of the first predicted license plate number. The feature fusioner of the target recognition model trained by the first stage is used to fuse the fifth sample image features, the sixth sample image features and the second semantic features to obtain third fusion features.

[0173] The third fusion feature is input into the first classifier of the target recognition model trained in the first stage to obtain an output fourth predicted license plate number of the sample vehicle; the third fusion feature is input into the second classifier of the target recognition model trained in the first stage to obtain a first predicted label of the first predicted license plate number of the sample vehicle; and the second fusion feature is input into the third classifier of the target recognition model trained in the first stage to obtain a second predicted label of the first predicted license plate number of the sample vehicle.

[0174] Based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label, the parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier in the target recognition model trained in the first stage are adjusted until a second end condition is met, and a trained target recognition model is obtained.

[0175] In some embodiments, the parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier in the target recognition model trained in the first stage are adjusted in the following manner:

[0176] Based on the fourth predicted license plate number and the sample license plate number, a third loss value is calculated, the third loss value representing the difference between the fourth predicted license plate number and the sample license plate number. Based on the third loss value, the parameters of the first classifier of the target recognition model trained in the first stage are adjusted so that the fourth predicted license plate number output by the first classifier is increasingly close to the sample license plate number.

[0177] Based on the first predicted label and the first sample label, a fourth loss value is calculated, the fourth loss value representing the difference between the first predicted label and the first sample label. Based on the fourth loss value, the parameters of the second classifier of the target recognition model trained in the first stage are adjusted so that the first predicted label output by the second classifier is increasingly close to the first sample label.

[0178] Based on the second predicted label and the second sample label, a fifth loss value is calculated, the fifth loss value representing the difference between the second predicted label and the second sample label. Based on the fifth loss value, the parameters of the third classifier of the target recognition model trained in the first stage are adjusted so that the second predicted label output by the third classifier is increasingly close to the second sample label.

[0179] The target loss value is calculated based on the third loss value, the fourth loss value and the fifth loss value. For example, a weighted sum of the third loss value, the fourth loss value and the fifth loss value is calculated to obtain the target loss value. The target loss value represents the difference between the fourth predicted license plate number and the sample license plate number, the difference between the first predicted label and the first sample label, and the difference between the second predicted label and the second sample label. Based on the target loss value, the parameters of the first feature extractor, the second feature extractor, the encoder and the feature fusioner of the target recognition model trained in the first stage are adjusted until the second end condition is met, and the trained target recognition model is obtained. The second end condition is set by the technician according to the requirements.

[0180] Based on the above processing, the target recognition model is trained in a two-stage training manner. The two-stage network optimization scheme can ensure that the fusion result output by the target recognition model (i.e., the license plate number output by the first classifier) is better than the license plate prediction result (i.e., the license plate number output by the first recognition model), improve the robustness of the target recognition model, and further improve the accuracy of license plate recognition.

[0181] Method two: The target recognition model to be trained includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner and a first classifier.

[0182] The first feature extractor of the target recognition model to be trained is used to extract image features of the first sample region to obtain seventh image features. The second feature extractor of the target recognition model to be trained is used to extract image features of the second sample region to obtain eighth image features. The encoder of the target recognition model to be trained is used to encode the first predicted license plate number to obtain third semantic features.

[0183] The feature fusioner of the target recognition model to be trained is used to perform feature fusion on the seventh image features, the eighth image features and the third semantic features to obtain fourth fusion features. The first classifier of the target recognition model to be trained is used to input the fourth fusion features to obtain the output sixth predicted license plate number of the sample vehicle. Based on the sixth predicted license plate number and the sample license plate number, a sixth loss value is calculated. Based on the sixth loss value, the parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner and the first classifier of the target recognition model to be trained are adjusted until the third end condition is reached, and the trained target recognition model is obtained. The second end condition is set by the technician according to the requirements.

[0184] In some embodiments, after the target recognition model training is completed, the first feature extractor in the trained target recognition model is used to replace the first feature extractor in the first recognition model, to obtain an updated first recognition model, further optimizing the performance of the first recognition model. The second feature extractor in the trained target recognition model is used to replace the second feature extractor in the second recognition model, to obtain an updated second recognition model, further optimizing the performance of the second recognition model.

[0185] With Figure 1 corresponding to the method embodiment of the present application, see Figure 8 , Figure 8 A structure diagram of a license plate number recognition device provided by an embodiment of the present application is shown in FIG. 1. The device comprises:

[0186] An image acquisition module 801 is configured to acquire a to-be-processed image of a target vehicle.

[0187] An image extraction module 802 is configured to extract a first to-be-processed region where a license plate number of the target vehicle is located and a second to-be-processed region where a magnification number is located from the to-be-processed image.

[0188] A first to-be-processed license plate number determination module 803 is configured to perform license plate number recognition on the first to-be-processed region, to obtain a first to-be-processed license plate number of the target vehicle.

[0189] A target license plate number determination module 804 is configured to input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle into a trained target recognition model, to perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by using the target recognition model, to obtain a target fusion feature; and to obtain a target license plate number of the target vehicle based on the target fusion feature; wherein the target recognition model is trained based on a first sample region where a license plate number of a sample vehicle in a sample image is located, a second sample region where a magnification number is located, a first predicted license plate number obtained by performing license plate number recognition on the first sample region, and a sample license plate number of the sample vehicle.

[0190] Optionally, the target license plate number determination module 804 is specifically configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number through the target recognition model, and obtain target fusion features; based on the target fusion features, obtain an output second to-be-processed license plate number of the target vehicle, a first correction label and a second correction label corresponding to the first to-be-processed license plate number; the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, determine an incorrect character indicated by the second correction label in the first to-be-processed license plate number; determine a target character at the same position in the second to-be-processed license plate number according to the position of the incorrect character in the first to-be-processed license plate number; replace the incorrect character in the first to-be-processed license plate number with the target character to obtain a target license plate number of the target vehicle; when the first correction label indicates that the first to-be-processed license plate number is correct, determine that the first to-be-processed license plate number is the target license plate number of the target vehicle.

[0191] Optionally, the trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier.

[0192] The target license plate number determination module 804 is specifically configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, extract image features of the first to-be-processed region through the first feature extractor to obtain first to-be-processed image features, extract image features of the second to-be-processed region through the second feature extractor to obtain second to-be-processed image features, encode the first to-be-processed license plate number through the encoder to obtain target semantic features of the first to-be-processed license plate number, and perform feature fusion on the first to-be-processed image features, the second to-be-processed image features, and the target semantic features through the feature fusioner to obtain target fusion features.

[0193] The target license plate number determination module is specifically configured to: input the target fusion features into the first classifier to obtain an output second to-be-processed license plate number of the target vehicle; input the target fusion features into the second classifier to obtain an output first correction label; and input the target fusion features into the third classifier to obtain an output second correction label.

[0194] Optionally, the first feature extractor comprises a first feature extraction module and a first encoding module; and the second feature extractor comprises a second feature extraction module and a second encoding module.

[0195] The target license plate number determination module 804 is specifically configured to: perform deep feature extraction on the first to-be-processed region by the first feature extraction module to obtain image features of the first to-be-processed region; and encode the extracted image features by the first encoding module to obtain first to-be-processed image features.

[0196] The target license plate number determination module 804 is specifically configured to: perform deep feature extraction on the second to-be-processed region by the second feature extraction module to obtain image features of the second to-be-processed region; and encode the extracted image features by the second encoding module to obtain second to-be-processed image features.

[0197] Optionally, the trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner and a first classifier.

[0198] The target license plate number determination module 804 is specifically configured to: input the first to-be-processed region, the second to-be-processed region and the first to-be-processed license plate number into the trained target recognition model; extract image features of the first to-be-processed region by the first feature extractor to obtain first to-be-processed image features; extract image features of the second to-be-processed region by the second feature extractor to obtain second to-be-processed image features; encode the first to-be-processed license plate number by the encoder to obtain target semantic features of the first to-be-processed license plate number; perform feature fusion on the first to-be-processed image features, the second to-be-processed image features and the target semantic features by the feature fusioner to obtain target fusion features; and input the target fusion features into the first classifier to obtain an output target license plate number of the target vehicle.

[0199] Based on the license plate number recognition device provided in the embodiments of the present application, the first to-be-processed region, the second to-be-processed region and the first to-be-processed license plate number are subjected to feature fusion by using the trained target recognition model to obtain target fusion features; and based on the target fusion features, a target license plate number of a target vehicle is obtained. Even if the license plate number is enlarged, damaged or blurred, the target fusion features obtained by the feature fusion of the first to-be-processed region, the second to-be-processed region and the first to-be-processed license plate number can accurately identify the license plate number of the vehicle, that is, the accuracy of license plate number recognition can be improved.

[0200] Corresponding to the method embodiments of the present application, see Figure 5 , Figure 9 , Figure 9A structural diagram of a recognition model training device provided by an embodiment of the present application is shown in FIG. 1. The device includes:

[0201] An image acquisition module 901 is configured to acquire a sample image containing a sample vehicle and a sample license plate number of the sample vehicle.

[0202] An image extraction module 902 is configured to extract, from the sample image, a first sample region in which the license plate number of the sample vehicle is located and a second sample region in which the characters are located.

[0203] A first predicted license plate number determination module 903 is configured to perform license plate number recognition on the first sample region to obtain a first predicted license plate number of the sample vehicle.

[0204] A sample label determination module 904 is configured to determine, based on the sample license plate number and the first predicted license plate number of the sample vehicle, a first sample label and a second sample label corresponding to the first predicted license plate number. The first sample label indicates whether the first predicted license plate number is correct, and the second sample label indicates whether each character in the first predicted license plate number is correct.

[0205] A training module 905 is configured to train a target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, to obtain a trained target recognition model.

[0206] Optionally, the target recognition model to be trained includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier.

[0207] The training module 905 is specifically configured to adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region, and the sample license plate number, until a first end condition is met, to obtain a first-stage trained target recognition model; and adjust parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier, and the third classifier of the first-stage trained target recognition model based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, until a second end condition is met, to obtain the trained target recognition model.

[0208] Optionally, the training module 905 is specifically configured to: based on the first sample region, the second sample region and the sample license plate number, adjust parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained until a preset iteration number is reached, to obtain a target recognition model trained in a first stage in a first sub-stage; based on the first sample region, the second sample region and the sample license plate number, adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage until a first end condition is met, to obtain the target recognition model trained in the first stage.

[0209] Optionally, the training module 905 is specifically configured to: use the first feature extractor of the target recognition model to be trained to extract image features of the first sample region, to obtain first sample image features; use the second feature extractor of the target recognition model to be trained to extract image features of the second sample region, to obtain second sample image features; use the feature fusioner of the target recognition model to be trained to perform feature fusion on the first sample image features, the second sample image features and a preset first semantic feature, to obtain first fusion features; input the first fusion features into the first classifier of the target recognition model to be trained, to obtain an output second predicted license plate number of the sample vehicle; based on the second predicted license plate number and the sample license plate number, calculate a first loss value; based on the first loss value, adjust parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained until a preset iteration number is reached, to obtain the target recognition model trained in the first stage in the first sub-stage.

[0210] Optionally, the training module 905 is specifically configured to: extract image features of the first sample region using a first feature extractor of the target recognition model trained in the first sub-stage to obtain third sample image features; extract image features of the second sample region using a second feature extractor of the target recognition model trained in the first sub-stage to obtain fourth sample image features; fuse the third sample image features, the fourth sample image features, and a preset first semantic feature using a feature fusioner of the target recognition model trained in the first sub-stage to obtain second fused features; input the second fused features into a first classifier of the target recognition model trained in the first sub-stage to obtain an output third predicted license plate number of the sample vehicle; calculate a second loss value based on the third predicted license plate number and the sample license plate number; and adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model trained in the first sub-stage based on the second loss value until a first end condition is met, so as to obtain a target recognition model trained in a second sub-stage of the first stage as the target recognition model trained in the first stage.

[0211] Optionally, the training module 905 is specifically configured to: extract image features of the first sample region using a first feature extractor of the target recognition model trained in the first stage to obtain fifth sample image features; extract image features of the second sample region using a second feature extractor of the target recognition model trained in the first stage to obtain sixth sample image features; encode the first predicted license plate number using an encoder of the target recognition model trained in the first stage to obtain second semantic features of the first predicted license plate number; fuse the fifth sample image features, the sixth sample image features, and the second semantic features using a feature fusioner of the target recognition model trained in the first stage to obtain third fused features; input the third fused features into a first classifier of the target recognition model trained in the first stage to obtain the fourth predicted license plate number of the sample vehicle as output; input the third fused features into a second classifier of the target recognition model trained in the first stage to obtain a first predicted label of the first predicted license plate number of the sample vehicle as output; input the third fused features into a third classifier of the target recognition model trained in the first stage to obtain a second predicted label of the first predicted license plate number of the sample vehicle as output; and adjust parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier, and the third classifier in the target recognition model trained in the first stage based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label until a second end condition is met, to obtain the target recognition model trained.

[0212] Optionally, the training module 905 is specifically configured to: calculate a third loss value based on the fourth predicted license plate number and the sample license plate number; adjust parameters of the first classifier of the target recognition model trained in the first stage based on the third loss value; calculate a fourth loss value based on the first predicted label and the first sample label; adjust parameters of the second classifier of the target recognition model trained in the first stage based on the fourth loss value; calculate a fifth loss value based on the second predicted label and the second sample label; adjust parameters of the third classifier of the target recognition model trained in the first stage based on the fifth loss value; calculate a target loss value based on the third loss value, the fourth loss value, and the fifth loss value; and adjust parameters of the first feature extractor, the second feature extractor, the encoder, and the feature fusioner of the target recognition model trained in the first stage based on the target loss value until the second end condition is met, to obtain the target recognition model trained.

[0213] Optionally, the apparatus further comprises: a model obtaining module, configured to, before the training module 905 performs training of the target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the trained target recognition model, perform obtaining of the trained first recognition model and the trained second recognition model; wherein the first recognition model is configured to recognize a license plate number; the second recognition model is configured to recognize a magnification number; the first recognition model comprises a first feature extractor and a classifier; and the second recognition model comprises a second feature extractor and a classifier.

[0214] a model constructing module, configured to determine the target recognition model to be trained based on the trained first recognition model and the trained second recognition model; wherein the first feature extractor in the target recognition model to be trained is the same as the first feature extractor in the first recognition model; and the second feature extractor in the target recognition model to be trained is the same as the second feature extractor in the second recognition model.

[0215] Optionally, the apparatus further comprises: a first recognition model updating module, configured to, after the training module 905 performs training of the target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the trained target recognition model, perform replacing of the first feature extractor in the first recognition model with the first feature extractor in the trained target recognition model to obtain an updated first recognition model.

[0216] a second recognition model updating module, configured to replace the second feature extractor in the second recognition model with the second feature extractor in the trained target recognition model to obtain an updated second recognition model.

[0217] Optionally, the image extracting module 902 is specifically configured to: extract a first image region in which a license plate number of the sample vehicle is located from the sample image, and adjust the first image region to obtain a first sample region; wherein the adjustment of the first image region comprises at least one of the following: truncation processing, and shielding processing; extract a second image region in which a magnification number of the sample vehicle is located from the sample image, and adjust the second image region to obtain a second sample region; wherein the adjustment of the second image region comprises at least one of the following: brightness adjustment, angle adjustment, color adjustment, and cropping processing.

[0218] Based on the recognition model training device provided in this application embodiment, a target recognition model is trained. Using the trained target recognition model, combined with the second processing area where the enlarged number is located, the first processing area where the license plate number is located, and the identified first license plate number, the target license plate number of the target vehicle is determined. Even if the enlarged number is missing or blurry, the vehicle's license plate number can be accurately identified by combining the first processing area where the license plate number is located and the identified first license plate number, thus improving the accuracy of license plate number recognition.

[0219] In the technical solution of this application, the acquisition, storage, use, processing, transmission, provision and disclosure of user personal information (license plate number) are all carried out with the user's authorization.

[0220] It should be noted that the recognition model in this embodiment is not a recognition model for a specific user and cannot reflect the personal information of a specific user.

[0221] It should be noted that the sample images containing license plate numbers and enlarged license plate numbers in this embodiment are from a publicly available dataset.

[0222] This application also provides an electronic device, such as... Figure 10 As shown, the device includes: a memory 1001 for storing computer programs; and a processor 1002 for executing the program stored in the memory 1001 to implement the steps of the license plate number recognition method or the recognition model training method in the aforementioned embodiments. Furthermore, the electronic device may also include a communication bus and / or a communication interface, and the processor 1002, the communication interface, and the memory 1001 communicate with each other through the communication bus.

[0223] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0224] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0225] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0226] The processor described above can be a general processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0227] In yet another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of any of the license plate number recognition methods or the steps of any of the recognition model training methods.

[0228] In yet another embodiment provided in the present application, a computer program product containing instructions is also provided, and when the computer program product is run on a computer, the computer is caused to execute any of the license plate number recognition methods or any of the recognition model training methods in the above embodiments.

[0229] In the embodiments described above, all or some of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or some of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into and executed by a computer, all or some of the processes or functions according to the embodiments described in the specification are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a solid state disk (SSD) and the like.

[0230] It should be noted that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0231] Each of the embodiments in the specification is described in a related manner, and the same or similar parts between each of the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device, electronic device, computer readable storage medium and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0232] The above merely provides the preferred embodiment of the present application, and not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A license plate number recognition method characterized by, The method comprises: acquiring a to-be-processed image of a target vehicle; extracting, from the to-be-processed image, a first to-be-processed region where a license plate number of the target vehicle is located and a second to-be-processed region where a magnification number is located; performing license plate number recognition on the first to-be-processed region to obtain a first to-be-processed license plate number of the target vehicle; inputting the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle into a trained target recognition model, performing feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model to obtain target fusion features, and obtaining a target license plate number of the target vehicle based on the target fusion features, wherein the target recognition model is trained based on a first sample region where a license plate number of a sample vehicle in a sample image is located, a second sample region where a magnification number is located, a first predicted license plate number obtained by performing license plate number recognition on the first sample region, and a sample license plate number of the sample vehicle.

2. The method of claim 1, wherein, The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number of the target vehicle are input into a trained target recognition model, and feature fusion is performed on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model to obtain target fusion features; obtaining a target license plate number of the target vehicle based on the target fusion features comprises: inputting the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, performing feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model to obtain target fusion features; obtaining, based on the target fusion features, an output second to-be-processed license plate number of the target vehicle, a first correction label and a second correction label corresponding to the first to-be-processed license plate number; wherein the first correction label indicates whether the first to-be-processed license plate number is correct; and the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, determining an error character indicated by the second correction label in the first to-be-processed license plate number, determining a target character in the same position in the second to-be-processed license plate number according to the position of the error character in the first to-be-processed license plate number, and replacing the error character in the first to-be-processed license plate number with the target character to obtain the target license plate number of the target vehicle; when the first correction label indicates that the first to-be-processed license plate number is correct, determining that the first to-be-processed license plate number is the target license plate number of the target vehicle.

3. The method of claim 2, wherein, The trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier. The first to be processed area, the second to be processed area, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed area are extracted through the first feature extractor, and first to be processed image features are obtained. The first to be processed area, the second to be processed area, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed area are extracted through the first feature extractor, and first to be processed image features are obtained. Image features of the second to be processed area are extracted through the second feature extractor, and second to be processed image features are obtained. The first to be processed license plate number is encoded through the encoder, and target semantic features of the first to be processed license plate number are obtained. The first to be processed image features, the second to be processed image features, and the target semantic features are fused through the feature fusioner, and target fusion features are obtained. The target fusion features are input into the first classifier, and the second to be processed license plate number of the target vehicle is output; the target fusion features are input into the second classifier, and the first correction label is output; and the target fusion features are input into the third classifier, and the second correction label is output. The first feature extractor includes a first feature extraction module and a first encoding module; and the second feature extractor includes a second feature extraction module and a second encoding module.

4. The method of claim 3, wherein, The first to be processed area, the second to be processed area, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed area are extracted through the first feature extractor, and first to be processed image features are obtained. The first to be processed area, the second to be processed area, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed area are extracted through the first feature extractor, and first to be processed image features are obtained. The first to be processed area, the second to be processed area, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed area are extracted through the first feature extractor, and first to be processed image features are obtained. The first to be processed area, the second to be processed area, and the first to be processed license plate number are input into the target recognition model trained, image features of the first to be processed area are extracted through the first feature extractor, and first to be processed image features are obtained. The target recognition model trained includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, and a first classifier.

5. The method of claim 1, wherein, ​ The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input into a trained target recognition model, feature fusion is performed on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model, and target fusion features are obtained; based on the target fusion features, a target license plate number of the target vehicle is obtained, including: The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input into a trained target recognition model, feature fusion is performed on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model, and target fusion features are obtained; based on the target fusion features, a target license plate number of the target vehicle is obtained, including: The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input into a trained target recognition model, feature fusion is performed on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model, and target fusion features are obtained; based on the target fusion features, a target license plate number of the target vehicle is obtained, including: The first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number are input into a trained target recognition model, feature fusion is performed on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number by the target recognition model, and target fusion features are obtained; based on the target fusion features, a target license plate number of the target vehicle is obtained, including: The target fusion features are input into a first classifier, and a target license plate number of the target vehicle is obtained as output. The method comprises:

6. A method of training a recognition model, the method comprising: Obtaining a sample image containing a sample vehicle and a sample license plate number of the sample vehicle; From the sample image, a first sample region where the sample license plate number of the sample vehicle is located and a second sample region where the sample license plate number is located are extracted; Performing license plate number recognition on the first sample region to obtain a first predicted license plate number of the sample vehicle; Based on the sample license plate number and the first predicted license plate number of the sample vehicle, a first sample label and a second sample label corresponding to the first predicted license plate number are determined; wherein the first sample label indicates whether the first predicted license plate number is correct; and the second sample label indicates whether each character in the first predicted license plate number is correct; Using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, a target recognition model to be trained is trained to obtain a trained target recognition model. The target recognition model to be trained comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier; 7. The method of claim 6, wherein, The method comprises: Based on the first sample region, the second sample region, and the sample license plate number, the parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model to be trained are adjusted until a first end condition is met, and a first stage trained target recognition model is obtained; ​ adjusting parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier of the target recognition model trained in the first stage based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number and the first sample label and the second sample label until the second end condition is met, to obtain the target recognition model trained.

8. The method of claim 7, wherein, The adjusting parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region and the sample license plate number until the first end condition is met to obtain the target recognition model trained in the first stage, comprises: adjusting parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region and the sample license plate number until a preset number of iterations is reached to obtain the target recognition model trained in the first stage of the first sub-stage; adjusting parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage based on the first sample region, the second sample region and the sample license plate number until the first end condition is met to obtain the target recognition model trained in the first stage.

9. The method of claim 8, wherein, The adjusting parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region and the sample license plate number until a preset number of iterations is reached to obtain the target recognition model trained in the first stage of the first sub-stage, comprises: extracting image features of the first sample region using the first feature extractor of the target recognition model to be trained to obtain first sample image features; extracting image features of the second sample region using the second feature extractor of the target recognition model to be trained to obtain second sample image features; performing feature fusion on the first sample image features, the second sample image features and a preset first semantic feature using the feature fusioner of the target recognition model to be trained to obtain first fusion features; inputting the first fusion features into the first classifier of the target recognition model to be trained to obtain a second predicted license plate number of the sample vehicle as output; calculating a first loss value based on the second predicted license plate number and the sample license plate number; adjusting parameters of the second feature extractor, the feature fusioner and the first classifier of the target recognition model to be trained based on the first loss value until a preset number of iterations is reached to obtain the target recognition model trained in the first stage of the first sub-stage.

10. The method of claim 8, wherein, The parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage are adjusted based on the first sample region, the second sample region and the sample license plate number until a first end condition is met, and a target recognition model trained in a first stage is obtained, including: The first feature extractor of the target recognition model trained in the first sub-stage is used to extract image features of the first sample region to obtain third sample image features; The second feature extractor of the target recognition model trained in the first sub-stage is used to extract image features of the second sample region to obtain fourth sample image features; The feature fusioner of the target recognition model trained in the first sub-stage is used to fuse the third sample image features, the fourth sample image features and a preset first semantic feature to obtain second fusion features; The second fusion features are input into the first classifier of the target recognition model trained in the first sub-stage to obtain a third predicted license plate number of the sample vehicle as output; A second loss value is calculated based on the third predicted license plate number and the sample license plate number; The parameters of the first feature extractor, the second feature extractor, the feature fusioner and the first classifier of the target recognition model trained in the first sub-stage are adjusted based on the third predicted license plate number and the sample license plate number until a first end condition is met, and a target recognition model trained in a second sub-stage of the first stage is obtained as the target recognition model trained in the first stage.

11. The method of claim 7, wherein, The parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier of the target recognition model trained in the first stage are adjusted based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label until a second end condition is met, and a trained target recognition model is obtained, including: The first feature extractor of the target recognition model trained in the first stage is used to extract image features of the first sample region to obtain fifth sample image features; The second feature extractor of the target recognition model trained in the first stage is used to extract image features of the second sample region to obtain sixth sample image features; The encoder of the target recognition model trained in the first stage is used to encode the first predicted license plate number to obtain a second semantic feature of the first predicted license plate number; The feature fusioner of the target recognition model trained in the first stage is used to fuse the fifth sample image features, the sixth sample image features and the second semantic feature to obtain third fusion features; inputting the third fusion feature into a first classifier of the target recognition model trained in the first stage to obtain an output fourth predicted license plate number of the sample vehicle; inputting the third fusion feature into a second classifier of the target recognition model trained in the first stage to obtain an output first predicted label of the first predicted license plate number of the sample vehicle; and inputting the third fusion feature into a third classifier of the target recognition model trained in the first stage to obtain an output second predicted label of the first predicted license plate number of the sample vehicle; adjusting parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier in the target recognition model trained in the first stage based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label until a second end condition is met, to obtain the target recognition model trained.

12. The method of claim 11, wherein, The adjusting parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier and the third classifier in the target recognition model trained in the first stage based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label until a second end condition is met, to obtain the target recognition model trained, comprises: calculating a third loss value based on the fourth predicted license plate number and the sample license plate number; and adjusting parameters of the first classifier of the target recognition model trained in the first stage based on the third loss value; calculating a fourth loss value based on the first predicted label and the first sample label; and adjusting parameters of the second classifier of the target recognition model trained in the first stage based on the fourth loss value; calculating a fifth loss value based on the second predicted label and the second sample label; and adjusting parameters of the third classifier of the target recognition model trained in the first stage based on the fifth loss value; calculating a target loss value based on the third loss value, the fourth loss value and the fifth loss value; and adjusting parameters of the first feature extractor, the second feature extractor, the encoder and the feature fusioner of the target recognition model trained in the first stage based on the target loss value until the second end condition is met, to obtain the target recognition model trained.

13. The method of claim 7, wherein, Before the training of the target recognition model to be trained based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label is performed to obtain the target recognition model trained, the method further comprises: obtaining a first recognition model and a second recognition model after training, wherein the first recognition model is used for recognizing a license plate number, the second recognition model is used for recognizing a magnification number, the first recognition model comprises a first feature extractor and a classifier, and the second recognition model comprises a second feature extractor and a classifier; determining a target recognition model to be trained based on the first recognition model and the second recognition model after training, wherein the first feature extractor in the target recognition model to be trained is the same as the first feature extractor in the first recognition model, and the second feature extractor in the target recognition model to be trained is the same as the second feature extractor in the second recognition model.

14. The method of claim 13, wherein, After the target recognition model to be trained is trained by using the first sample area, the second sample area, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, the method further comprises: replacing the first feature extractor in the first recognition model with the first feature extractor in the target recognition model after training to obtain an updated first recognition model; replacing the second feature extractor in the second recognition model with the second feature extractor in the target recognition model after training to obtain an updated second recognition model.

15. The method of claim 6, wherein, The first sample area and the second sample area are extracted from the sample image, and the first sample area and the second sample area comprise: The first sample area and the second sample area are extracted from the sample image, and the first sample area and the second sample area comprise: The first sample area and the second sample area are extracted from the sample image, and the first sample area and the second sample area comprise:

16. A license plate number recognition apparatus characterized by comprising: The device comprises: an image acquisition module configured to acquire a to-be-processed image of a target vehicle; an image extraction module configured to extract, from the to-be-processed image, a first to-be-processed area where a license plate number of the target vehicle is located and a second to-be-processed area where a magnification number of the target vehicle is located; a first to-be-processed license plate number determination module configured to perform license plate number recognition on the first to-be-processed area to obtain a first to-be-processed license plate number of the target vehicle; and a second to-be-processed license plate number determination module configured to perform license plate number recognition on the second to-be-processed area to obtain a second to-be-processed license plate number of the target vehicle. The target license plate number determination module is configured to:

17. The apparatus of claim 16, wherein, input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number through the target recognition model, and obtain target fusion features; based on the target fusion features, obtain an output second to-be-processed license plate number of the target vehicle, a first correction label, and a second correction label corresponding to the first to-be-processed license plate number; the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, determine an error character indicated by the second correction label in the first to-be-processed license plate number; determine a target character at the same position in the second to-be-processed license plate number according to the position of the error character in the first to-be-processed license plate number; replace the error character in the first to-be-processed license plate number with the target character to obtain the target license plate number of the target vehicle; when the first correction label indicates that the first to-be-processed license plate number is correct, determine that the first to-be-processed license plate number is the target license plate number of the target vehicle; and / or, The trained target recognition model includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier. The target license plate number determination module is configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number through the target recognition model, and obtain target fusion features; based on the target fusion features, obtain an output second to-be-processed license plate number of the target vehicle, a first correction label, and a second correction label corresponding to the first to-be-processed license plate number; the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, determine an error character indicated by the second correction label in the first to-be-processed license plate number; determine a target character at the same position in the second to-be-processed license plate number according to the position of the error character in the first to-be-processed license plate number; replace the error character in the first to-be-processed license plate number with the target character to obtain the target license plate number of the target vehicle; when the first correction label indicates that the first to-be-processed license plate number is correct, determine that the first to-be-processed license plate number is the target license plate number of the target vehicle; and / or, The trained target recognition model includes a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier. The target license plate number determination module is configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into a trained target recognition model, perform feature fusion on the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number through the target recognition model, and obtain target fusion features; based on the target fusion features, obtain an output second to-be-processed license plate number of the target vehicle, a first correction label, and a second correction label corresponding to the first to-be-processed license plate number; the first correction label indicates whether the first to-be-processed license plate number is correct; the second correction label indicates whether each character in the first to-be-processed license plate number is correct; when the first correction label indicates that the first to-be-processed license plate number is incorrect, determine an error character indicated by the second correction label in the first to-be-processed license plate number; determine a target character at the same position in the second to-be-processed license plate number according to the position of the error character in the first to-be-processed license plate number; replace the error character in the first to-be-processed license plate number with the target character to obtain the target license plate number of the target vehicle; when the first correction label indicates that the first to-be-processed license plate number is correct, determine that the first to-be-processed license plate number is the target license plate number of the target vehicle; and / or, The target license plate number determination module is specifically configured to: input the target fusion feature into the first classifier to obtain an output of a second to-be-processed license plate number of the target vehicle; input the target fusion feature into a second classifier to obtain an output of a first correction label; and input the target fusion feature into a third classifier to obtain an output of a second correction label; and / or The first feature extractor comprises a first feature extraction module and a first encoding module; and the second feature extractor comprises a second feature extraction module and a second encoding module. The target license plate number determination module is specifically configured to: perform deep feature extraction on the first to-be-processed region by the first feature extraction module to obtain image features of the first to-be-processed region; and perform encoding on the extracted image features by the first encoding module to obtain first to-be-processed image features. The target license plate number determination module is specifically configured to: perform deep feature extraction on the second to-be-processed region by the second feature extraction module to obtain image features of the second to-be-processed region; and perform encoding on the extracted image features by the second encoding module to obtain second to-be-processed image features. and / or The trained target recognition model comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, and a first classifier. The target license plate number determination module is specifically configured to: input the first to-be-processed region, the second to-be-processed region, and the first to-be-processed license plate number into the trained target recognition model, extract image features of the first to-be-processed region by the first feature extractor to obtain first to-be-processed image features; extract image features of the second to-be-processed region by the second feature extractor to obtain second to-be-processed image features; perform encoding on the first to-be-processed license plate number by the encoder to obtain target semantic features of the first to-be-processed license plate number; perform feature fusion on the first to-be-processed image features, the second to-be-processed image features, and the target semantic features by the feature fusioner to obtain target fusion features; input the target fusion features into the first classifier to obtain an output of a target license plate number of the target vehicle. 18.A recognition model training apparatus, characterized by comprising: The device comprises: an image acquisition module configured to acquire a sample image containing a sample vehicle and a sample license plate number of the sample vehicle; an image extraction module configured to extract, from the sample image, a first sample region in which the sample license plate number of the sample vehicle is located and a second sample region in which a license plate number is located; a first predicted license plate number determination module configured to perform license plate number recognition on the first sample region to obtain a first predicted license plate number of the sample vehicle; a sample label determination module configured to determine, based on the sample license plate number and the first predicted license plate number of the sample vehicle, a first sample label and a second sample label corresponding to the first predicted license plate number; wherein the first sample label indicates whether the first predicted license plate number is correct, and the second sample label indicates whether each character in the first predicted license plate number is correct. The training module is configured to train the target recognition model to be trained by using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, and obtain the trained target recognition model.

19. The apparatus of claim 18, wherein, The target recognition model to be trained comprises a first feature extractor, a second feature extractor, an encoder, a feature fusioner, a first classifier, a second classifier, and a third classifier. The training module is specifically configured to: adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region, and the sample license plate number, until a first end condition is met, and obtain a first-stage trained target recognition model; adjust parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier, and the third classifier of the first-stage trained target recognition model based on the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label, until a second end condition is met, and obtain the trained target recognition model; and / or, The training module is specifically configured to: adjust parameters of the second feature extractor, the feature fusioner, and the first classifier of the target recognition model to be trained based on the first sample region, the second sample region, and the sample license plate number, until a preset iteration number is reached, and obtain a first-stage first-sub-stage trained target recognition model; adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the first-stage first-sub-stage trained target recognition model based on the first sample region, the second sample region, and the sample license plate number, until the first end condition is met, and obtain the first-stage trained target recognition model; and / or, The training module is specifically configured to: extract image features of the first sample region by using the first feature extractor of the target recognition model to be trained, and obtain first sample image features; extract image features of the second sample region by using the second feature extractor of the target recognition model to be trained, and obtain second sample image features; perform feature fusion on the first sample image features, the second sample image features, and a preset first semantic feature by using the feature fusioner of the target recognition model to be trained, and obtain first fusion features; input the first fusion features into the first classifier of the target recognition model to be trained, to obtain a second predicted license plate number of the sample vehicle as output; calculate a first loss value based on the second predicted license plate number and the sample license plate number; adjust parameters of the second feature extractor, the feature fusioner, and the first classifier of the target recognition model to be trained based on the first loss value, until the preset iteration number is reached, and obtain the first-stage first-sub-stage trained target recognition model; and / or, The training module is specifically configured to: extract image features of the first sample region using the first feature extractor of the target recognition model trained in the first sub-stage, to obtain third sample image features; extract image features of the second sample region using the second feature extractor of the target recognition model trained in the first sub-stage, to obtain fourth sample image features; fuse the third sample image features, the fourth sample image features, and the preset first semantic features using the feature fusioner of the target recognition model trained in the first sub-stage, to obtain second fusion features; input the second fusion features into the first classifier of the target recognition model trained in the first sub-stage, to obtain an output third predicted license plate number of the sample vehicle; calculate a second loss value based on the third predicted license plate number and the sample license plate number; adjust parameters of the first feature extractor, the second feature extractor, the feature fusioner, and the first classifier of the target recognition model trained in the first sub-stage based on the second loss value, until a first end condition is met, to obtain a target recognition model trained in a second sub-stage of the first stage as a target recognition model trained in the first stage; and / or, The training module is specifically configured to: extract image features of the first sample region using the first feature extractor of the target recognition model trained in the first stage, to obtain fifth sample image features; extract image features of the second sample region using the second feature extractor of the target recognition model trained in the first stage, to obtain sixth sample image features; encode the first predicted license plate number using the encoder of the target recognition model trained in the first stage, to obtain second semantic features of the first predicted license plate number; fuse the fifth sample image features, the sixth sample image features, and the second semantic features using the feature fusioner of the target recognition model trained in the first stage, to obtain third fusion features; input the third fusion features into the first classifier of the target recognition model trained in the first stage, to obtain an output fourth predicted license plate number of the sample vehicle; input the third fusion features into the second classifier of the target recognition model trained in the first stage, to obtain an output first predicted label of the first predicted license plate number of the sample vehicle; and input the third fusion features into the third classifier of the target recognition model trained in the first stage, to obtain an output second predicted label of the first predicted license plate number of the sample vehicle; adjust parameters of the first feature extractor, the second feature extractor, the encoder, the feature fusioner, the first classifier, the second classifier, and the third classifier in the target recognition model trained in the first stage based on the fourth predicted license plate number and the sample license plate number, the first predicted label and the first sample label, and the second predicted label and the second sample label, until a second end condition is met, to obtain a target recognition model trained. and / or, The training module is specifically configured to: calculating a third loss value based on the fourth predicted license plate number and the sample license plate number; and adjusting parameters of the first classifier of the target recognition model trained in the first stage based on the third loss value; calculating a fourth loss value based on the first predicted label and the first sample label; and adjusting parameters of the second classifier of the target recognition model trained in the first stage based on the fourth loss value; calculating a fifth loss value based on the second predicted label and the second sample label; and adjusting parameters of the third classifier of the target recognition model trained in the first stage based on the fifth loss value; calculating a target loss value based on the third loss value, the fourth loss value and the fifth loss value; and adjusting parameters of the first feature extractor, the second feature extractor, the encoder and the feature fusioner of the target recognition model trained in the first stage based on the target loss value until a second end condition is met, to obtain the trained target recognition model; and / or, The device further comprises: a model obtaining module, configured to, before the training module performs training on the target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the trained target recognition model, perform obtaining a first recognition model and a second recognition model trained; the first recognition model is configured to recognize a license plate number; the second recognition model is configured to recognize an enlarged number; the first recognition model comprises a first feature extractor and a classifier; and the second recognition model comprises a second feature extractor and a classifier; a model construction module, configured to determine the target recognition model to be trained based on the first recognition model and the second recognition model trained; the first feature extractor in the target recognition model to be trained is the same as the first feature extractor in the first recognition model; and the second feature extractor in the target recognition model to be trained is the same as the second feature extractor in the second recognition model; and / or, The device further comprises: a first recognition model updating module, configured to, after the training module performs training on the target recognition model to be trained using the first sample region, the second sample region, the sample license plate number, the first predicted license plate number, and the first sample label and the second sample label to obtain the trained target recognition model, perform replacing the first feature extractor in the first recognition model with the first feature extractor in the trained target recognition model to obtain an updated first recognition model; a second recognition model updating module, configured to replace the second feature extractor in the second recognition model with the second feature extractor in the trained target recognition model to obtain an updated second recognition model; and / or, The image extraction module is specifically configured to: From the sample image, a first image region where the license plate number of the sample vehicle is located is extracted, and the first image region is adjusted to obtain a first sample region; wherein the adjustment of the first image region includes at least one of the following: truncation processing, shielding processing; From the sample image, a second image region where the enlarged number of the sample vehicle is located is extracted, and the second image region is adjusted to obtain a second sample region; wherein the adjustment of the second image region includes at least one of the following: brightness adjustment, angle adjustment, color adjustment, and cropping processing.

20. An electronic device, comprising: Comprise: a memory for storing a computer program; a processor for executing the program stored on the memory to realize the method of claims 1-5, or any one of claims 6-15.

21. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the method of claims 1-5, or any one of claims 6-15.

Images