A license plate recognition method, device, equipment and medium

Abstract

The present invention provides a license plate recognition method, device, equipment and medium, which are used to solve the problem of inaccurate recognition that is easy to occur when recognizing a vehicle license plate in the prior art. In the embodiment of the present invention, by determining the regional shape information of the license plate area in the image, if the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the license plate area is corrected through perspective transformation, and the license plate area after perspective transformation is identified to contain the license plate information, so that when an abnormality occurs in the license plate area, the license plate area can be accurately identified, thereby improving the accuracy of license plate recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a license plate recognition method, device, equipment and medium. Background Art

[0002] With the rapid growth of the number of cars in my country, urban traffic problems are becoming more and more frequent, such as illegal parking. If a vehicle is parked illegally, the illegally parked vehicle will occupy the sidewalk or emergency passage for a long time, causing road obstruction and inconvenience for pedestrians and passing vehicles. And when traffic problems occur, there are also huge safety hazards.

[0003] In the prior art, image recognition technology is usually used to identify the license plate number of the vehicle that has violated the law, so as to issue some punishment measures. However, the image to be identified is collected by an image acquisition device at a fixed position, and the license plate at various angles and large angles will appear in the field of view of the image acquisition device at a fixed position.

[0004] Figure 1 A schematic diagram of an image captured by an image acquisition device in the prior art, Figure 1 It can be seen that the license plate of the vehicle in the picture is unclear or the angle is too large. Figure 1 When recognizing the license plate in the image, the recognition is usually inaccurate. Therefore, the existing technology usually cuts out the area where the vehicle is located in the image and performs recognition on the cut-out image. Figure 2 The image after the vehicle is cut out in the prior art is obtained by Figure 2 It can be seen that Figure 2 The angle of the license plate in the image is too large, so if we directly recognize Figure 2 The recognition result of the license plate is still not accurate enough. Summary of the invention

[0005] The present invention provides a license plate recognition method, device, equipment and medium, which are used to solve the problem of inaccurate recognition that is easy to occur when recognizing a vehicle license plate in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a license plate recognition method, the method comprising:

[0007] For each received frame of image, obtain each license plate area in the frame of image;

[0008] For each license plate area, determining area shape information of the license plate area;

[0009] If the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the license plate area is perspective transformed, and the license plate information contained in the license plate area after the perspective transformation is identified, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

[0010] Furthermore, for each license plate area, determining the area shape information of the license plate area includes:

[0011] For each license plate area, the aspect ratio of the license plate area is determined, and the aspect ratio is used as the area shape information of the license plate area.

[0012] Furthermore, the matching of the regional shape information of the license plate area with the regional shape information of the abnormal license plate area includes:

[0013] The aspect ratio of the license plate area is smaller than the preset threshold.

[0014] Furthermore, obtaining the license plate area of ​​each license plate in the frame image includes:

[0015] Obtain the license plate area of ​​each license plate in the frame image through the trained license plate detection model;

[0016] The license plate detection model is trained in the following way:

[0017] Obtain any sample image in the sample set, and a sample license plate area of ​​each license plate contained in the sample image;

[0018] Inputting the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image;

[0019] Training the original license plate detection model according to the sample license plate area and the identified license plate area to obtain an intermediate license plate detection model;

[0020] Unstructured pruning is performed on the intermediate license plate detection model to obtain a trained license plate detection model.

[0021] Furthermore, the performing unstructured pruning on the intermediate license plate detection model includes:

[0022] Obtain the sub-weight corresponding to each block in each corresponding single-channel convolution kernel when the intermediate license plate detection model training is completed; and for each block, determine the mask of the block according to the sub-weight of the block;

[0023] According to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, the position of the block to be pruned in the single-channel convolution kernel is determined, and the block at this position in each single-channel convolution kernel is pruned.

[0024] Furthermore, before performing perspective transformation on the license plate area, the method further includes:

[0025] According to the license plate area, determine the width and height of the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model;

[0026] According to a preset number of coordinate points in the target position information, sequentially determining the sum of distances between the preset number of coordinate points and a preset number of preset coordinate points;

[0027] Determine the position information of the coordinate points corresponding to the minimum distance and the minimum distance, and perform subsequent perspective transformation operations on the license plate area.

[0028] Further, the step of sequentially determining the sum of distances between the preset number of coordinate points and the preset number of preset coordinate points according to the preset number of coordinate points in the target position information includes:

[0029] Determine, in a preset order, coordinate sets corresponding to different orders of coordinate points in the target position information;

[0030] For each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance to the preset coordinate point corresponding to each preset position, and determine the distance and value.

[0031] In a second aspect, an embodiment of the present invention further provides a license plate recognition device, the device comprising:

[0032] An acquisition module, for acquiring a license plate area of ​​each license plate in each frame of image acquired by the image acquisition device;

[0033] A determination module, for determining the area shape information of each license plate area;

[0034] A processing module is used to perform perspective transformation on the license plate area if the area shape information of the license plate area matches the area shape information of the abnormal license plate area, and identify the license plate information contained in the license plate area after the perspective transformation, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

[0035] Furthermore, the determination module is specifically used to determine the aspect ratio of each license plate area, and use the aspect ratio as the area shape information of the license plate area.

[0036] Furthermore, the acquisition module is specifically used to acquire the license plate area of ​​each license plate in the frame image through the trained license plate detection model;

[0037] The processing module is also used to obtain any sample image in the sample set and the sample license plate area of ​​each license plate contained in the sample image; input the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image; train the original license plate detection model according to the sample license plate area and the recognized license plate area to obtain an intermediate license plate detection model; perform unstructured pruning on the intermediate license plate detection model to obtain a trained license plate detection model.

[0038] Furthermore, the processing module is specifically used to obtain the sub-weight corresponding to each block in each single-channel convolution kernel when the training of the intermediate license plate detection model is completed; and for each block, determine the mask of the block according to the sub-weight of the block; determine the position of the block to be pruned in the single-channel convolution kernel according to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, and prune the block at this position in each single-channel convolution kernel.

[0039] Furthermore, the processing module is also used to determine the width and height of the license plate area based on the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model; determine the sum of distances between the preset number of coordinate points and the preset number of preset coordinate points in sequence according to a preset number of coordinate points in the target position information; and determine the position information of the coordinate points that correspond in sequence when the sum of the distances is minimum.

[0040] Furthermore, the processing module is specifically used to determine, in a preset order, coordinate sets corresponding to different orders of coordinate points in the target position information; for each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance to the preset coordinate point corresponding to each preset position, and determine the distance and value.

[0041] In a third aspect, an embodiment of the present invention further provides an electronic device, which includes at least a processor and a memory, and the processor is used to execute the steps of the license plate recognition method described in any one of the claims when executing a computer program stored in the memory.

[0042] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the license plate recognition method described in any one of the above claims.

[0043] In the embodiment of the present invention, by determining the regional shape information of the license plate area in the image, if the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the license plate area is corrected through perspective transformation, and it is recognized that the license plate area after the perspective transformation contains the license plate information, so that when an abnormality occurs in the license plate area, the license plate area can be accurately identified, thereby improving the accuracy of license plate recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of an image captured by an image acquisition device in the prior art;

[0046] Figure 2 The image after the vehicle is cut out in the prior art;

[0047] Figure 3 A schematic diagram of a license plate recognition process provided by an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of a license plate detection model training process provided by an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of a convolution kernel block in a network provided by an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of an enlarged license plate area provided in an embodiment of the present invention;

[0051] Figure 7a A schematic diagram of a license plate area provided by an embodiment of the present invention;

[0052] Figure 7b A schematic diagram of a license plate area after perspective transformation provided by an embodiment of the present invention;

[0053] Figure 8 A detailed schematic diagram of a license plate recognition process provided by an embodiment of the present invention;

[0054] Fig. 9 A schematic diagram of the structure of a license plate recognition device provided by an embodiment of the present invention;

[0055] Fig.10 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0057] Embodiment 1:

[0058] Figure 3 A schematic diagram of a license plate recognition process provided by an embodiment of the present invention, the process includes the following steps:

[0059] S301: For each received frame of image, obtain each license plate area in the frame of image.

[0060] The license plate recognition method provided by the embodiment of the present invention is applied to an electronic device, which may be an intelligent device such as an image acquisition device, a PC or a server.

[0061] In the embodiment of the present invention, the image acquisition device acquires each frame of image and sends each acquired frame of image to the electronic device, and the electronic device acquires each license plate area in the frame of image for each frame of image acquired by the image acquisition device. Specifically, the acquired license plate area can be the position information of the license plate marked by a rectangular frame in the frame of image.

[0062] S302: For each license plate area, determine the area shape information of the license plate area.

[0063] After acquiring each license plate area in the frame image, for each license plate area in the frame image, the regional shape information of the license plate area in the image is determined, wherein the regional shape information may refer to information such as the width and height of the license plate area.

[0064] S303: If the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, a perspective transformation is performed on the license plate area, and the license plate information contained in the license plate area after the perspective transformation is identified, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

[0065] In order to determine whether the license plate area is an abnormal license plate area, in an embodiment of the present invention, the area shape information of the abnormal license plate area is pre-saved, and the abnormal license plate area is a pre-configured license plate area that meets the requirements, specifically a license plate area in the image captured by the image acquisition device whose deformation degree from the standard license plate shape meets the requirements.

[0066] After determining the regional shape information of the license plate area, determine whether the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area. If the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, then the license plate area is determined to be an abnormal license plate area, and the license plate is perspectively transformed, and the license plate information after perspective transformation is identified through character recognition. Specifically, in the embodiment of the present invention, perspective transformation of the license plate according to the license plate area is a prior art and will not be described in detail here. That is to say, if the license plate area is determined to be an abnormal license plate area, the license plate area is corrected through perspective transformation, so that the characters of the license plate area are accurately determined through character recognition, that is, the license plate number of the license plate area is determined.

[0067] Moreover, in an embodiment of the present invention, if the regional shape information of the license plate area does not match the regional shape information of the abnormal license plate area, it means that the license plate area is not placed at an angle. In order to identify the license plate information of the license plate area, in an embodiment of the present invention, the area where the license plate is located is intercepted, and the intercepted license plate area is identified through character recognition to obtain the license plate information.

[0068] In the embodiment of the present invention, by determining the regional shape information of the license plate area in the image, if the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the license plate area is corrected through perspective transformation, and it is recognized that the license plate area after the perspective transformation contains the license plate information, so that when an abnormality occurs in the license plate area, the license plate area can be accurately identified, thereby improving the accuracy of license plate recognition.

[0069] Embodiment 2:

[0070] In order to accurately determine the regional shape information of the license plate area, based on the above embodiment, in an embodiment of the present invention, for each license plate area, determining the regional shape information of the license plate area includes:

[0071] For each license plate area, the aspect ratio of the license plate area is determined, and the aspect ratio is used as the area shape information of the license plate area.

[0072] In an embodiment of the present invention, the regional shape information of the license plate area may refer to the aspect ratio of the license plate area. Specifically, the process of determining the regional shape information of the license plate area is: based on the license plate area, determine the width of the license plate area in the image and the height of the license plate area in the image, and determine that the ratio of the width to the height of the license plate area in the image is the aspect ratio of the license plate area in the image, and determine the determined aspect ratio as the regional shape information of the license plate area.

[0073] In order to accurately determine the license plate information contained in the license plate area, based on the above embodiments, in an embodiment of the present invention, the matching of the regional shape information of the license plate area with the regional shape information of the abnormal license plate area includes:

[0074] The aspect ratio of the license plate area is smaller than the preset threshold.

[0075] In an embodiment of the present invention, after determining the aspect ratio of the license plate area in the image, it is determined whether the aspect ratio of the license plate is less than a preset threshold value. If the aspect ratio of the license plate is less than the preset threshold value, it means that the width of the license plate area in the image is relatively small, which means that the license plate area in the image is a large-angle non-upright license plate, that is, the regional shape information of the license plate area matches that of the abnormal license plate area. In this embodiment of the present invention, the preset threshold value can be 1, or 0.5, etc.

[0076] Embodiment 3:

[0077] In order to accurately determine each license plate area in the image, based on the above embodiment, in an embodiment of the present invention, obtaining the license plate area of ​​each license plate in the frame image includes:

[0078] Obtain the license plate area of ​​each license plate in the frame image through the trained license plate detection model;

[0079] The license plate detection model is trained in the following way:

[0080] Obtain any sample image in the sample set, and a sample license plate area of ​​each license plate contained in the sample image;

[0081] Inputting the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image;

[0082] Training the original license plate detection model according to the sample license plate area and the identified license plate area to obtain an intermediate license plate detection model;

[0083] Unstructured pruning is performed on the intermediate license plate detection model to obtain a trained license plate detection model.

[0084] In an embodiment of the present invention, when acquiring each license plate area in an image, for each frame of image, the frame image is input into a pre-trained license plate detection model to obtain the output of the license plate detection model. The output of the license plate detection model includes the license plate area of ​​each license plate in the frame image, and the acquired license plate area may be the position information of the license plate marked by a rectangular frame in the frame image.

[0085] In order to train the license plate detection model, a sample set for training the license plate detection model is stored in an embodiment of the present invention, and the sample set includes a large number of sample images, wherein the sample images are images collected in different scenes and different time periods. In order to facilitate the training of the license plate detection model, the sample set also stores, for each sample image, a sample license plate region of each license plate contained in the sample image in the sample image, wherein the sample license plate region can be the position information of the license plate marked by a rectangular frame in the sample image.

[0086] In an embodiment of the present invention, after obtaining any sample image in the sample set and the sample license plate area of ​​the license plate contained in the sample image, the sample image is input into the license plate detection model, and the license plate detection model outputs the recognized license plate area of ​​the license plate contained in the sample image.

[0087] After obtaining the output of the license plate detection model, the electronic device determines that the output of the license plate detection model is the recognized license plate area of ​​the license plate contained in the sample image. The electronic device adjusts the parameters of the license plate detection model according to the sample license plate area in the sample image and the recognized license plate area output by the license plate detection model, thereby training the license plate detection model and obtaining an intermediate license plate detection model.

[0088] The license plate detection model is trained in the above manner, and when the preset conditions are met, an intermediate license plate detection model is obtained. The preset conditions may be that the number of sample images in the sample set whose identification license plate area is consistent with the sample license plate area obtained after training with the original license plate detection model is greater than a set number; or the number of iterations for training the original license plate detection model reaches a set maximum number of iterations, etc. Specifically, the embodiments of the present invention do not limit this.

[0089] In an embodiment of the present invention, the license plate detection model can be a YoloV3 target detection model. Since the YoloV3 target detection model has excellent algorithm performance, includes 3 detection layers, 9 scales of anchors, and adopts the structure of Feature Pyramid Networks (FPN), it is very suitable for detecting multi-scale targets. However, the original model is large and difficult to be deployed on a device with limited computing power such as an image acquisition device. Therefore, the license plate detection model in the embodiment of the present invention is based on the YoloV3 target detection algorithm with excellent performance, and performs unstructured pruning on the backbone network. The traditional unstructured pruning method will cause the network structure to be irregular, and the hardware device cannot be used for parallel computing. Although the model is reduced, the problem of slowing down the operation speed will occur. The unstructured pruning method of the embodiment of the present invention can sparse the network while ensuring that the local network structure still has the characteristics of being able to facilitate parallel computing of the hardware device. The pruning is more refined and can ensure the accuracy of the model. While ensuring the performance of the algorithm, the network model is further lightweight and the algorithm operation speed is improved. Specifically, how to perform unstructured pruning is a prior art and will not be repeated here.

[0090] Moreover, in an embodiment of the present invention, when pruning the license plate detection model, it is necessary to prune blocks with smaller parameter values ​​in the single-channel convolution kernel block, where the parameter includes weights. Therefore, in an embodiment of the present invention, in order to make the weights of the pruned blocks small enough so as not to affect the normal operation of the license plate detection model, while ensuring that the identified license plate area output by the license plate detection model is accurate enough, it is also necessary to ensure that the weight of each block in the single-channel convolution kernel is small enough.

[0091] Therefore, in the embodiment of the present invention, the license plate detection model is trained according to the sample license plate area and the identified license plate area, and the parameters of the license plate detection model can be adjusted through two trainings. Specifically, the process includes: first, the parameters of the license plate detection model can be adjusted according to whether there is a deviation between the sample license plate area and the identified license plate area. The specific adjusted parameters can be weights. After the training is completed, the license plate detection model with adjusted weights is retrained through a preset loss function so that the weight of the license plate detection model is small enough.

[0092] Specifically, the preset loss function may be:

[0093]

[0094] Among them, f(W; b) represents the original loss function of the license plate detection model during the first training; R represents the regularization term used to sparse the network structure; λ represents the weight for adjusting the sparsity, and λ is a preset value; α i (t) Represents the corresponding convolution kernel W iA set of penalty values.

[0095] Among them, W i =[W i1 ,W i2 ,...,W iK ],

[0096] therefore and

[0097] Among them, ε is a small value pre-set based on experience. According to the above formula, α ijn (t) represents the block corresponding to the convolution kernel [W ij ] t h，w The penalty value, α ijn (t) *[W ij ] t h，w It represents the result of the dot multiplication of the weight of the block of the convolution kernel and the penalty value corresponding to the weight. According to the above description, when the weight of the block of the convolution kernel is larger, the penalty value of the corresponding weight is smaller; when the weight of the block of the convolution kernel is smaller, the penalty value of the corresponding weight is larger, thereby further adjusting the weight of the license plate detection model.

[0098] In order to accurately train the license plate detection model, based on the above embodiments, in an embodiment of the present invention, the unstructured pruning of the intermediate license plate detection model includes:

[0099] Obtain the sub-weight corresponding to each block in each corresponding single-channel convolution kernel when the intermediate license plate detection model training is completed; and for each block, determine the mask of the block according to the sub-weight of the block;

[0100] According to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, the position of the block to be pruned in the single-channel convolution kernel is determined, and the block at this position in each single-channel convolution kernel is pruned.

[0101] In an embodiment of the present invention, when pruning, blocks at the same position in different single-channel convolution kernels are pruned. Specifically, the process is: after the training of the intermediate license plate detection model is completed, the sub-weight corresponding to each block in each single-channel convolution kernel is obtained, and the mask of the block is determined according to the sub-weight corresponding to each block. Specifically, how to determine the mask according to the weight is a prior art and will not be repeated here.

[0102] According to the mask of each block in each single-channel convolution kernel, the pruned block is determined. Specifically, the process is as follows: determine the absolute value of the mask of each block, and determine the average value of the absolute value of the mask of the blocks at the same position in each single-channel convolution kernel. According to the set pruning number and the average value of the absolute value of the mask of the blocks at the same position, determine the position of the pruned block in the single-channel convolution kernel, and prune the block at this position in each single-channel convolution kernel. Specifically, the average value determined by the removed block is smaller.

[0103] Figure 4 A schematic diagram of a license plate detection model training process provided by an embodiment of the present invention.

[0104] Depend on Figure 4 It can be seen that the license plate detection model is first trained according to the sample set, and the values ​​of the license plate detection model parameters are adjusted, wherein the parameters include weights. After obtaining the license plate detection model with adjusted parameters, the license plate detection model with adjusted parameters is sparsely trained by adding a regularization term to the loss function to obtain an intermediate license plate detection model. After obtaining the trained intermediate license plate detection model, the mask of each block of the single-channel convolution kernel contained in the convolution kernel block of each layer of the network is determined. In an embodiment of the present invention, each convolution kernel block can contain 8 single-channel convolution kernels, and the same positions of all single-channel convolution kernels in each convolution kernel block participate in pruning as a whole. Taking a 3x3 single-channel convolution kernel as an example, each single-channel convolution kernel has 9 blocks, so there are 9 wholes in each single-channel convolution kernel. The size of each whole is the weight mean within the whole. Each layer has several wholes. The masks determined by all blocks are arranged in descending order according to size. According to the set pruning rate, those blocks with smaller masks are used as pruning targets. After pruning, the shape distribution of each single-channel convolution kernel is exactly the same, which facilitates more accurate determination of the license plate area.

[0105] Figure 5 A schematic diagram of a convolution kernel block in a network provided in an embodiment of the present invention.

[0106] Depend on Figure 5 It can be seen that each convolution kernel block contains multiple single-channel convolution kernels, wherein each single-channel convolution kernel includes 3*3 blocks. In the embodiment of the present invention, when pruning, what is removed is the block in the single-channel convolution kernel.

[0107] Embodiment 4:

[0108] In order to accurately identify the license plate, based on the above embodiments, in an embodiment of the present invention, before the perspective transformation is performed on the license plate area, the method further includes:

[0109] According to the license plate area, determine the width and height of the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model;

[0110] According to a preset number of coordinate points in the target position information, sequentially determining the sum of distances between the preset number of coordinate points and a preset number of preset coordinate points;

[0111] Determine the position information of the coordinate points corresponding to the minimum distance and the minimum distance, and perform subsequent perspective transformation operations on the license plate area.

[0112] In an embodiment of the present invention, if the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, it means that there is an abnormality in the license plate area, that is, the license plate is not placed in the upright position. Therefore, if the license plate area is directly transformed in perspective, the width and height of the license plate area after the perspective transformation may be relatively small, causing the characters in the license plate area to be stacked together, thereby causing the subsequent problem of inaccurate license plate recognition. It is also possible that the area occupied by the license plate area in the image is too small, resulting in inaccurate license plate recognition.

[0113] Therefore, in an embodiment of the present invention, if the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the area where the license plate area is located in the image is enlarged, wherein the process of enlarging the area where the license plate area is located in the image is: according to the license plate area, determine the width and height of the license plate area in the image, enlarge the width and height of the license plate area in the image, and in an embodiment of the present invention, the multiples of the width enlargement and the multiples of the height enlargement are usually different, and usually the multiples of the width enlargement are greater than the multiples of the height enlargement, to prevent the characters of the license plate from being stacked together after the subsequent perspective transformation. Since the license plate area can be the position information of the license plate in the image marked by a rectangular frame, the width and height of the license plate area in the image can be accurately determined.

[0114] Specifically, if the license plate area is the position information of the license plate in the image marked by a rectangular frame, the process of enlarging the license plate area in the image can be: reducing the horizontal coordinate of the coordinate value of the upper left corner in the position information by a preset multiple, reducing the vertical coordinate of the coordinate value of the upper left corner by a preset multiple, increasing the horizontal coordinate of the coordinate value of the lower right corner in the position information by a preset multiple, and increasing the vertical coordinate of the coordinate value of the lower right corner by a preset multiple. Since the coordinate system established in the image usually takes the pixel point in the upper left corner of the image as the origin, the direction to the right is the positive direction of the x-axis, and the vertical downward is the positive direction of the y-axis, the license plate area is determined according to the coordinate system. Therefore, reducing the horizontal and vertical coordinate values ​​of the upper left corner in the position information of the rectangular frame marking the license plate area by a preset multiple, and increasing the horizontal and vertical coordinate values ​​of the lower right corner in the position information of the rectangular frame marking the license plate area by a preset multiple can achieve the expansion of the width and height of the license plate area in the image.

[0115] In an embodiment of the present invention, the license plate area is a rectangular area. The rectangular area where the license plate is located is represented by the coordinate values ​​of the pixel point in the upper left corner of the rectangular area and the coordinate values ​​of the pixel point in the lower right corner of the rectangular area as an example. If the coordinate values ​​of the pixel point in the upper left corner have a horizontal coordinate of xmin and a vertical coordinate of ymin, and the coordinate values ​​of the pixel point in the lower right corner have a horizontal coordinate of xmax and a vertical coordinate of ymax, then xmin and ymin are respectively reduced by 0.1 times, and xmax and ymax are respectively increased by 0.1 times, thereby achieving the expansion of the width and height of the license plate area.

[0116] Figure 6 This is a schematic diagram of an enlarged license plate area provided in an embodiment of the present invention.

[0117] in, Figure 6 The smaller rectangular area in the figure is the license plate area output by the license plate detection model. Figure 6 The larger rectangular area in is the license plate area after the license plate area is enlarged. Figure 6 It can be seen that after the license plate area is enlarged, the license plate information is clearer and can be identified more accurately.

[0118] And when performing perspective transformation, it is necessary to determine the position information of a preset number of corner points in the license plate area. Therefore, in an embodiment of the present invention, the width and height of the license plate area are intercepted and enlarged, and the intercepted image is input into a pre-trained key point detection model, the output of the key point detection model is obtained, and the output of the model is determined to be the position information of a preset number of corner points in the intercepted image, that is, the target position information. Among them, the key point detection model outputs the position information of a preset number of corner points in the intercepted image, which is the position information of a preset number of corner points in clockwise order or the position information of a preset number of corner points in counterclockwise order. And in an embodiment of the present invention, the preset number of corner points are usually the upper left corner, upper right corner, lower right corner and lower left corner of the license plate.

[0119] Because it is necessary to determine the coordinate value corresponding to each corner point when performing perspective transformation, and when the angle of the license plate area in the image is too large, it can only be determined that the output position information contains the coordinate values ​​of a preset number of corner points, but it cannot be determined which corner points the coordinate values ​​in the output position information correspond to in sequence, that is, it cannot be ensured that the output position information is the coordinate values ​​of the pixel points of the upper left corner, upper right corner, lower right corner and lower left corner in sequence. Therefore, in an embodiment of the present invention, the specific process of the electronic device determining which corner point each coordinate point corresponds to is: according to different coordinate points in the target position information of the rectangular frame identifying the license plate area, determine a coordinate set composed of coordinate points of different orders, and determine the sum of the distances between each coordinate point in the preset number of coordinate points in each coordinate set and the preset coordinate point in sequence, and obtain the position information of the coordinate point that is first determined when the sum of the distances is the smallest.

[0120] Furthermore, in an embodiment of the present invention, since the position information output by the key point detection model can be the position information of a clockwise coordinate point or the position information of a counterclockwise coordinate point, in order to quickly obtain which corner point each coordinate point in the position information output by the key point detection model corresponds to, in an embodiment of the present invention, a preset number of preset coordinate points are clockwise coordinate points or counterclockwise coordinate points in sequence. Specifically, if the position information output by the key point detection model is the position information of a clockwise coordinate point, then the preset number of preset coordinate points are clockwise coordinate points. Therefore, in an embodiment of the present invention, after obtaining the position information of the coordinate point that is first determined when the distance sum is minimum, the position information is determined to be the position information of the corner point in the preset direction, and since the coordinate points in the target position information contained in the license plate area output by the key point detection model are clockwise or counterclockwise coordinate points, after determining the corner point corresponding to the position information that is first determined when the distance sum is minimum, the corner points corresponding to other position information can be determined.

[0121] Since the preset number of preset coordinate points are coordinate points in different directions, the coordinate points corresponding in the coordinate set with the smallest distance in the distance values ​​determined in different coordinate sets are the corner points in the direction of the preset coordinate points. For example, the preset number of preset coordinate points are the four coordinate points of the upper left corner, upper right corner, lower right corner, and lower left corner in the intercepted image. Then, when the sum of the distances is the smallest, the corner points corresponding to the corresponding coordinate points are the upper left corner, upper right corner, lower right corner, and lower left corner. In other words, the preset number of preset coordinate points are several coordinate points in different directions. Therefore, specifically, the coordinate point corresponding to the corner point in the direction has a smaller distance from the preset coordinate point in the direction. Therefore, it is possible to effectively determine which corner point of the license plate area the different coordinate points specifically correspond to.

[0122] In an embodiment of the present invention, the preset coordinate point may be a preset number of position information of the license plate area output by the license plate detection model, that is, the coordinate values ​​of the upper left corner, upper right corner, lower right corner, and lower left corner of the rectangular frame output by the license plate detection model. If the preset number of preset coordinate points is the result output by the license plate detection model, the target position information output by the key point detection model is converted into the position information of the corner point in the original image where the license plate area is located. Specifically, since the coordinate value of the upper left corner of the license plate area after enlargement is known in the image containing the license plate, and the upper left corner of the license plate area in the intercepted image is known as the origin, the position information of the preset number of corner points in the original image can be obtained according to the position information of the license plate area in the intercepted image and the original image and the position information of the preset number of corner points of the license plate area in the intercepted image. Therefore, according to the position information of the preset number of corner points in the original image and the position information output by the license plate detection model, it is determined which coordinate value corresponds to which corner point.

[0123] In order to accurately determine the coordinate value of each corner point, based on the above embodiments, in an embodiment of the present invention, the distances between the preset number of coordinate points and the preset number of preset coordinate points are sequentially determined according to the preset number of coordinate points in the target position information, including:

[0124] Determine, in a preset order, coordinate sets corresponding to different orders of coordinate points in the target position information;

[0125] For each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance to the preset coordinate point corresponding to each preset position, and determine the distance and value.

[0126] In an embodiment of the present invention, the coordinate value of each corner point is determined by the distance and value. Specifically, the distance and value are determined by the order of the coordinate points included in different coordinate sets. When determining the distance and value, for each coordinate set, the distance value corresponding to the coordinate set is determined. When determining the coordinate set, since the position information output by the key point detection model can be the position information of the clockwise coordinate point or the counterclockwise coordinate point, and the direction corresponding to the preset number of preset coordinate points is the same as the clockwise direction corresponding to the position information output by the key point detection model, that is, if the preset number of preset coordinate points are clockwise coordinate points in sequence, the position information output by the key point detection model is the position information of the clockwise coordinate point. Therefore, when determining the coordinate set, the different coordinate points in the target position information can be used as the first coordinate point to be determined, and the corresponding positions of other coordinate points in the target position information in the coordinate set can be determined clockwise. For example, the coordinate points corresponding to the target position information are a, b, and c, respectively. Then the first coordinate points determined in the different coordinate sets are a, b, and c, respectively. In the coordinate set where the first coordinate point determined is a, the other coordinate points determined clockwise are b and c; in the coordinate set where the first coordinate point determined is b, the other coordinate points determined clockwise are c and a; in the coordinate set where the first coordinate point determined is c, the other coordinate points determined clockwise are a and b.

[0127] And the process of determining the distance and value corresponding to the coordinate set according to the coordinate set is: for each coordinate point of the preset position in the coordinate set, determine the distance between the coordinate point of the preset position and the preset coordinate point corresponding to the preset position, and determine the distance and value based on the distance determined for the coordinate point of each preset position in the coordinate set.

[0128] For example, the position information output by the license plate detection model starts from the upper left corner and goes clockwise as follows (x 0, y0)、(x 1, y1)、(x 2, y2), (x 3, y3), that is to say, the position information output by the license plate detection model is the coordinate points corresponding to the upper left corner, upper right corner, lower right corner, and lower left corner. After the key point detection model outputs 4 coordinate points and converts them to the original image, they are clockwise as follows: Specifically, in the embodiment of the present invention, the determined coordinate set is as well as

[0129] In one possible implementation, when determining the sum of the distances between each coordinate point in the first coordinate set and each coordinate point in the position information output by the license plate detection model, determine the distance between the first coordinate point in the coordinate set and the first coordinate point in the position information output by the license plate detection model, the distance between the second coordinate point in the coordinate set and the second coordinate point in the position information output by the license plate detection model, the distance between the third coordinate point in the coordinate set and the third coordinate point in the position information output by the license plate detection model, and the distance between the fourth coordinate point in the coordinate set and the fourth coordinate point in the position information output by the license plate detection model, and determine the sum of the distances, and determine the sum of the distances corresponding to other coordinate sets in turn, and determine the corresponding coordinate values ​​in the coordinate set with the smallest sum of distances.

[0130] Specifically, since the square of the distance needs to be determined first when determining the distance between the coordinate points, in the embodiment of the present invention, when determining the distance sum, the sum of the squares of the distances may also be determined, wherein the formula for determining the sum of the squares of the distances of the first coordinate set is:

[0131]

[0132] That is to say, determine the square of the distance between the first coordinate point in the first coordinate set and the first coordinate point in the position information output by the license plate detection model as the first square value; determine the square of the distance between the second coordinate point in the first coordinate set and the second coordinate point in the position information output by the license plate detection model as the second square value; determine the square of the distance between the third coordinate point in the first coordinate set and the third coordinate point in the position information output by the license plate detection model as the third square value; determine the square of the distance between the fourth coordinate point in the first coordinate set and the fourth coordinate point in the position information output by the license plate detection model as the fourth square value, and determine the sum of the first square value, the second square value and the third square value as d0.

[0133] And the result d1 is obtained by determining the sum of the squares of the distances between the coordinate points in the second coordinate set and the coordinate points of the position information output by the license plate detection model through the above determination method, and then determining d2 corresponding to the third coordinate set and d3 corresponding to the fourth coordinate set respectively based on the above method, and determining that the coordinate points in the coordinate set corresponding to the smallest d are the coordinate points of the pixel points of the upper left corner, upper right corner, lower right corner, and lower left corner of the license plate respectively, so that the position information of the corner points obtained can be more accurately transformed in perspective.

[0134] In order to accurately obtain the license plate information, in an embodiment of the present invention, after obtaining the position information of the coordinate points corresponding to the minimum distance sum, the license plate area can be perspective transformed according to the position information of the coordinate points corresponding to the minimum distance sum, and the license plate information in the transformed license plate area can be determined by character recognition.

[0135] Figure 7a A schematic diagram of a license plate area provided in an embodiment of the present invention.

[0136] Figure 7a The aspect ratio of the license plate area in is less than the preset value, that is Figure 7a For the license plate placed diagonally, Figure 7a The larger rectangular box in the figure is the result of the license plate detection model recognition.

[0137] Figure 7b A schematic diagram of a license plate area after perspective transformation provided by an embodiment of the present invention.

[0138] Figure 7b for Figure 7a The image obtained after the perspective transformation of the license plate area in Figure 7b It can be seen that after the perspective transformation, the license plate area in the image is corrected, and the license plate information can be accurately recognized through the corrected license plate area.

[0139] Figure 8 A detailed schematic diagram of the license plate recognition process provided by an embodiment of the present invention, the specific process is as follows:

[0140] First, the image is input into the license plate detection model to obtain the location information of the license plate area. The aspect ratio of the license plate area in the image is obtained according to the location information of the license plate area. If the aspect ratio of the license plate in the image is greater than the preset threshold, it means that the license plate is not a large-angle license plate. Then the area where the license plate is located is intercepted, and the intercepted license plate information is identified;

[0141] If the aspect ratio of the license plate in the image is less than a preset threshold, it means that the license plate is a large-angle license plate. The key point detection model is used to obtain the target position information of a preset number of corner points, and the sum of the distances between the preset number of coordinate points and the preset number of preset coordinate points are determined in sequence according to the preset number of coordinate points in the target position information; the license plate area is perspectively transformed according to the position information of the coordinate points that correspond in sequence when the sum of the distances is minimized, and the transformed license plate information is identified.

[0142] Embodiment 4:

[0143] Fig. 9 A schematic diagram of the structure of a license plate recognition device provided by an embodiment of the present invention, the device comprises:

[0144] An acquisition module 901 is used to acquire a license plate area of ​​each license plate in each frame of image acquired by an image acquisition device;

[0145] A determination module 902 is used to determine the area shape information of each license plate area.

[0146] Processing module 903 is used to perform perspective transformation on the license plate area if the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, and identify the license plate information contained in the license plate area after the perspective transformation, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

[0147] In a possible implementation, the determination module 902 is specifically configured to determine, for each license plate region, the aspect ratio of the license plate region, and use the aspect ratio as the region shape information of the license plate region.

[0148] In a possible implementation, the acquisition module 901 is specifically configured to acquire a license plate region of each license plate in the frame image through a trained license plate detection model;

[0149] The processing module 903 is also used to obtain any sample image in the sample set and the sample license plate area of ​​each license plate contained in the sample image; input the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image; train the original license plate detection model according to the sample license plate area and the recognized license plate area to obtain an intermediate license plate detection model; perform unstructured pruning on the intermediate license plate detection model to obtain a trained license plate detection model.

[0150] In one possible implementation, the processing module 903 is specifically used to obtain the sub-weight corresponding to each block in each corresponding single-channel convolution kernel when the training of the intermediate license plate detection model is completed; and for each block, determine the mask of the block according to the sub-weight of the block; determine the position of the block to be pruned in the single-channel convolution kernel according to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, and prune the block at this position in each single-channel convolution kernel.

[0151] In a possible implementation, the processing module 903 is also used to determine the width and height of the license plate area based on the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model; determine the sum of distances between the preset number of coordinate points and the preset number of preset coordinate points in sequence according to a preset number of coordinate points in the target position information; and determine the position information of the coordinate points that correspond in sequence when the sum of the distances is minimum.

[0152] In a possible implementation, the processing module 903 is specifically used to determine, in a preset order, coordinate sets corresponding to different orders of coordinate points in the target position information; for each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance to the preset coordinate point corresponding to each preset position, and determine the distance and value.

[0153] Embodiment 5:

[0154] Based on the above embodiments, Fig.10 A schematic diagram of an electronic device provided by an embodiment of the present invention, such as Fig.10 As shown, it includes: a processor 1001 , a communication interface 1002 , a memory 1003 and a communication bus 1004 , wherein the processor 1001 , the communication interface 1002 , and the memory 1003 communicate with each other via the communication bus 1004 .

[0155] The memory 1003 stores a computer program. When the program is executed by the processor 1001, the processor 1001 performs the following steps:

[0156] For each received frame of image, obtain each license plate area in the frame of image;

[0157] For each license plate area, determining area shape information of the license plate area;

[0158] If the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the license plate area is perspective transformed, and the license plate information contained in the license plate area after the perspective transformation is identified, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

[0159] In a possible implementation manner, determining the region shape information of each license plate region includes:

[0160] For each license plate area, the aspect ratio of the license plate area is determined, and the aspect ratio is used as the area shape information of the license plate area.

[0161] In a possible implementation manner, matching the regional shape information of the license plate region with the regional shape information of the abnormal license plate region includes:

[0162] The aspect ratio of the license plate area is smaller than the preset threshold.

[0163] In a possible implementation manner, acquiring the license plate area of ​​each license plate in the frame image includes:

[0164] Obtain the license plate area of ​​each license plate in the frame image through the trained license plate detection model;

[0165] The license plate detection model is trained in the following way:

[0166] Obtain any sample image in the sample set, and a sample license plate area of ​​each license plate contained in the sample image;

[0167] Inputting the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image;

[0168] Training the original license plate detection model according to the sample license plate area and the identified license plate area to obtain an intermediate license plate detection model;

[0169] Unstructured pruning is performed on the intermediate license plate detection model to obtain a trained license plate detection model.

[0170] In a possible implementation, the performing unstructured pruning on the intermediate license plate detection model includes:

[0171] Obtain the sub-weight corresponding to each block in each corresponding single-channel convolution kernel when the intermediate license plate detection model training is completed; and for each block, determine the mask of the block according to the sub-weight of the block;

[0172] According to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, the position of the block to be pruned in the single-channel convolution kernel is determined, and the block at this position in each single-channel convolution kernel is pruned.

[0173] In a possible implementation, before performing perspective transformation on the license plate area, the method further includes:

[0174] According to the license plate area, determine the width and height of the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model;

[0175] According to a preset number of coordinate points in the target position information, sequentially determining the sum of distances between the preset number of coordinate points and a preset number of preset coordinate points;

[0176] Determine the position information of the coordinate points corresponding to the minimum distance and the minimum distance, and perform subsequent perspective transformation operations on the license plate area.

[0177] In a possible implementation manner, determining the sum of distances between the preset number of coordinate points and the preset number of preset coordinate points in sequence according to the preset number of coordinate points in the target position information includes:

[0178] Determine, in a preset order, coordinate sets corresponding to different orders of coordinate points in the target position information;

[0179] For each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance to the preset coordinate point corresponding to each preset position, and determine the distance and value.

[0180] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0181] The communication interface 1002 is used for communication between the above electronic device and other devices.

[0182] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0183] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0184] Embodiment 6:

[0185] On the basis of the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by an electronic device, and when the program is run on the electronic device, the electronic device implements the following steps when executing:

[0186] The memory stores a computer program, and when the program is executed by the processor, the processor performs the following steps:

[0187] For each received frame of image, obtain each license plate area in the frame of image;

[0188] For each license plate area, determining area shape information of the license plate area;

[0189] If the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, the license plate area is perspective transformed, and the license plate information contained in the license plate area after the perspective transformation is identified, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

[0190] In a possible implementation manner, determining the region shape information of each license plate region includes:

[0191] For each license plate area, the aspect ratio of the license plate area is determined, and the aspect ratio is used as the area shape information of the license plate area.

[0192] In a possible implementation manner, matching the regional shape information of the license plate region with the regional shape information of the abnormal license plate region includes:

[0193] The aspect ratio of the license plate area is smaller than the preset threshold.

[0194] In a possible implementation manner, acquiring the license plate area of ​​each license plate in the frame image includes:

[0195] Obtain the license plate area of ​​each license plate in the frame image through the trained license plate detection model;

[0196] The license plate detection model is trained in the following way:

[0197] Obtain any sample image in the sample set, and a sample license plate area of ​​each license plate contained in the sample image;

[0198] Inputting the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image;

[0199] Training the original license plate detection model according to the sample license plate area and the identified license plate area to obtain an intermediate license plate detection model;

[0200] Unstructured pruning is performed on the intermediate license plate detection model to obtain a trained license plate detection model.

[0201] In a possible implementation, the performing unstructured pruning on the intermediate license plate detection model includes:

[0202] Obtain the sub-weight corresponding to each block in each corresponding single-channel convolution kernel when the intermediate license plate detection model training is completed; and for each block, determine the mask of the block according to the sub-weight of the block;

[0203] According to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, the position of the block to be pruned in the single-channel convolution kernel is determined, and the block at this position in each single-channel convolution kernel is pruned.

[0204] In a possible implementation, before performing perspective transformation on the license plate area, the method further includes:

[0205] According to the license plate area, determine the width and height of the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model;

[0206] According to a preset number of coordinate points in the target position information, sequentially determining the sum of distances between the preset number of coordinate points and a preset number of preset coordinate points;

[0207] Determine the position information of the coordinate points corresponding to the minimum distance and the minimum distance, and perform subsequent perspective transformation operations on the license plate area.

[0208] In a possible implementation manner, determining the sum of distances between the preset number of coordinate points and the preset number of preset coordinate points in sequence according to the preset number of coordinate points in the target position information includes:

[0209] Determine, in a preset order, coordinate sets corresponding to different orders of coordinate points in the target position information;

[0210] For each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance to the preset coordinate point corresponding to each preset position, and determine the distance and value.

[0211] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0212] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0213] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0214] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0215] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A license plate recognition method, characterized in that: The method comprises: For each frame of image received, the license plate area of ​​each license plate in the frame of image is obtained through the trained license plate detection model; For each license plate area, determining the aspect ratio of the license plate area, and using the aspect ratio as area shape information of the license plate area; If the regional shape information of the license plate area matches the regional shape information of the abnormal license plate area, determine the width and height of the license plate area according to the license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model; determine the coordinate sets corresponding to the coordinate points in different orders in the target position information in a preset order; for each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance from the preset coordinate point corresponding to each preset position, and determine the distance sum value; determine the position information of the coordinate points corresponding in sequence when the distance sum is minimum, perform perspective transformation on the license plate area, and identify the license plate information contained in the license plate area after the perspective transformation, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

2. The method according to claim 1, characterized in that The matching of the regional shape information of the license plate area and the regional shape information of the abnormal license plate area includes: The aspect ratio of the license plate area is smaller than the preset threshold.

3. The method according to claim 1, characterized in that: The license plate detection model is trained in the following way: Obtain any sample image in the sample set, and a sample license plate area of ​​each license plate contained in the sample image; Inputting the sample image into the original license plate detection model to obtain the recognized license plate area of ​​each license plate contained in the sample image; Training the original license plate detection model according to the sample license plate area and the identified license plate area to obtain an intermediate license plate detection model; Unstructured pruning is performed on the intermediate license plate detection model to obtain a trained license plate detection model.

4. The method according to claim 3, characterized in that The unstructured pruning of the intermediate license plate detection model includes: Obtain the sub-weight corresponding to each block in each corresponding single-channel convolution kernel when the intermediate license plate detection model training is completed; and for each block, determine the mask of the block according to the sub-weight of the block; According to the mask of the block at the same position in each single-channel convolution kernel and the preset pruning rate, the position of the block to be pruned in the single-channel convolution kernel is determined, and the block at this position in each single-channel convolution kernel is pruned.

5. A license plate recognition device, characterized in that: The device comprises: An acquisition module is used to acquire the license plate area of ​​each license plate in each frame of image acquired by the image acquisition device through the trained license plate detection model; A determination module, for determining the aspect ratio of the license plate area of ​​each license plate, and using the aspect ratio as the area shape information of the license plate area; A processing module is used to determine the width and height of the license plate area according to the license plate area if the area shape information of the license plate area matches the area shape information of the abnormal license plate area, enlarge the width and the height, and obtain the target position information of the license plate key points of the license plate area after the width and height are enlarged in the image through the trained key point detection model; determine the coordinate sets corresponding to the coordinate points in different orders in the target position information in a preset order; for each determined coordinate set, determine the coordinate point corresponding to each preset position in the coordinate set, the distance from the preset coordinate point corresponding to each preset position, and determine the distance sum value; determine the position information of the coordinate points corresponding in sequence when the distance sum is minimum, perform perspective transformation on the license plate area, and identify the license plate information contained in the license plate area after the perspective transformation, wherein the abnormal license plate area is a license plate area whose deformation degree of the standard license plate shape meets the requirements.

6. An electronic device, characterized in that: The electronic device comprises at least a processor and a memory, and the processor is used to execute the steps of the license plate recognition method described in any one of claims 1 to 4 when executing the computer program stored in the memory.

7. A computer-readable storage medium, characterized in that: It stores a computer program, and when the computer program is executed by a processor, the steps of the license plate recognition method described in any one of claims 1 to 4 are executed.

Images