License plate detection methods, devices, computer equipment and storage media

Abstract

This application relates to the field of image processing technology, and in particular to a license plate detection method, apparatus, computer device, and storage medium. The method includes: acquiring a test image including a target vehicle; identifying the test image based on a target detection model to obtain the target vehicle's 3D bounding box and its visibility information; wherein the visibility information of the target 3D bounding box is used to mark the plane containing the front or rear of the target vehicle within the target 3D bounding box; extracting the visible region corresponding to the visibility information of the target 3D bounding box from the test image; and detecting the visible region based on the license plate detection model to obtain the license plate location information of the target vehicle. This method can improve the accuracy of license plate recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a license plate detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the rapid development of society and the economy, automobiles have become a common means of transportation for people's daily travel, and the resulting pressure on traffic management is also increasing. Automated license plate recognition technology is of great significance to traffic management and the development of intelligent transportation. License plate detection is the foundation of license plate recognition, and factors such as the vehicle's driving environment, lighting, and driving status have a significant impact on license plate detection.

[0003] Traditional methods for license plate region detection and localization mainly include edge detection-based methods, color segmentation-based methods, wavelet transform-based methods, genetic algorithm-based methods, license plate localization based on mathematical morphology, and methods based on grayscale image texture feature analysis.

[0004] However, in different application scenarios, the direction of vehicle travel varies, which often causes the license plate to be tilted at a large angle in the image. In scenarios where the license plate is tilted at a large angle, the proportion of the vehicle body in the image is much smaller than in regular scenarios, which leads to an increase in the false negative rate and a decrease in the accuracy of the above-mentioned license plate detection algorithm. Summary of the Invention

[0005] Therefore, it is necessary to provide a license plate detection method, apparatus, computer equipment, and storage medium that can improve the accuracy of license plate detection, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a license plate detection method, the method comprising:

[0007] Acquire the image to be tested, including the target vehicle;

[0008] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0009] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0010] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0011] In one embodiment, a target 3D bounding box is marked in the image to be tested. From the image to be tested, the visible region corresponding to the visibility information of the target 3D bounding box is extracted, including:

[0012] The image to be tested marked with the target 3D bounding box is compared with the initial image to be tested to determine the projection direction;

[0013] Based on the projection direction, the plane marked by the visibility information of the target's 3D bounding box is projected into the target's 3D bounding box to obtain the projection range;

[0014] Based on the projection range, the visible region is extracted from the image to be tested.

[0015] In one embodiment, based on a license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle, including:

[0016] Based on the target standard template, perform perspective transformation on the visible area;

[0017] Based on the license plate detection model, the visible area after perspective transformation is detected to obtain the license plate location information of the target vehicle.

[0018] In one embodiment, perspective transformation is performed on the visible area based on a target standard template, including:

[0019] Based on the plane marked by the visibility information of the target 3D bounding box, select the target standard template from the front and rear standard templates;

[0020] Based on the target standard template, perform perspective transformation on the visible area.

[0021] In one embodiment, the license plate detection method further includes:

[0022] Acquire sample data; wherein, the sample data includes sample images containing sample vehicles, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes.

[0023] The initial model is trained based on the sample images, the point cloud information of the sample vehicles, the sample 3D bounding boxes corresponding to the sample vehicles, and the visibility information of the sample 3D bounding boxes to obtain the target detection model.

[0024] In one embodiment, the license plate detection method further includes: acquiring a training image containing a reference license plate, and the position information of the reference license plate in the training image;

[0025] The initial license plate detection model is trained based on the training images and the location information of reference license plates in the training images to obtain the license plate detection model.

[0026] Secondly, this application also provides a license plate recognition device, which includes:

[0027] The acquisition module is used to acquire the image to be tested, including the target vehicle;

[0028] The generation module is used to identify the target image based on the target detection model to obtain the target 3D bounding box of the target vehicle and the visibility information of the target 3D bounding box; wherein, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box;

[0029] The extraction module is used to extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0030] The detection module is used to detect the visible area based on the license plate detection model to obtain the license plate location information of the target vehicle.

[0031] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0032] Acquire the image to be tested, including the target vehicle;

[0033] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0034] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0035] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0037] Acquire the image to be tested, including the target vehicle;

[0038] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0039] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0040] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0041] Fifthly, this application also provides a computer program product comprising a computer program that, when executed by a processor, performs the following steps:

[0042] Acquire the image to be tested, including the target vehicle;

[0043] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0044] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0045] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0046] The aforementioned license plate detection method, apparatus, computer equipment, and storage medium utilize a target detection model to generate a target 3D bounding box for the target vehicle in the image to be tested. This bounding box accurately extracts the features of the entire target vehicle, avoiding missed detections of the license plate and improving detection accuracy under various license plate shapes. Then, if the front or rear of the target vehicle is visible in the image to be tested, the plane containing the front or rear of the target vehicle is marked using the visibility information of the target 3D bounding box. The visible region is then extracted from the image to be tested using this visibility information. This visible region only includes the front or rear region of the target vehicle containing the license plate, avoiding interference from other features of the target vehicle for license plate recognition. Finally, the visible region is input into the license plate detection model for recognition. Through the corresponding license plate detection algorithm, accurate extraction of the license plate location information can be achieved, improving the accuracy of license plate detection. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating a license plate detection method in one embodiment;

[0048] Figure 2 This is a schematic diagram of the training process for an object detection model in one embodiment;

[0049] Figure 3 This is a flowchart illustrating the process of extracting the visible region corresponding to the 3D bounding box of a target in one embodiment.

[0050] Figure 4 This is a schematic diagram of the process for detecting visible areas in one embodiment;

[0051] Figure 5 This is a flowchart illustrating the license plate detection method in another embodiment;

[0052] Figure 6 This is a structural block diagram of a license plate recognition device in one embodiment;

[0053] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The license plate detection method provided in this application is applicable to intelligent transportation scenarios, specifically license plate recognition scenarios. Optionally, this method can be executed by a computer device, which can be a server or a terminal. Specifically, in one embodiment, as... Figure 1 As shown, the method specifically includes the following steps:

[0056] S101, acquire the image to be tested, including the target vehicle.

[0057] The target vehicle in the image to be tested can be a moving vehicle or a stationary vehicle. There can be one or more target vehicles. The image to be detected is a camera-based image, specifically an image acquired by a monocular camera.

[0058] S102, based on the target detection model, identifies the image to be tested and obtains the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box.

[0059] The target detection model is a monocular 3D vehicle detection model used to detect the three-dimensional structure of vehicles. Optionally, this monocular 3D vehicle detection model is designed using the Monodle algorithm based on the Centernet network. It is understood that the main objective of 3D target detection is to estimate the target's position, size, azimuth, and pose from the input information. In this embodiment, the input information is the image to be tested, and the output information is the target's three-dimensional bounding box and the visibility information of the target's three-dimensional bounding box.

[0060] Optionally, before inputting the image to be detected into the target detection model, in order to improve the accuracy of image detection, the image to be detected can be preprocessed. Preprocessing can specifically be image enhancement processing, which can mainly include random multi-scale scaling, flipping, gamma transformation, color conversion, etc.

[0061] Among them, the target three-dimensional bounding box refers to a three-dimensional bounding box. The bounding box is the smallest cuboid that completely surrounds the target vehicle and whose sides are parallel to any coordinate axis. Specifically, any coordinate axis is the coordinate axis in the coordinate system in which the target vehicle is located, such as the x-axis, y-axis and z-axis in the spatial coordinate system.

[0062] Understandably, when performing license plate recognition, the area of ​​interest is typically the front or rear of the vehicle (i.e., the area containing the license plate). However, since the image to be tested in this embodiment is a single image taken from a single angle, the target vehicle in the image cannot simultaneously contain both the front and rear areas. Therefore, the front or rear area of ​​the vehicle in the image to be tested can be extracted using the target's 3D bounding box. That is, the visibility information of the target's 3D bounding box can be used to determine whether the extracted image is the front or rear area.

[0063] The visibility information of the target 3D bounding box is used to mark the plane containing the front or rear of the target vehicle within the bounding box. Specifically, if the image to be tested contains the front region of the target vehicle, a "front plane" is marked on the plane corresponding to the front position in the generated target 3D bounding box; similarly, if the image to be tested contains the front region of the target vehicle, a "rear plane" is marked on the plane corresponding to the rear position in the generated target 3D bounding box. In this case, the front region can be extracted from the image to be tested using the "front plane," or the rear region can be extracted using the "rear plane."

[0064] Optionally, to extract the visibility information of the target's 3D bounding box, a prediction structure can be added to the target model. This prediction structure can be designed based on the heatmap concept in the CenterNet network. The CenterNet network detects the target as a point, that is, it uses the center point of the target box to represent the target. It predicts the center point offset and size of the target to obtain the actual object box, while the heatmap represents classification information. The output of this prediction structure is the classification information of the front face or the rear face of the target's 3D bounding box. This classification information is the aforementioned visibility information.

[0065] Furthermore, in this embodiment, the target's 3D bounding box can also be labeled with the target's spatial position information and attitude information. The target's spatial position information refers to the spatial position information of the 12 vertices within the bounding box, and the target's attitude information refers to the heading angle, roll angle, and pitch angle of the target's 3D bounding box. Specifically, to generate the spatial position information of the target's 3D bounding box, training samples labeled with spatial position information can be used as input to the target detection model during training. To extract the attitude information of the target's 3D bounding box, pitch and roll angle predictions can be added to the Monodle algorithm's monocular 3D vehicle detection algorithm to adapt to various camera angle scenarios.

[0066] S103, extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested.

[0067] The visible region is the area in the image to be tested corresponding to the aforementioned "front plane" or "rear plane," i.e., the front or rear region of the target vehicle. Specifically, when extracting the visible region, a mapping relationship between the "front plane" or "rear plane" and the image to be tested can be established, and the visible region is obtained by projection based on this mapping relationship.

[0068] S104, based on the license plate detection model, detects the visible area to obtain the license plate location information of the target vehicle.

[0069] The license plate detection model can be a model based on the YOLOv6 object detection algorithm; optionally, the license plate location information is the coordinate information of the license plate image in the visible area; the license plate image can be extracted based on the license plate location information.

[0070] In the aforementioned license plate detection method, a target detection model is used to generate a target 3D bounding box for the target vehicle in the image to be tested. This bounding box accurately extracts the features of the entire target vehicle, avoiding missed detections of the license plate and improving the detection accuracy under various license plate shapes. Then, if the front or rear of the target vehicle is visible in the image to be tested, the plane containing the front or rear of the target vehicle is marked using the visibility information of the target 3D bounding box. The visible region is then extracted from the image to be tested using this visibility information. This visible region only includes the front or rear region of the target vehicle containing the license plate, avoiding interference from other features of the target vehicle for license plate recognition. Finally, the visible region is input into the license plate detection model for recognition. Through the corresponding license plate detection algorithm, accurate extraction of the license plate location information can be achieved, improving the accuracy of license plate detection.

[0071] like Figure 2 As shown in the figure, this embodiment provides a training method for an object detection model, and the specific implementation process may include:

[0072] S201, Obtain sample data.

[0073] The sample data includes sample images containing sample vehicles, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes.

[0074] Specifically, since license plate detection and recognition are typically used in scenarios such as road checkpoints, electronic police systems, and parking lots, it is essential to collect sufficient image data from different application scenarios. Therefore, when generating sample data:

[0075] First, sample images containing sample vehicles were collected from checkpoints, electronic police systems, parking lot entrances and exits, and on-street parking areas, along with the corresponding point cloud information. A total of 100,000 data sets were collected, and the internal and external parameters of the cameras were recorded. The point cloud information was acquired using a 3D LiDAR device. LiDAR point cloud data is a dataset of spatial points scanned by a 3D LiDAR device. Each point contains 3D coordinate information, namely X, Y, and Z elements, and some also include color information, reflection intensity information, and echo count information. Vehicle information collected by 3D LiDAR in application scenarios such as checkpoints provides relatively accurate 3D spatial information of the sample vehicles.

[0076] Secondly, for any of the above sample images, using the LiDAR point cloud information from the above steps, the 3D position of the sample vehicle (i.e., the point cloud information of the sample vehicle) and the sample 3D bounding box of the sample vehicle are marked, and whether the front and rear of the sample 3D bounding box are visible (i.e., the visibility information of the sample 3D bounding box) are marked. Furthermore, if the target detection model needs to identify the vehicle's pose information, the pose information of the sample 3D bounding box can also be marked. After the above marking is completed, the annotation information corresponding to the sample image is generated.

[0077] Then, each sample image is labeled to form a 3D vehicle detection dataset. 30% (30,000 sets) of the data are randomly selected as the validation set, and the remainder is used as the training set. This training set is the sample data.

[0078] S202, the initial model is trained based on the sample image, the point cloud information of the sample vehicle, the sample 3D bounding box corresponding to the sample vehicle, and the visibility information of the sample 3D bounding box to obtain the target detection model.

[0079] Specifically, training stops when the initial model's recognition accuracy reaches a preset value, i.e., the loss function is less than the corresponding threshold.

[0080] In this embodiment, marking the sample image with point cloud information makes the generated target 3D bounding box more accurate.

[0081] like Figure 3 As shown, in one embodiment, the target 3D bounding box is marked in the image to be tested. This embodiment provides an optional method for extracting the visible region corresponding to the visibility information of the target 3D bounding box from the image to be tested, that is, a method for refining S103. The specific implementation process may include:

[0082] S301, compare the image to be tested marked with the target 3D bounding box with the initial image to be tested to determine the projection direction.

[0083] The target detection model generates a 3D bounding box of the target vehicle in space based on the target vehicle in the image to be tested, containing the target vehicle's parameters / attributes. Therefore, by comparing the image to be tested marked with the target bounding box with the initial image, the projection direction can be determined. For example, if the plane of the image to be tested is (0, y, z), and the plane of the target bounding box is (x, y, z), then when projecting the front or rear plane of the 3D target bounding box, the projection direction is the x-axis direction.

[0084] S302, Based on the projection direction, project the plane marked by the visibility information of the target's 3D bounding box into the target's 3D bounding box to obtain the projection range.

[0085] In this process, at least four vertices of the front or rear plane of the vehicle are projected along the projection direction. For example, if the coordinates of one vertex are a(x0, y0, z0), the coordinates obtained after projection are a'(0, y0, z0). The projection range is obtained based on the line connecting the at least four projected coordinates.

[0086] S303, extract the visible region from the image to be tested based on the projection range.

[0087] The visible area is obtained by extracting each pixel within the projection range from the image to be tested.

[0088] In this embodiment, the visible area is determined from the image to be tested by projection, which can avoid interference from other areas of the target vehicle on license plate detection.

[0089] Understandably, due to camera manufacturing... Distortions introduced by assembly process deviations, or issues such as angle, rotation, and scaling during photo capture, may cause distortion in the image under test, resulting in distortion of the extracted visible area. The quality of the raw data will have a certain impact on the recognition accuracy of the license plate detection model. To improve the recognition accuracy of the license plate detection model, the image can be corrected before using the license plate detection model to detect the visible area, such as... Figure 4 As shown, this embodiment provides an optional method for detecting the visible area based on a license plate detection model to obtain the license plate location information of a target vehicle, that is, a method for refining S104. The specific implementation process may include:

[0090] S401 performs perspective transformation on the visible area based on the target standard template.

[0091] Perspective transformation transforms a two-dimensional plane into a new two-dimensional viewing plane. Specifically, it involves using the collinearity of the projection center, the imaging point of the original image plane, and the target point of the new image plane, and rotating the image plane around the trace line by a certain angle according to the law of perspective rotation. This disrupts the original projection ray beam while maintaining the original geometric shape of the projection on the image plane. In this embodiment, perspective transformation is used to correct distortion in the visible area to repair the aforementioned distortion.

[0092] Optionally, in this embodiment, different standard templates are used when performing perspective transformation on the front and rear areas of the vehicle. That is, the target standard templates in this embodiment include a front standard template and a rear standard template. Specifically, based on the plane marked by the visibility information of the target 3D bounding box, a target standard template is selected from the front and rear standard templates; based on the target standard template, perspective transformation is performed on the visible area.

[0093] The standard template for the front of the vehicle can be a standard image template obtained by feature learning from the front region of the vehicle in multiple historical images; the standard template for the rear of the vehicle can be a standard image template obtained by feature learning from the rear region of the vehicle in multiple historical images. If the plane marked by the visibility information is the "front plane", then the target standard template is the front standard template; if the plane marked by the visibility information is the "rear plane", then the target standard template is the rear standard template.

[0094] For example, taking a standard vehicle front template as an example, after obtaining the visible area, feature detection is performed on both the visible area and the standard vehicle front template to identify and record the feature information and coordinate information of each feature point. Feature matching is performed between the feature points in the visible area and the feature points in the standard vehicle front template, and the successfully matched feature points and their coordinate information on the visible area and the standard vehicle front template are recorded. Two successfully matched feature points and their corresponding coordinate information are marked as a feature point group. At least four feature point groups are selected, and the perspective transformation matrix is ​​calculated using the perspective transformation formula. The perspective transformation matrix is ​​used to calculate the transformed pixel values ​​of each pixel in the visible area, thereby obtaining the visible area after perspective transformation.

[0095] S402, based on the license plate detection model, detects the visible area after perspective transformation to obtain the license plate location information of the target vehicle.

[0096] In this embodiment, the visible region after perspective transformation is input into the license plate detection model to obtain the license plate image output by the model, as well as the coordinate information of the license plate image within the visible region. In this embodiment, the input to the license plate detection model is the visible region after perspective transformation. Therefore, the training samples used in training the license plate detection model can also be images after perspective transformation. The model is trained based on these training samples until it converges.

[0097] Specifically, the license plate detection method further includes: acquiring a training image containing a reference license plate, and the position information of the reference license plate in the training image; training an initial license plate detection model based on the training image and the position information of the reference license plate in the training image to obtain the license plate detection model.

[0098] Furthermore, the license plate detection method also includes: determining the position information of the visible area based on the position information of the target's 3D bounding box; and determining the position information of the license plate image based on the position information of the visible area and the license plate position information.

[0099] For example, based on the above embodiments, this embodiment provides an optional example of a license plate detection method. For instance... Figure 5 As shown, the specific implementation process includes:

[0100] S501, acquire the image to be tested, including the target vehicle.

[0101] S502, based on the target detection model, identifies the image to be tested and obtains the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box.

[0102] The visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located within the target 3D bounding box.

[0103] S503 compares the image to be tested marked with the target 3D bounding box with the initial image to be tested to determine the projection direction.

[0104] S504, based on the projection direction, project the plane marked by the visibility information of the target's 3D bounding box into the target's 3D bounding box to obtain the projection range.

[0105] S505, extracts the visible region from the image to be tested based on the projection range.

[0106] S506 performs perspective transformation on the visible area based on the target standard template.

[0107] Specifically, based on the plane marked by the visibility information of the target 3D bounding box, the target standard template is selected from the front standard template and the rear standard template.

[0108] S507, based on a license plate detection model, detects the visible area after perspective transformation to obtain the license plate location information of the target vehicle.

[0109] The specific processes of S501-S507 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0110] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0111] Based on the same inventive concept, this application also provides a license plate recognition device for implementing the license plate detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more license plate recognition device embodiments provided below can be found in the limitations of the license plate detection method described above, and will not be repeated here.

[0112] In one embodiment, such as Figure 6As shown, a license plate recognition device 1 is provided, including: an acquisition module 11, a generation module 12, an extraction module 13, and a detection module 14, wherein:

[0113] The acquisition module 11 is used to acquire a test image including the target vehicle;

[0114] The generation module 12 is used to identify the target image based on the target detection model to obtain the target 3D bounding box of the target vehicle and the visibility information of the target 3D bounding box; wherein, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0115] Extraction module 13 is used to extract the visible region corresponding to the visibility information of the target's three-dimensional bounding box from the image to be tested;

[0116] The detection module 14 is used to detect the visible area based on the license plate detection model to obtain the license plate location information of the target vehicle.

[0117] In one embodiment, a target 3D bounding box is marked in the image to be measured, and the extraction module 13 includes:

[0118] The parsing submodule is used to compare the image to be tested marked with the target 3D bounding box with the initial image to be tested to determine the projection direction;

[0119] The projection submodule is used to project the plane marked by the visibility information of the target 3D bounding box into the target 3D bounding box based on the projection direction, so as to obtain the projection range;

[0120] The delineation submodule is used to extract the visible region from the image under test based on the projection range.

[0121] In one embodiment, the detection module 14 includes:

[0122] The transformation submodule is used to perform perspective transformation on the visible area based on the target standard template;

[0123] The recognition submodule is used to detect the visible area after perspective transformation based on the license plate detection model to obtain the license plate location information of the target vehicle.

[0124] In one embodiment, the transformation submodule is further configured to: select a target standard template from the front standard template and the rear standard template based on the plane marked by the visibility information of the target 3D bounding box;

[0125] Based on the target standard template, perform perspective transformation on the visible area.

[0126] In one embodiment, the license plate recognition device further includes a first training module, which is used for:

[0127] Acquire sample data; wherein, the sample data includes sample images containing sample vehicles, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes.

[0128] The initial model is trained based on the sample images, the point cloud information of the sample vehicles, the sample 3D bounding boxes corresponding to the sample vehicles, and the visibility information of the sample 3D bounding boxes to obtain the target detection model.

[0129] In one embodiment, the license plate recognition device further includes a second training module, which is used to: acquire a training image containing a reference license plate, and the position information of the reference license plate in the training image;

[0130] The initial license plate detection model is trained based on the training images and the location information of reference license plates in the training images to obtain the license plate detection model.

[0131] The various modules in the aforementioned license plate recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0132] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a license plate detection method.

[0133] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0135] Acquire the image to be tested, including the target vehicle;

[0136] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0137] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0138] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0139] In one embodiment, a target 3D bounding box is marked in the image to be tested. When the processor executes the logic of the computer program to extract the visible region corresponding to the visibility information of the target 3D bounding box from the image to be tested, the specific steps are as follows: the image to be tested marked with the target 3D bounding box is compared with the initial image to be tested to determine the projection direction; based on the projection direction, the plane marked by the visibility information of the target 3D bounding box is projected to obtain the projection range; and the visible region is extracted from the image to be tested according to the projection range.

[0140] In one embodiment, when the processor executes the logic of a computer program to detect the visible area based on a license plate detection model and obtain the license plate location information of the target vehicle, the specific steps are as follows: performing perspective transformation on the visible area based on a target standard template; and detecting the visible area after perspective transformation based on the license plate detection model to obtain the license plate location information of the target vehicle.

[0141] In one embodiment, when the processor executes the logic of the computer program to perform perspective transformation on the visible area based on the target standard template, the following steps are specifically implemented: selecting the target standard template from the front and rear standard templates according to the plane marked by the visibility information of the target 3D bounding box; and performing perspective transformation on the visible area based on the target standard template.

[0142] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring sample data; wherein the sample data includes sample images containing sample vehicles, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes; and training an initial model based on the sample images, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes to obtain an object detection model.

[0143] In one embodiment, a training image containing a reference license plate and the location information of the reference license plate in the training image are obtained; based on the training image and the location information of the reference license plate in the training image, an initial license plate detection model is trained to obtain a license plate detection model.

[0144] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0145] Acquire the image to be tested, including the target vehicle;

[0146] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0147] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0148] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0149] In one embodiment, the target 3D bounding box is marked in the image to be tested. When the logic of the computer program extracting the visible region corresponding to the visibility information of the target 3D bounding box from the image to be tested is executed by the processor, the following steps are specifically implemented:

[0150] The image to be tested marked with the target 3D bounding box is compared with the initial image to be tested to determine the projection direction;

[0151] Based on the projection direction, the plane marked by the visibility information of the target's 3D bounding box is projected into the target's 3D bounding box to obtain the projection range;

[0152] Based on the projection range, the visible region is extracted from the image to be tested.

[0153] In one embodiment, when the logic of a computer program detecting the visible area based on a license plate detection model to obtain the license plate location information of a target vehicle is executed by the processor, the specific steps are as follows: performing perspective transformation on the visible area based on a target standard template; and detecting the visible area after perspective transformation based on the license plate detection model to obtain the license plate location information of the target vehicle.

[0154] In one embodiment, when the logic of the computer program performing perspective transformation on the visible area based on the target standard template is executed by the processor, the following steps are specifically implemented: selecting the target standard template from the front and rear standard templates according to the plane marked by the visibility information of the target 3D bounding box; and performing perspective transformation on the visible area based on the target standard template.

[0155] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring sample data; wherein the sample data includes sample images containing sample vehicles, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes; and training an initial model based on the sample images, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes to obtain an object detection model.

[0156] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a training image containing a reference license plate and the position information of the reference license plate in the training image; training an initial license plate detection model based on the training image and the position information of the reference license plate in the training image to obtain a license plate detection model.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0158] Acquire the image to be tested, including the target vehicle;

[0159] Based on the target detection model, the image to be tested is identified to obtain the target vehicle's 3D bounding box and the visibility information of the target 3D bounding box; among which, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box.

[0160] Extract the visible region corresponding to the visibility information of the target's 3D bounding box from the image to be tested;

[0161] Based on the license plate detection model, the visible area is detected to obtain the license plate location information of the target vehicle.

[0162] In one embodiment, the target 3D bounding box is marked in the image to be tested. When the logic of the computer program extracting the visible region corresponding to the visibility information of the target 3D bounding box from the image to be tested is executed by the processor, the following steps are specifically implemented:

[0163] The image to be tested marked with the target 3D bounding box is compared with the initial image to be tested to determine the projection direction;

[0164] Based on the projection direction, the plane marked by the visibility information of the target's 3D bounding box is projected into the target's 3D bounding box to obtain the projection range;

[0165] Based on the projection range, the visible region is extracted from the image to be tested.

[0166] In one embodiment, when the logic of a computer program detecting the visible area based on a license plate detection model to obtain the license plate location information of a target vehicle is executed by the processor, the specific steps are as follows: performing perspective transformation on the visible area based on a target standard template; and detecting the visible area after perspective transformation based on the license plate detection model to obtain the license plate location information of the target vehicle.

[0167] In one embodiment, when the logic of the computer program performing perspective transformation on the visible area based on the target standard template is executed by the processor, the following steps are specifically implemented: selecting the target standard template from the front and rear standard templates according to the plane marked by the visibility information of the target 3D bounding box; and performing perspective transformation on the visible area based on the target standard template.

[0168] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring sample data; wherein the sample data includes sample images containing sample vehicles, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes; and training an initial model based on the sample images, point cloud information of sample vehicles, sample 3D bounding boxes corresponding to sample vehicles, and visibility information of sample 3D bounding boxes to obtain an object detection model.

[0169] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a training image containing a reference license plate and the position information of the reference license plate in the training image; training an initial license plate detection model based on the training image and the position information of the reference license plate in the training image to obtain a license plate detection model.

[0170] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0171] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0172] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting license plates, characterized in that, The method includes: Acquire the image to be tested, including the target vehicle; Based on the target detection model, the image to be tested is identified to obtain the target 3D bounding box of the target vehicle and the visibility information of the target 3D bounding box; wherein, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box; The image to be tested marked with the target 3D bounding box is compared with the initial image to be tested to determine the projection direction; Based on the projection direction, the plane marked by the visibility information of the target 3D bounding box is projected into the target 3D bounding box to obtain the projection range; Based on the projection range, the visible region is extracted from the image to be tested; Based on the target standard template, a perspective transformation is performed on the visible area; Based on the license plate detection model, the visible area after perspective transformation is detected to obtain the license plate location information of the target vehicle.

2. The method according to claim 1, characterized in that, The perspective transformation of the visible area based on the target standard template includes: Based on the plane marked by the visibility information of the target 3D bounding box, select the target standard template from the front standard template and the rear standard template; Based on the target standard template, a perspective transformation is performed on the visible area.

3. The method according to claim 2, characterized in that, When the target standard template is a vehicle front standard template, the step of performing perspective transformation on the visible area based on the target standard template includes: Feature detection is performed on the visible area and the standard template of the vehicle front, and the coordinate information of each detected feature point is recorded. The feature points in the visible area are matched with the feature points in the standard template of the vehicle front, and the successfully matched feature points and their coordinate information on the visible area and the standard template of the vehicle front are recorded. Each successfully matched feature point and its corresponding coordinate information are marked as a feature point group; Select at least four feature point groups, and determine the perspective transformation matrix based on the selected feature point groups using the perspective transformation formula; The perspective transformation matrix is ​​used to determine the transformed pixel values ​​of each pixel in the visible region, so as to obtain the visible region after perspective transformation.

4. The method according to claim 1, characterized in that, The method further includes: Based on the position information of the target's three-dimensional bounding box, determine the position information of the visible area; The location information of the license plate image is determined based on the location information of the visible area and the location information of the license plate.

5. The method according to claim 1, characterized in that, The method further includes: Acquire sample data; wherein the sample data includes sample images containing sample vehicles, point cloud information of the sample vehicles, sample 3D bounding boxes corresponding to the sample vehicles, and visibility information of the sample 3D bounding boxes; The initial model is trained based on the sample image, the point cloud information of the sample vehicle, the sample 3D bounding box corresponding to the sample vehicle, and the visibility information of the sample 3D bounding box to obtain the target detection model.

6. The method according to claim 1, characterized in that, The method further includes: Obtain a training image containing a reference license plate, and the location information of the reference license plate in the training image; The initial license plate detection model is trained based on the training image and the position information of the reference license plate in the training image to obtain the license plate detection model.

7. A license plate recognition device, characterized in that, The device includes: The acquisition module is used to acquire the image to be tested, including the target vehicle; The generation module is used to identify the image to be tested based on the target detection model to obtain the target 3D bounding box of the target vehicle and the visibility information of the target 3D bounding box; wherein, the visibility information of the target 3D bounding box is used to mark the plane where the front or rear of the target vehicle is located in the target 3D bounding box; The extraction module is used to extract the visible region corresponding to the visibility information of the target's three-dimensional bounding box from the image to be tested; The detection module is used to detect the visible area based on the license plate detection model to obtain the license plate location information of the target vehicle. The extraction module includes: The parsing submodule is used to compare the image to be tested marked with the target 3D bounding box with the initial image to be tested to determine the projection direction; The projection submodule is used to project the plane marked by the visibility information of the target 3D bounding box into the target 3D bounding box based on the projection direction, so as to obtain the projection range; The delineation submodule is used to extract the visible region from the image under test based on the projection range; The detection module includes: The transformation submodule is used to perform perspective transformation on the visible area based on the target standard template; The recognition submodule is used to detect the visible area after perspective transformation based on the license plate detection model to obtain the license plate location information of the target vehicle.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images