A method for solving the problem that license plates cannot be detected due to night vision exposure

Abstract

The application provides a method for solving the problem that license plates cannot be detected due to night vision exposure, comprising the following steps: S1, training a license plate detection model: S1.1, labeling license plate detection data; S1.2, training the license plate detection model; S2, training a car light detection model: S2.1, labeling car light detection data; S2.2, training the car light detection model; S3, adding movement detection: S3.1, the movement detection uses two frames of gray-scale images to make frame difference; S3.2, obtaining a difference binary image; S3.3, obtaining a rectangular frame of a moving target on the binary image, i.e. all moving objects; S4, outputting a moving car light target: S4.1, performing IOU operation on the results of car light detection and movement detection; S4.2, when IOU value / car light area>threshold 2 or IOU value / movement area>threshold 2, determining that the car light target is a moving car light target, and then outputting the car light target; and S5, the device performs light collection operation. The method effectively solves the problem that license plates cannot be detected due to night vision exposure.

Description

Technical Field

[0001] This invention belongs to the field of intelligent video processing technology, and specifically relates to a method for solving the problem of license plate detection failure caused by night vision exposure. Background Technology

[0002] With the development of computer technology and the widespread application of computer vision principles, the method of using computer image processing technology to identify targets in real time has become increasingly popular. As people's living standards in my country gradually improve, motor vehicles have become one of the most important modes of transportation. Therefore, license plate recognition systems (LPRS) are widely used in various automated management systems such as intelligent transportation systems and intelligent monitoring systems. Furthermore, license plate recognition also has broad application value in intelligent park management and highway toll collection systems. The main task of license plate recognition is to extract and identify moving vehicle license plates from complex backgrounds, which is one of the core problems in the field of computer vision. Due to the different colors and types of license plates, coupled with interference from factors such as imaging and occlusion during movement, license plate recognition has always been one of the most challenging problems in the field of machine vision.

[0003] License plate recognition technology is becoming increasingly widely used, leading to a surge in related demands. License plate recognition can be applied to systems at the entrances of residential communities, such as automatic parking time-based billing. This involves automatically recognizing vehicle license plates when a vehicle enters the community and automatically charging based on the parking duration when the vehicle leaves.

[0004] However, in existing technologies, under night vision exposure, the existing license plate attribute recognition suffers from a decrease in recognition rate due to excessive light interference with license plate data, which can lead to the inability to detect license plates normally. Summary of the Invention

[0005] To address the aforementioned issues, the purpose of this application is to: perform a light-removal operation on the exposed vehicle so that the vehicle's headlights no longer obscure the license plate due to exposure, thereby enabling the license plate to be extracted for license plate detection.

[0006] Specifically, the present invention provides a method for solving the problem of license plate detection failure due to night vision exposure, the method comprising the following steps:

[0007] S1. Train the license plate detection model;

[0008] S2. Train the vehicle headlight detection model;

[0009] S3. Add motion detection:

[0010] S3.1, the motion detection algorithm is used to detect moving targets in the video stream: The motion detection here uses the method of calculating the frame difference between two grayscale images. A vector of size 4 is used to store 4 grayscale images. The image resolution can be modified according to the actual situation. The image width and height are 640x360. The number of frames between the frames can be set as needed. Since the difference between two adjacent frames is small, it is assumed that there is a two-frame interval. Then the difference between the fourth frame and the first frame is used. Let the fourth frame be A and the first frame be B. The frame difference between the fourth frame and the first frame is AB.

[0011] S3.2, Binarize the difference map from step S3.1. The threshold 1 can be adjusted according to the requirements. Assuming the threshold 1 is set to 20, values ​​greater than 20 are set to 1, and values ​​less than 20 are set to 0, thereby obtaining the difference binary map.

[0012] S3.3, Obtain the bounding box of the moving target from the binary image in step S3.2, i.e., the area with a pixel value of 1. First, find the edge contour with a pixel value of 1, and then obtain the smallest bounding box of the edge contour. This gives the bounding box of the area where the image changes. The smallest bounding box here is the bounding box of the area where the image changes, i.e., all moving objects. The width and height can be set and filtered. Assuming it is set to 20, boxes smaller than 20 can be filtered out. The movement time is 10ms.

[0013] S4. Output the target of the moving vehicle lights:

[0014] The purpose of filtering stationary vehicle lights is to filter out stationary lights that cause interference, and this further includes:

[0015] S4.1 Perform an IOU operation on the results of the vehicle light detection in step S2 and the motion detection in step S3. The formula for calculating IOU is as follows:

[0016] IOU=Area Of Overlap / Area Of Union

[0017] Where, Area Of Overlap represents the intersection area of ​​the headlight frame and the moving frame; Area Of Union represents the area of ​​the union of the headlight frame and the moving frame; the value of IOU is between 0 and 1, where: 0 means that the two bounding boxes do not overlap; 1 means that the two bounding boxes completely overlap.

[0018] S4.2, when the IOU value / headlight area > threshold 2 or the IOU value / moving area > threshold 2, the headlight target is determined to be a moving headlight target. Here, the headlight area is the width multiplied by the height of the headlight frame, and the moving area is the width multiplied by the height of the moving frame. Then, this headlight target is output. The threshold 2 can be adjusted according to the actual situation. Here, it is set to 0.5.

[0019] S5. The equipment performs light collection operation:

[0020] When there is a moving vehicle light detection output in step S4, the device, i.e., the camera, performs a light receiving operation to reveal the license plate that was blocked by the light, thus enabling the license plate to be detected.

[0021] Step S1 further includes:

[0022] S1.1, Mark license plate detection data

[0023] The annotation method uses rectangular boxes to annotate license plates, and labelImg is used for annotation. labelImg is a tool for annotating data, and the data is the coco2017 dataset.

[0024] S1.2, Training the license plate detection model

[0025] The detection framework used here is YOLOv7, and the network selected is YOLOv7-tiny. YOLOv7 is a multi-class deep learning open-source framework for object detection. This method is not limited to the YOLOv7 framework; all object detection frameworks can be used, as long as it is object detection. License plate detection is a type of object detection.

[0026] Step S2 further includes:

[0027] S2.1, Marking vehicle headlight test data

[0028] The annotation method uses rectangular boxes to annotate the vehicle lights, and labelImg is used for annotation data. LabelImg is a tool for annotating data. It annotates two lights of a vehicle light as one light, which can extract the characteristics of two lights of the vehicle light and also prevent false detection of roadside lights to a certain extent. When two lights of a vehicle are combined into one light, the entire light can be annotated, and labelImg is used for annotation data.

[0029] S2.2, Training the vehicle headlight detection model

[0030] The detection framework used here is YOLOv7, and the network selected is YOLOv7-tiny. YOLOv7 is a multi-class deep learning open-source framework for object detection. This method is not limited to the YOLOv7 framework; all object detection frameworks can be used, as long as it is object detection. License plate detection is a type of object detection.

[0031] The lamps that cause interference in step S4 include streetlights and street-side building lighting.

[0032] Therefore, the advantage of this application is that it effectively solves the problem of license plates being undetectable due to night vision exposure. Attached Figure Description

[0033] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0034] Figure 1 This is a schematic diagram of the vehicle shown in the embodiments of this application.

[0035] Figure 2 This is a schematic diagram of the vehicle after light collection in an embodiment of this application.

[0036] Figure 3 This is a schematic diagram in this application embodiment where the license plate is marked with a rectangular frame in red.

[0037] Figure 4 This is a schematic diagram in an embodiment of this application, in which two lights of a vehicle headlight are marked as one vehicle headlight and indicated by a green box.

[0038] Figure 5 This is a schematic diagram of the method flow of this application. Detailed Implementation

[0039] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0040] This application proposes a method to solve the problem of license plate detection failure due to night vision exposure. Specifically, it involves performing a light-receiving operation on the exposed vehicle. If the exposed vehicle is like... Figure 1 As shown, under the current exposure, the license plate is not visible; only the headlights are visible. After the light is removed, the vehicle appears... Figure 2 As shown, the headlights no longer obscure the license plate, allowing the license plate to be extracted for detection.

[0041] like Figure 5 As shown, the main implementation steps are as follows:

[0042] Step S1. Train the license plate detection model:

[0043] S1.1, Mark license plate detection data

[0044] The labeling method involves marking the license plate with a rectangular frame, such as... Figure 3 The data in the red box is labeled using labelImg, a tool for labeling data, and the data is the COCO2017 dataset.

[0045] S1.2, Training the license plate detection model

[0046] The detection framework used here is YOLOv7, and the network selected is YOLOv7-tiny (YOLOv7 is a multi-class deep learning open-source framework for object detection). This method is not limited to the YOLOv7 framework; any object detection framework can be used, as long as it is object detection. License plate detection is a type of object detection.

[0047] Step S2. Train the headlight detection model:

[0048] S2.1, Marking vehicle headlight test data

[0049] Annotation method as follows Figure 4 The green box indicates areas labeled using labelImg, a data annotation tool. LabelImg labels two headlights as a single headlight, allowing for the extraction of characteristics from both headlights and mitigating the possibility of falsely identifying roadside lights. When two headlights converge into one, the entire headlight can be labeled using labelImg.

[0050] S2.2, Training the vehicle headlight detection model

[0051] The detection framework used here is YOLOv7, and the network selected is YOLOv7-tiny (YOLOv7 is a multi-class deep learning open-source framework for object detection). This method is not limited to the YOLOv7 framework; any object detection framework can be used, as long as it is object detection. License plate detection is a type of object detection.

[0052] Step S3. Add motion detection

[0053] S3.1, motion detection is an algorithm that detects moving targets in a video stream. Here, motion detection uses the method of calculating the frame difference between two grayscale images. A vector of size 4 is used to store 4 grayscale images, with an image width and height of 640x360 (the image resolution can be modified according to the actual situation). The difference between the fourth frame and the first frame is calculated (the interval of two frames is because the difference between two adjacent frames is small, and the number of interval frames can also be set according to the requirements). Assuming the fourth frame is A and the first frame is B, the frame difference between the fourth frame and the first frame is AB.

[0054] S3.2, Binarize the difference map from step S3.1, set the threshold to 20 (the threshold can be adjusted according to requirements), set values ​​>20 to 1, and values ​​less than 20 to 0, thereby obtaining the difference binary map.

[0055] S3.3, on the binary image from step S3.2, obtain the bounding box of the moving target, i.e., the area with a pixel value of 1. First, find the edge contour with a pixel value of 1, then obtain the smallest bounding box of the edge contour. This gives the bounding boxes of the areas where the image changes, i.e., all moving objects. The smallest bounding box here is the bounding box of the areas where the image changes. The width and height can be set, and filtering can be performed. For example, setting it to 20 will filter out boxes smaller than 20. This movement time is approximately 10ms.

[0056] Step S4. Output the moving vehicle headlight target.

[0057] The purpose of filtering stationary vehicle headlights is to filter out stationary lights such as streetlights and street-side building lighting. The specific steps are as follows:

[0058] S4.1, Perform an IOU operation on the results of the vehicle headlight detection in step S2 and the motion detection in step S3. IOU (Intersection over Union) measures the degree of overlap between the ground truth bounding box and the predicted bounding box, and is a commonly used evaluation metric in object detection tasks. The formula for calculating IOU is as follows:

[0059] IOU=Area Of Overlap / Area Of Union

[0060] Wherein, Area Of Overlap represents the intersection area of ​​the headlight frame and the moving frame. Area Of Union represents the union area of ​​the headlight frame and the moving frame. The value of IOU is between 0 and 1, where: 0 indicates that the two bounding boxes do not overlap, and 1 indicates that the two bounding boxes completely overlap.

[0061] S4.2, when the IOU value / headlight area > 0.5 or the IOU value / moving area > 0.5, the headlight target is determined to be a moving headlight target, and this headlight target is output. Here, the headlight area is the width x height of the headlight frame, and the moving area is the width x height of the moving frame. 0.5 can be adjusted according to the actual situation.

[0062] Step S5. The equipment performs light collection operation.

[0063] When there is a moving vehicle light detection output in step S3, the device, i.e., the camera, performs a light-receiving operation to reveal the license plate that was obscured by the flashing light. Figure 2 As shown, this is how the license plate can be detected.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for solving the problem of license plate detection failure due to night vision exposure, characterized in that, The method includes the following steps: S1. Train the license plate detection model; S2. Train the vehicle headlight detection model; S3. Add motion detection: S3.1, the motion detection algorithm is used to detect moving targets in the video stream: The motion detection here uses the method of calculating the frame difference between two grayscale images. A vector of size 4 is used to store 4 grayscale images. The image resolution can be modified according to the actual situation. The image width and height are 640x360. The number of frames between the frames can be set as needed. Since the difference between two adjacent frames is small, it is assumed that there is a two-frame interval. Then the difference between the fourth frame and the first frame is used. Let the fourth frame be A and the first frame be B. The frame difference between the fourth frame and the first frame is AB. S3.2, Binarize the difference map from step S3.

1. The threshold 1 can be adjusted according to the requirements. Assuming the threshold 1 is set to 20, values ​​greater than 20 are set to 1, and values ​​less than 20 are set to 0, thereby obtaining the difference binary map. S3.3, Obtain the bounding box of the moving target from the binary image in step S3.2, i.e., the area with a pixel value of 1. First, find the edge contour with a pixel value of 1, and then obtain the smallest bounding box of the edge contour. This gives the bounding box of the area where the image changes. The smallest bounding box here is the bounding box of the area where the image changes, i.e., all moving objects. The width and height can be set and filtered. Assuming it is set to 20, boxes smaller than 20 can be filtered out. The movement time is 10ms. S4. Output the target of the moving vehicle lights: The purpose of filtering stationary vehicle lights is to filter out stationary lights that cause interference, and this further includes: S4.1 Perform an IOU operation on the results of the vehicle light detection in step S2 and the motion detection in step S3. The formula for calculating IOU is as follows: IOU=Area Of Overlap / Area Of Union Where Area Of Overlap represents the intersection area of ​​the headlight frame and the moving frame; Area Of Union represents the union area of ​​the headlight frame and the moving frame; the value of IOU is between 0 and 1, where: 0 means that the two bounding boxes do not overlap; 1 means that the two bounding boxes completely overlap. S4.2, when the IOU value / headlight area > threshold 2 or the IOU value / moving area > threshold 2, the headlight target is determined to be a moving headlight target. Here, the headlight area is the width multiplied by the height of the headlight frame, and the moving area is the width multiplied by the height of the moving frame. Then, this headlight target is output. The threshold 2 can be adjusted according to the actual situation. Here, it is set to 0.

5. S5. The equipment performs light collection operation: When there is a moving vehicle light detection output in step S4, the device performs a light collection operation to reveal the license plate that was blocked by the exploding light, thus enabling the license plate to be detected.

2. The method for solving the problem of license plate detection failure due to night vision exposure as described in claim 1, characterized in that, Step S1 further includes: S1.1, Mark license plate detection data The annotation method uses rectangular boxes to annotate license plates, and labelImg is used for annotation. labelImg is a tool for annotating data, and the data is the coco2017 dataset. S1.2, Training the license plate detection model The detection framework used here is YOLOv7, and the network selected is YOLOv7-tiny. YOLOv7 is a multi-class deep learning open-source framework for object detection. This method is not limited to the YOLOv7 framework; all object detection frameworks can be used, as long as it is object detection. License plate detection is a type of object detection.

3. The method for solving the problem of license plate detection failure due to night vision exposure as described in claim 1, characterized in that, Step S2 further includes: S2.1, Marking vehicle headlight test data The annotation method uses rectangular boxes to annotate the vehicle lights, and labelImg is used for annotation data. LabelImg is a tool for annotating data. It annotates two lights of a vehicle light as one light, which can extract the characteristics of two lights of the vehicle light and also prevent false detection of roadside lights to a certain extent. When two lights of a vehicle are combined into one light, the entire light can be annotated, and labelImg is used for annotation data. S2.2, Training the vehicle headlight detection model The detection framework used here is YOLOv7, and the network selected is YOLOv7-tiny. YOLOv7 is a multi-class deep learning open-source framework for object detection. This method is not limited to the YOLOv7 framework; all object detection frameworks can be used, as long as it is object detection. License plate detection is a type of object detection.

4. The method for solving the problem of license plate detection failure due to night vision exposure as described in claim 1, characterized in that, The lamps that cause interference in step S4 include streetlights and street-side building lighting.

Images