A false license plate determination method and device

Abstract

The application provides a false license plate determination method, which comprises the following steps: collecting or inputting a video image, performing license plate detection on the video image to obtain a license plate region; performing outward expansion on the license plate region to obtain a region of interest, and scaling the region of interest to a fixed size to obtain a scaled region of interest; inputting sample images of real license plates and false license plates into an improved yolov8n network to perform training and obtain a trained false license plate detection model; and inputting the scaled region of interest into the trained false license plate detection model to output a license plate detection result. Compared with existing license plate recognition technologies, the application can detect false license plates.

Description

Technical Field

[0001] This invention relates to image processing, smart parking, and license plate recognition, and particularly to a method and apparatus for identifying fake license plates. Background Technology

[0002] With the increasing modernization of cities and the continuous increase in the number of motor vehicles, intelligent transportation systems are playing an increasingly important role. License plate recognition technology is an important component of modern intelligent transportation systems and has a wide range of applications.

[0003] The primary task of existing license plate recognition technology is to detect and identify license plates and their characters, without intelligently integrating anti-counterfeiting functions. Currently, license plate recognition systems face the challenge of "fake license plate" attacks. For example, attackers may use mobile phones or tablets to display images of real license plates, or a single genuine license plate (not mounted on a vehicle), or a temporary license plate, to create a convincing illusion, causing the system to mistakenly believe it is a whitelisted license plate and thus evade payment.

[0004] In conclusion, there is an urgent need to propose a method for identifying fake license plates to solve the above problems. Summary of the Invention

[0005] In view of this, the main objective of the present invention is to provide a method for identifying fake license plates that can achieve both accuracy and speed.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for determining fake license plates is provided, the method comprising:

[0007] The first step is to acquire or input video images, perform license plate detection on the video images, and obtain the license plate area;

[0008] The second step is to expand the license plate area to obtain the region of interest, and then scale the region of interest to a fixed size to obtain the scaled region of interest.

[0009] The third step is to input sample images of real and fake license plates into the improved yolov8n network, train it, and obtain a trained fake license plate detection model.

[0010] The fourth step is to input the scaled region of interest into the trained fake license plate detection model and output the license plate detection results.

[0011] Furthermore, the second step includes:

[0012] The steps for obtaining the region of interest are as follows: Calculate the width (Width) of the license plate area; Expand outward from the center point of the license plate area by expanding the horizontal coordinate of the center point to the left by T1*Width, expanding it to the right by T2*Width, expanding the vertical coordinate of the center point to the top by T3*Width, and expanding the lower boundary to the bottom by T4*Width; Use the expanded license plate area as the region of interest.

[0013] The region of interest scaling step involves scaling the region of interest to a fixed size to obtain the scaled region of interest.

[0014] Wherein, T1, T2, T3, and T4 refer to the expansion coefficients of the left boundary, right boundary, upper boundary, and lower boundary, respectively.

[0015] Furthermore, the value range of T1 is 2~4, the value range of T2 is 2~4, the value range of T3 is 2~3, and the value range of T4 is 1.1~2. The fixed scale refers to a width of 288 pixels and a height of 192 pixels.

[0016] The improved YOLOv8n network in the third step refers to adding a classification branch to the YOLOv8n network and adding cross-entropy loss calculation to the input image. Specifically, it adds a 128-channel 1*1 convolution + ReLU, a 128-channel 3*3 convolution + ReLU with stride=1, a GAP layer, a liner layer, a softmax layer, and a cross-entropy loss (CE loss) to the P5 layer of the backbone. The neck in the original detection network structure is changed from PAN-FPN to FPN, the output layers with stride16 and stride32 are removed from the head, the softmax of the DFL branch in the head is changed to sigmoid, and reg_max is changed from 16 to 8. The liner layer is a fully connected layer with 2 output channels, performing binary classification of fake and real license plates.

[0017] Furthermore, the fourth step includes: inputting the scaled region of interest, performing detection on the detection branch of the trained fake license plate detection model, and directly outputting "fake license plate" if the detection result is a fake license plate, and inputting "real license plate" into the classification branch of the trained fake license plate detection model for classification if the classification result is a fake license plate, and outputting "real license plate" otherwise.

[0018] According to another aspect of the present invention, a device for detecting fake license plates is provided, the device comprising:

[0019] The license plate region acquisition module is used to acquire or input video images, perform license plate detection on the video images, and acquire the license plate region.

[0020] The Region of Interest (ROI) acquisition and scaling module is used to expand the license plate area outward, acquire the ROI, scale the ROI to a fixed size, and acquire the scaled ROI.

[0021] The fake license plate detection model training module is used to train a fake license plate detection model in an improved YOLOv8N network by taking sample images of real and fake license plates as input.

[0022] The fake license plate detection module is used to input the scaled region of interest into the trained fake license plate detection model and output the license plate detection results.

[0023] Furthermore, the region of interest acquisition and scaling module includes:

[0024] The region of interest (ROI) acquisition module is used to calculate the width (Width) of the license plate area; it expands outward from the center point of the license plate area by expanding the horizontal coordinate of the center point to the left by T1*Width, to the right by T2*Width, expanding the vertical coordinate of the center point to the top by T3*Width, and expanding the lower boundary to the bottom by T4*Width; the expanded license plate area is then used as the ROI.

[0025] The Region of Interest (ROI) scaling module is used to scale the ROI to a fixed size and obtain the scaled ROI.

[0026] Wherein, T1, T2, T3, and T4 refer to the expansion coefficients of the left boundary, right boundary, upper boundary, and lower boundary, respectively. Further, the value range of T1 is 2~4, the value range of T2 is 2~4, the value range of T3 is 2~3, and the value range of T4 is 1.1~2.

[0027] The fixed scale in the region of interest scaling module refers to a width of 288 pixels and a height of 192 pixels.

[0028] The improved YOLOv8n network in the fake license plate detection model training module refers to adding a classification branch to the YOLOv8n network and adding cross-entropy loss calculation to the input image. Specifically, it adds a 128-channel 1*1 convolution + ReLU, a 128-channel 3*3 convolution + ReLU with stride=1, a GAP layer, a liner layer, a softmax layer, and a cross-entropy loss (CE loss) to the P5 layer of the backbone. The neck in the original detection network structure is changed from PAN-FPN to FPN, the output layers with stride16 and stride32 are removed from the head, the softmax of the DFL branch in the head is changed to sigmoid, and reg_max is changed from 16 to 8. The liner layer is a fully connected layer with 2 output channels, performing binary classification of fake and real license plates.

[0029] The sample images of real and fake license plates refer to license plate images that have been labeled with real and fake license plates, respectively. The width and height of the sample images of real and fake license plates are 288 pixels and 192 pixels, respectively.

[0030] Furthermore, the fake license plate detection module includes: inputting a scaled region of interest, performing detection on the detection branch of the trained fake license plate detection model, and directly outputting "fake license plate" if the detection result is a fake license plate, and inputting "real license plate" into the classification branch of the trained fake license plate detection model for classification if the classification result is a fake license plate, and outputting "real license plate" otherwise.

[0031] Compared with existing license plate detection technologies, the advantage of the fake license plate determination method of the present invention is that it can effectively detect fake license plates by using an improved yolov8n network. Attached Figure Description

[0032] Figure 1 A flowchart of a method for identifying fake license plates according to the present invention is shown.

[0033] Figure 2 A frame diagram of a fake license plate detection device according to the present invention is shown. Implementation

[0034] To enable those skilled in the art to further understand the structure, features and other objectives of the present invention, the following detailed description is provided in conjunction with the accompanying preferred embodiments. The preferred embodiments described are only for illustrating the technical solutions of the present invention and are not intended to limit the present invention.

[0035] Figure 1A flowchart of a method for detecting fake license plates according to the present invention is provided. Figure 1 As shown, a method for determining fake license plates according to the present invention includes:

[0036] The first step, S1, involves acquiring or inputting video images, performing license plate detection on the video images, and obtaining the license plate area.

[0037] The second step, S2, involves expanding the license plate area to obtain the region of interest, scaling the region of interest to a fixed size, and obtaining the scaled region of interest.

[0038] The third step, S3, involves inputting sample images of real and fake license plates into the improved yolov8n network, training it, and obtaining a trained fake license plate detection model.

[0039] In the fourth step S4, the scaled region of interest is input into the trained fake license plate detection model, and the license plate detection result is output.

[0040] In the first step S1, video images can be acquired using an image acquisition device or input using a data transmission device. For example, in a parking lot scenario, a camera is installed at the entrance, facing the entrance, to acquire video images in that direction.

[0041] The license plate detection can be implemented using existing license plate localization methods, including one or more combinations of the following: license plate detection or localization based on color segmentation, license plate detection or localization based on pattern recognition, license plate detection or localization based on classifiers, license plate detection or localization based on neural networks, license plate detection or localization based on deep learning, etc. For example, a license plate localization method based on color segmentation and texture analysis (e.g., "License Plate Localization Method Based on Color and Texture Analysis. Guo Jie, Shi Pengfei. Journal of Image and Graphics, 2002, 7(5):472-476") is used to locate the license plate in the video image, obtain the license plate position, and take the area in the video image corresponding to the license plate position as the license plate area.

[0042] Further, the second step S2 includes:

[0043] Step S21 for obtaining the region of interest: Calculate the width Width of the license plate region; expand outward from the center point of the license plate region by expanding the horizontal coordinate of the center point of the license plate region to the left by T1*Width, expanding it to the right by T2*Width, expanding the vertical coordinate of the center point of the license plate region to the top by T3*Width, and expanding the lower boundary to the bottom by T4*Width; use the expanded license plate region as the region of interest.

[0044] Step S22 involves scaling the region of interest to a fixed size to obtain the scaled region of interest.

[0045] Wherein, T1, T2, T3, and T4 refer to the expansion coefficients of the left boundary, right boundary, upper boundary, and lower boundary, respectively. Further, the value range of T1 is 2~4, the value range of T2 is 2~4, the value range of T3 is 2~3, and the value range of T4 is 1.1~2. For example, T1, T2, T3, and T4 are selected as 3, 3, 2.5, and 1.5 respectively, and the left boundary of the expanded license plate area is x. l ´=x c -3 * Width, right boundary is x r ´=x c +3*Width, with the upper boundary being y. u ´=y c -2.5 * Width, with the lower boundary being y. d ´=y c +1.5*Width, x c y c These represent the x-coordinate and y-coordinate of the center point of the license plate area, respectively.

[0046] The fixed scale in the region of interest scaling step S22 refers to a width of 288 pixels and a height of 192 pixels.

[0047] In the third step S3, the improved YOLOv8n network refers to adding a classification branch to the YOLOv8n network and adding cross-entropy loss calculation to the input image. Specifically, it adds a 128-channel 1*1 convolution + ReLU, a 128-channel 3*3 convolution + ReLU with stride=1, a GAP layer (global average pooling layer), a liner layer, a softmax layer, and a cross-entropy loss (CE loss) to the P5 layer of the backbone. The neck in the original detection network structure is changed from PAN-FPN to FPN, the output layers with stride16 and stride32 are removed from the head, the softmax of the DFL branch in the head is changed to sigmoid, and reg_max is changed from 16 to 8. The liner layer is a fully connected layer with 2 output channels, performing binary classification of fake and real license plates.

[0048] The sample images of real and fake license plates refer to license plate images that have been labeled with real and fake license plates, respectively. The width and height of the sample images of real and fake license plates are 288 pixels and 192 pixels, respectively.

[0049] Furthermore, the fourth step S4 includes: inputting the scaled region of interest, performing detection on the detection branch of the trained fake license plate detection model, and directly outputting "fake license plate" if the detection result is a fake license plate, and inputting "real license plate" into the classification branch of the trained fake license plate detection model for classification if the classification result is a fake license plate, and outputting "real license plate" otherwise.

[0050] Figure 2 A framework diagram of a fake license plate detection device according to the present invention is provided. Figure 2 As shown, a fake license plate detection device according to the present invention includes:

[0051] License plate region acquisition module 1 is used to acquire or input video images, perform license plate detection on the video images, and acquire the license plate region.

[0052] The Region of Interest (ROI) acquisition and scaling module 2 is used to expand the license plate area outward, acquire the ROI, scale the ROI to a fixed size, and acquire the scaled ROI.

[0053] The fake license plate detection model training module 3 is used to train the improved yolov8n network by taking sample images of real and fake license plates as input, and to obtain the trained fake license plate detection model.

[0054] The fake license plate detection module 4 is used to input the scaled region of interest into the trained fake license plate detection model and output the license plate detection result.

[0055] Furthermore, the region of interest acquisition and scaling module 2 includes:

[0056] The region of interest acquisition module 21 is used to calculate the width of the license plate region; it expands outward from the center point of the license plate region by expanding the horizontal coordinate of the center point of the license plate region to the left by T1*Width, expanding it to the right by T2*Width, expanding the vertical coordinate of the center point of the license plate region to the top by T3*Width, and expanding the lower boundary to the bottom by T4*Width; the expanded license plate region is taken as the region of interest.

[0057] The region of interest scaling module 22 is used to scale the region of interest to a fixed size and obtain the scaled region of interest.

[0058] Wherein, T1, T2, T3, and T4 refer to the expansion coefficients of the left boundary, right boundary, upper boundary, and lower boundary, respectively. Further, the value range of T1 is 2~4, the value range of T2 is 2~4, the value range of T3 is 2~3, and the value range of T4 is 1.1~2.

[0059] The fixed scale in the region of interest scaling module 22 refers to a width of 288 pixels and a height of 192 pixels.

[0060] The improved YOLOv8n network in the training module 3 of the fake license plate detection model refers to adding a classification branch to the YOLOv8n network and adding cross-entropy loss calculation to the input image. Specifically, it adds a 128-channel 1*1 convolution + ReLU, a 128-channel 3*3 convolution + ReLU with stride=1, a GAP layer (global average pooling layer), a liner layer, a softmax layer, and a cross-entropy loss (CE loss) to the P5 layer of the backbone. The neck in the original detection network structure is changed from PAN-FPN to FPN, the output layers with stride16 and stride32 are removed from the head, the softmax of the DFL branch in the head is changed to sigmoid, and reg_max is changed from 16 to 8. The liner layer is a fully connected layer with 2 output channels, performing binary classification of fake and real license plates.

[0061] The sample images of real and fake license plates refer to license plate images that have been labeled with real and fake license plates, respectively. The width and height of the sample images of real and fake license plates are 288 pixels and 192 pixels, respectively.

[0062] Furthermore, the fake license plate detection module 4 includes: inputting a scaled region of interest, performing detection on the detection branch of the trained fake license plate detection model, and directly outputting a fake license plate if the detection result is a fake license plate, and inputting the detection result to the classification branch of the trained fake license plate detection model for classification if the classification result is a fake license plate, and outputting a real license plate otherwise.

[0063] Compared with existing license plate detection technologies, the advantage of the fake license plate determination method of the present invention is that it can effectively detect fake license plates by using an improved yolov8n network.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It should be understood that the present invention is not limited to the implementations described herein. These implementations are described to help those skilled in the art practice the present invention. Any person skilled in the art can easily make further improvements and modifications without departing from the spirit and scope of the present invention. Therefore, the present invention is only limited by the content and scope of the claims, and is intended to cover all alternatives and equivalents included within the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A false license plate determination method characterized by comprising: The method comprises: A first step of collecting or inputting a video image, performing license plate detection on the video image, and obtaining a license plate region; A second step of performing outer expansion on the license plate region, obtaining a region of interest, scaling the region of interest to a fixed size, and obtaining a scaled region of interest; A third step of inputting sample images of real license plates and fake license plates into the improved yolov8n network, training, and obtaining a trained fake license plate detection model; A fourth step of inputting the scaled region of interest into the trained fake license plate detection model and outputting a license plate detection result; further, the improved yolov8n network refers to adding a classification branch to the yolov8n network, adding cross-entropy loss calculation to the input image, specifically, adding a 1*1 convolution of 128 channels + relu, a 3*3 convolution of 128 channels and stride = 1 + relu, a GAP layer, a liner layer, a softmax, and a cross-entropy loss (CE loss) to the P5 of the backbone; the neck in the original detection network structure is changed from PAN-FPN to FPN, the output layer of stride16 and stride32 in the head is removed, the softmax in the DFL branch in the head is changed to sigmoid, and reg_max is changed from 16 to 8. The liner layer is a fully connected layer, the output channel is 2, and the fake license plate and the real license plate are classified.

2. The method of claim 1, wherein, The second step comprises: A region of interest obtaining step of calculating the width Width of the license plate region, performing outer expansion from the center point of the license plate region, performing left outer expansion T1*Width and right outer expansion T2*Width on the horizontal coordinate of the center point of the license plate region, performing upper outer expansion T3*Width on the vertical coordinate of the center point of the license plate region, and performing lower outer expansion T4*Width on the lower boundary; the expanded license plate region is taken as the region of interest; A region of interest scaling step of scaling the region of interest to a fixed size to obtain a scaled region of interest; The T1, T2, T3, and T4 are respectively left boundary, right boundary, upper boundary, and lower boundary expansion coefficients.

3. The method of claim 2, wherein, The value range of T1 is 2-4, the value range of T2 is 2-4, the value range of T3 is 2-3, and the value range of T4 is 1.1-2; the fixed size refers to a width of 288 pixels and a height of 192 pixels.

4. The method of claim 1, wherein, The fourth step comprises: inputting the scaled region of interest, detecting in the detection branch of the trained fake license plate detection model, if the detection result is a fake license plate, directly outputting the fake license plate, if the detection result is a real license plate, inputting into the classification branch of the trained fake license plate detection model for classification, if the classification result is a fake license plate, directly outputting the fake license plate, otherwise outputting the real license plate.

5. A false license plate determination device characterized by comprising: The device comprises: The license plate region acquisition module is configured to collect or input a video image, perform license plate detection on the video image, and acquire a license plate region; the region of interest acquisition and scaling module is configured to expand the license plate region outward, acquire a region of interest, scale the region of interest to a fixed size, and acquire a scaled region of interest; The false license plate detection model training module is configured to input sample images of real license plates and false license plates into the improved yolov8n network, perform training, and acquire a trained false license plate detection model; The false license plate detection module is configured to input the scaled region of interest into the trained false license plate detection model and output a license plate detection result. Further, the improved yolov8n network refers to adding a classification branch to the yolov8n network, adding cross-entropy loss calculation to the input image, specifically, adding 1*1 convolution of 128 channels + relu, 3*3 convolution of 128 channels and stride = 1 + relu, 1 GAP layer, 1 liner layer, 1 softmax, and 1 cross-entropy loss (CE loss) to P5 of the backbone; changing the neck in the original detection network structure from PAN-FPN to FPN, removing the output layers of stride16 and stride32 in the head, changing the softmax in the DFL branch in the head to sigmoid, and changing reg_max from 16 to 8. The liner layer is a fully connected layer, and the output channel is 2, for binary classification of real license plates and false license plates.

6. The apparatus of claim 5, wherein, The region of interest acquisition and scaling module includes: The region of interest acquisition module is configured to calculate the width Width of the license plate region, expand outward from the center point of the license plate region, expand the horizontal coordinate of the center point of the license plate region leftward by T1*Width and rightward by T2*Width, expand the vertical coordinate of the center point of the license plate region upward by T3*Width, and expand the lower boundary downward by T4*Width, and take the expanded license plate region as the region of interest. The region of interest scaling module is configured to scale the region of interest to a fixed size and acquire a scaled region of interest. The T1, T2, T3, and T4 are respectively the expansion coefficients of the left boundary, the right boundary, the upper boundary, and the lower boundary.

7. The apparatus of claim 6, wherein, The value range of T1 is 2-4, the value range of T2 is 2-4, the value range of T3 is 2-3, and the value range of T4 is 1.1-2; the fixed size refers to a width of 288 pixels and a height of 192 pixels.

8. The apparatus of claim 5, wherein, The false license plate detection module includes: inputting the scaled region of interest, detecting in the detection branch of the trained false license plate detection model, if the detection result is a false license plate, directly outputting the false license plate, if the detection result is a real license plate, inputting into the classification branch of the trained false license plate detection model for classification, if the classification result is a false license plate, directly outputting the false license plate, otherwise outputting the real license plate.

Images