True and false license plate determination method and device, electronic equipment and storage medium

Abstract

The invention provides an authenticity license plate determination method and device, electronic equipment and a storage medium, and relates to the technical field of license plate detection, and the method comprises the steps: obtaining a video monitoring image which is collected by a barrier gate camera and comprises a license plate; performing image difference processing on the video monitoring image to obtain a difference feature map; determining an authenticity identification result of the license plate based on the difference feature map and a license plate classification model; wherein the license plate classification model is used for identifying the authenticity of the license plate based on the difference feature map. According to the technical scheme provided by the invention, the false license plate displayed in media such as terminal equipment and paper can be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of license plate detection, and in particular to a method, device, electronic equipment and storage medium for determining an authentic license plate. Background Art

[0002] In parks, residential areas, parking lots and other places where vehicles enter and exit, cameras will be installed at the entrances and exits to collect license plate information of vehicles entering and exiting, in order to manage entry and exit permissions, parking charges, etc.

[0003] For the entry and exit gates, as long as the license plate information collected by the camera is an authorized license plate or a license plate that has paid parking fees, the vehicle will be allowed to pass. However, if the owner takes an image of the authorized license plate in advance and displays the license plate image in front of the gate camera through a terminal device, paper or other media, the corresponding gate can also be opened, which will make the gate meaningless. Therefore, how to identify such fake license plates is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The invention provides a method and device for determining a genuine or fake license plate, an electronic device and a storage medium, which are used to solve the problem of how to identify a genuine or fake license plate in the prior art.

[0005] The present invention provides a method for determining a genuine or fake license plate, comprising:

[0006] Obtain the video surveillance image containing the license plate captured by the gate camera;

[0007] Performing image difference processing on the video surveillance image to obtain a difference feature map;

[0008] The authenticity recognition result of the license plate is determined based on the differential feature map and the license plate classification model; the license plate classification model is used to identify the authenticity of the license plate based on the differential feature map.

[0009] According to a method for determining a genuine license plate provided by the present invention, the method for determining the genuine license plate recognition result based on the differential feature map and the license plate classification model includes:

[0010] Inputting the differential feature map into the license plate classification model to obtain the license plate classification confidence output by the license plate classification model;

[0011] Determining the authenticity recognition result of the license plate based on the license plate classification confidence;

[0012] The license plate classification model is obtained by training an initial license plate classification model based on a sample differential feature map corresponding to a sample video surveillance image and a license plate classification label corresponding to the sample differential feature map.

[0013] A method for determining genuine and fake license plates provided by the present invention, wherein the sample differential feature map includes a first sample differential feature map and a second sample differential feature map, and the license plate classification label includes a first label and a second label; the license plate classification model is trained based on the following steps:

[0014] Obtain the video frame image of the real vehicle passing video stream collected by the barrier camera to obtain the first sample video surveillance image;

[0015] Obtain the video frame image when the license plate image is displayed using the media container collected by the barrier camera to obtain the second sample video surveillance image;

[0016] Perform image differential processing on the first sample video surveillance image to obtain the first sample differential feature map, and mark the first label on the first sample differential feature map;

[0017] Perform image differential processing on the second sample video surveillance image to obtain the second sample differential feature map, and mark the second label on the second sample differential feature map;

[0018] Train the initial license plate classification model based on the first sample differential feature map, the first label, the second sample differential feature map, and the second label to obtain the license plate classification model.

[0019] According to a method for determining genuine and fake license plates provided by the present invention, the determining the authenticity recognition result of the license plate based on the differential feature map and the license plate classification model includes:

[0020] Obtain the optical flow feature information of the license plate in the video surveillance image;

[0021] Input the optical flow feature information and the differential feature map into the license plate classification model to obtain the classification confidence level output by the license plate classification model;

[0022] Wherein, the license plate classification model is trained by training the initial license plate classification model based on the sample differential feature map corresponding to the sample video surveillance image, the license plate classification label corresponding to the sample differential feature map, and the sample optical flow feature information of the sample license plate in the sample video surveillance image.

[0023] According to a method for determining genuine and fake license plates provided by the present invention, the obtaining the optical flow feature information of the license plate in the video surveillance image includes:

[0024] Extract the optical flow feature points of the license plate in the video surveillance image;

[0025] For each of the optical flow feature points, determine the horizontal optical flow and the vertical optical flow corresponding to the optical flow feature point;

[0026] Determine the optical flow strain value corresponding to the optical flow feature point based on the horizontal optical flow and the vertical optical flow;

[0027] Determine the horizontal optical flow, the vertical optical flow, and the optical flow strain value corresponding to each of the optical flow feature points as the optical flow feature information of the license plate in the video surveillance image.

[0028] According to a method for determining genuine and fake license plates provided by the present invention, the optical flow feature points of the license plate include the four corner points of the license plate.

[0029] According to a method for determining genuine and fake license plates provided by the present invention, the license plate classification model is trained based on the following steps:

[0030] Obtain a sample video surveillance image, and perform image difference processing on the sample video surveillance image to obtain a sample difference feature map;

[0031] Extract the sample optical flow feature points of the sample license plate in each of the sample video surveillance images, and determine the optical flow features of each of the sample optical flow feature points to obtain the sample optical flow feature information;

[0032] Train the initial license plate classification model based on the sample difference feature map, the sample optical flow feature information, and the license plate classification label corresponding to the sample difference feature map to obtain the license plate classification model.

[0033] According to a method for determining genuine and fake license plates provided by the present invention, the performing image difference processing on the video surveillance image to obtain a difference feature map includes:

[0034] Perform grayscale processing on the video surveillance image to obtain a grayscale image;

[0035] Perform image difference processing on the grayscale image, and perform binarization processing on the image difference processing result to obtain the difference feature map.

[0036] The present invention also provides a device for determining genuine and fake license plates, including:

[0037] An image acquisition module, configured to acquire a video surveillance image containing a license plate collected by a barrier gate camera;

[0038] A difference processing module, configured to perform image difference processing on the video surveillance image to obtain a difference feature map;

[0039] A genuine and fake license plate determination module, configured to determine the genuine and fake recognition result of the license plate based on the difference feature map and the license plate classification model; the license plate classification model is used to recognize the genuineness and fakeness of the license plate based on the difference feature map.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for determining the authenticity of a license plate as described in any one of the above is implemented.

[0041] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for determining the authenticity of a license plate as described in any one of the above is implemented.

[0042] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the method for determining the authenticity of a license plate as described in any one of the above is implemented.

[0043] For the method, device, electronic device, and storage medium for determining the authenticity of a license plate provided by the present invention, first, a video surveillance image containing a license plate collected by a barrier gate camera is obtained. Then, image difference processing is performed on the video surveillance image to obtain a difference feature map. Next, based on the difference feature map and a license plate classification model for identifying the authenticity of a license plate based on the difference feature map, an authenticity recognition result of the license plate is determined. Thus, the identification of the authenticity of a license plate is realized by using the global image in the field of view of the barrier gate camera. The global image can cover the frames of terminal devices and media containers such as paper for displaying license plate photos, facilitating the distinction between genuine and fake license plates. Moreover, background interference is suppressed through image difference processing, improving the accuracy of authenticating license plates. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 is a flowchart of the method for determining the authenticity of a license plate provided by an embodiment of the present invention;

[0046] Figure 2 is a flowchart of performing image difference processing on a video surveillance image in an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a video surveillance image and a corresponding difference feature map in a paper license plate scenario in an embodiment of the present invention;

[0048] Figure 4 is a schematic diagram of a video surveillance image and a corresponding difference feature map in a mobile phone displaying license plate scenario in an embodiment of the present invention;

[0049] Figure 5 It is a schematic diagram of a video surveillance image and a corresponding differential feature map in the real license plate scenario in the embodiment of the present invention;

[0050] Figure 6 It is one of the flowcharts of the training method of the license plate classification model in the embodiment of the present invention;

[0051] Figure 7 It is the second flowchart of the training method of the license plate classification model in the embodiment of the present invention;

[0052] Figure 8 It is a schematic diagram of the structural principle of the license plate classification model in the embodiment of the present invention;

[0053] Figure 9 It is a schematic diagram of the structure of the genuine and fake license plate determination device provided in the embodiment of the present invention;

[0054] Figure 10 It is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention. Detailed implementation manners

[0055] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts belong to the scope of protection of the present invention.

[0056] It should be noted that the serial numbers assigned to the objects described in the present invention itself, such as "first", "second", etc., are only used to distinguish the described objects and do not have any sequential or technical meaning.

[0057] At the vehicle entrance and exit, if the vehicle owner takes a picture of the authorized license plate in advance and displays the license plate image in front of the camera of the barrier gate through media such as terminal devices and papers, it is very easy to open the corresponding barrier gate, making the barrier gate lose its established meaning.

[0058] In the related technologies for identifying such fake license plates, the authenticity of the license plate can be judged by detecting the frames of media containers such as terminal devices and papers used to display license plate photos. However, this method cannot defend against the attack of license plate photos without container frames, and moreover, different sizes and shapes of papers will also have a certain impact on the detection effect.

[0059] In the related art, a license plate anti-counterfeiting method based on border classification and image classification of the area around the license plate can be adopted. This method requires expanding based on the license plate area, and it is difficult to control the expansion size, making it unable to meet the attacks of fake license plates of various sizes and different distances. When the expanded area is too small, it cannot cover the borders of media containers such as terminal devices and papers used to display license plate photos. When the expanded area is too large, it will include too much background area, affecting the classification effect of the license plate.

[0060] Based on this, in the embodiments of the present invention, the complete video surveillance image under the field of view angle of the camera is used as the analysis object. The video surveillance image is subjected to image difference processing, and based on the difference feature map obtained from the image difference processing and the trained license plate classification model for identifying the authenticity of the license plate according to the difference feature map, the authenticity of the license plate in the video surveillance image is identified. During the image analysis process, the borders of media containers such as terminal devices and papers used to display license plate photos can be covered, and at the same time, background interference can be suppressed, so as to cope with fake license plate attacks in various scenarios and forms.

[0061] The following combines Figures 1-8 to describe the method for determining the authenticity of license plates of the present invention. The method for determining the authenticity of license plates can be applied to an electronic device, which includes a monitoring device or a terminal device, a server, etc. that are communicatively connected to the monitoring device. Among them, the terminal device can include a mobile phone, a computer, a tablet computer, a wearable device, etc.; the server can include an independent server, a cluster server, or a cloud server, etc. The method for determining the authenticity of license plates can also be applied to a device for determining the authenticity of license plates provided in the electronic device, and the device for determining the authenticity of license plates can be implemented through software, hardware, or a combination of both.

[0062] Figure 1 Exemplarily shows a schematic flowchart of the method for determining the authenticity of license plates provided by the embodiments of the present invention. Referring to Figure 1 as shown, the method for determining the authenticity of license plates can include the following steps 110 to 130.

[0063] Step 110: Obtain a video surveillance image containing a license plate collected by a barrier gate camera.

[0064] The barrier gate camera can perform real-time monitoring on its monitoring area to obtain a video stream. The electronic device can process the video stream frame by frame, detect whether there is a license plate in the video frame image, and when a license plate is detected, obtain at least two video frame images containing the license plate from the video stream to obtain a video surveillance image.

[0065] Exemplarily, at least two video frame images containing license plates obtained from a video stream can be at least two consecutive video frame images containing license plates, or one video frame image can be taken every first preset number of frames to obtain at least two video frame images. The first preset number can be 1 frame, 2 frames, etc. For example, one video frame image containing a license plate can be taken every other frame.

[0066] Exemplarily, for each video frame image in the video stream, it can be determined whether the video frame image includes a license plate based on an object detection algorithm. For example, each video frame image can be input into a pre-trained first object detection model to obtain a target prediction box output by the first object detection model. When the target prediction box is a license plate prediction box, it is determined that the video frame image includes a license plate.

[0067] Among them, the first object detection model can be obtained by training a first initial object detection model based on the first sample image and the first label data corresponding to the first sample image. Among them, the first sample image includes a sample image containing a license plate. The first initial object detection model can include a neural network model with structures such as YOLO (You Only Look Once), FCOS (Fully Convolutional One-Stage), etc.

[0068] Exemplarily, each video frame image can be input into a pre-trained second object detection model to obtain a target prediction box output by the second object detection model. When the target prediction box includes a vehicle and a license plate prediction box, it is determined that the video frame image includes a license plate.

[0069] Among them, the second object detection model can be obtained by training a second initial object detection model based on the second sample image and the second label data corresponding to the second sample image. Among them, the second sample image includes a sample image containing a vehicle and its license plate. The second initial object detection model can include a neural network model with structures such as YOLO, FCOS, etc.

[0070] Step 120: Perform image difference processing on the video surveillance image to obtain a difference feature map.

[0071] For the video surveillance image containing a license plate collected by the barrier camera, image difference processing such as frame difference, three-frame difference, or background difference can be performed on the video surveillance image to obtain a difference feature map.

[0072] Exemplarily, step 120 can include: performing grayscale processing on the video surveillance image to obtain a grayscale image; performing image difference processing on the grayscale image and performing binarization processing on the image difference processing result to obtain a difference feature map.

[0073] Optionally, after performing binary processing on the result of image difference processing, morphological processing can also be performed on the binary image obtained through binary processing to obtain a difference feature map. Through morphological processing, the contour lines in the binary image can be made clearer, improving the image quality of the binary image. The morphological processing can include at least one of corrosion operation, noise filtering, dilation operation, etc., but is not limited thereto.

[0074] For example, Figure 2 An exemplary flowchart of image difference processing for video surveillance images is shown. Taking frame difference as an example, refer to Figure 2 As shown, for each frame of video surveillance image, the video surveillance image can first be grayscale processed to obtain a grayscale image, and then the grayscale image of the current frame of video surveillance image and the grayscale image of the previous frame of video surveillance image are subjected to frame difference operation to obtain the result of image difference processing. Then, the result of image difference processing is binary processed to obtain a binary image. Then, the binary image is successively subjected to corrosion processing and dilation processing to obtain a difference feature map for storage. It can be understood that for the first frame of video surveillance image containing a license plate, there is no corresponding previous frame of video surveillance image containing a license plate, so only the first frame of video surveillance image containing a license plate can be grayscale processed, and the obtained grayscale image can be saved for frame difference operation with the grayscale image of the next frame of video surveillance image.

[0075] For difference feature maps in different scenarios, different image features will be exhibited. For example, Figure 3 An exemplary schematic diagram of a video surveillance image and the corresponding difference feature map in the scenario of a paper license plate is shown, Figure 4 An exemplary schematic diagram of a video surveillance image and the corresponding difference feature map in the scenario of a mobile phone displaying a license plate is shown, Figure 5 An exemplary schematic diagram of a video surveillance image and the corresponding difference feature map in the scenario of a real license plate is shown. Combining Figures 3-5 , by comparing the difference feature maps in the three scenarios of paper license plate, mobile phone screen displaying license plate, and real license plate, it can be seen that compared with the difference feature map in the real license plate scenario, in the difference feature maps of the paper license plate and mobile phone screen displaying license plate scenarios, due to the influence of secondary imaging, the vehicle contour features are not obvious and there is significant noise interference. Moreover, there are obvious container boundary contours in the difference feature map, that is, there are obvious paper boundary contours and mobile phone boundary contours. At the same time, the influence of the moving speed of the proximal part of the picture on the image features after difference is more obvious. The difference feature maps in the paper license plate and mobile phone screen displaying license plate scenarios have thicker contour lines and obvious ghosting phenomena. Based on this, the authenticity of the license plate can be identified according to this difference shown by the pseudo license plate and the real license plate in the difference feature map.

[0076] Step 130: Determine the true or false recognition result of the license plate based on the differential feature map and the license plate classification model.

[0077] The license plate classification model is used to identify the authenticity of the license plate based on the differential feature map.

[0078] In one exemplary embodiment, after obtaining the differential feature map, the license plate classification model can be used to identify the authenticity of the license plate for the differential feature map. Specifically, the license plate classification model can be trained based on the sample differential feature map corresponding to the sample video surveillance image and the license plate classification label corresponding to the sample differential feature map. The sample video surveillance image includes a first sample video surveillance image and a second sample video surveillance image. The first sample video surveillance image is a video frame image of the real vehicle passing video stream collected by the barrier gate camera, and the second sample video surveillance image is a video frame image of the video stream collected by the barrier gate camera when the license plate image is displayed using media containers such as paper and terminal devices.

[0079] In another exemplary embodiment, the optical flow feature information of the license plate in the video surveillance image can also be obtained. The optical flow feature information can reflect the motion information such as the motion speed and acceleration of the license plate in the video surveillance image. After obtaining the differential feature map, the differential feature map and the optical flow feature information can be input into the license plate classification model together, and the license plate classification model is used to identify the authenticity of the license plate. In this way, the accuracy of license plate authenticity identification can be improved by combining the optical flow feature information.

[0080] Specifically, the license plate classification model can be trained based on the sample differential feature map corresponding to the sample video surveillance image, the license plate classification label corresponding to the sample differential feature map, and the sample optical flow feature information corresponding to the sample license plate in the sample video surveillance image. The sample video surveillance image includes a first sample video surveillance image and a second sample video surveillance image. The first sample video surveillance image is a video frame image of the real vehicle passing video stream collected by the barrier gate camera, and the second sample video surveillance image is a video frame image of the video stream collected by the barrier gate camera when the license plate image is displayed using media containers such as paper and terminal devices.

[0081] The true and false license plate determination method provided by the embodiments of the present invention first obtains a video surveillance image containing a license plate collected by a barrier gate camera, then performs image difference processing on the video surveillance image to obtain a differential feature map, and then determines the true or false recognition result of the license plate based on the differential feature map and a license plate classification model for identifying the authenticity of the license plate based on the differential feature map. Thus, the identification of true and false license plates is realized by using the global image in the field of view of the barrier gate camera. The global image can cover the frames of media containers such as terminal devices and paper used to display license plate photos, facilitating the distinction between true and false license plates. Moreover, background interference is suppressed through image difference processing, improving the accuracy of true and false license plate identification.

[0082] Based on Figure 1 Figure 1 For the genuine and fake license plate determination method corresponding to the embodiment, in an exemplary embodiment, determining the genuine and fake recognition result of the license plate based on the differential feature map and the license plate classification model may include: inputting the differential feature map into the license plate classification model to obtain the license plate classification confidence output by the license plate classification model; determining the genuine and fake recognition result of the license plate based on the license plate classification confidence; wherein, the license plate classification model is obtained by training the initial license plate classification model based on the sample differential feature map corresponding to the sample video surveillance image and the license plate classification label corresponding to the sample differential feature map.

[0083] Exemplarily, determining the genuine and fake recognition result of the license plate based on the license plate classification confidence may include: determining the license plate classification type corresponding to the highest license plate classification confidence in the license plate classification confidence as the genuine and fake recognition result of the license plate; or, sorting the license plate classification confidence from largest to smallest, obtaining the first second preset number of target license plate classification confidences in the sorting result, and determining the license plate classification type with the largest proportion in the target license plate classification confidence as the genuine and fake recognition result of the license plate. Wherein, the license plate classification type includes genuine license plate and fake license plate.

[0084] Exemplarily, Figure 6 FIG. shows one of the flowcharts of the training method of the license plate classification model. Refer to Figure 6 As shown, the license plate classification model can be obtained by training based on the following steps 610 to 620.

[0085] Step 610: Obtain a sample video surveillance image, and perform image difference processing on the sample video surveillance image to obtain a sample differential feature map.

[0086] Among them, the sample video surveillance image includes a first sample video surveillance image and a second sample video surveillance image. The first sample video surveillance image is a video frame image of the real vehicle passing video stream collected by the barrier gate camera, and the second sample video surveillance image is a video frame image of the video stream collected by the barrier gate camera when using media containers such as paper and terminal devices to display the license plate image.

[0087] Correspondingly, the sample differential feature map includes a first sample differential feature map corresponding to the first sample video surveillance image and a second sample differential feature map corresponding to the second sample video surveillance image.

[0088] Exemplarily, in a real application scenario, considering that there may be situations such as environmental light changes and interference from other moving targets during the process of the barrier gate camera collecting images of the monitoring area, for the collected first sample video surveillance image and second sample video surveillance image, image enhancement processing can be performed to obtain the sample video surveillance image. The image enhancement processing therein may include adding random foreground noise, brightness adjustment, etc., but is not limited thereto.

[0089] Exemplarily, the image difference processing may include image difference processing methods such as inter-frame difference, three-frame difference, or background difference. For example, the inter-frame difference processing as shown in Figure 2 may be performed on the sample video surveillance image to obtain a sample difference feature map.

[0090] Step 620: Train the initial license plate classification model based on the sample difference feature map and the license plate classification label corresponding to the sample difference feature map to obtain a license plate classification model.

[0091] Among them, the license plate classification label is used to mark the classification type of the sample license plate, and may include a first label and a second label. The first label is used to mark a genuine license plate, and the second label is used to mark a fake license plate. The first sample difference feature map corresponding to the first sample video surveillance image may be marked with the first label, and the second sample difference feature map corresponding to the second sample video surveillance image may be marked with the second label. After obtaining the sample difference feature map, the initial license plate classification model may be iteratively trained using the sample difference feature map and the corresponding license plate classification label until the model converges to obtain a trained license plate classification model. That is, the initial license plate classification model is trained based on the first sample difference feature map, the first label, the second sample difference feature map, and the second label to obtain the license plate classification model.

[0092] The initial license plate classification model therein may include a neural network model with a structure such as YOLO, Residual Neural Network (ResNet), Region-based Convolutional Neural Networks (R-CNN), etc.

[0093] In this way, after obtaining the trained license plate classification model, the license plate classification model may be used to identify the authenticity of the license plate for the difference feature map corresponding to the video surveillance image.

[0094] In this exemplary embodiment, by performing image difference processing on the sample video surveillance image, a sample difference feature map is obtained, and the sample difference feature map is labeled according to the collected sample video type to ensure the accuracy of the sample label. At the same time, manual calibration of the sample is not required, saving the labor cost of sample calibration.

[0095] In practical application scenarios, there are certain differences between the motion states of license plates displayed on media containers such as paper and terminal devices in the video surveillance images captured by the barrier gate camera and the motion states of real vehicles when they normally drive through the surveillance area of the barrier gate camera in the video surveillance images. Specifically, real vehicles appear to move from far to near and at a slow and stable speed in the video surveillance images; while the motion directions of license plates displayed on media containers such as paper and terminal devices are not fixed in the video surveillance images, the speed is usually fast and unstable, with speed jumps, and the paper may bend and change during the movement, etc.

[0096] Based on this, in an exemplary embodiment of the present invention, the optical flow features of the license plate in the video surveillance image can be extracted, and the authenticity of the license plate can be identified by combining the optical flow features and the differential features Figure 1 to improve the accuracy of license plate authenticity identification.

[0097] Specifically, determining the authenticity identification result of the license plate based on the differential feature map and the license plate classification model may include: obtaining the optical flow feature information of the license plate in the video surveillance image; inputting the optical flow feature information and the differential feature map into the license plate classification model to obtain the classification confidence level output by the license plate classification model; wherein, the license plate classification model is obtained by training an initial license plate classification model based on the sample differential feature map corresponding to the sample video surveillance image, the license plate classification label corresponding to the sample differential feature map, and the sample optical flow feature information of the sample license plate in the sample video surveillance image.

[0098] Among them, the video surveillance image is a continuous video frame image containing the license plate captured by the barrier gate camera. The optical flow feature information can reflect the motion information such as the motion speed and acceleration of the license plate in the video surveillance image, and the optical flow method can be used to obtain the optical flow feature information of the license plate in the video surveillance image. Exemplarily, the optical flow feature information may include optical flow strain values, horizontal optical flow, and vertical optical flow, etc.

[0099] Exemplarily, obtaining the optical flow feature information of the license plate in the video surveillance image may include: extracting the optical flow feature points of the license plate in the video surveillance image; for each optical flow feature point, determining the horizontal optical flow and vertical optical flow corresponding to the optical flow feature point; determining the optical flow strain value corresponding to the optical flow feature point based on the horizontal optical flow and vertical optical flow; and determining the horizontal optical flow, vertical optical flow, and optical flow strain values corresponding to each optical flow feature point as the optical flow feature information of the video surveillance image.

[0100] For example, the TV-L1 energy functional can be used to extract the optical flow features of the license plate in the video surveillance image. Specifically, for each optical flow feature point of the license plate in the video surveillance image, assuming that u represents the horizontal component of the optical flow (i.e., the horizontal optical flow) and v represents the vertical component of the optical flow (i.e., the vertical optical flow), then u and v can be expressed by the following formula (1):

[0101]

[0102] Among them, dx represents the pixel change along the x dimension (i.e., the horizontal dimension), dy represents the pixel change along the y dimension (i.e., the vertical dimension), and dt represents the time change.

[0103] Based on u and v, the optical flow strain ε can be determined using the following formula (2):

[0104]

[0105] Among them, the optical flow strain components (ε xx , ε yy ) on the diagonal are the normal strain components, and (ε xy , ε yx ) are the shear strain components.

[0106] Based on each optical flow strain component in ε, the optical flow strain value |ε| of the optical flow feature point can be determined using the following formula (3):

[0107]

[0108] After obtaining u, v, and |ε|, the three optical flow feature components u, v, and |ε| corresponding to each optical flow feature point can be concatenated by channel superposition and used as the optical flow feature information of the video surveillance image, together with the differential feature Figure 1 input into the license plate classification model of this exemplary embodiment to obtain the classification confidence level output by the license plate classification model. Furthermore, the authenticity recognition result of the license plate in the video surveillance image can be determined based on the license plate classification confidence level.

[0109] Exemplarily, the optical flow feature points of the license plate may include two corner points on a diagonal of the license plate, or may include the four corner points of the license plate, or may further include other feature points such as the center point of the license plate.

[0110] In this exemplary embodiment, the optical flow feature information of the license plate is considered during the license plate authenticity recognition process. Correspondingly, during the training process of the license plate classification model, the training samples include the sample optical flow feature information of the sample license plates. Specifically, Figure 7 Exemplarily shows the second flowchart of the training method of the license plate classification model. Refer to Figure 7 as shown, the license plate classification model can be trained based on the following steps 710 to step 730.

[0111] Step 710: Obtain a sample video surveillance image and perform image differential processing on the sample video surveillance image to obtain a sample differential feature map.

[0112] Specifically, the specific implementation process of step 710 can refer to the above step 610 and will not be elaborated here.

[0113] Step 720: Extract the sample optical flow feature points of the sample license plates in each sample video surveillance image, and determine the optical flow features of each sample optical flow feature point to obtain the sample optical flow feature information.

[0114] Among them, the sample optical flow feature points may include two corner points on a diagonal line of the sample license plate, or may include the four corner points of the sample license plate, or may also include other feature points such as the center point of the sample license plate. In this exemplary embodiment, the case of including the four corner points of the sample license plate is taken as an example.

[0115] The sample optical flow feature information may include the sample optical flow strain value, the sample horizontal optical flow, the sample vertical optical flow, etc.

[0116] In step 720, the specific process of extracting the sample optical flow feature points of the sample license plates in each sample video surveillance image and determining the optical flow features of each sample optical flow feature point may refer to the above process of obtaining the optical flow feature information of the license plates in the video surveillance image. For example, the TV-L1 energy functional can be used to extract the optical flow features of the sample license plates in the sample video surveillance image, which will not be elaborated here.

[0117] Step 730: Train the initial license plate classification model based on the sample difference feature map, the sample optical flow feature information, and the license plate classification label corresponding to the sample difference feature map to obtain the license plate classification model.

[0118] Among them, the license plate classification label is used to mark the classification type of the sample license plate, and may include a first label and a second label. The first label is used to mark the genuine license plate, and the second label is used to mark the fake license plate. After obtaining the first sample difference feature map corresponding to the first sample video surveillance image and the second sample difference feature map corresponding to the second sample video surveillance image, the first sample difference feature map can be marked with the first label, and the second sample difference feature map can be marked with the second label.

[0119] After obtaining the sample difference feature map and the sample optical flow feature information of the sample video surveillance image, the initial license plate classification model can be iteratively trained based on the sample difference feature map, the sample optical flow feature information, and the license plate classification label corresponding to the sample difference feature map until the model converges to obtain the trained license plate classification model. That is, the initial license plate classification model can be trained based on the first sample difference feature map, the first label, the second sample difference feature map, the second label, the first sample optical flow feature information corresponding to the first sample video surveillance image, and the second sample optical flow feature information corresponding to the second sample video surveillance image to obtain the license plate classification model.

[0120] The initial license plate classification model among them may include a neural network model with structures such as YOLO, ResNet, R-CNN, etc.

[0121] In this way, after obtaining the trained license plate classification model, the differential feature map and optical flow feature information of the video surveillance image can be respectively input into the license plate classification model, and the license plate classification model can identify genuine and fake license plates.

[0122] For example, Figure 8 An exemplary structural principle diagram of the license plate classification model is shown. Referring to Figure 8 as shown, the license plate classification model may include a conversion layer, a backbone network layer, and a classification layer. The optical flow feature information can be input into the conversion layer of the license plate classification model, and the conversion layer converts the optical flow feature information into data with the same structure and form as the differential feature map and inputs it into the backbone network layer. At the same time, the corresponding differential feature map is input into the backbone network layer. The backbone network layer extracts features from the differential feature map and the converted optical flow feature information, and then inputs the extracted feature vectors into the classification layer to obtain the classification confidence output by the classification layer.

[0123] Among them, the classification layer is a binary classification layer, and the classification types may include fake license plates and genuine license plates. The classification layer can output the classification confidence 1 corresponding to the fake license plate and the classification confidence 2 corresponding to the genuine license plate. The backbone network layer is, for example, a ResNet50 neural network, but is not limited thereto.

[0124] In this way, by combining the sample differential feature map and the sample optical flow feature information to jointly train the license plate classification model, the feature discrimination ability of the license plate classification model can be improved. Furthermore, when using the license plate classification model to identify genuine and fake license plates, the accuracy of genuine and fake license plate identification can be improved.

[0125] On the one hand, the method for determining genuine and fake license plates provided by the embodiments of the present invention can, by analyzing the global image within the field of view angle of the barrier gate camera, ensure the coverage of the borders of media containers such as papers and terminal devices displaying license plates, enhance the subsequent discrimination effect of genuine and fake license plates, and have good applicability to media containers of different sizes and shapes. At the same time, by performing image difference processing on the video surveillance image containing the license plate collected by the barrier gate camera and using the obtained differential feature map to identify genuine and fake license plates, the interference of the background can be excluded, and it also has an effective identification effect on fake license plates when the background image contains real vehicle targets. On the other hand, by combining the differential feature map with the optical flow feature information of the license plate in the video surveillance image to jointly identify genuine and fake license plates, the accuracy of genuine and fake license plate identification can be improved.

[0126] Next, the device for determining genuine and fake license plates provided by the present invention will be described. The device for determining genuine and fake license plates described below can be mutually referred to in correspondence with the method for determining genuine and fake license plates described above.

[0127] Figure 9The structural schematic diagram of the genuine and fake license plate determination device provided by the embodiment of the present invention is exemplarily shown. Refer to Figure 9 As shown, the genuine and fake license plate determination device may include: an image acquisition module 910, configured to acquire a video surveillance image including a license plate collected by a barrier camera; a differential processing module 920, configured to perform image differential processing on the video surveillance image to obtain a differential feature map; a genuine and fake license plate determination module 930, configured to determine a genuine and fake recognition result of the license plate based on the differential feature map and a license plate classification model; wherein, the license plate classification model is used to recognize the genuineness of the license plate based on the differential feature map.

[0128] In an exemplary embodiment, the genuine and fake license plate determination module 930 may include: a confidence determination unit, configured to input the differential feature map into the license plate classification model to obtain the license plate classification confidence output by the license plate classification model; a genuine and fake license plate determination unit, configured to determine the genuine and fake recognition result of the license plate based on the license plate classification confidence; wherein, the license plate classification model is obtained by training an initial license plate classification model based on a sample differential feature map corresponding to a sample video surveillance image and a license plate classification label corresponding to the sample differential feature map.

[0129] In an exemplary embodiment, the sample differential feature map includes a first sample differential feature map and a second sample differential feature map, and the license plate classification label includes a first label and a second label; the genuine and fake license plate determination device may further include a first model training module, and the first model training module is configured to train the license plate classification model based on the following steps: acquiring video frame images of a real vehicle passing video stream collected by the barrier camera to obtain a first sample video surveillance image; acquiring video frame images when a license plate image is displayed using a media container collected by the barrier camera to obtain a second sample video surveillance image; performing image differential processing on the first sample video surveillance image to obtain a first sample differential feature map, and marking the first label on the first sample differential feature map; performing image differential processing on the second sample video surveillance image to obtain a second sample differential feature map, and marking the second label on the second sample differential feature map; training the initial license plate classification model based on the first sample differential feature map, the first label, the second sample differential feature map, and the second label to obtain the license plate classification model.

[0130] In an exemplary embodiment, the genuine and fake license plate determination module 930 may include: an acquisition unit, configured to acquire the optical flow feature information of the license plate in the video surveillance image; a determination unit, configured to input the optical flow feature information and the differential feature map into the license plate classification model to obtain the classification confidence output by the license plate classification model; wherein, the license plate classification model is obtained by training an initial license plate classification model based on a sample differential feature map corresponding to a sample video surveillance image, a license plate classification label corresponding to the sample differential feature map, and sample optical flow feature information corresponding to a sample license plate in the sample video surveillance image.

[0131] In an exemplary embodiment, the obtaining unit may be specifically configured to: extract the optical flow feature points of the license plate in the video surveillance image; for each optical flow feature point, determine the horizontal optical flow and the vertical optical flow corresponding to the optical flow feature point; determine the optical flow strain value corresponding to the optical flow feature point based on the horizontal optical flow and the vertical optical flow; and determine the horizontal optical flow, the vertical optical flow, and the optical flow strain value corresponding to each optical flow feature point as the optical flow feature information of the license plate in the video surveillance image.

[0132] In an exemplary embodiment, the optical flow feature points of the license plate include the four corner points of the license plate.

[0133] In an exemplary embodiment, the genuine and fake license plate determination device may further include a second model training module, which is configured to train a license plate classification model based on the following steps: obtain sample video surveillance images, perform image difference processing on the sample video surveillance images to obtain sample difference feature maps; extract the sample optical flow feature points of the sample license plates in the sample video surveillance images, and determine the optical flow features of each sample optical flow feature point to obtain sample optical flow feature information; and train an initial license plate classification model based on the sample difference feature maps, the sample optical flow feature information, and the license plate classification labels corresponding to the sample difference feature maps to obtain the license plate classification model.

[0134] In an exemplary embodiment, the difference processing module 920 may include: a first processing unit configured to perform grayscale processing on the video surveillance image to obtain a grayscale image; and a second processing unit configured to perform image difference processing on the grayscale image and perform binarization processing on the image difference processing result to obtain a difference feature map.

[0135] Figure 10 An exemplary structural diagram of an electronic device is shown as Figure 10 shown. The electronic device may include: a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040. Among them, the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the genuine and fake license plate determination method provided in any of the above method embodiments. The method may include, for example: obtaining a video surveillance image containing a license plate collected by a barrier gate camera; performing image difference processing on the video surveillance image to obtain a difference feature map; and determining the genuine and fake recognition result of the license plate based on the difference feature map and the license plate classification model. Among them, the license plate classification model is used to recognize the genuineness of the license plate based on the difference feature map.

[0136] In addition, when the logical instructions in the above-mentioned memory 1030 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0137] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the genuine and fake license plate determination method provided in each of the above method embodiments. This method can, for example, include: obtaining a video surveillance image containing a license plate collected by a barrier gate camera; performing image difference processing on the video surveillance image to obtain a difference feature map; determining a genuine and fake recognition result of the license plate based on the difference feature map and a license plate classification model; wherein, the license plate classification model is used to recognize the genuineness of the license plate based on the difference feature map.

[0138] In yet another aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the genuine and fake license plate determination method provided in each of the above method embodiments. This method can, for example, include: obtaining a video surveillance image containing a license plate collected by a barrier gate camera; performing image difference processing on the video surveillance image to obtain a difference feature map; determining a genuine and fake recognition result of the license plate based on the difference feature map and a license plate classification model; wherein, the license plate classification model is used to recognize the genuineness of the license plate based on the difference feature map.

[0139] By way of example, the computer-readable storage medium includes a non-transitory computer-readable storage medium.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0141] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining the authenticity of license plates, characterized in that, it includes: Obtain a video surveillance image containing a license plate collected by a barrier gate camera; Perform image difference processing on the video surveillance image to obtain a difference feature map; Based on the difference feature map and a license plate classification model, determine the authenticity recognition result of the license plate; The license plate classification model is used to recognize the authenticity of the license plate based on the difference feature map.

2. The method for determining the authenticity of license plates according to claim 1, characterized in that, The step of determining the authenticity recognition result of the license plate based on the difference feature map and the license plate classification model includes: Input the difference feature map into the license plate classification model to obtain the license plate classification confidence level output by the license plate classification model; Based on the license plate classification confidence level, determine the authenticity recognition result of the license plate; wherein, the license plate classification model is obtained by training an initial license plate classification model based on a sample difference feature map corresponding to a sample video surveillance image and a license plate classification label corresponding to the sample difference feature map.

3. The method for determining the authenticity of license plates according to claim 2, characterized in that, The sample difference feature map includes a first sample difference feature map and a second sample difference feature map, and the license plate classification label includes a first label and a second label; the license plate classification model is trained based on the following steps: Obtain the video frame image of the real vehicle passing video stream collected by the barrier gate camera to obtain a first sample video surveillance image; Obtain the video frame image when a license plate image is displayed using a media container collected by the barrier gate camera to obtain a second sample video surveillance image; Perform image difference processing on the first sample video surveillance image to obtain the first sample difference feature map, and mark the first label on the first sample difference feature map; Perform image difference processing on the second sample video surveillance image to obtain the second sample difference feature map, and mark the second label on the second sample difference feature map; Based on the first sample difference feature map, the first label, the second sample difference feature map and the second label, train the initial license plate classification model to obtain the license plate classification model.

4. The method for determining the authenticity of license plates according to claim 1, characterized in that, The step of determining the authenticity recognition result of the license plate based on the difference feature map and the license plate classification model includes: Obtain the optical flow feature information of the license plate in the video surveillance image; Input the optical flow feature information and the difference feature map into the license plate classification model to obtain the classification confidence level output by the license plate classification model; wherein, the license plate classification model is obtained by training the initial license plate classification model based on a sample difference feature map corresponding to a sample video surveillance image, a license plate classification label corresponding to the sample difference feature map, and sample optical flow feature information of a sample license plate in the sample video surveillance image.

5. The method for determining the authenticity of license plates according to claim 4, characterized in that, The step of obtaining the optical flow feature information of the license plate in the video surveillance image includes: Extract the optical flow feature points of the license plate in the video surveillance image. For each of the optical flow feature points, determine the horizontal optical flow and the vertical optical flow corresponding to the optical flow feature point; Based on the horizontal optical flow and the vertical optical flow, determine the optical flow strain value corresponding to the optical flow feature point; Determine the horizontal optical flow, the vertical optical flow, and the optical flow strain value corresponding to each of the optical flow feature points as the optical flow feature information of the license plate in the video surveillance image.

6. The method for determining genuine and fake license plates according to claim 4, wherein, The license plate classification model is trained based on the following steps: Obtain a sample video surveillance image, and perform image difference processing on the sample video surveillance image to obtain a sample difference feature map; Extract the sample optical flow feature points of the sample license plate in each of the sample video surveillance images, and determine the optical flow features of each of the sample optical flow feature points to obtain the sample optical flow feature information; Train the initial license plate classification model based on the sample difference feature map, the sample optical flow feature information, and the license plate classification label corresponding to the sample difference feature map to obtain the license plate classification model.

7. The method for determining genuine and fake license plates according to any one of claims 1 to 6, wherein, The performing image difference processing on the video surveillance image to obtain a difference feature map includes: Perform grayscale processing on the video surveillance image to obtain a grayscale image; Perform image difference processing on the grayscale image, and perform binarization processing on the image difference processing result to obtain the difference feature map.

8. A device for determining genuine and fake license plates, wherein, It includes: An image acquisition module, configured to acquire a video surveillance image containing a license plate collected by a barrier gate camera; A difference processing module, configured to perform image difference processing on the video surveillance image to obtain a difference feature map; A genuine and fake license plate determination module, configured to determine the genuine and fake recognition result of the license plate based on the difference feature map and the license plate classification model; The license plate classification model is used to recognize the genuineness of the license plate based on the difference feature map.

9. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the computer program, the method for determining genuine and fake license plates according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, on which a computer program is stored, wherein, When the computer program is executed by a processor, the method for determining genuine and fake license plates according to any one of claims 1 to 7 is implemented.

Images