A dynamic vision sensor-based license plate detection binocular recognition method and system

Abstract

The application discloses a license plate detection binocular recognition method based on a dynamic visual sensor, and comprises the following steps: step 10, a CMOS image sensor outputs a color image frame set, and simultaneously, a dynamic visual sensor outputs an event information stream; step 20, the event information stream is analyzed and processed to obtain an event image frame set; step 30, a first convolutional neural network is used to detect the color image frame set and the event image frame set, and corresponding license plate region position information is output; step 40, a second convolutional neural network is used to recognize information in a region corresponding to the license plate region position information, and license plate information is output; and step 50, a license plate image and the license plate information are displayed. The application further discloses a license plate detection binocular recognition system based on a dynamic visual sensor. The application can guarantee a high target recognition accuracy, make up for the fact that a traditional camera cannot normally detect and recognize license plate information under strong inverse light, and improve imaging detection and recognition efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a license plate detection binocular recognition method and system based on a dynamic vision sensor. BACKGROUND

[0002] The automatic license plate recognition (ALPR) system of the expressway is used for recognizing vehicles from images containing license plates. One ALPR system can be divided into the following steps: image acquisition, license plate detection, and character recognition. The traditional image acquisition module is mainly composed of a CCD / COMS traditional camera, which is a key part of the ALPR system for license plate recognition. However, the traditional acquisition module cannot effectively extract, or in the worst case, cannot extract the license plate information, so that the license plate information cannot be correctly extracted, thereby making the recognition invalid.

[0003] The traditional CCD / COMS camera has a slow imaging speed (20-50 FPS) and a small dynamic range (40-60 dB) based on energy integral imaging, and often cannot acquire license plate information (produced by the small dynamic range) when the visibility is poor, especially in backlight, so that the monitoring and license plate recognition functions cannot be realized. At the same time, the traditional imaging system does not distinguish between foreground and background, has a large data load, and brings a huge pressure to the backend detection and recognition algorithm. In addition, the imaging environment on the current expressway is complex, the light interference is strong, and the target signal is weak, so additional light is often needed to improve the accuracy of image acquisition. However, the reflection ratio of light of different vehicle parts is greatly different, and the dynamic range of the traditional camera is small, which may cause the license plate to be covered or not to reach the imaging condition, and the license plate information cannot be effectively captured. There is a high requirement for the speed of vehicles on the expressway, and the imaging speed of the traditional optical camera is slow, which is easy to produce blurred images under high-speed conditions, greatly affecting the image acquisition of the license plate.

[0004] Siqi Li et al. in their published paper "Event Stream Super-Resolution via Spatiotemporal Constraint Learning" proposed an end-to-end framework of event stream super-resolution based on a spiking neural network. The framework can generate a high-resolution event stream from an input low-resolution event stream, and has achieved good results in object classification, image reconstruction and other applications. The deficiency of this method is that the spiking neural network used is difficult to train, has a large amount of calculation, and the target detection and recognition speed is slow, which cannot realize real-time target detection and recognition, and the detection efficiency is low. SUMMARY

[0005] In order to solve the above problems in the prior art, the application provides a license plate detection binocular recognition method and system based on a dynamic vision sensor. The technical problem to be solved by the application is solved by the following technical scheme:

[0006] The first aspect of the embodiment of the present application provides a dynamic visual sensor-based license plate detection binocular recognition method, comprising the following steps:

[0007] Step 10, a CMOS image sensor outputs a color image frame set of a collected license plate image, and a dynamic visual sensor outputs an event information stream of an acquired license plate image; wherein the event information stream indicates a signal of a gray scale information change of the license plate image;

[0008] Step 20, the event information stream is parsed and processed to obtain an event image frame set;

[0009] Step 30, a first convolutional neural network is used to detect the color image frame set and the event image frame set, and output corresponding first license plate region position information and second license plate region position information;

[0010] Step 40, a second convolutional neural network is used to recognize information in regions corresponding to the first license plate region position information and the second license plate region position information, and output license plate information;

[0011] Step 50, the license plate image and the license plate information are displayed according to the color image frame set, the event image frame set and the license plate information.

[0012] In an embodiment of the present application, the color image frame set is represented as: P={P l |0<l<L},P l represents the lth frame of the license plate image, and L represents the total number of frames of the license plate image;

[0013] The event information stream is represented as: S={S l |0<l<L},wherein S l represents the lth frame of the license plate image P l corresponding to the sub-event information stream, S l ={s l,r |(x l,r ,y l,r ,g l,r ,t l,r )},0<r<N l , wherein N l represents the number of events contained in the sub-event information stream, x l,r is the row address of the triggered event pixel, y l,r is the column address of the triggered event pixel, g l,r is the gray scale value, and t l,r is the time of the gray scale information change of s l,r .

[0014] In one embodiment of the present invention, step 20 includes:

[0015] Step 21, construct a zero matrix M = zeros(H, W); where H and W are the numbers of column pixels and row pixels of the dynamic vision sensor respectively;

[0016] Step 22, parse the event information stream into multiple events; where each event S l,t = {s l,r,t |(x l,r,t , y l,r,t , g l,r,t , t l,r,t )}, 0 < t < T, T represents the acquisition time of the event information stream; x l,r,t represents the row address information of each event pixel, y l,r,t represents the column address information of each event pixel, g l,r,t represents the gray-scale information of each event, t l,r,t represents the trigger time of each event;

[0017] Step 23, within the acquisition time T of the event information stream, fill each event S into the zero matrix M according to the number of events, to obtain the set of event image frames, and the set of event image frames is represented as: I = {I l |0 < l < L}.

[0018] In one embodiment of the present invention, step 30 includes:

[0019] Step 31, label the positions of the license plate areas in the vehicle sample image dataset to obtain the first image training set D1;

[0020] Step 32, construct the first convolutional neural network C1;

[0021] Step 33, input the first image training set D1 into the first convolutional neural network C1 for model training of license plate area position detection; the total loss value of the first convolutional neural network is:

[0022] Loss total = λ1L cls + λ2L obj + λ3L loc ; <00...​​​​​​​​​​​

[0024] Step 34, the gradient descent method is used to pass the total loss value Loss total The parameters of the first convolutional neural network C1 are updated, and the parameter update formula is:

[0025]

[0026]

[0027] Wherein, ω u Indicates the parameters of the updated convolutional layer, ω u ' +1 Indicates the parameters of the updated convolutional layer, θ u Indicates the parameters of the updated full connection layer, θ' u+1 Indicates the parameters of the updated full connection layer.

[0028] Step 35, the first convolutional neural network C1 is iteratively trained to obtain a detection convolutional neural network;

[0029] Step 36, the color image frame set and the event image frame set are input into the detection convolutional neural network, and the corresponding first license plate region position information and second license plate region position information are output.

[0030] In an embodiment of the present application, the step 40 comprises:

[0031] Step 41, the license plate information of the license plate sample image data set is labeled to obtain a second image training set D2;

[0032] Step 42, the second image training set D2 is subjected to tilt correction and rotation transformation;

[0033] Step 43, the second convolutional neural network C2 is constructed;

[0034] Step 44, the second image training set D2 after tilt correction and rotation transformation is input into the second convolutional neural network C2 for model training of license plate information recognition, to obtain an identification convolutional neural network;

[0035] Step 45, the images corresponding to the first license plate region position information and the second license plate region position information are input into the identification convolutional neural network, and license plate information result is output;

[0036] Step 46, it is judged whether the license plate information result satisfies result >= F, if yes, the license plate information is output, if not, step 36 is continuously executed;

[0037] Wherein, F represents the recognition confidence.

[0038] The second aspect of the embodiment of the present application provides a dynamic visual sensor-based license plate detection binocular recognition system, comprising:

[0039] The CMOS image sensor is configured to output a color image frame set of the collected license plate image.

[0040] The dynamic visual sensor is configured to output an event information stream of the acquired license plate image.

[0041] The processor is configured to analyze and process the event information stream to obtain an event image frame set.

[0042] The license plate image detection module is configured to detect the color image frame set and the event image frame set through a first convolutional neural network, and output corresponding first license plate region position information and second license plate region position information.

[0043] The license plate information recognition module is configured to recognize information in regions corresponding to the first license plate region position information and the second license plate region position information through a second convolutional neural network, and output license plate information.

[0044] The visualization module is configured to display an image of the license plate and the license plate information according to the color image frame set, the event image frame set and the license plate information.

[0045] The present application has the following advantages:

[0046] 1. The present application simultaneously uses a dynamic visual sensor to collect license plate image data, which can compensate for the limitations of a CMOS image sensor in terms of exposure time and dynamic range, effectively capturing license plate information in strong backlights such as the sun and car lights, and accurately capturing in high-speed state. Since the dynamic visual sensor only generates event pulses when the pixel points of the pixel array perceive changes in external light intensity, not only does it greatly reduce the amount of data, but it also greatly reduces power consumption, effectively reducing the amount of data input into the network and improving detection speed.

[0047] 2. The present application uses a high-performance binocular camera system that combines traditional high-speed imaging technology and bionic dynamic imaging technology, which can achieve time-space synchronous imaging, allowing the traditional high-speed camera and the bionic dynamic camera to perform information fusion on the imaging data of the same scene at the same time, thereby achieving high temporal resolution, high spatial resolution and large dynamic range imaging.

[0048] 3. The present application uses a convolutional neural network for license plate detection and recognition tasks, which can achieve higher computational efficiency on CPU or GPU, greatly reducing the amount of data compared to current pulse neural network processing of event data, making the system faster and more efficient in detection and recognition, and reducing system power consumption.

[0049] The application will be described in further detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A flowchart of a license plate detection binocular recognition method based on a dynamic visual sensor is provided for an embodiment of the application. DETAILED DESCRIPTION

[0051] The application will be described in further detail below with reference to the drawings and embodiments.

[0052] Embodiment one

[0053] As shown in the figure, the first aspect of the embodiment of the application provides a license plate detection binocular recognition method based on a dynamic visual sensor, including the following steps: Figure 1

[0054] Step 10: The CMOS image sensor outputs a color image frame set of the collected license plate image, and at the same time, the dynamic visual sensor outputs an event information stream of the acquired license plate image. The event information stream indicates the signal of the gray scale information change of the license plate image.

[0055] In this embodiment, the CMOS image sensor collects the license plate image after focusing of the lens and outputs a color image frame set, the dynamic visual sensor is used for sensing the gray scale information change of the license plate position and outputs an event information stream, and the lens used is a visible light fixed focus lens. The license plate image is a vehicle image containing a license plate.

[0056] In this embodiment, the color image frame set is represented as: P = {P l |0 < l < L}, P l represents the lth frame license plate image, and L represents the total number of frames of the license plate image;

[0057] The event information stream is represented as: S = {S l |0 < l < L}, wherein each frame of the license plate image corresponds to a piece of sub-event information stream S l , the license plate image has L frames, the dynamic visual sensor outputs L pieces of sub-event information stream, and the set of the L pieces of sub-event information stream S l is the event information stream S. S l represents the lth frame license plate image P l corresponding sub-event information stream, S l = {s l,r |(x l,r ,y l,r ,g l,r ,t l,r )}, 0 < r < N l ​wherein N l represents the number of events contained in the sub-event information stream, x l,r is the row address of the triggered event pixel, y l,r is the column address of the triggered event pixel, g l,r is the gray value, t l,r is s l,r The time of gray information change.

[0058] In a feasible implementation, the photosensitive column pixels of the dynamic vision sensor have H = 800 pixels, the row pixels have W = 1280 pixels, and the sensor has 4096 gray levels, so that 0 < x l,r < W, 0 < y l,r < H, 0 < g l,r < 4096.

[0059] Step 20, the event information stream is parsed and processed to obtain an event image frame set. The event information stream output by the dynamic vision sensor is parsed into multiple events containing address information, gray information and triggering time of the event, and an event image frame is generated through data compression.

[0060] Specifically, step 20 includes steps 21-23:

[0061] Step 21, a zero matrix M = zeros(H, W) is constructed.

[0062] Wherein H and W are the number of column pixels and row pixels of the dynamic vision sensor respectively.

[0063] Step 22, the event information stream is parsed into multiple events. Each piece of sub-event information stream of the event information stream is parsed to obtain N l events S l,t .

[0064] Wherein each event S l,t = {s l,r,t |(x l,r,t , y l,r,t , g l,r,t , t l,r,t )}, 0 < t < T, T represents the collection time of the event information stream; x l,r,t represents the row address information of each event pixel, y l,r,t represents the column address information of each event pixel, g l,r,t represents the gray information of each event, t l,r,t represents the triggering time of each event.

[0065] Step 23, according to the number of events within the collection time T of the event information stream, each event S l,tfilled into the all-zero matrix M, to obtain an event image frame set, denoted as: I = {I l |0<l<L}.

[0066] Step 30, detecting the color image frame set and the event image frame set by the first convolutional neural network, and outputting corresponding first license plate region position information and second license plate region position information.

[0067] Specifically, step 30 includes steps 31-36:

[0068] Step 31, constructing a training set: labeling the position of the license plate region of the vehicle sample image data set, taking the coordinates of the upper left corner and the lower right corner of the license plate region as the label value, to obtain a first image training set D1;

[0069] Step 32, constructing a first convolutional neural network C1;

[0070] Step 33, inputting the first image training set D1 into the first convolutional neural network C1 for model training of license plate region position detection; the total loss value of the first convolutional neural network is:

[0071] Loss total = λ1L cls + λ2L obj + λ3L loc ;

[0072] Wherein, L cls is a detection classification loss, L obj is a confidence loss of the license plate region position, L loc is a positioning loss of the license plate region position, and λ1, λ2 and λ3 are different weight values.

[0073] Step 34, updating the parameters of the first convolutional neural network C1 by the gradient descent method through the total loss value Loss total , and the parameter update formula is:

[0074]

[0075]

[0076] Wherein, ω u represents the parameters of the convolutional layer before updating, ω u ' +1 represents the parameters of the convolutional layer after updating, θ u represents the parameters of the fully connected layer before updating, and θ' u+1 represents the parameters of the fully connected layer after updating.

[0077] Step 35, iteratively train the first convolutional neural network C1 to obtain a detection convolutional neural network. In this step, the maximum number of iterations is E = 1000. The detection convolutional neural network can quickly and effectively detect the position of the license plate region.

[0078] Step 36, input the color image frame set and the event image frame set into the detection convolutional neural network, and output corresponding first license plate region position information and second license plate region position information.

[0079] The first license plate region position information is the detection result corresponding to the color image frame set, and the second license plate region position information is the detection result corresponding to the event image frame set. The license plate region position information is expressed in the form of coordinates.

[0080] Step 40, identify the information in the region corresponding to the first license plate region position information and the second license plate region position information through the second convolutional neural network, and output the license plate information. Through the second convolutional neural network, the characters in the detected license plate region are recognized, and the recognition result, i.e., the license plate information, is output. The license plate information is the license plate number.

[0081] Specifically, step 40 includes steps 41-46:

[0082] Step 41, label the license plate information of the license plate sample image dataset to obtain a second image training set D2;

[0083] Step 42, perform tilt correction and rotation transformation on the second image training set D2;

[0084] Step 43, construct a second convolutional neural network C2;

[0085] Step 44, input the second image training set D2 after tilt correction and rotation transformation into the second convolutional neural network C2, and perform model training for license plate information recognition using CTC (Connectionist Temporal Classification) Loss to calculate the loss value, to obtain an identification convolutional neural network;

[0086] Step 45, input the images corresponding to the first license plate region position information and the second license plate region position information into the identification convolutional neural network respectively, and output the recognition result result, i.e., the license plate information;

[0087] Step 46, determine whether the recognition result result satisfies result >= F. If yes, output the recognition result (license plate information); if not, continue to execute step 36.

[0088] Wherein, F represents the recognition confidence.

[0089] Step 50, displaying the license plate image and the license plate information according to the color image frame set, the event image frame set and the license plate information. In this step, the image frame and the license plate information are visualized, and the imaging result and the recognition result are output.

[0090] In this step, the color image frame set P={P l |0<l<L} and the event image frame set I={I l |0<l<L} are processed to generate and display the license plate image, and the content of the license plate information is displayed.

[0091] The license plate detection binocular recognition method based on the dynamic visual sensor in the embodiment can guarantee high target recognition accuracy, make up for the inability of the traditional camera to normally detect and recognize license plate information under strong back light, and improve imaging detection and recognition efficiency.

[0092] The dynamic visual sensor is used to collect license plate image data in the embodiment, which can make up for the limitation of the CMOS image sensor in exposure time and dynamic range, effectively capture license plate information under strong back light such as the sun and car light, and accurately capture in a high-speed state. Since the dynamic visual sensor generates an event pulse only when the pixel point of the pixel array senses the change of external light intensity, not only the data volume is greatly reduced, but also the power consumption is greatly reduced, the data volume input into the network is effectively reduced, and the detection speed is improved.

[0093] The high-performance binocular camera system fusing the traditional high-speed imaging technology and the bionic dynamic imaging technology is used in the embodiment, which can realize time-space synchronous imaging, so that the traditional high-speed camera and the bionic dynamic camera can perform information fusion on the imaging data of the same scene at the same time, thereby realizing high-time resolution, high-space resolution and large dynamic range imaging.

[0094] The convolutional neural network is used for the license plate detection and recognition task in the embodiment, which can have higher calculation efficiency of CPU or GPU, greatly reduce the data volume compared with the current pulse neural network processing event data, make the system have faster detection and recognition speed and higher efficiency, and reduce the power consumption of the system.

[0095] Embodiment two

[0096] The second aspect of the embodiment of the application provides a license plate detection binocular recognition system based on a dynamic visual sensor, which comprises:

[0097] A CMOS image sensor is used to output a color image frame set of the collected license plate image;

[0098] A dynamic visual sensor is used to output an event information stream of the acquired license plate image;

[0099] a processor configured to parse and process the event information stream to obtain a set of event image frames;

[0100] a license plate image detection module configured to detect the set of color image frames and the set of event image frames by using a first convolutional neural network, and output corresponding first license plate region position information and second license plate region position information;

[0101] a license plate information recognition module configured to recognize information in regions corresponding to the first license plate region position information and the second license plate region position information by using a second convolutional neural network, and output license plate information;

[0102] a visualization module configured to display license plate images and license plate information according to the set of color image frames, the set of event image frames, and the license plate information.

[0103] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", and the like indicate the orientation or positional relationship shown in the drawings, and are only used for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0104] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features referred to. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0105] In the present application, unless otherwise specifically defined and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be broadly understood, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0106] In the present application, unless specifically stated and limited otherwise, the first feature "on" or "under" the second feature can include that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, the first feature "on", "above" and "over" the second feature includes that the first feature is directly above and obliquely above the second feature, or only means that the first feature is higher than the second feature in horizontal height. The first feature "under", "below" and "underneath" the second feature includes that the first feature is directly below and obliquely below the second feature, or only means that the first feature is lower than the second feature in horizontal height.

[0107] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and integrate different embodiments or examples described in the present application.

[0108] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For those skilled in the art, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be considered as falling within the protection scope of the present application.

Claims

1. A dynamic vision sensor-based license plate detection binocular recognition method, characterized in that, The method comprises the following steps: Step 10, the CMOS image sensor outputs a color image frame set of the collected license plate image, and meanwhile, the dynamic vision sensor outputs an event information stream of the acquired license plate image; wherein the event information stream indicates signals of gray information changes of the license plate image; The color image frame set is represented as: , represents the first frame license plate image, represents the total number of frames of the license plate image; The event information flow is represented as: wherein, represents the first frame license plate image corresponding sub-event information flow, , wherein, represents the number of events contained in the sub-event information flow, is the row address of the triggered event pixel, is the column address of the triggered event pixel, is the gray value, is the time of the change in gray information; Step 20, the event information stream is parsed and processed to obtain an event image frame set; The step 20 comprises: Step 21, constructing all-zero matrix ; wherein, are the number of column pixels and row pixels of the dynamic vision sensor, respectively. Step 22, parsing the event information stream into a plurality of events; wherein each event , , represents the acquisition time of the event information stream; represents the row address information of each event pixel, represents the column address information of each event pixel, represents the gray scale information of each event, represents the trigger time of each event; Step 23, filling the events into the all-zero matrix according to the number of events in the event information stream at the capture time of the event information stream Step 24, obtaining the event image frame set according to the event image frame set Step 25, filling the events into the all-zero matrix according to the number of events in the event information stream at the capture time of the event information stream Step 26, obtaining the event image frame set according to the event image frame set ; Step 30, the color image frame set and the event image frame set are detected by a first convolutional neural network to output corresponding first license plate region position information and second license plate region position information; Step 40, information in regions corresponding to the first license plate region position information and the second license plate region position information is identified by a second convolutional neural network to output license plate information; Step 50, the license plate image and the license plate information are displayed according to the color image frame set, the event image frame set and the license plate information.

2. The dynamic vision sensor based license plate detection binocular recognition method according to claim 1, wherein, The step 30 comprises: Step 31, labeling the position of the license plate region of the vehicle sample image data set to obtain a first image training set ; Step 32, constructing the first convolutional neural network ; Step 33, training the first image set inputting the first convolutional neural network a model for detecting the position of the license plate region; and a total loss value of the first convolutional neural network is: ； wherein, is a detection classification loss, is a confidence loss for the position of the license plate region, is a positioning loss for the position of the license plate region, are different weight values, respectively; Step 34, updating parameters of the first convolutional neural network by using gradient descent method through the total loss value updating parameters of the first convolutional neural network ​ wherein, denotes the parameters of the convolutional layer before update, denotes the parameters of the convolutional layer after update, denotes the parameters of the fully connected layer before update, denotes the parameters of the fully connected layer after update; Step 35, iteratively training the first convolutional neural network to obtain a detection convolutional neural network; Step 36, the color image frame set and the event image frame set are input into the detection convolutional neural network to output the first license plate region position information and the second license plate region position information. 3.The dynamic vision sensor based license plate detection binocular recognition method of claim 1, wherein, The step 40 comprises: Step 41, labeling the license plate information of the license plate sample image dataset to obtain a second image training set ; Step 42, training the second image set performing tilt correction and rotation transformation; Step 43, constructing the second convolutional neural network ; Step 44, training the second image set after inclination correction and rotation transformation inputting the second convolutional neural network model training for license plate information recognition, to obtain an identification convolutional neural network Step 45, input the image corresponding to the first license plate region position information and the second license plate region position information into the recognition convolutional neural network respectively, and output license plate information ; Step 46, judging whether the license plate information satisfies , if yes, outputting the license plate information, if not, continuing to execute step 36; wherein, represents a recognition confidence.

4. A dynamic vision sensor based dual-view recognition system for license plate detection, characterized in that, The system for implementing the method for license plate detection and binocular recognition based on a dynamic vision sensor according to any one of claims 1-3 comprises: The CMOS image sensor is configured to output a color image frame set of a collected license plate image; The dynamic vision sensor is configured to output an event information stream of an acquired license plate image; The processor is configured to parse and process the event information stream to obtain an event image frame set; The license plate image detection module is configured to detect the color image frame set and the event image frame set by a first convolutional neural network to output corresponding first license plate region position information and second license plate region position information; The license plate information identification module is configured to identify information in regions corresponding to the first license plate region position information and the second license plate region position information by a second convolutional neural network to output license plate information; The visualization module is configured to display a license plate image and the license plate information according to the color image frame set, the event image frame set and the license plate information.

Images