Image processing method for license plate recognition and camera

Abstract

The invention relates to an image processing method for license plate recognition and a camera, and the method comprises the steps: configuring an image sensor into a double-gain mode, and employing different gain modes for two adjacent frames; calculating a fusion region according to the gain ratio; reading the data of two adjacent frames collected by the image sensor, and combining the brightness maximum value and the brightness minimum value corresponding to the fusion region; carrying out license plate ROI region identification on the fused data; taking the target brightness as a brightness adjustment target of the license plate ROI region of the fused data, calculating an exposure parameter of an image sensor through an automatic exposure algorithm, and setting the image sensor through the calculated exposure parameter; and updating the gain ratio based on the calculation result of the automatic exposure algorithm, and fusing two adjacent frames of data acquired next time. According to the invention, external environment interference is reduced, and the requirement of license plate recognition in multiple scenes can be met.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method and a camera for license plate recognition. Background Art

[0002] Night license plate recognition uses the principle of light reflection. After the infrared light is reflected at the license plate, the reflected license plate information is transmitted back to the camera. The camera combines image processing technology to restore clear license plate information. Currently, there are many solutions for night license plate recognition. While they solve some problems, they also have the following problems:

[0003] (1) Based on single automatic gain solution: This is the most common solution at present. After the camera receives the signal, the AE algorithm calculates the exposure time and gain and sends it to the sensor. This solution relies heavily on the intensity of infrared light and external scene light. It is not compatible with different backlighting scenes of near / far headlights, nor is it compatible with license plates of different reflective materials.

[0004] (2) Based on the dual automatic exposure control scheme: The synthesis of two frames with long / short exposure time is used to solve the problem of being incompatible with different backlighting scenes of near / far headlights and license plates with different reflective materials. However, this scheme introduces the problem of license plate blur when driving at high speed, because the longer the exposure time, the more serious the motion blur caused by fast-moving objects.

[0005] (3) Based on single automatic gain and infrared strobe light solution: By changing the intensity of infrared light, the camera can more compatibly collect information about license plates that are too dark or too bright, solving the compatibility issues of license plates made of different reflective materials, wide dynamic range of low / high beam lights, and blurred license plates at high speeds. However, this solution introduces the phenomenon of overexposure / underexposure of some frames, requiring the algorithm layer to screen the best license plate effects of frames with different brightness, which also affects the service life of the infrared light.

[0006] (4) Based on a two-sensor single automatic gain solution: Two image sensors are used, one with a smaller exposure and gain to mainly collect images of vehicles with high beams / strong license plates, and the other with a larger exposure time and gain to mainly collect images of vehicles with low beams / weak license plates / no reflective license plates. This solution solves the compatibility problem in various scenarios, but the addition of an image sensor increases costs and requires the reconstruction of the hardware structure. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes an image processing method and a camera for license plate recognition.

[0008] The specific plan is as follows:

[0009] An image processing method for license plate recognition comprises the following steps:

[0010] S1: The image sensor is configured in dual gain mode, and two adjacent frames use different gain modes;

[0011] S2: Initialize the gain ratio, fusion times k=1, and target brightness;

[0012] S3: Calculate the maximum brightness and the minimum brightness corresponding to the fusion area according to the gain ratio;

[0013] S4: reading the data of the 2k-1th frame and the 2kth frame collected by the image sensor, and fusing the data of the 2k-1th frame and the 2kth frame in combination with the maximum brightness value and the minimum brightness value corresponding to the fusion area;

[0014] S5: Identify the license plate ROI area on the fused data;

[0015] S6: taking the target brightness as the brightness adjustment target of the license plate ROI area of ​​the fused data, calculating the exposure parameters of the image sensor through the automatic exposure algorithm, and setting the image sensor according to the calculated exposure parameters;

[0016] S7: updating the gain ratio based on the calculation result of the automatic exposure algorithm;

[0017] S8: Increment k by 1 and return to S3.

[0018] Furthermore, the gain ratio is initially set to 4:1.

[0019] Furthermore, the maximum brightness value and the minimum brightness value corresponding to the fusion area are calculated according to the ratio of the preset brightness range to the gain ratio.

[0020] Furthermore, the frames using the HCG mode are used as high-gain frames, and the frames using the LCG mode are used as low-gain frames; when performing fusion, low-gain frame data is used for pixels whose brightness is greater than the maximum brightness value corresponding to the fusion area, and high-gain frame data is used for pixels whose brightness is less than the minimum brightness value corresponding to the fusion area. For other areas, high-gain frame data and low-gain frame data are superimposed.

[0021] An image processing camera for license plate recognition comprises an image sensor, a processor, a memory and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described above in the embodiment of the present invention are implemented.

[0022] The present invention adopts the above technical solution and uses a single image sensor to achieve dual automatic gain, which has the following beneficial effects:

[0023] (1) It is implemented based on a single image sensor, without adding hardware costs;

[0024] (2) The automatic exposure algorithm mainly counts the brightness information of the current image based on the license plate ROI area, increases the exposure weight of the license plate ROI area, and reduces interference from the external environment.

[0025] (3) While ensuring that the exposure time does not cause motion blur, the applicability of the scene is improved by automatically controlling two different gain values. Targeted optimization can be performed for different environments to meet the needs of license plate recognition in multiple scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Shown is a flow chart of Embodiment 1 of the present invention.

[0027] Figure 2 The figure is a schematic diagram of the weight setting of the license plate region of interest in the automatic exposure algorithm in this embodiment. DETAILED DESCRIPTION

[0028] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0029] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0030] Embodiment 1:

[0031] The embodiment of the present invention provides an image processing method for license plate recognition, such as Figure 1 As shown, the method comprises the following steps:

[0032] S1: The image sensor is configured in dual gain mode, and two adjacent frames use different gain modes.

[0033] In this embodiment, the configuration of the dual gain mode is performed in the sensor DCG, and the odd frame is set to use the HCG (Hight Conversion Gain) mode and the even frame is set to use the LCG (Low Conversion Gain) mode, and both frames use the same exposure time. The frame using the HCG mode is used as a high gain frame, and the frame using the LCG mode is used as a low gain frame.

[0034] S2: Initialize the gain ratio, fusion times k=1, and target brightness.

[0035] The gain ratio is the ratio of the gain of the high-gain frame to the gain of the low-gain frame, and is initially set to 4:1 in this embodiment. Based on this gain ratio, when the gain of the high-gain frame is 16x, the gain of the low-gain frame is 4x.

[0036] The target brightness is the brightness that the user wants the license plate ROI area to reach, and can be set according to needs.

[0037] S3: Calculate the maximum brightness and the minimum brightness corresponding to the fusion area according to the gain ratio.

[0038] The maximum brightness and minimum brightness corresponding to the fusion zone are calculated according to the ratio of the preset brightness range to the gain ratio, specifically the ratio of the upper and lower limits corresponding to the brightness range divided by the gain ratio. In this embodiment, since the maximum brightness is 4096 and the minimum brightness is 0, the brightness range is set to 1024-3072 based on the brightness turning point. Therefore, the upper limit corresponding to the brightness range is 3072, and the maximum brightness corresponding to the fusion zone is 3072 / 4=768; the lower limit corresponding to the brightness range is 1024, and the minimum brightness corresponding to the fusion zone is 10242 / 4=256. Those skilled in the art can set the above brightness range according to actual needs, and there is no limitation here.

[0039] S4: Read the data of the 2k-1th frame and the 2kth frame, and fuse the data of the 2k-1th frame and the 2kth frame in combination with the maximum brightness value and the minimum brightness value corresponding to the fusion area.

[0040] The fusion of high-gain frame and low-gain frame data can be achieved through step S4. In this embodiment, when fusion is performed, low-gain frame data is used for pixel points whose brightness is greater than the maximum brightness corresponding to the fusion area (overexposed area), high-gain frame data is used for pixel points whose brightness is less than the minimum brightness corresponding to the fusion area (underexposed area), and high-gain frame data and low-gain frame data are superimposed for other areas, thereby achieving the technical effect of being compatible with different dynamic range scenes and license plates of different reflective materials.

[0041] S5: Identify the license plate ROI (region of interest) on the fused data.

[0042] The license plate ROI area recognition can be performed using existing algorithms, which are not limited here.

[0043] S6: Using the target brightness as a brightness adjustment target for the license plate ROI area of ​​the fused data, calculating exposure parameters of the image sensor through an automatic exposure (AE) algorithm, and setting the image sensor through the calculated exposure parameters.

[0044] The automatic exposure algorithm counts the brightness of the license plate ROI area of ​​the fused data, and calculates the deviation between the counted brightness and the target brightness. In order to ensure the clarity of high-speed vehicles, the exposure time cannot be set too large. It is necessary to set a certain gain ratio between the high-gain frame and the low-gain frame, and then obtain the exposure time of the compensated brightness and the gain values ​​of the high-gain frame and the low-gain frame respectively. The exposure time and the gain values ​​of the high-gain frame and the low-gain frame are sent to the image sensor to form a closed loop. The above exposure parameters include the exposure time and the gain values ​​of the high-gain frame and the low-gain frame respectively.

[0045] When the automatic exposure algorithm performs brightness statistics on the license plate ROI area, it passes the license plate ROI area to the AE algorithm. The AE algorithm divides the image into 32x32 areas and manually or automatically weights the area based on the license plate ROI area. At this time, the brightness statistics of the image are mainly based on the license plate ROI area. Figure 2 shown.

[0046] S7: updating the gain ratio based on the calculation result of the automatic exposure algorithm.

[0047] The exposure parameters calculated in step S6 include the gain values ​​of the high-gain frame and the low-gain frame respectively. The gain ratio is the ratio of the gain value of the high-gain frame to the gain value of the low-gain frame. The gain ratio parameter is updated by this ratio for calculating the maximum brightness and the minimum brightness corresponding to the fusion area during the next fusion.

[0048] S8: Increment k by 1 and return to S3.

[0049] After identifying the license plate ROI area, the embodiment of the present invention uses an automatic exposure algorithm to calculate the deviation between the current brightness of the license plate ROI area and the target brightness to obtain the gain value, and then generates a high-gain frame and a low-gain frame through a certain gain ratio. Finally, the high-gain frame and the low-gain frame are fused to obtain an effect with good compatibility with various environments.

[0050] Through the processing of this embodiment, an image with balanced brightness can be obtained, and then the appropriate contrast effect can be adjusted by digital wide dynamic, the image can be sharpened to extract the license plate outline, the sharpening intensity of large edges can be enhanced, the sharpening intensity of detail texture areas and platform areas can be weakened, and gauss noise can be removed through spatiotemporal noise reduction. Finally, an image effect with a clear license plate, balanced brightness and a clean picture can be obtained. Finally, the obtained license plate image is used for license plate recognition.

[0051] Embodiment 2:

[0052] The present invention also provides a camera for image processing for license plate recognition, comprising an image sensor, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned method embodiment of the first embodiment of the present invention when executing the computer program.

[0053] In this embodiment, the processor includes a DSP (Digital Signal Processing) and an ISP (Image Signal Processing), wherein the DSP receives and distinguishes high-gain frame data and low-gain frame data through the virtual channel (VC) of the MIPI interface, and then sends the received data to the ISP, which performs operations such as data fusion and license plate ROI area recognition.

[0054] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. An image processing method for license plate recognition, characterized in that: The following steps are involved: S1: The image sensor is configured in dual gain mode, and two adjacent frames use different gain modes; S2: Initialize the gain ratio, fusion times k=1, and target brightness; S3: Calculate the maximum brightness and the minimum brightness corresponding to the fusion area according to the gain ratio; S4: reading the data of the 2k-1th frame and the 2kth frame collected by the image sensor, and fusing the data of the 2k-1th frame and the 2kth frame in combination with the maximum brightness value and the minimum brightness value corresponding to the fusion area; S5: Identify the license plate ROI area on the fused data; S6: taking the target brightness as the brightness adjustment target of the license plate ROI area of ​​the fused data, calculating the exposure parameters of the image sensor through the automatic exposure algorithm, and setting the image sensor according to the calculated exposure parameters; S7: updating the gain ratio based on the calculation result of the automatic exposure algorithm; S8: Increment k by 1 and return to S3.

2. The image processing method for license plate recognition according to claim 1, characterized in that: The initial setting gain ratio is 4:

1.

3. The image processing method for license plate recognition according to claim 1, characterized in that: The maximum brightness value and the minimum brightness value corresponding to the fusion area are calculated according to the ratio of the preset brightness range to the gain ratio.

4. The image processing method for license plate recognition according to claim 1, characterized in that: The frames using the HCG mode are used as high-gain frames, and the frames using the LCG mode are used as low-gain frames. When performing fusion, low-gain frame data is used for pixels whose brightness is greater than the maximum brightness value corresponding to the fusion area, and high-gain frame data is used for pixels whose brightness is less than the minimum brightness value corresponding to the fusion area. For other areas, high-gain frame data and low-gain frame data are superimposed.

5. A camera for image processing for license plate recognition, characterized in that: The method comprises an image sensor, a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the methods as claimed in claims 1 to 4 when executing the computer program.

Images