An image processing method, apparatus, electronic device, and storage medium

Abstract

This application provides an image processing method, apparatus, electronic device, and storage medium, relating to the field of image processing. For the license plate area of ​​a target vehicle, two license plate area images with different brightness are obtained, and the two images are fused according to fusion weights corresponding to different positions of the license plate areas to obtain a target license plate image. The fused license plate image is easier to recognize and has better visual effects. The method includes: obtaining a first license plate area image and a second license plate area image with different brightness for the license plate area of ​​the target vehicle; the brightness of the first license plate area image is greater than the brightness of the second license plate area image; and fusing the first license plate area image and the second license plate area image according to fusion weights corresponding to different positions of the license plate areas to obtain the target license plate image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to an image processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] As an important part of image recognition in the field of traffic management, license plate recognition is widely used in parking lot charge management, factory vehicle access management, and highway toll management.

[0003] However, due to the influence of the scene, weather, and light compensation, the license plate region in the image is prone to overexposure, which may cause the font of the license plate to become lighter, the font to become thicker, and the characters to have serious jaggedness. If the license plate region in the image is too dark, the edge of the license plate may be blurred or even disappear. These problems may result in poor visual effects of the license plate and further cause the recognition effect of the license plate to be less than ideal. SUMMARY

[0004] The present application provides an image processing method and device, an electronic device, and a storage medium. For the license plate region of a target vehicle, two license plate region images with different brightness are obtained, and the two images are fused according to the fusion weight corresponding to the different positions of the license plate region to obtain a target license plate image. The license plate image obtained after fusion is easier to recognize, and the visual effect of the image is better.

[0005] In a first aspect, the present application provides an image processing method, which includes: obtaining a first license plate region image and a second license plate region image with different brightness for the license plate region of a target vehicle; the brightness in the first license plate region image is greater than the brightness in the second vehicle region image; and fusing the first license plate region image and the second license plate region image according to the fusion weight corresponding to the different positions of the license plate region to obtain a target license plate image.

[0006] It can be understood that, generally, when the brightness of the license plate image is large, the license plate edge is clear, but the license plate font part may appear overexposure, and the license plate font will appear thicker compared to the normal font; and when the brightness of the license plate image is small, the license plate font is thin, but the license plate edge will appear more blurred compared to the normal license plate edge. Therefore, for the target vehicle, a first license plate region image with large brightness and a second license plate region image with small brightness are obtained. The license plate edge of the first license plate image with large brightness is relatively clear, but the license plate font is thick and blurred, and the license plate font of the second license plate image with small brightness is relatively clear, but the license plate edge is relatively blurred. Then, the first license plate region image and the second license plate region image are fused according to the fusion weights corresponding to the different positions of the license plate region, for example, the fusion weight corresponding to the edge position is set to the fusion proportion of the first license plate region image being greater than the fusion proportion of the second license plate region image, and the fusion weight corresponding to the font position is set to the fusion proportion of the first license plate region image being less than the fusion proportion of the second license plate region image, so that the fusion obtains a target license plate image with clear edge and clear font.

[0007] The method of the embodiment can solve the problems of unclear and thick license plate font and unclear license plate edge in the vehicle image. The license plate image obtained by the method is easier to recognize, and the visual effect of the image is better.

[0008] In addition, in the case of overexposure of the license plate image, the method of the embodiment can still process the overexposed image to obtain a target license plate image with clear license plate font and clear license plate edge.

[0009] In some embodiments, the above fusion of the first license plate region image and the second license plate region image according to the fusion weights corresponding to the different positions of the license plate region to obtain the target license plate image includes: fusing the first license plate region image and the second license plate region image according to the fusion weight corresponding to the edge position of the license plate region to obtain a target license plate image with clear edge; and the fusion weight corresponding to the edge position indicates that the fusion proportion of the first license plate region image is greater than the fusion proportion of the second license plate region image.

[0010] In some embodiments, the above fusion of the first license plate region image and the second license plate region image according to the fusion weights corresponding to the different positions of the license plate region to obtain the target license plate image includes:

[0011] fusing the first license plate region image and the second license plate region image according to the fusion weight corresponding to the font position of the license plate region to obtain a target license plate image with clear font; and the fusion weight corresponding to the font position indicates that the fusion proportion of the first license plate region image is less than the fusion proportion of the second license plate region image.

[0012] In some embodiments, the above-mentioned method of fusing the first license plate region image and the second license plate region image according to the fusion weights corresponding to different positions of the license plate region to obtain a target license plate image includes: obtaining a weight factor of the target license plate image based on the height of the license plate of the target vehicle; the weight factor represents the degree to which the pixel value of the target license plate image changes with the height of the license plate; filtering the first license plate region image and the second license plate region image respectively to obtain a first filtered image and a second filtered image; determining the fusion weights for image fusion based on the weight factor, the RGB parameters of the first filtered image and the RGB parameters of the second filtered image; and fusing the first license plate region image and the second license plate region image according to the fusion weights to obtain a target license plate image with optimized license plate font clarity and license plate edge clarity.

[0013] In some embodiments, filtering the first license plate area image to obtain a first filtered image includes: performing RGB maximum value channel filtering on the first license plate image to obtain a first processed image; RGB maximum value channel filtering is used to obtain the first processed image corresponding to the channel with the largest pixel value among the three color channels of RGB in the image; performing neighborhood maximum value filtering on the first processed image to obtain a first filtered image after target feature region enhancement, the target feature region including a rivet region, and the neighborhood maximum value filtering includes arranging the pixel values ​​in the neighborhood of the first processed image with a preset value as the radius in an orderly manner, and replacing the center pixel value with the sorted maximum pixel value.

[0014] In some embodiments, filtering the second license plate region image to obtain a second filtered image includes: performing RGB maximum value channel filtering on the second license plate image to obtain a second processed image; RGB maximum value channel filtering is used to obtain the second processed image corresponding to the channel with the largest pixel value among the three RGB color channels in the image; performing neighborhood maximum value filtering on the second processed image to obtain a first maximum value image, wherein neighborhood maximum value filtering includes arranging the pixel values ​​in the neighborhood of the second processed image in an ordered manner with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value; performing blur filtering on the second license plate image in a preset neighborhood with a preset value as the radius to obtain a second maximum value image with smoothed pixels in the preset neighborhood; and fusing the first maximum value image and the second maximum value image according to a weighting factor to obtain a second filtered image after target feature region enhancement.

[0015] In some embodiments, when the background color of the license plate of the target vehicle is any one of a green gradient, yellow, and yellow-green, the method includes: obtaining a brightness ratio between a first license plate area image and a second license plate area image; the brightness ratio being the ratio between the average brightness of the first license plate area image and the average brightness of the second license plate area image; converting the target license plate image into a grayscale image of the corresponding color according to the brightness ratio; and converting the grayscale image into a license plate optimized image with an optimized license plate background color according to the license plate background color.

[0016] In some embodiments, when the background color of the license plate of the target vehicle is a green gradient, the above-mentioned conversion of the grayscale image into a license plate optimized image based on the background color of the license plate includes: converting the grayscale image into a green license plate image based on the background color of the license plate; determining the color gradient coefficient based on the height of the license plate of the target vehicle; and converting the green license plate image into a license plate optimized image based on the color gradient coefficient.

[0017] In some embodiments, when the background color of the license plate of the target vehicle is yellow-green, the above-mentioned conversion of the grayscale image into a license plate optimized image based on the background color of the license plate includes: determining a color segmentation coefficient based on the height and width of the license plate of the target vehicle; and converting the grayscale image into a license plate optimized image based on the color segmentation coefficient and the background color of the license plate.

[0018] In some embodiments, obtaining first license plate area images and second license plate area images with different brightness for the license plate area of ​​a target vehicle includes: obtaining a first original image of the target vehicle; performing different image processing on the first original image when the exposure of the first original image is less than or equal to a preset exposure to obtain first vehicle images and second vehicle images with different brightness; obtaining a second original image of the target vehicle with an exposure less than or equal to a preset exposure when the exposure of the first original image is greater than the preset exposure; the second original image is obtained after the first original image is obtained, and the first original image is the first vehicle image and the second original image is the second vehicle image; and obtaining the first license plate area image in the first vehicle image and the second license plate area image in the second vehicle image.

[0019] In some embodiments, after converting the grayscale image into a license plate optimized image based on the license plate background color, the process includes: when the exposure of the first original image is less than or equal to a preset exposure, fusing the license plate optimized image with the license plate region of the first vehicle image to obtain a first target image; performing edge smoothing processing on the license plate region of the first target image to obtain a target vehicle image; when the exposure of the first original image is greater than the preset exposure, fusing the license plate optimized image with the license plate region of the second vehicle image to obtain a first target image; and performing edge smoothing processing on the license plate region of the first target image to obtain a target vehicle image.

[0020] Secondly, this application provides an image processing apparatus, which includes: an image acquisition unit, an image processing unit, and an image fusion unit.

[0021] The image acquisition unit can obtain a first license plate area image and a second license plate area image with different brightness for the license plate area of ​​the target vehicle; the brightness of the first license plate area image is greater than the brightness of the second license plate area image.

[0022] The image fusion unit can fuse the first license plate region image and the second license plate region image according to the fusion weights corresponding to different positions in the license plate region to obtain the target license plate image.

[0023] In some embodiments, the image processing unit can obtain a weighting factor for the target license plate image based on the height of the target vehicle's license plate; the weighting factor represents the degree to which the pixel value of the target license plate image changes with the height of the license plate; the image processing unit performs filtering processing on the first license plate region image and the second license plate region image respectively to obtain a first filtered image and a second filtered image; the image processing unit can also determine a fusion weight for image fusion based on the weighting factor, the RGB parameters of the first filtered image and the RGB parameters of the second filtered image; the image fusion unit can also fuse the first license plate region image and the second license plate region image according to the fusion weight to obtain a target license plate image with optimized license plate font clarity and license plate edge clarity.

[0024] In some embodiments, the image processing unit may further perform RGB maximum value channel filtering on the first license plate image to obtain a first processed image; RGB maximum value channel filtering is used to obtain the first processed image corresponding to the channel with the largest pixel value among the three color channels of RGB in the image; the image processing unit may further perform neighborhood maximum value filtering on the first processed image to obtain a first filtered image after target feature region enhancement, the target feature region including the rivet region, and the neighborhood maximum value filtering includes the operation of arranging the pixel values ​​in the neighborhood of the first processed image with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value.

[0025] In some embodiments, the image processing unit may further perform RGB maximum value channel filtering on the second license plate image to obtain a second processed image; RGB maximum value channel filtering is used to obtain the second processed image corresponding to the channel with the largest pixel value among the three color channels of RGB in the image; the image processing unit may further perform neighborhood maximum value filtering on the second processed image to obtain a first maximum value image, the neighborhood maximum value filtering includes arranging the pixel values ​​in the neighborhood of the second processed image in an ordered manner with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value; the image processing unit may further perform blur filtering on the second license plate image in a preset neighborhood with a preset value as the radius to obtain a second maximum value image with smooth pixels in the preset neighborhood; the image fusion unit may further fuse the first maximum value image and the second maximum value image according to a weight factor to obtain a second filtered image after target feature region enhancement.

[0026] In some embodiments, the image processing apparatus further includes a color correction unit.

[0027] In some embodiments, the image fusion unit can also obtain the brightness ratio between the first license plate area image and the second license plate area image; the brightness ratio is the ratio between the average brightness of the first license plate area image and the average brightness of the second license plate area image; the color correction unit can convert the target license plate image into a grayscale image of the corresponding color according to the brightness ratio; the color correction unit can also convert the grayscale image into an optimized license plate image with an optimized license plate background color according to the license plate background color.

[0028] In some embodiments, the color correction unit can also convert the grayscale image into a green license plate image based on the background color of the license plate; the image processing unit can also determine the color gradient coefficient based on the height of the license plate of the target vehicle; and the color correction unit can convert the green license plate image into an optimized license plate image based on the color gradient coefficient.

[0029] In some embodiments, the image processing unit may also determine the color segmentation coefficient based on the height and width of the license plate of the target vehicle; the color correction unit may convert the grayscale image into an optimized license plate image based on the color segmentation coefficient and the background color of the license plate.

[0030] In some embodiments, the image acquisition unit can also acquire a first original image of the target vehicle. If the exposure of the first original image is less than or equal to a preset exposure, the image processing unit can further perform different image processing on the first original image to obtain a first vehicle image and a second vehicle image with different brightness. If the exposure of the first original image is greater than the preset exposure, the image acquisition unit can also acquire a second original image of the target vehicle with an exposure less than or equal to the preset exposure. The second original image is acquired after the first original image is acquired, and the first original image is the first vehicle image, and the second original image is the second vehicle image. The system acquires a first license plate area image from the first vehicle image and a second license plate area image from the second vehicle image.

[0031] In some embodiments, the image fusion unit may also fuse the license plate optimized image with the license plate area of ​​the first vehicle image to obtain a first target image when the exposure of the first original image is less than or equal to a preset exposure; the image processing unit may also perform edge smoothing processing on the license plate area of ​​the first target image to obtain a target vehicle image.

[0032] In some embodiments, the image fusion unit may also fuse the license plate optimized image with the license plate area of ​​the second vehicle image to obtain a first target image if the exposure of the first original image is greater than the preset exposure. The image processing unit may also perform edge smoothing processing on the license plate area of ​​the first target image to obtain a target vehicle image.

[0033] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code including computer instructions; wherein, when the processor executes the computer instructions, it causes an image processing device to perform an image processing method as described in the first aspect and any possible design thereof.

[0034] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an image processing apparatus, cause the image processing apparatus to implement the method described in the first aspect.

[0035] Fifthly, this application provides a computer program product that, when run on an image processing apparatus, causes the image processing apparatus to perform the steps of the relevant method described in the first aspect above, so as to implement the method of the first aspect above.

[0036] The beneficial effects of the second to fifth aspects mentioned above can be referred to the corresponding description of the first aspect, and will not be repeated here. Attached Figure Description

[0037] Figure 1 A schematic diagram of an image processing system provided in this application;

[0038] Figure 2 A schematic diagram of the scene for collecting images of the target vehicle provided in this application;

[0039] Figure 3 A schematic diagram of the structure of an image processing device provided in this application;

[0040] Figure 4 A flowchart illustrating an image processing method provided in this application;

[0041] Figure 5 A flowchart illustrating yet another image processing method provided in this application;

[0042] Figure 6 A flowchart illustrating yet another image processing method provided in this application;

[0043] Figure 7 A flowchart illustrating yet another image processing method provided in this application;

[0044] Figure 8 A flowchart illustrating yet another image processing method provided in this application;

[0045] Figure 9 A flowchart illustrating yet another image processing method provided in this application;

[0046] Figure 10 A flowchart illustrating yet another image processing method provided in this application;

[0047] Figure 11 A flowchart illustrating yet another image processing method provided in this application;

[0048] Figure 12 A flowchart illustrating a color correction method provided in this application;

[0049] Figure 13 A flowchart illustrating yet another color correction method provided in this application;

[0050] Figure 14 A flowchart illustrating yet another color correction method provided in this application;

[0051] Figure 15 A flowchart illustrating yet another color correction method provided in this application;

[0052] Figure 16 A flowchart illustrating yet another color correction method provided in this application;

[0053] Figure 17 A flowchart illustrating yet another image processing method provided in this application;

[0054] Figure 18 A flowchart illustrating yet another image processing method provided in this application;

[0055] Figure 19 This is a schematic diagram of the hardware structure of an image processing device provided in this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0058] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0059] To facilitate understanding, a brief introduction to the relevant concepts involved in this application will be provided first.

[0060] 1. YUV format: YUV refers to a pixel format in which luminance and chrominance parameters are represented separately. "Y" represents luminance (Luminance or Luma), which is the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma).

[0061] 2. Red, Green, Blue (RGB) format: RGB format is a color standard format that uses variations in the three color channels (red (R), green (G), and blue (B)) and their superposition to obtain images in various color formats.

[0062] 3. Image Single Process Pipeline (ISPPIPE): Composed of a series of image processing modules, it mainly performs a series of image processing operations. A classic image processing pipeline mainly includes lens correction, color correction, followed by black level compensation (BLC), lens shading correction, bad pixel correction (BPC), demosaic interpolation (CIP), denoising (DNS), automatic white balance (AWB), color correction, gamma correction, color space conversion (RGB to YUV), and finally outputting data in YUV (or RGB) format.

[0063] 4. Unsharpen Mask (USM) is a commonly used algorithm for image sharpening in image convolution processing. This sharpening method first applies a Gaussian blur to the original image, then subtracts a coefficient from the original image multiplied by the Gaussian blur, and finally controls the value to the range of 0 to 255 RGB pixel values.

[0064] License plate recognition, as an important component of image recognition in the field of traffic management, is widely used in parking lot fee management, factory area vehicle access management, and highway toll management.

[0065] Because license plates typically use directional, retroreflective, and reflective materials—meaning they reflect light in the direction it's illuminated—they possess excellent reflectivity. Therefore, with appropriate supplemental lighting, capturing an image of the target vehicle easily places the license plate in the brightest area of ​​the image. However, due to the influence of the scene, weather, and supplemental lighting, license plates are very prone to overexposure. For example, overexposure of a blue license plate with white lettering can cause the blue lettering to appear lighter, thicker, and exhibit severe jagged edges. For new energy vehicle license plates with green gradients, yellow license plates, and yellow-green license plates for new energy buses, overexposure can lead to overexposed or even washed-out backgrounds. Conversely, an overly dark license plate area in the image can result in blurred or even disappeared edges. These issues all contribute to poor visual quality of the license plate, leading to less than ideal recognition results.

[0066] To address the aforementioned issues, some current solutions enhance the clarity of the license plate area's text when the image containing the license plate is overexposed. However, this results in thicker text, leading to suboptimal license plate recognition. Alternatively, other solutions adjust the color and brightness of the license plate area to address overexposed images, but this also leaves the text thick and unclear, resulting in unsatisfactory recognition performance.

[0067] To address this issue, this application provides an image processing method, apparatus, electronic device, and storage medium. It is known that, generally, when the brightness of a license plate image is high, the license plate edges are clear, but the font may be overexposed, and the font may appear thicker than normal fonts. Conversely, when the brightness of a license plate image is low, the license plate font is thinner, but the license plate edges appear blurry compared to normal license plate edges. Therefore, for a target vehicle, a first license plate area image with high brightness and a second license plate area image with low brightness are obtained. The first license plate image with high brightness has clearer license plate edges, but the font is thicker and blurrier; the second license plate image with low brightness has clearer license plate fonts, but the license plate edges are blurrier. Then, according to the fusion weights corresponding to different positions in the license plate area, the first license plate area image and the second license plate area image are fused. For example, the fusion weight corresponding to the edge position is set so that the fusion ratio of the first license plate area image is greater than that of the second license plate area image, and the fusion weight corresponding to the font position is set so that the fusion ratio of the first license plate area image is less than that of the second license plate area image. This results in a target license plate image with clear edges and clear fonts. The method in this embodiment can solve the problems of unclear and thick license plate fonts and unclear license plate edges in vehicle images. The license plate image obtained by this method is easier to recognize and has better visual effects.

[0068] In addition, even when the license plate image is overexposed, the method in this embodiment can still process the overexposed image to obtain a target license plate image with clear font and clear edges.

[0069] The license plates of the target vehicles in this application embodiment may include various types, such as blue background with white characters, yellow background with black characters, green gradient background with black characters, and yellow-green double-tone background with black characters.

[0070] The image processing method provided in this application embodiment can be applied to, for example... Figure 1 The image processing system shown. For example... Figure 1 As shown, the image processing system may include an image processing device 11 and an image capturing device 12.

[0071] The capturing device 12 can acquire images of the target vehicle in a real-world scenario. Examples include cameras installed at parking lot entrances, road checkpoints, highway toll stations, and overhead electronic cameras on city roads. For instance,... Figure 2 As shown, when the shooting device 12 is a snapshot camera, it can capture images of the front or rear of the target vehicle passing through the shooting area or target area.

[0072] The image processing device 11 can acquire an image of the target vehicle from the shooting device 12 and process the image of the target vehicle.

[0073] like Figure 3 As shown, the image processing device 11 includes an image acquisition unit 101, an image processing unit 102, an image fusion unit 103, and a color correction unit 104.

[0074] In some embodiments, the image acquisition unit 101 can acquire images of the target vehicle captured in the actual scene from the shooting device 12.

[0075] In some other embodiments, the image acquisition unit 101 may also read the image of the target vehicle from a database storing images acquired at historical time points. This embodiment does not impose any specific limitations.

[0076] The image acquisition unit 101 can acquire a first license plate area image and a second license plate area image with different brightness for the license plate area of ​​the target vehicle; the brightness in the first license plate area image is greater than the brightness in the second license plate area image.

[0077] The image fusion unit 103 can fuse the first license plate region image and the second license plate region image according to the fusion weights corresponding to different positions of the license plate region to obtain the target license plate image.

[0078] In some embodiments, the image fusion unit 103 can fuse the first license plate region image and the second license plate region image according to the fusion weight corresponding to the edge position of the license plate region to obtain a target license plate image with clear edges; the fusion weight corresponding to the edge position indicates that the fusion ratio of the first license plate region image is greater than the fusion ratio of the second license plate region image.

[0079] In some embodiments, the image fusion unit 103 can fuse the first license plate region image and the second license plate region image according to the fusion weight corresponding to the font position of the license plate region to obtain a target license plate image with clear font; the fusion weight corresponding to the font position indicates that the fusion ratio of the first license plate region image is less than the fusion ratio of the second license plate region image.

[0080] In some embodiments, the image processing unit 102 can obtain a weighting factor of the target license plate image based on the height of the target vehicle's license plate; the weighting factor represents the degree to which the pixel value of the target license plate image changes with the height of the license plate; the image processing unit 102 performs filtering processing on the first license plate region image and the second license plate region image respectively to obtain a first filtered image and a second filtered image; the image processing unit 102 can also determine a fusion weight for image fusion based on the weighting factor, the RGB parameters of the first filtered image and the RGB parameters of the second filtered image; the image fusion unit 103 can also fuse the first license plate region image and the second license plate region image according to the fusion weight to obtain a target license plate image with optimized license plate font clarity and license plate edge clarity.

[0081] In some embodiments, the image processing unit 102 may further perform RGB maximum value channel filtering on the first license plate image to obtain a first processed image; RGB maximum value channel filtering is used to obtain the first processed image corresponding to the channel with the largest pixel value among the three color channels of RGB in the image; the image processing unit 102 may further perform neighborhood maximum value filtering on the first processed image to obtain a first filtered image after target feature region enhancement, the target feature region including the rivet region, and the neighborhood maximum value filtering includes the operation of orderly arranging the pixel values ​​in the neighborhood of the first processed image with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value.

[0082] In some embodiments, the image processing unit 102 can further perform RGB maximum value channel filtering on the second license plate image to obtain a second processed image; RGB maximum value channel filtering is used to obtain the second processed image corresponding to the channel with the largest pixel value among the three color channels of RGB in the image; the image processing unit 102 can further perform neighborhood maximum value filtering on the second processed image to obtain a first maximum value image, the neighborhood maximum value filtering includes arranging the pixel values ​​in the neighborhood of the second processed image with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value; the image processing unit 102 can further perform blur filtering on the second license plate image with a preset neighborhood with a preset value as the radius to obtain a second maximum value image with smooth pixels in the preset neighborhood; the image fusion unit 103 can further fuse the first maximum value image and the second maximum value image according to the weight factor to obtain a second filtered image after target feature region enhancement.

[0083] In some embodiments, the image fusion unit 103 can also fuse the first license plate area image and the second license plate area image to obtain a target fused image; the image fusion unit 103 can also obtain the brightness ratio between the first license plate area image and the second license plate area image; the brightness ratio is the ratio between the average brightness of the first license plate area image and the average brightness of the second license plate area image; the color correction unit 104 can convert the target license plate image into a grayscale image of the corresponding color according to the brightness ratio; the color correction unit 104 can also convert the grayscale image into a license plate optimized image after the license plate background color is optimized according to the license plate background color.

[0084] In some embodiments, the color correction unit 104 can also convert the grayscale image into a green license plate image based on the background color of the license plate; the image processing unit 102 can also determine the color gradient coefficient based on the height of the license plate of the target vehicle; and the color correction unit 104 can convert the green license plate image into an optimized license plate image based on the color gradient coefficient.

[0085] In some embodiments, the image processing unit 102 may also determine the color segmentation coefficient based on the height and width of the license plate of the target vehicle; the color correction unit 104 may convert the grayscale image into an optimized license plate image based on the color segmentation coefficient and the background color of the license plate.

[0086] In some embodiments, the image acquisition unit 101 can also acquire a first original image of the target vehicle. If the exposure of the first original image is less than or equal to a preset exposure, the image processing unit 102 can further perform different image processing on the first original image to obtain a first vehicle image and a second vehicle image with different brightness. If the exposure of the first original image is greater than the preset exposure, the image acquisition unit 101 can also acquire a second original image of the target vehicle with an exposure less than or equal to the preset exposure. The second original image is acquired after the acquisition time of the first original image, and the first original image is the first vehicle image, and the second original image is the second vehicle image. A first license plate area image in the first vehicle image and a second license plate area image in the second vehicle image are also acquired.

[0087] In some embodiments, the image fusion unit 103 may also fuse the license plate optimized image with the license plate area of ​​the first vehicle image to obtain a first target image when the exposure of the first original image is less than or equal to a preset exposure; the image processing unit 102 may also perform edge smoothing processing on the license plate area of ​​the first target image to obtain a target vehicle image.

[0088] In some embodiments, the image fusion unit 103 may also fuse the license plate optimized image with the license plate area of ​​the second vehicle image to obtain a first target image if the exposure of the first original image is greater than the preset exposure. The image processing unit 102 may also perform edge smoothing processing on the license plate area of ​​the first target image to obtain a target vehicle image.

[0089] In addition, in some embodiments, the image acquisition unit 101 in the image processing apparatus 11 may include a shooting unit, which acquires images of the target vehicle in the actual scene. In this case, the image processing method provided in the embodiments of this application can also be applied to the image processing apparatus 11, and the functions that the image processing apparatus 11 can implement and the unit structure it includes can be found in the foregoing embodiments.

[0090] Figure 4 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. Exemplarily, the image processing method provided in this embodiment can be applied to... Figure 1 The image processing system shown or Figure 2 The image processing device shown. (e.g.) Figure 4 As shown, the image processing method provided in this application embodiment may specifically include the following steps S101-S102.

[0091] S101. For the license plate area of ​​the target vehicle, obtain a first license plate area image and a second license plate area image with different brightness.

[0092] The brightness of the first license plate area image is greater than that of the second vehicle area image. Therefore, the first license plate area image can be called the bright frame image, and the second license plate area image can be called the dark frame image.

[0093] Because license plates typically use directional, retroreflective, and reflective materials—meaning they reflect light in the direction it's illuminated—they possess excellent reflectivity. Therefore, with appropriate supplemental lighting, capturing an image of the target vehicle can easily place the license plate in the brightest area of ​​the image. However, due to the influence of the scene, weather, and supplemental lighting, license plates are very prone to overexposure. For example, for blue license plates with white lettering, excessive brightness or overexposure can cause the blue lettering to appear lighter, thicker, or exhibit severe jagged edges.

[0094] Therefore, generally speaking, when the license plate image is bright, the edges of the license plate are clear, but there may be exposure issues in the license plate area, resulting in thicker and blurrier fonts compared to normal fonts. Thus, the first license plate image with higher brightness has clearer edges, but the fonts are thicker and blurrier.

[0095] When the license plate image is less bright, the font is thinner and clearer compared to the first license plate image. However, the edges of the license plate appear blurry compared to normal license plate edges, and may even disappear. Therefore, the second license plate image, with less brightness, has clearer fonts but blurrier edges.

[0096] S102. According to the fusion weights corresponding to different positions of the license plate area, fuse the first license plate area image and the second license plate area image to obtain the target license plate image.

[0097] In some embodiments, the first license plate region image and the second license plate region image can be fused according to the fusion weight corresponding to the edge position of the license plate region to obtain a target license plate image with clear edges. The fusion weight corresponding to the edge position indicates that the fusion ratio of the first license plate region image is greater than that of the second license plate region image. In this embodiment, the license plate edge of the brighter first license plate image is clearer than that of the darker second license plate image. Therefore, setting the fusion weight corresponding to the edge position to have a higher fusion ratio of the first license plate region image than the second license plate region image can make the license plate edge of the fused target license plate image clearer.

[0098] In other embodiments, the first license plate region image and the second license plate region image can be fused according to the fusion weight corresponding to the font position in the license plate region to obtain a target license plate image with clear font. The fusion weight corresponding to the font position indicates that the fusion ratio of the first license plate region image is less than that of the second license plate region image. In this embodiment, the font of the second license plate image with lower brightness is clearer than the font of the first license plate image with higher brightness. Therefore, setting the fusion weight corresponding to the font position to be less than the fusion ratio of the first license plate region image can make the license plate edge of the fused target license plate image clearer.

[0099] For example, the first license plate region image and the second license plate region image are fused according to the fusion weight corresponding to the edge position of the license plate region and the fusion weight corresponding to the font position to obtain a target license plate image with clear edges and clear fonts; wherein, the fusion weight corresponding to the edge position indicates that the fusion ratio of the first license plate region image is greater than that of the second license plate region image, and the fusion weight corresponding to the font position indicates that the fusion ratio of the first license plate region image is less than that of the second license plate region image.

[0100] Understandably, when a license plate image is bright, the edges are sharp, but the text may be overexposed and appear thicker than normal. Conversely, when the image is dim, the text is thin, but the edges are blurry. Therefore, for the target vehicle, a first license plate area image with higher brightness and a second license plate area image with lower brightness are obtained. The first image has sharper edges but thicker, blurry text, while the second image has sharper text but blurrier edges. Then, the first and second license plate areas are fused according to different fusion weights for different positions within the license plate area. For example, the fusion weight for edge positions is set to be greater for the first image than the second, and the fusion weight for text positions is set to be less for the first image than the second. This results in a target license plate image with both sharp edges and sharp text. The method in this embodiment can solve the problems of unclear and thick license plate fonts and unclear license plate edges in vehicle images. Furthermore, even when the license plate image is overexposed, it can still be processed to obtain a target license plate image with clear fonts and sharp edges. The license plate image obtained by this method is easier to identify and has better visual effects.

[0101] In some embodiments, such as Figure 5 As shown, obtaining a first license plate area image and a second license plate area image with different brightness for the license plate area of ​​the target vehicle may include the following steps Sa1-Sa4.

[0102] Sa1: Obtain the first raw image of the target vehicle.

[0103] The first original image can be either an overexposed image or a normally exposed image. Overexposure refers to the abnormal image phenomenon caused by excessively large aperture during image acquisition, excessively high film sensitivity, excessively long exposure time, or excessively strong ambient light. Specifically, it can manifest as a washed-out image or an image that is too bright.

[0104] Under normal circumstances, because license plates are highly reflective, appropriate supplemental lighting is usually used to capture the image of the target vehicle, making the license plate a highlight area of ​​the image. However, when the ambient light is too strong, the license plate area is more likely to be overexposed.

[0105] For example, when the target vehicle is a truck with a high-intensity light installed above its license plate while driving in the dark, the license plate area in the first raw image of the target vehicle is likely to be overexposed due to the excessive light emitted by the high-intensity light.

[0106] In some embodiments, the first original image may be an image of the target vehicle captured when passing through the shooting area in an actual scene. The actual scene in which the first original image is captured may be a parking lot entrance / exit, a road checkpoint, a highway toll station, etc. This embodiment does not impose specific limitations.

[0107] In other embodiments, the first original image may also be an image of the target vehicle that was collected at historical time points and stored in a database, and then read from the database when needed.

[0108] Sa2. When the exposure of the first original image is less than or equal to the preset exposure, different image processing is performed on the first original image to obtain a first vehicle image and a second vehicle image with different brightness.

[0109] The brightness of the first vehicle image is greater than that of the second vehicle image.

[0110] In some embodiments, image processing can be image processing via ISP PIPE, and the image parameters after image processing will be partially improved, such as increased sharpness, improved brightness, and more saturated colors.

[0111] Understandably, obtaining a brighter first vehicle image ensures the overall quality of the captured image (the license plate area may be overexposed), while obtaining a less bright second vehicle image ensures that the license plate area in the image is not overexposed and that the color and contrast are good (the license plate area edge module and the overall brightness of the license plate area may be dark / black).

[0112] Sa3. If the exposure of the first original image is greater than the preset exposure, obtain a second original image of the target vehicle with an exposure less than or equal to the preset exposure.

[0113] In this case, the second original image is acquired after the first original image is acquired, and the first original image is the first vehicle image, while the second original image is the second vehicle image. Since the exposure of the first original image is greater than a preset exposure, and the exposure of the second original image is less than or equal to the preset exposure, the brightness of the first vehicle image is greater than the brightness of the second vehicle image.

[0114] If the exposure of the first original image is greater than the preset exposure, it can be considered that the first original image is overexposed. Because under overexposure, the image will lose too much detail. In order to obtain more image details, a second original image with an exposure less than or equal to the preset exposure can be obtained.

[0115] Sa4. Obtain the first license plate area image from the first vehicle image and the second license plate area image from the second vehicle image.

[0116] The brightness of the first vehicle image is greater than that of the second vehicle image.

[0117] Generally, when the brightness of a vehicle image is high, the license plate font in the image will appear thicker and blurrier compared to normal fonts. Therefore, in the first vehicle image, the license plate font in the license plate area will appear thicker and blurrier compared to normal fonts.

[0118] If the brightness of the vehicle image is low, the edges of the license plate will appear blurry compared to normal license plate edges, or even disappear. Therefore, the license plate edges in the second vehicle image are relatively blurry.

[0119] Because the license plate font in the license plate area of ​​the first vehicle image appears thicker and blurrier than normal font, the license plate font in the first license plate area image obtained from the first vehicle image is thicker and blurrier than normal font; the license plate edge in the license plate area of ​​the second vehicle image is relatively blurry, so the license plate edge in the second license plate area image obtained from the second vehicle image is relatively blurry.

[0120] When the exposure of the first original image is less than or equal to a preset exposure, and different image processing is applied to the first original image to obtain first and second vehicle images with different brightness, if the image processing is ISPPIPE, the first original image can be processed by setting a normal brightness parameter to obtain a first vehicle image in YUV format with normal brightness, and processed by setting a low brightness parameter to obtain a second vehicle image in YUV format with low brightness. Then, the license plate region in the first vehicle image and the license plate region in the second vehicle image are obtained respectively, and the first and second license plate region images are merged to obtain the merged target license plate image.

[0121] In the case where the exposure of the first original image is greater than the preset exposure, and a second original image of the target vehicle is obtained with an exposure less than or equal to the preset exposure, such as... Figure 7As shown, ISP PIPE image processing can be performed on the first and second original images to obtain a first vehicle image and a second vehicle image. Then, the license plate region in the first vehicle image and the license plate region in the second vehicle image are obtained respectively, resulting in a first license plate region image and a second license plate region image. These images are then fused to obtain the fused target license plate image. It is understandable that the image parameters will be improved after image processing, such as increased sharpness, improved brightness, and more saturated colors. Since the exposure of the first original image is greater than the preset exposure, and the exposure of the second original image is less than or equal to the preset exposure, the brightness of the first vehicle image obtained after ISP PIPE image processing is still greater than the brightness of the second vehicle image.

[0122] In the above embodiments, if the exposure of the first original image of the target vehicle is less than or equal to a preset exposure, the first original image is subjected to two different image processing steps to obtain a bright frame image and a dark frame image. If the exposure of the first original image of the target vehicle is greater than the preset exposure, a second original image with an exposure less than or equal to the preset exposure is then acquired to obtain a bright frame image and a dark frame image. This method can perform fusion processing on vehicle images that are either overexposed or not overexposed, thereby obtaining a license plate image with clear font and unblurred edges.

[0123] In some embodiments, such as Figure 8 As shown in (a), if the exposure of the first original image is less than or equal to the preset exposure, then after obtaining the target license plate image, the following steps Sb1 may be included; Figure 8 As shown in (b), if the exposure of the first original image is greater than the preset exposure, the following steps Sb2 can be included after obtaining the target license plate image.

[0124] Sb1. When the exposure of the first original image is less than or equal to the preset exposure, the target license plate image is fused with the license plate area of ​​the first vehicle image to obtain the first target image.

[0125] In some embodiments, before fusing the target license plate image with the license plate region of the first vehicle image, the target license plate image may be sharpened, for example, by using the USM sharpening algorithm, so that the visual effect of the target license plate image can be improved and the details can be more obvious.

[0126] When the exposure of the first original image is less than or equal to the preset exposure, the overall vehicle body effect of the first vehicle image with higher brightness is better. Therefore, the target license plate image is fused with the license plate area of ​​the first vehicle image to obtain a first target image with clear license plate font, clear license plate edges, and excellent overall vehicle body effect.

[0127] Sb2. When the exposure of the first original image is greater than the preset exposure, the target license plate image is fused with the license plate area of ​​the second vehicle image to obtain the first target image.

[0128] In some embodiments, before fusing the target license plate image with the license plate region of the second vehicle image, the target license plate image can be processed by the USM sharpening algorithm, thereby improving the visual effect of the target license plate image and making the details more obvious.

[0129] When the exposure of the first original image is greater than the preset exposure, the brighter first vehicle image suffers from overexposure. Compared to the first vehicle image, the second vehicle image has a better overall vehicle body effect and more detail. Therefore, the target license plate image is fused with the license plate area of ​​the second vehicle image to obtain a first target image with clear license plate font, sharp license plate edges, and excellent overall vehicle body effect.

[0130] In some embodiments, as described above Figure 8 As shown in (a) and (b), after acquiring the first target image, step Sb3 can be performed.

[0131] Sb3. Smooth the edges of the license plate area in the first target image to obtain the target vehicle image.

[0132] Understandably, when the exposure of the first original image is less than or equal to the preset exposure, the overall appearance of the vehicle in the first vehicle image is better. Therefore, the target license plate image is fused with the license plate area of ​​the first vehicle image. When the exposure of the first original image is greater than the preset exposure, the overall appearance of the vehicle in the second vehicle image is better. Therefore, it is also possible to fuse the target license plate image with the license plate area of ​​the second vehicle image. Further edge smoothing processing of the fused image yields a target vehicle image with optimized license plate font clarity, optimized license plate edge clarity, and excellent overall vehicle appearance. This allows for improved recognition of the target vehicle based on the target image.

[0133] In some embodiments, such as Figure 9 As shown, the above-mentioned fusion of the first license plate region image and the second license plate region image according to the fusion weight corresponding to different positions of the license plate region to obtain the target license plate image may include the following steps S201-S204.

[0134] S201. Obtain the weighting factor of the target license plate image based on the height of the target vehicle's license plate.

[0135] The weighting factor represents the degree to which the pixel value of the target license plate image changes with the height of the license plate.

[0136] In some embodiments, the method for obtaining the weighting factor is as shown in formula (1):

[0137] fac=min(256×abs(y-height / 2) / (height / 4),256) (1)

[0138] Where fac represents the weight factor, with a value between 0 and 256; y represents the height of the current pixel, and height represents the height of the target license plate. According to the formula, fac is at its minimum when y is the median value of the target license plate height; fac is at its maximum when y is equal to 0 or equal to the target license plate height.

[0139] For example, when height equals 15cm (the target license plate height is 15cm), when y equals 7.5 (the current pixel is at a height of 7.5cm, i.e., the current pixel is the license plate font area), fac equals zero; when y equals 0 or 15 (the current pixel is at a height of 0cm or 15cm, i.e., the current pixel is the upper and lower edge areas of the license plate), fac is 256, which is the maximum value.

[0140] Understandably, the weight factor obtained using the above formula (1) is at its minimum when the current pixel is at the midpoint of the target license plate height; the weight factor will gradually increase when the height of the current pixel is higher than or lower than the midpoint of the target license plate height; and the weight factor will reach its maximum when the height of the current pixel is 0 or the height of the target license plate (i.e., the current pixel is located in the upper or lower edge area of ​​the target license plate).

[0141] S202. Filter the first license plate area image and the second license plate area image respectively to obtain the first filtered image and the second filtered image.

[0142] The first filtered image and the second filtered image are used to determine the fusion weights for fusing the first license plate region image and the second license plate region image.

[0143] In some embodiments, such as Figure 10 As shown, filtering is performed on the first license plate area image and the second license plate area image respectively to obtain the first filtered image and the second filtered image, which may include the following steps Sc1-Sc4.

[0144] Sc1. Perform RGB maximum value channel filtering on the first license plate image and the second license plate image respectively to obtain the first processed image and the second processed image.

[0145] The RGB maximum value channel filter is used to obtain the first processed image corresponding to the channel with the largest pixel value among the three RGB color channels in the first license plate image, and to obtain the second processed image corresponding to the channel with the largest pixel value among the three RGB color channels in the second license plate image.

[0146] The first image to be processed can be denoted as max_chn norm The second processed image is denoted as max_chn dark .

[0147] Understandably, the channel with the largest pixel value corresponds to an image with more image features. For example, the rivet area of ​​the license plate is more obvious in the first processed image and the font of the license plate is clearer in the second processed image.

[0148] Sc2. Perform neighborhood maximum filtering on the first processed image and the second processed image respectively to obtain the first filtered image corresponding to the first processed image after target feature region enhancement and the first maximum image corresponding to the second processed image.

[0149] The target feature region includes at least one of the rivet region, sludge region, and dust region. The neighborhood maximum value filtering includes sorting the pixel values ​​in the first processed image and the second processed image in an ordered manner within a neighborhood with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value.

[0150] The first filtered image can be denoted as Let the image of the first maximum value be denoted as

[0151] Sc3. Perform blur filtering on the second processed image within a preset neighborhood with a preset radius to obtain the second maximum value image of pixel smoothing within the preset neighborhood.

[0152] The second maximum value image can be denoted as Images processed with blur filtering can preserve the detailed texture of the rivet area and make the image transition smooth.

[0153] Sc4. Based on the weighting factor, fuse the first maximum value image and the second maximum value image to obtain the second filtered image after target feature region enhancement.

[0154] In some embodiments, the method for obtaining the fusion weights is as shown in formula (2):

[0155]

[0156] Where, max__chn′ dark This is the second filtered image.

[0157] According to formula (3), the larger fac is, the greater the weight of the second maximum value image in the fusion of the second filtered image. This ensures that the rivet area can maintain its detailed texture after the image is fused, and makes the image transition smoother and the license plate font clearer.

[0158] S203. Based on the weighting factor, the RGB parameters of the first filtered image, and the RGB parameters of the second filtered image, determine the fusion weights for image fusion.

[0159] In some embodiments, the method for obtaining the fusion weights is as shown in formula (3):

[0160] wt = min(fac × max(max_chn) norm -max__chn′ dark ,0),256) (3)

[0161] Where wt represents the fusion weight, and its value ranges from 0 to 256.

[0162] According to formula (3), the larger fac is, the larger wt is.

[0163] S204. According to the fusion weight, the first license plate area image and the second license plate area image are fused to obtain the target license plate image with optimized license plate font clarity and license plate edge clarity.

[0164] In some embodiments, the method for obtaining the fusion weights is as shown in formula (4):

[0165] rgb fus =(rgb) norm ×wt+rgb dark ×(256-wt)) / 256 (4)

[0166] Among them, rgb fus Represents the target license plate image, RGB norm Represents the bright frame image (first vehicle image), RGB dark This represents the dark frame image (the second vehicle image).

[0167] According to formula (4), the larger wt is, the greater the weight of the bright frame image in the fusion of the target license plate image, and the smaller wt is, the greater the weight of the dark frame image in the fusion of the target license plate image.

[0168] like Figure 11As shown, the fusion process of bright frame images and dark frame images mainly includes the following steps S11 to S15.

[0169] S11. Perform RGB maximum value channel processing and neighborhood maximum value filtering on the bright frame image in sequence to obtain the first filtered image.

[0170] S12. Obtain the license plate height and set the weight factor based on the pixel height.

[0171] S13. Perform RGB maximum value channel processing and neighborhood maximum value filtering on the dark frame image in sequence to obtain the first processed image. Perform blur filtering on the first processed image to obtain the second processed image. According to the weight factor, fuse the first processed image and the second processed image to obtain the second filtered image.

[0172] S14. Based on the weighting factor, the RGB parameters of the first filtered image, and the RGB parameters of the second filtered image, determine the fusion weights used for image fusion.

[0173] S15. Based on the fusion weights, the bright frame image and the dark frame image are fused to obtain the target license plate image with optimized font clarity and edge clarity.

[0174] In some cases, the rivets or license plate surface of the target vehicle may be covered with thick mud or dust, which usually appears at the top and bottom edges of the license plate. Since the reflectivity of these areas is weaker than that of a normal license plate surface, the resulting license plate image is prone to appearing black, gray, or dark at the top and bottom edges. Therefore, based on formulas (1) to (4), it can be seen that when the pixel position is located in the top or bottom edge area of ​​the target license plate, the weight factor fac and wt reach their maximum values. That is, the bright frame image can take the maximum weight in the fusion of the top or bottom edge area. In this way, the license plate image obtained will have clearer top and bottom edges, retain the detailed texture of the rivet area, and make the image transition smooth. The black, gray, or dark areas (e.g., mud or dust areas) in the top or bottom edge area will become clearer and more obvious, and the overall visual effect of the license plate image will be better.

[0175] It is understandable that, according to formula (1), the weight factor fac corresponding to the pixels in the middle height area of ​​the target license plate can reach the minimum value, that is, the weight factor fac corresponding to the pixels in the font area of ​​the target license plate can reach the minimum value. According to formula (4), when fac is at its minimum value, wt is also at its minimum value. Therefore, in this case, the dark frame image has the largest weight in the fusion of the target license plate image. The license plate image obtained in this way will have clearer license plate font and the font will not be too thick.

[0176] According to formula (1), the weight factor fac corresponding to the pixels in the upper or lower edge region of the target license plate can reach the maximum value. According to formula (4), when fac is at its maximum value, wt is also at its maximum value. Therefore, in this case, the bright frame image has the largest weight in the fusion of the target license plate image. In this way, the license plate image obtained will have a clearer upper and lower edge region, and the features of the rivet region will be more obvious. The black, gray or dark areas in the upper or lower edge region (e.g., mud or dust areas) will become clearer and more obvious.

[0177] Therefore, when the dark frame image has the largest weight in the fusion of the target license plate font area, and the bright frame image has the largest weight in the fusion of the upper or lower edge area, the fused license plate image has the following characteristics: the upper and lower edge areas of the license plate are clearer, the features of the rivet area are more obvious, and the license plate font is clear and of moderate thickness, resulting in a better overall visual effect.

[0178] In some embodiments, such as Figure 12 As shown, when the background color of the license plate of the target vehicle is any one of green gradient, yellow, and yellow-green, the first license plate area image and the second license plate area image are fused according to the fusion weight corresponding to different positions of the license plate area to obtain the target license plate image. After that, i.e. after S102, the license plate area can also be color corrected. The color correction process can include the following steps S103 to S105.

[0179] S103. Obtain the brightness ratio between the first license plate area image and the second license plate area image.

[0180] The brightness ratio is the ratio between the average brightness of the first license plate area image and the average brightness of the second license plate area image.

[0181] Since the target vehicle can be photographed in a bright daytime or a dimly lit nighttime, obtaining the brightness ratio between the first license plate area image and the second license plate area image is equivalent to obtaining the brightness of the target vehicle in its actual photographing environment.

[0182] S104. Based on the brightness ratio, convert the target license plate image into a grayscale image of the corresponding color.

[0183] Because RGB format images have a large amount of data, subsequent processing is slow. Grayscale images, on the other hand, have less data. To improve processing speed, the target license plate image in RGB format is converted to a grayscale image of the corresponding color. This improves image processing speed and ensures that the image colors can be optimized later.

[0184] In some embodiments, during the process of converting the target fused image into a grayscale image, the brightness ratio is multiplied by a grayscale parameter to obtain a grayscale image.

[0185] Understandably, converting the target license plate image into a grayscale image of the corresponding color based on the brightness ratio can make the grayscale image have the brightness characteristics of the target vehicle in the actual shooting environment. This makes the visual effect of the license plate image more consistent with the image in the actual shooting environment.

[0186] S105. Based on the license plate background color, convert the grayscale image into an optimized license plate image with optimized background color.

[0187] It is understandable that when the license plate background color of the target vehicle is any of the following: green gradient, yellow, or yellowish-green, the license plate color is relatively light, which may lead to the license plate being too light or disappearing due to overexposure or over-exposure of the first original image. Therefore, the method in this embodiment can not only restore the color features of the license plate, but also ensure that the background color of the license plate is clean and the characters are clear. Furthermore, by converting the grayscale image to the corresponding color license plate image according to the brightness ratio, the brightness of the obtained target license plate image can be more consistent with the brightness conditions of the shooting environment. Thus, based on the target license plate image, the accuracy of target vehicle recognition can be improved.

[0188] It should be noted that, Figure 13 The illustrated embodiment is given by example that steps S103 to S105 are executed after S102. In other embodiments, steps S103 to S105 may be executed independently of step S102.

[0189] For example, such as Figure 13 As shown, the method may include steps S301 to S303. Step S301 can be referred to the description of step S103 above, step S302 can be referred to the description of step S104 above, and step S303 can be referred to the description of step S105 above.

[0190] like Figure 14 As shown, if the background color of the license plate of the target vehicle is a green gradient, obtaining the target license plate image may also include the following steps Sd1 to Sd3.

[0191] Sd1. Convert the grayscale image to a green license plate image based on the background color of the license plate.

[0192] Sd2. Determine the color gradient coefficient based on the height of the target vehicle's license plate.

[0193] Since the background color of the target vehicle's license plate is a green gradient, and the corresponding green color parameter value changes with the height of the target vehicle's license plate, the color gradient coefficient corresponding to the green gradient license plate can be determined based on the height of the target vehicle's license plate.

[0194] Sd3. Based on the color gradient coefficient, convert the green license plate image into an optimized license plate image.

[0195] Understandably, since the target vehicle's license plate has a green gradient background, the grayscale image can be converted into a green license plate image first, and then the green license plate image can be converted into a color-optimized green gradient target license plate image based on the color gradient coefficient. This method can still ensure that the license plate background is clean and the characters are clear, thereby improving the accuracy of target vehicle recognition based on the target license plate image.

[0196] like Figure 15 As shown, if the background color of the license plate of the target vehicle is yellow-green, obtaining the target license plate image may also include the following steps Se1 to Se2.

[0197] Se1. Determine the color division coefficient based on the height and width of the target vehicle's license plate.

[0198] The color division coefficient is used to separate the yellow and green areas in the license plate.

[0199] Since the target vehicle's license plate has a yellow-green background, meaning it's a two-tone yellow and green plate, the width and height of the yellow and green areas are related to the plate's height and width. Therefore, it's necessary to determine the color division coefficient based on the target vehicle's license plate height and width.

[0200] For example, the color division coefficient can be determined based on the ratio of the yellow to the green area in the license plate, or the color division coefficient can be determined according to the template of the yellow-green license plate.

[0201] Se2. Based on the color segmentation coefficient and the license plate background color, convert the grayscale image into an optimized license plate image.

[0202] Understandably, since the background color of the target vehicle's license plate is yellow-green, the color segmentation coefficient can be determined based on the height and width of the license plate. Then, based on the color segmentation coefficient and brightness ratio, the grayscale image can be converted into a color-optimized yellow-green target license plate image. This method can still ensure that the background color of the license plate is clean and the characters are clear, thereby improving the accuracy of target vehicle recognition based on the target license plate image.

[0203] like Figure 16As shown, when the background color of the license plate of the target vehicle is any one of green gradient, yellow, and yellow-green, and the license plate color is too light or the license plate color is almost invisible, the color correction process for the license plate area may include the following steps S21 to S27.

[0204] S21. Obtain the average brightness ratio between the bright frame image and the dark frame image corresponding to the license plate area of ​​the target vehicle.

[0205] S22. Obtain the target fused image after fusing the bright frame image and the dark frame image.

[0206] S23. Convert the RGB format target fusion image into a grayscale image multiplied by the brightness ratio.

[0207] S24. Determine the color of the license plate.

[0208] S25. When the license plate color is a green gradient, convert the grayscale image into a green license plate image. Determine the color gradient coefficient based on the height of the license plate of the target vehicle. Based on the color gradient coefficient, convert the green license plate image into a color-optimized green gradient license plate image.

[0209] S26. When the license plate color is yellow-green, determine the color segmentation coefficient based on the height and width of the target vehicle's license plate, and convert the grayscale image into a color-optimized yellow-green license plate image based on the color segmentation coefficient and brightness ratio.

[0210] S27. If the license plate color is yellow, convert the grayscale image into a color-optimized yellow license plate image.

[0211] It is understandable that when the background color of the target vehicle's license plate is any of the following: green gradient, yellow, or yellowish-green, the license plate color is relatively light, which can lead to problems such as the license plate being too light or disappearing due to image overexposure or overexposure. Therefore, the color correction processing in this embodiment can not only restore the color features of the license plate, but also ensure that the background color of the license plate is clean and the characters are clear, thereby improving the accuracy of target vehicle recognition based on the target license plate image.

[0212] In some embodiments, steps S401 to S403 may be performed after step S105. For example... Figure 17 As shown in (a), if the exposure of the first original image is less than or equal to the preset exposure, then after obtaining the license plate optimized image, the following step S401 may be included; Figure 17 As shown in (b), if the exposure of the first original image is greater than the preset exposure, the following step S402 can be included after obtaining the license plate optimized image.

[0213] S401. When the exposure of the first original image is less than or equal to the preset exposure, the license plate optimized image is fused with the license plate area of ​​the first vehicle image to obtain the first target image.

[0214] S402. When the exposure of the first original image is greater than the preset exposure, the license plate optimized image is fused with the license plate area of ​​the second vehicle image to obtain the first target image.

[0215] In some embodiments, as described above Figure 17 As shown in (a) and (b), after acquiring the first target image, step S403 can be performed.

[0216] S403. Perform edge smoothing processing on the license plate area of ​​the first target image to obtain the target vehicle image.

[0217] Understandably, when the exposure of the first original image is less than or equal to the preset exposure, the overall appearance of the vehicle in the first vehicle image is better. Therefore, the optimized license plate image is fused with the license plate area of ​​the first vehicle image. When the exposure of the first original image is greater than the preset exposure, the overall appearance of the vehicle in the second vehicle image is better. Therefore, the optimized license plate image is fused with the license plate area of ​​the second vehicle image. Further edge smoothing processing of the fused image yields a target vehicle image with optimized license plate font clarity, optimized license plate edge clarity, optimized license plate background color, and excellent overall vehicle appearance. This allows for improved recognition of the target vehicle based on the target image.

[0218] In some embodiments, such as Figure 18 As shown, acquiring an image of a target vehicle with clear license plate font, clear license plate edges, and a clear overall vehicle image may also include the following steps S31 to S36.

[0219] S31. For the target vehicle, obtain the vehicle image after image processing.

[0220] S32. Obtain the bright frame image and dark frame image corresponding to the license plate area of ​​the target vehicle.

[0221] S33. According to the fusion weights corresponding to different positions in the license plate area, the bright frame image and the dark frame image are fused to obtain the target license plate image after the license plate font clarity is optimized and the license plate edge clarity is optimized.

[0222] S34. Perform color correction processing on the target license plate image to obtain a color-optimized license plate image.

[0223] S35. Perform image sharpening processing on the license plate optimized image to obtain the license plate enhanced image.

[0224] S36. Merge the enhanced license plate image with the license plate area in the above vehicle image to obtain the target vehicle image.

[0225] Understandably, this embodiment fuses the bright and dark frames of the license plate area of ​​the target vehicle according to the fusion weights corresponding to different positions in the license plate area. Then, it performs color correction on the fused license plate image, resulting in an optimized license plate image with improved font clarity, edge clarity, and color. Next, it sharpens the license plate image to obtain an enhanced license plate image, thus improving the visual effect and making details more prominent. Finally, the enhanced license plate image is fused with the license plate area in the vehicle image to obtain a target vehicle image with excellent overall vehicle quality. Furthermore, even when the vehicle image is overexposed, it can still be processed to obtain a target vehicle image with optimized font clarity, edge clarity, color, and overall vehicle clarity. The vehicle image obtained by this method is easier to recognize and has better visual quality.

[0226] In the case of implementing the functions of the integrated units described above in hardware, embodiments of this application provide a schematic diagram of the hardware composition of an electronic device, such as... Figure 19 As shown, the electronic device also includes: a processor 301, a communication interface 302, and a bus 304. Optionally, the electronic device may also include a memory 303.

[0227] Processor 301 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0228] Communication interface 302 is used to connect with other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0229] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0230] As one possible implementation, the memory 303 can exist independently of the processor 301. The memory 303 can be connected to the processor 301 via a bus 304 and is used to store instructions or program code. When the processor 301 calls and executes the instructions or program code stored in the memory 303, it can implement the image processing method provided in the embodiments of this application.

[0231] In another possible implementation, the memory 303 can also be integrated with the processor 301.

[0232] Bus 304 can be an extended industry standard architecture (EISA) bus, etc. Bus 304 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 19 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0233] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the image processing device can be divided into different functional modules to complete all or part of the functions described above.

[0234] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The computer-readable storage medium can also be an external storage device of the image processing apparatus, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the image processing apparatus. Further, the computer-readable storage medium can include both internal storage units of the image processing apparatus and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the image processing apparatus. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0235] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the image processing method provided in the above embodiments.

[0236] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0237] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0238] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: For the license plate area of ​​the target vehicle, a first license plate area image and a second license plate area image with different brightness are obtained; the brightness of the first license plate area image is greater than the brightness of the second license plate area image. According to the fusion weights corresponding to different positions in the license plate area, the first license plate area image and the second license plate area image are fused to obtain the target license plate image; According to the fusion weights corresponding to different positions in the license plate area, the first license plate area image and the second license plate area image are fused to obtain the target license plate image, including: Based on the height of the license plate of the target vehicle, a weighting factor for the target license plate image is obtained; the weighting factor represents the degree to which the pixel value of the target license plate image changes with the height of the license plate. The first license plate area image and the second license plate area image are filtered respectively to obtain a first filtered image and a second filtered image; Based on the weighting factor, the RGB parameters of the first filtered image, and the RGB parameters of the second filtered image, the fusion weights for image fusion are determined. Based on the fusion weights, the first license plate region image and the second license plate region image are fused to obtain the target license plate image.

2. The method according to claim 1, characterized in that, The step of fusing the first license plate region image and the second license plate region image according to the fusion weights corresponding to different positions in the license plate region to obtain the target license plate image includes: According to the fusion weight corresponding to the edge position of the license plate area, the first license plate area image and the second license plate area image are fused to obtain a target license plate image with clear edges; the fusion weight corresponding to the edge position indicates that the fusion ratio of the first license plate area image is greater than the fusion ratio of the second license plate area image.

3. The method according to claim 1 or 2, characterized in that, The step of fusing the first license plate region image and the second license plate region image according to the fusion weights corresponding to different positions in the license plate region to obtain the target license plate image includes: According to the fusion weight corresponding to the font position in the license plate area, the first license plate area image and the second license plate area image are fused to obtain a target license plate image with clear font; the fusion weight corresponding to the font position indicates that the fusion ratio of the first license plate area image is less than the fusion ratio of the second license plate area image.

4. The method according to claim 1, characterized in that, The image of the first license plate area is filtered to obtain a first filtered image, including: The first license plate area image is subjected to RGB maximum value channel filtering to obtain a first processed image; the RGB maximum value channel filtering is used to obtain the first processed image corresponding to the channel with the largest pixel value among the three RGB color channels in the image. The first processed image is subjected to neighborhood maximum filtering to obtain a first filtered image after target feature region enhancement. The target feature region includes a rivet region. The neighborhood maximum filtering includes sorting the pixel values ​​in the first processed image within a neighborhood with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value.

5. The method according to claim 1, characterized in that, The second license plate area image is filtered to obtain a second filtered image, including: The second license plate area image is subjected to RGB maximum value channel filtering to obtain a second processed image; the RGB maximum value channel filtering is used to obtain the second processed image corresponding to the channel with the largest pixel value among the three RGB color channels in the image. The second processed image is subjected to neighborhood maximum value filtering to obtain a first maximum value image. The neighborhood maximum value filtering includes sorting the pixel values ​​in the neighborhood of the second processed image with a preset value as the radius, and replacing the center pixel value with the sorted maximum pixel value. The second processed image is subjected to blur filtering processing within a preset neighborhood with a preset value as the radius to obtain the second maximum value image of pixel smoothing within the preset neighborhood. Based on the weighting factor, the first maximum value image and the second maximum value image are fused to obtain a second filtered image after target feature region enhancement.

6. The method according to claim 1, characterized in that, When the background color of the license plate of the target vehicle is any one of green gradient, yellow, and yellow-green, it also includes: Obtain the brightness ratio between the first license plate area image and the second license plate area image; the brightness ratio is the ratio between the average brightness of the first license plate area image and the average brightness of the second license plate area image; Based on the brightness ratio, the target license plate image is converted into a grayscale image of the corresponding color; Based on the license plate background color, the grayscale image is converted into an optimized license plate image with an optimized background color.

7. The method according to claim 6, characterized in that, When the license plate background color of the target vehicle is a green gradient, the step of converting the grayscale image into a license plate color-optimized image based on the license plate background color includes: Based on the background color of the license plate, the grayscale image is converted into a green license plate image; The color gradient coefficient is determined based on the height of the license plate of the target vehicle; Based on the color gradient coefficient, the green license plate image is converted into an optimized license plate image.

8. The method according to claim 6, characterized in that, When the background color of the license plate of the target vehicle is yellow-green, the step of converting the grayscale image into a license plate optimized image based on the background color of the license plate includes: The color division coefficient is determined based on the height and width of the license plate of the target vehicle; Based on the color division coefficient and the license plate background color, the grayscale image is converted into an optimized license plate image.

9. The method according to claim 6, characterized in that, The process of obtaining first and second license plate region images with different brightness for the license plate region of the target vehicle includes: For the target vehicle, obtain a first original image of the target vehicle; When the exposure of the first original image is less than or equal to the preset exposure, different image processing is performed on the first original image to obtain a first vehicle image and a second vehicle image with different brightness. If the exposure of the first original image is greater than the preset exposure, a second original image corresponding to the target vehicle with an exposure less than or equal to the preset exposure is obtained; the second original image is obtained after the first original image is obtained, and the first original image is the first vehicle image, and the second original image is the second vehicle image. Obtain the first license plate area image from the first vehicle image and the second license plate area image from the second vehicle image.

10. The method according to claim 9, characterized in that, Based on the license plate background color, after converting the grayscale image into a license plate color-optimized image, the process includes: When the exposure of the first original image is less than or equal to the preset exposure, the license plate optimized image is fused with the license plate area of ​​the first vehicle image to obtain the first target image; The license plate area of ​​the first target image is smoothed to obtain the target vehicle image; If the exposure of the first original image is greater than the preset exposure, the license plate optimized image is fused with the license plate area of ​​the second vehicle image to obtain the first target image; The license plate area of ​​the first target image is smoothed to obtain the target vehicle image.

11. An image processing apparatus, characterized in that, The device includes: an image acquisition unit and an image fusion unit; The image acquisition unit is used to obtain a first license plate area image and a second license plate area image with different brightness for the license plate area of ​​the target vehicle; the brightness of the first license plate area image is greater than the brightness of the second license plate area image. The image fusion unit is used to fuse the first license plate area image and the second license plate area image to obtain a target license plate image with optimized license plate font clarity and license plate edge clarity without overexposure. According to the fusion weights corresponding to different positions in the license plate area, the first license plate area image and the second license plate area image are fused to obtain the target license plate image, including: Based on the height of the license plate of the target vehicle, a weighting factor for the target license plate image is obtained; the weighting factor represents the degree to which the pixel value of the target license plate image changes with the height of the license plate. The first license plate area image and the second license plate area image are filtered respectively to obtain a first filtered image and a second filtered image; Based on the weighting factor, the RGB parameters of the first filtered image, and the RGB parameters of the second filtered image, the fusion weights for image fusion are determined. Based on the fusion weights, the first license plate region image and the second license plate region image are fused to obtain the target license plate image.

12. An electronic device, characterized in that, The device includes a processor and a memory, the memory being used to store computer instructions, and the processor being used to retrieve and execute the computer instructions from the memory to implement the image processing method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: computer software instructions; when the computer software instructions are executed in an image processing apparatus, the image processing apparatus causes the image processing apparatus to implement the image processing method as described in any one of claims 1-10.

Images