A license plate image processing method, device and equipment

Abstract

The application provides a license plate image processing method, device and equipment, the method comprising: obtaining a first intermediate image corresponding to a first original image and a second intermediate image corresponding to a second original image, the first original image and the second original image both comprising a license plate of a moving vehicle; generating a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image and a noise level corresponding to the first intermediate image; performing frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image to obtain a frame difference weight fusion image; and generating a target image to be output based on the frame difference weight fusion image. Through the technical scheme of the application, image fusion is performed in combination with the displacement difference image, the information of bright areas and dark areas of each image can be fully utilized, and the visual effect of the fused image is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a license plate image processing method, device and equipment. BACKGROUND

[0002] Under the irradiation of a strong light source (such as sunlight, a lamp or a reflector), there are both high-brightness areas and low-brightness areas (such as shadows or backlight) in the image, that is, the bright areas in the image become white due to overexposure, and the dark areas become black due to underexposure, which seriously affects the image quality. The camera has limitations in representing the brightest areas and the darker areas in the same scene, and this limitation is the "dynamic range".

[0003] Due to the limitation of the dynamic range, the image cannot simultaneously take into account the bright areas and the dark areas, such as the problem of underexposure in the dark areas of the image and the problem of overexposure in the bright areas of the image. In order to solve the problem of being unable to simultaneously take into account the bright areas and the dark areas, multiple images of the same scene can be acquired by using different exposure amounts, and the multiple images can be fused into a high-dynamic-range image (also referred to as a wide-dynamic-range image), which can maintain the color and details of the bright areas and the dark areas. Compared with ordinary images, the high-dynamic-range image can provide more dynamic range and image details, and provide a better visual experience for users.

[0004] Although the high-dynamic-range image can maintain useful information such as the color and details of the bright areas and the dark areas, since the high-dynamic-range image is obtained by fusing multiple images, when the multiple images are fused, the information of the bright areas and the dark areas of each image cannot be fully utilized, and useful detail information of overexposed areas and overdark areas is still lost, and the fused image still has problems such as poor visual effect. SUMMARY

[0005] The present application provides a license plate image processing method, which comprises:

[0006] obtaining a first intermediate image corresponding to a first original image and a second intermediate image corresponding to a second original image, the exposure amount of the first original image being greater than the exposure amount of the second original image; wherein the first original image and the second original image both comprise a license plate of a moving vehicle;

[0007] generating a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image and a noise level corresponding to the first intermediate image;

[0008] performing frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image, to obtain a frame difference weight fusion image;

[0009] generating a target image to be output based on the frame difference weight fusion image.

[0010] The application provides a license plate image processing device, the device comprising:

[0011] The acquisition module is configured to acquire a first intermediate image corresponding to a first original image and a second intermediate image corresponding to a second original image, the exposure of the first original image being greater than the exposure of the second original image, wherein the first original image and the second original image both comprise a license plate of a moving vehicle.

[0012] The processing module is configured to generate a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and a noise level corresponding to the first intermediate image, perform frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image, and obtain a frame difference weight fusion image, and generate a target image to be output based on the frame difference weight fusion image.

[0013] The application provides an electronic device, comprising a processor and a machine readable storage medium, the machine readable storage medium storing machine executable instructions capable of being executed by the processor, and the processor is configured to execute the machine executable instructions to implement the license plate image processing method of the above examples of the application.

[0014] As can be seen from the above technical solutions, in the embodiments of the application, the first intermediate image and the second intermediate image are subjected to frame difference fusion based on the displacement difference image between the first intermediate image and the second intermediate image, to obtain a frame difference weight fusion image, so that the image fusion is combined with the displacement difference image, the information of the bright area and the dark area of each image can be fully utilized, the useful detail information of the overexposed area and the dark area can be fully utilized, and the visual effect of the fused image is better. For example, the displacement difference of the license plate in different images is determined based on the displacement difference image, the weight of fusion is determined through the displacement difference, and thus the problem of license plate fusion ghosting is solved. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application or the prior art. Obviously, the drawings in the following description are only some embodiments described in the application, and other drawings can also be obtained by those skilled in the art according to these drawings of the embodiments of the application.

[0016] Figure 1 is a flowchart of the license plate image processing method in an embodiment of the application;

[0017] Figure 2 is a high dynamic range image fused based on brightness in an embodiment of the present application;

[0018] Figure 3 is a flowchart of a license plate image processing method in an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of brightness fusion on the first intermediate image and the second intermediate image;

[0020] Figure 5 is a schematic diagram of frame difference fusion on the first intermediate image and the second intermediate image;

[0021] Figure 6 is a schematic diagram of fusion on the brightness weight fused image and the frame difference weight fused image;

[0022] Figure 7 is a structural schematic diagram of a license plate image processing device in an embodiment of the present application;

[0023] Figure 8 is a hardware structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The terminology used in the present application merely for the purpose of describing particular embodiments, and is not intended to be limiting of the present application. The singular forms "a", "an", and "the" used in the present application and its claims are intended to include plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0025] It should be understood that although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order of the information. These terms are used merely to distinguish one type of information from another. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present application. Furthermore, the word "if" can be interpreted as meaning "when" or "upon" or "in response to determining," depending on the context.

[0026] This application proposes a license plate image processing method, which can be applied to front-end devices (such as network cameras, analog cameras, and cameras) and back-end devices (such as servers, management devices, and storage devices). If the method is applied to a front-end device, the front-end device acquires a first original image and a second original image of the same target scene. Based on the first and second original images, the front-end device can perform license plate image processing using the scheme of this application. If the method is applied to a back-end device, the front-end device acquires a first original image and a second original image of the same target scene and sends the first and second original images to the back-end device. Based on the first and second original images, the back-end device can perform license plate image processing using the scheme of this application.

[0027] See Figure 1 The diagram shown is a flowchart of the license plate image processing method, which may include:

[0028] Step 101: Obtain the first intermediate image corresponding to the first original image and the second intermediate image corresponding to the second original image. The exposure of the first original image can be greater than the exposure of the second original image. Both the first and second original images include the license plate of the moving vehicle; therefore, both the first and second original images can be referred to as license plate images. (The first and second original images are used as examples.)

[0029] For example, temporal denoising can be performed on the first original image to obtain a temporally denoised image, and the exposure difference ratio can be determined based on the exposure of the first original image and the exposure of the second original image. The temporally denoised image can be adjusted (such as image alignment adjustment) based on the exposure difference ratio to obtain an adjusted image; and a first intermediate image corresponding to the first original image can be generated based on the adjusted image.

[0030] For example, spatial domain denoising can be performed on the second original image to obtain a spatially denoised image, and a second intermediate image corresponding to the second original image can be generated based on the spatially denoised image.

[0031] Step 102: Based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image, generate a displacement difference image between the first intermediate image and the second intermediate image.

[0032] For example, generating a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image may include, but is not limited to: for each pixel in the displacement difference image, determining the motion value corresponding to the pixel in the displacement difference image based on the first pixel value corresponding to the pixel in the first intermediate image, the second pixel value corresponding to the pixel in the second intermediate image, and the noise level corresponding to the first pixel value.

[0033] The larger the motion value, the greater the displacement difference. The noise level can be obtained by querying a mapping relationship using the first pixel value. This mapping relationship represents the relationship between pixel values ​​and noise levels, with larger pixel values ​​corresponding to lower noise levels.

[0034] Step 103: Based on the displacement difference image between the first intermediate image and the second intermediate image, perform frame difference fusion on the first intermediate image and the second intermediate image to obtain the frame difference weighted fused image.

[0035] For example, based on the displacement difference image between the first intermediate image and the second intermediate image, frame difference fusion is performed on the first intermediate image and the second intermediate image to obtain a frame difference weighted fusion image. This may include, but is not limited to: for each pixel in the frame difference weighted fusion image, if the target motion value corresponding to the pixel in the displacement difference image is less than a first displacement difference threshold, then the target pixel value corresponding to the pixel in the frame difference weighted fusion image is determined based on the first pixel value corresponding to the pixel in the first intermediate image; if the target motion value is greater than a second displacement difference threshold, then the target pixel value is determined based on the second pixel value corresponding to the pixel in the second intermediate image; if the target motion value is between the first displacement difference threshold and the second displacement difference threshold, then the first pixel value and the second pixel value are weighted to obtain the target pixel value.

[0036] Step 104: Generate the target image to be output based on the frame difference weighted image fusion.

[0037] In one possible implementation, after obtaining the frame difference weighted fused image, this frame difference weighted fused image can be used as the target image. Alternatively, based on the brightness information corresponding to the first intermediate image or the second intermediate image, brightness fusion can be performed on the first intermediate image and the second intermediate image to obtain a brightness weighted fused image. The brightness weighted fused image and the frame difference weighted fused image can then be fused to obtain a fused image, and the target image to be output can be generated based on the fused image. For example, the fused image can be used as the target image, or the target image can be obtained by processing the fused image.

[0038] For example, when fusing a luminance-weighted fused image and a frame difference-weighted fused image, a weighted operation can be performed on the luminance-weighted fused image and the frame difference-weighted fused image to obtain the fused image. For instance, a weighted operation can be performed on the luminance-weighted fused image, its weight coefficients, the frame difference-weighted fused image, and its weight coefficients to obtain the fused image. Alternatively, a temporally denoised image corresponding to the first original image can be obtained, and the motion region corresponding to the moving vehicle in the first original image can be obtained based on the temporally denoised image. Based on the motion region corresponding to the moving vehicle in the first original image, motion region fusion can be performed on the luminance-weighted fused image and the frame difference-weighted fused image to obtain the fused image. Of course, the above are just two examples of fusion methods and are not intended to limit the process.

[0039] For example, based on the brightness information corresponding to the first intermediate image, brightness fusion is performed on the first intermediate image and the second intermediate image to obtain a brightness weighted fused image. This may include, but is not limited to: for each pixel in the brightness weighted fused image, if the first pixel value corresponding to the pixel in the first intermediate image is less than a third brightness threshold A1, then the target pixel value corresponding to the pixel in the brightness weighted fused image can be determined based on the first pixel value. Alternatively, if the first pixel value is greater than a fourth brightness threshold B1, then the target pixel value can be determined based on the second pixel value corresponding to the pixel in the second intermediate image. Alternatively, if the first pixel value is between the values ​​of the third brightness threshold A1 and the fourth brightness threshold B1, then a weighted operation can be performed on the first pixel value and the second pixel value to obtain the target pixel value.

[0040] For example, based on the brightness information corresponding to the second intermediate image, brightness fusion is performed on the first and second intermediate images to obtain a brightness weighted fused image. This may include, but is not limited to: for each pixel in the brightness weighted fused image, if the first pixel value corresponding to the pixel in the second intermediate image is less than a third brightness threshold A2 (the third brightness threshold A2 may be the same as or different from the third brightness threshold A1), then the target pixel value corresponding to the pixel in the brightness weighted fused image can be determined based on the first pixel value. Alternatively, if the first pixel value is greater than a fourth brightness threshold B2 (the fourth brightness threshold B2 may be the same as or different from the fourth brightness threshold B1), then the target pixel value can be determined based on the second pixel value corresponding to the pixel in the second intermediate image. Alternatively, if the first pixel value is between the values ​​of the third brightness threshold A2 and the fourth brightness threshold B2, then a weighted operation can be performed on the first pixel value and the second pixel value to obtain the target pixel value.

[0041] For example, based on the motion region corresponding to the moving vehicle in the first original image, motion region fusion is performed on the brightness weighted fusion image and the frame difference weighted fusion image to obtain the fused image, i.e., the motion region fused image. For instance, for each pixel in the fused image, if the pixel is not located within the motion region, the target pixel value in the fused image is determined based on the first pixel value corresponding to the pixel in the brightness weighted fusion image. If the pixel is located within the motion region, and the reference pixel value corresponding to the pixel in the first intermediate image is less than a first brightness threshold, the target pixel value is determined based on the first pixel value; if the reference pixel value is greater than a second brightness threshold, the target pixel value is determined based on the second pixel value corresponding to the pixel in the frame difference weighted fusion image; if the reference pixel value is between the first brightness threshold and the second brightness threshold, the first pixel value and the second pixel value are weighted to obtain the target pixel value; wherein, if the moving speed of the moving vehicle within the motion region is greater than a speed threshold, the weight coefficient of the second pixel value is greater than the weight coefficient of the first pixel value; if the moving speed of the moving vehicle is less than a speed threshold, the weight coefficient of the first pixel value is greater than the weight coefficient of the second pixel value.

[0042] As can be seen from the above technical solutions, in this embodiment, based on the brightness information corresponding to the first intermediate image, brightness fusion is performed on the first intermediate image and the second intermediate image to obtain a brightness weighted fused image. Based on the displacement difference image between the first intermediate image and the second intermediate image, frame difference fusion is performed on the first intermediate image and the second intermediate image to obtain a frame difference weighted fused image. Based on the motion region corresponding to the first original image, motion region fusion is performed on the brightness weighted fused image and the frame difference weighted fused image to obtain a motion region fused image. The above method combines displacement difference image and motion region fusion, which can make full use of the information of the bright and dark areas of each image, and make full use of the useful detail information of the overexposed and underexposed areas, resulting in a better visual effect of the fused image. For example, the displacement difference image can be used to determine the displacement difference of the license plate in different images, and the fusion weight can be determined by the displacement difference, thereby solving the problem of license plate ghosting. As another example, fusion is performed based on motion region, thereby achieving the goal of obtaining the license plate and headlight halo from the short exposure image, and the vehicle body and surrounding environment from the long exposure image, solving the problem of ghosting of fast-moving vehicles, reducing headlight halo without sacrificing the brightness of the surrounding environment, and improving the dynamic range of the image.

[0043] The technical solutions of the embodiments of this application will be described below in conjunction with specific application scenarios.

[0044] To address the challenge of simultaneously capturing both bright and dark areas, multiple images of the same scene can be captured at varying exposure levels and then fused into a high dynamic range (HDR) image (also known as a wide dynamic range image). This HDR image preserves color and detail information in both bright and dark areas. Compared to ordinary images, HDR images offer greater dynamic range and image detail, providing a better visual experience for users. However, while HDR images retain valuable information like color and detail in both bright and dark areas, because they are derived from the fusion of multiple images, the fusion process cannot fully utilize the bright and dark information from each individual image. Useful details in overexposed and underexposed areas are still lost, resulting in a poor visual effect in the fused image.

[0045] See Figure 2 As shown, this is a high dynamic range image based on brightness fusion. In this fusion method, fast-moving vehicles exhibit displacement differences, such as two outlines above the license plate font. The halo around the headlights is not well suppressed, mainly because the brightness of the halo around the headlights is close to the ambient brightness. During brightness fusion, the brightness around the headlights and the ambient brightness increase or decrease simultaneously. To maintain overall brightness, the headlight halo will be larger, resulting in poor suppression of the halo around the headlights.

[0046] To address the aforementioned findings, this application's embodiments combine displacement difference images and motion regions for fusion. This fully utilizes information from the bright and dark areas of each image, as well as useful details in overexposed and underexposed areas, resulting in a better visual effect for the fused image. For example, displacement difference images are used to determine the displacement difference of license plates in different images, and the fusion weight is determined by this displacement difference, thus resolving the issue of license plate ghosting. Fusion based on motion regions achieves the goal of having license plates and headlight halos derived from short-exposure images, while the vehicle body and surrounding environment are derived from long-exposure images. This solves the ghosting problem of fast-moving vehicles and reduces headlight halos without sacrificing ambient brightness, thereby improving the image's dynamic range.

[0047] This application proposes a license plate image processing method, which can be applied to front-end devices (such as network cameras, analog cameras, cameras, etc.) or back-end devices (such as servers, management devices, storage devices, etc.). See [link to relevant documentation]. Figure 3 The diagram shown is a flowchart of a license plate image processing method, which includes:

[0048] Step 301: Acquire the first original image and the second original image. The exposure of the first original image can be greater than that of the second original image. For example, the exposure duration of the first original image can be greater than that of the second original image, such as when the first original image is a long frame image and the second original image is a short frame image. Another example is that the gain of the first original image can be greater than the gain of the second original image.

[0049] For example, both the first original image and the second original image can include the license plate of the moving vehicle; therefore, both the first original image and the second original image can be referred to as license plate images.

[0050] For example, shutter speed and gain are automatic exposure parameters. Shutter speed controls the exposure time; a larger shutter speed results in a longer exposure time and more sufficient exposure. Under the same conditions, an image with a larger shutter speed is brighter. Gain controls the amplification of the photosensitive pixels; under the same conditions, an image with a larger gain is brighter. In summary, by controlling shutter speed and / or gain, the exposure of the first original image is made greater than that of the second original image. For example, the exposure time of the first original image (determined by the shutter speed) is greater than the exposure time of the second original image, and / or, the gain of the first original image is greater than the gain of the second original image. For ease of description, let's take the example where the exposure time of the first original image is greater than that of the second original image, i.e., the first original image is a long frame image and the second original image is a short frame image.

[0051] For example, when the sensor is in DOL mode or line-by-line mode, it can be controlled to perform long and short frame exposures to obtain long frame images and short frame images. The long frame image is recorded as the first original image, and the short frame image is recorded as the second original image. Because the long frame image and the short frame image have different exposure times and different exposure orders, there are brightness differences and displacement differences between them.

[0052] Step 302: Perform temporal denoising on the first original image to obtain the temporally denoised image.

[0053] For example, since the first original image is a long frame image with a good signal-to-noise ratio, temporal denoising can be performed on the first original image to obtain a temporally denoised image. For instance, the 3DNR algorithm can be used to perform 3D temporal denoising on the first original image, that is, only temporal denoising is performed without spatial denoising. In this way, the spatial details of the first original image will not be lost, while removing jittery noise.

[0054] Step 303: Determine the motion region corresponding to the first original image based on the temporally denoised image.

[0055] For example, after obtaining the temporally denoised image, the region where the moving vehicle is located can be determined based on the temporally denoised image, i.e., the region where the moving vehicle is located in the first original image, and the coordinates of the region where the moving vehicle is located can be output, i.e., the motion region corresponding to the first original image. For example, when the region where the moving vehicle is located is a rectangular region, the coordinates of the four vertices of the rectangular region can be output, or the coordinates of one vertex (such as the coordinates of the top left vertex, or the top right vertex, or the bottom left vertex, or the bottom right vertex) plus the length and width information. When the region where the moving vehicle is located is a circular region, the coordinates of the center point of the circular region plus the radius can be output.

[0056] For example, when determining the location of a moving vehicle based on a time-domain denoised image, the time-domain denoised image can be analyzed directly to obtain the location of the moving vehicle, or the time-domain denoised image can be input into a neural network, and the neural network can output the location of the moving vehicle. There is no limitation on this.

[0057] Step 304: Determine the exposure difference ratio based on the exposure of the first original image and the exposure of the second original image, and adjust the temporal denoised image based on the exposure difference ratio (such as image alignment adjustment) to obtain the adjusted image, and generate the first intermediate image corresponding to the first original image based on the adjusted image.

[0058] For example, since the exposure of the first original image is greater than that of the second original image, such as the first original image having a longer exposure time than the second original image, in order to fuse the first original image and the second original image in the same dimension, it is necessary to align the first original image and the second original image, so that the first original image is aligned to the dimension of the second original image for fusion.

[0059] To align the first original image and the second original image, the exposure difference ratio can be determined based on the exposure of the first original image and the exposure of the second original image. For example, the ratio of the exposure of the first original image to the exposure of the second original image can be used as the exposure difference ratio. Taking exposure time as an example, the ratio of the exposure time of the first original image to the exposure time of the second original image can be used as the exposure difference ratio.

[0060] After obtaining the exposure difference ratio, the temporally denoised image can be adjusted based on this ratio to align the adjusted image to the dimension of the second original image. That is, the dimension of the adjusted image will be the same as the dimension of the second original image. This alignment can be achieved by dividing the temporally denoised image by the exposure difference ratio. For example, dividing the pixel value of each pixel in the temporally denoised image by the exposure difference ratio.

[0061] After adjusting the temporally denoised image to obtain the adjusted image, a first intermediate image corresponding to the first original image can be generated based on the adjusted image. For example, the adjusted image can be used as the first intermediate image, or the adjusted image can be processed to obtain the first intermediate image. There are no restrictions on this.

[0062] In summary, based on steps 302-304, the first intermediate image corresponding to the first original image can be obtained.

[0063] Step 305: Perform spatial domain denoising on the second original image to obtain a spatially denoised image, and generate a second intermediate image corresponding to the second original image based on the spatially denoised image.

[0064] For example, since the second original image is a short-frame image, the exposure of a short-frame image is relatively small, such as a short exposure time. This results in a darker image with a poor signal-to-noise ratio, thus requiring noise reduction. Furthermore, because the short-frame image is dark, temporal denoising can easily lead to motion decision errors, which result in loss of detail. Therefore, spatial denoising can be performed on the second original image (i.e., the short-frame image) to obtain a spatially denoised image. For instance, using the 2DNR algorithm to perform 2D spatial denoising on the second original image can remove some large-grained noise while preserving smaller noise and details.

[0065] After obtaining the spatially denoised image, a second intermediate image corresponding to the second original image can be generated based on the spatially denoised image. For example, the spatially denoised image can be used as the second intermediate image, or the spatially denoised image can be processed to obtain the second intermediate image. There are no restrictions on this.

[0066] In summary, based on step 305, the second intermediate image corresponding to the second original image can be obtained.

[0067] Step 306: Based on the brightness information corresponding to the first intermediate image or the second intermediate image, perform brightness fusion on the first intermediate image and the second intermediate image to obtain a brightness weighted fused image.

[0068] In one possible implementation, the brightness of the first intermediate image and the second intermediate image can be fused based on the brightness information corresponding to the first intermediate image to obtain a brightness weighted fused image.

[0069] For example, for each pixel in the luminance weighted fusion image, taking pixel (x, y) as an example, the pixel value corresponding to pixel (x, y) in the first intermediate image is recorded as the first pixel value, the pixel value corresponding to pixel (x, y) in the second intermediate image is recorded as the second pixel value, and the pixel value corresponding to pixel (x, y) in the luminance weighted fusion image is recorded as the target pixel value. The target pixel value can be determined based on the first pixel value and / or the second pixel value. After obtaining the target pixel value corresponding to each pixel, the target pixel values ​​corresponding to all pixels can be combined to obtain the luminance weighted fusion image.

[0070] To determine the target pixel value corresponding to pixel (x, y) in the brightness weighted fused image, the following methods are employed: If the first pixel value is less than the third brightness threshold (which can be configured empirically, such as th1), then the target pixel value corresponding to pixel (x, y) is determined based on the first pixel value; for example, the first pixel value is used as the target pixel value corresponding to pixel (x, y). Alternatively, if the first pixel value is greater than the fourth brightness threshold (which can be configured empirically, such as th2, where the fourth brightness threshold is greater than the third brightness threshold), then the target pixel value corresponding to pixel (x, y) is determined based on the second pixel value; for example, the second pixel value is used as the target pixel value corresponding to pixel (x, y). Alternatively, if the first pixel value is between the third and fourth brightness thresholds (e.g., the first pixel value is not less than the third brightness threshold and not greater than the fourth brightness threshold), then a weighted operation is performed on the first pixel value and the second pixel value to obtain the target pixel value corresponding to pixel (x, y).

[0071] For example, see Figure 4 As shown, this is a schematic diagram of brightness fusion of the first intermediate image and the second intermediate image. The horizontal axis Yin represents the first pixel value corresponding to the pixel point (x, y) in the first intermediate image. That is, the first pixel value is used as a reference to determine how to determine the target pixel value corresponding to the pixel point (x, y). The vertical axis Yout represents the target pixel value corresponding to the pixel point (x, y) in the brightness weighted fusion image.

[0072] For each pixel in the brightness weighted fusion image, taking pixel (x, y) as an example, if the first pixel value Yin corresponding to pixel (x, y) is less than the third brightness threshold th1, then the target pixel value Yout corresponding to pixel (x, y) is Ylong, where Ylong is the pixel value (first pixel value) corresponding to pixel (x, y) in the first intermediate image. If the first pixel value Yin corresponding to pixel (x, y) is greater than the fourth brightness threshold th2, then the target pixel value Yout corresponding to pixel (x, y) is Yshort, where Yshort is the pixel value (second pixel value) corresponding to pixel (x, y) in the second intermediate image. If the first pixel value Yin corresponding to pixel (x, y) is between the third brightness threshold th1 and the fourth brightness threshold th2, then the target pixel value Yout corresponding to pixel (x, y) is Yblend, where Yblend is the weighted pixel value.

[0073] For example, Yblend can be calculated using the following formula: Yblend = w1*a + w2*b. Where a represents the first pixel value (x, y) in the first intermediate image, b represents the second pixel value (x, y) in the second intermediate image, Yblend represents the target pixel value (x, y) in the brightness weighted fusion image, w1 represents the weight coefficient of the first pixel value, and w2 represents the weight coefficient of the second pixel value. w1 and w2 can be configured empirically and are not restricted. For example, the sum of w1 and w2 can be 1, w1 can be greater than w2, w1 can be equal to w2, and w1 can be less than w2.

[0074] For example, if the first pixel value corresponding to pixel (x, y) is less than (th1 + th2) / 2, then w1 can be greater than w2, and the closer the first pixel value corresponding to pixel (x, y) is to th1, the larger w1 is. If the first pixel value corresponding to pixel (x, y) is equal to (th1 + th2) / 2, then w1 can be equal to w2. If the first pixel value corresponding to pixel (x, y) is greater than (th1 + th2) / 2, then w1 can be less than w2, and the closer the first pixel value corresponding to pixel (x, y) is to th2, the larger w2 is.

[0075] In another possible implementation, the brightness information corresponding to the second intermediate image can be used to fuse the brightness of the first intermediate image and the second intermediate image to obtain a brightness weighted fused image. The brightness fusion method is similar to the brightness fusion method based on the first intermediate image, except that the pixel value corresponding to the pixel point (x, y) in the second intermediate image is recorded as the first pixel value, and the other processes are similar.

[0076] Step 307: Based on the displacement difference image between the first intermediate image and the second intermediate image, perform frame difference fusion on the first intermediate image and the second intermediate image to obtain the frame difference weighted fused image.

[0077] For example, since there is a displacement difference between the first original image and the second original image, and a displacement difference between the first intermediate image and the second intermediate image, in order to make the displacement regions come from the same frame of data, frame difference fusion can be performed on the long and short frames. That is, based on the displacement difference image between the first intermediate image and the second intermediate image, frame difference fusion is performed on the first intermediate image and the second intermediate image to obtain the frame difference weighted fusion image.

[0078] In one possible implementation, frame difference fusion can be performed using the following steps:

[0079] Step 3071: Based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image, generate a displacement difference image, which includes the motion value of each pixel.

[0080] For example, for each pixel in the displacement difference image, taking pixel (x, y) as an example, the pixel value corresponding to pixel (x, y) in the first intermediate image is recorded as the first pixel value, the pixel value corresponding to pixel (x, y) in the second intermediate image is recorded as the second pixel value, and the pixel value corresponding to pixel (x, y) in the displacement difference image is recorded as the motion value. The motion value corresponding to pixel (x, y) can be determined based on the first pixel value, the second pixel value, and the noise level corresponding to the first pixel value. After obtaining the motion value corresponding to each pixel, the motion values ​​corresponding to all pixels are combined to obtain the displacement difference image.

[0081] For example, for a pixel (x, y) in a displacement difference image, the motion value of the pixel (x, y) in the displacement difference image is determined based on the first pixel value corresponding to the pixel (x, y) in the first intermediate image, the second pixel value corresponding to the pixel (x, y) in the second intermediate image, and the noise level corresponding to the first pixel value. A larger motion value indicates a larger displacement difference, representing more motion.

[0082] For example, the motion value of a pixel (x, y) in the displacement difference image can be determined using the following formula: motion = ||ab| - noiselevel|. In this formula, motion represents the motion value of a pixel (x, y) in the displacement difference image. A larger motion value indicates a larger displacement difference, and when fusing based on displacement differences, a larger motion value means more short frames are selected. 'a' represents the first pixel value of the pixel (x, y) in the first intermediate image, and 'b' represents the second pixel value of the pixel (x, y) in the second intermediate image.

[0083] `noiselevel` represents the noise level corresponding to the first pixel value. In one possible implementation, the noise level can be pre-configured, and all pixel values ​​correspond to the same noise level; therefore, the noise level corresponding to the first pixel value can be obtained. In another possible implementation, a mapping relationship can be pre-configured to represent the relationship between pixel values ​​and noise levels, where a larger pixel value corresponds to a smaller noise level. Of course, the above are just examples of mapping relationships, and there are no limitations on this mapping relationship, as long as it reflects the relationship between pixel values ​​and noise levels. Based on this, the noise level corresponding to the first pixel value can be obtained by querying the mapping relationship using the first pixel value.

[0084] In summary, this embodiment, when determining frame difference, needs to consider the impact of noise. Therefore, a noise estimation variable, noiselevel, can be introduced. Only when the absolute value of the difference between long and short frames (i.e., the difference between a and b) is greater than the noise estimation variable noiselevel, is a true motion displacement difference between the long and short frames considered to exist. The noise estimation variable noiselevel is not a fixed value; it changes with the value of the first pixel and has different values ​​under different brightness levels. This allows for a better evaluation of displacement differences under different brightness levels.

[0085] Step 3072: After obtaining the displacement difference image, perform frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image to obtain the frame difference weighted fused image, that is, the image after frame difference fusion.

[0086] For example, for each pixel in the frame difference weighted fusion image, taking pixel (x, y) as an example, the pixel value corresponding to pixel (x, y) in the first intermediate image is recorded as the first pixel value, the pixel value corresponding to pixel (x, y) in the second intermediate image is recorded as the second pixel value, the pixel value corresponding to pixel (x, y) in the frame difference weighted fusion image is recorded as the target pixel value, and the pixel value corresponding to pixel (x, y) in the displacement difference image is recorded as the target motion value. Based on this, the target pixel value is determined based on the first pixel value, the second pixel value, and the target motion value. After obtaining the target pixel value corresponding to each pixel, the target pixel values ​​corresponding to all pixels can be combined to obtain the frame difference weighted fusion image.

[0087] For example, to determine the target pixel value corresponding to pixel (x, y) in the frame difference weighted fusion image, then: if the target motion value corresponding to pixel (x, y) in the displacement difference image is less than a first displacement difference threshold (which can be configured empirically, such as th1), then the target pixel value corresponding to pixel (x, y) is determined based on the first pixel value; for example, the first pixel value is used as the target pixel value corresponding to pixel (x, y). Alternatively, if the target motion value corresponding to pixel (x, y) in the displacement difference image is greater than a second displacement difference threshold (which can be configured empirically, such as th2, where the second displacement difference threshold is greater than the first displacement difference threshold), then the target pixel value corresponding to pixel (x, y) is determined based on the second pixel value; for example, the second pixel value is used as the target pixel value corresponding to pixel (x, y). Alternatively, if the target motion value corresponding to pixel (x, y) in the displacement difference image is between the first displacement difference threshold and the second displacement difference threshold (e.g., the target motion value is not less than the first displacement difference threshold and not greater than the second displacement difference threshold), then the first pixel value and the second pixel value are weighted to obtain the target pixel value corresponding to pixel (x, y).

[0088] For example, see Figure 5 As shown, this is a schematic diagram of frame difference fusion of the first intermediate image and the second intermediate image. The horizontal axis motion represents the target motion value corresponding to the pixel (x, y) in the displacement difference image. That is, the target motion value is used as a reference to determine how to determine the target pixel value corresponding to the pixel (x, y). The vertical axis Yout represents the target pixel value corresponding to the pixel (x, y) in the frame difference weighted fusion image.

[0089] For each pixel in the frame difference weighted fused image, taking pixel (x, y) as an example, if the target motion value motion corresponding to pixel (x, y) is less than the first displacement difference threshold th1, then the target pixel value Yout corresponding to pixel (x, y) is Ylong, where Ylong is the first pixel value corresponding to pixel (x, y) in the first intermediate image. If the target motion value motion corresponding to pixel (x, y) is greater than the second displacement difference threshold th2, then the target pixel value Yout corresponding to pixel (x, y) is Yshort, where Yshort is the second pixel value corresponding to pixel (x, y) in the second intermediate image. If the target motion value motion corresponding to pixel (x, y) is between the first displacement difference threshold th1 and the second displacement difference threshold th2, then the target pixel value Yout corresponding to pixel (x, y) is Yblend, where Yblend is the weighted pixel value.

[0090] For example, Yblend can be calculated using the following formula: Yblend = w1*a + w2*b. Where a represents the first pixel value (x, y) in the first intermediate image, b represents the second pixel value (x, y) in the second intermediate image, Yblend represents the target pixel value (x, y) in the frame difference weighted fusion image, w1 represents the weight coefficient of the first pixel value, and w2 represents the weight coefficient of the second pixel value. w1 and w2 can be configured empirically and are not restricted. For example, the sum of w1 and w2 can be 1, w1 can be greater than w2, w1 can be equal to w2, and w1 can be less than w2.

[0091] For example, if the target motion value corresponding to pixel (x, y) is less than (th1 + th2) / 2, then w1 can be greater than w2, and the closer the target motion value corresponding to pixel (x, y) is to th1, the larger w1 can be. If the target motion value corresponding to pixel (x, y) is equal to (th1 + th2) / 2, then w1 can be equal to w2. If the target motion value corresponding to pixel (x, y) is greater than (th1 + th2) / 2, then w1 can be less than w2, and the closer the target motion value corresponding to pixel (x, y) is to th2, the larger w2 can be.

[0092] Step 308: Based on the motion region corresponding to the first original image, perform motion region fusion on the brightness weighted fusion image and the frame difference weighted fusion image to obtain the motion region fusion image, i.e. the fused image.

[0093] For example, after performing luminance fusion on the first intermediate image and the second intermediate image to obtain a luminance weighted fused image, and performing frame difference fusion on the first intermediate image and the second intermediate image to obtain a frame difference weighted fused image, the luminance weighted fused image has a good signal-to-noise ratio in the dark areas, but there is a ghosting problem in the motion area. The frame difference weighted fused image does not have a motion ghosting problem, but the signal-to-noise ratio in the dark areas is poor. Therefore, the luminance weighted fused image and the frame difference weighted fused image can be fused so that the dark areas of the fused image have a good signal-to-noise ratio and there is no ghosting problem in the motion area.

[0094] For example, for each pixel in the motion region fusion image, taking pixel (x, y) as an example, the pixel value corresponding to pixel (x, y) in the brightness weighted fusion image is recorded as the first pixel value, the pixel value corresponding to pixel (x, y) in the frame difference weighted fusion image is recorded as the second pixel value, the pixel value corresponding to pixel (x, y) in the motion region fusion image is recorded as the target pixel value, and the pixel value corresponding to pixel (x, y) in the first intermediate image is recorded as the reference pixel value. The target pixel value can be determined based on the reference pixel value, the first pixel value, and the second pixel value. After obtaining the target pixel value corresponding to each pixel, the target pixel values ​​corresponding to all pixels are combined to obtain the motion region fusion image.

[0095] For example, to determine the target pixel value corresponding to pixel (x, y) in the motion region fused image, we can first determine whether pixel (x, y) is located within the motion region. For instance, since the motion region corresponding to the first original image has already been obtained in step 303, such as the coordinates of the four vertices of a rectangular region, we can determine whether pixel (x, y) is located within this motion region. If pixel (x, y) is not located within this motion region, then the target pixel value corresponding to pixel (x, y) is determined based on the first pixel value. For example, the first pixel value can be used as the target pixel value corresponding to pixel (x, y).

[0096] If pixel (x, y) is located within the motion region, then: If the reference pixel value corresponding to pixel (x, y) in the first intermediate image is less than a first brightness threshold (which can be configured empirically, such as th3), then the target pixel value corresponding to pixel (x, y) can be determined based on the first pixel value. For example, the first pixel value can be used as the target pixel value corresponding to pixel (x, y). Alternatively, if the reference pixel value corresponding to pixel (x, y) in the first intermediate image is greater than a second brightness threshold (which can be configured empirically, such as th4, where the second brightness threshold is greater than the first brightness threshold), then the target pixel value corresponding to pixel (x, y) can be determined based on the second pixel value. For example, the second pixel value can be used as the target pixel value corresponding to pixel (x, y). Alternatively, if the reference pixel value corresponding to pixel (x, y) in the first intermediate image is between the first brightness threshold and the second brightness threshold (e.g., the reference pixel value is not less than the first brightness threshold and not greater than the second brightness threshold), then a weighted operation can be performed on the first pixel value and the second pixel value to obtain the target pixel value corresponding to pixel (x, y). Specifically, when performing a weighted calculation on the first pixel value and the second pixel value, if the moving speed of the vehicle within the moving area is greater than the speed threshold, the weight coefficient of the second pixel value can be greater than the weight coefficient of the first pixel value; if the moving speed of the vehicle is less than the speed threshold, the weight coefficient of the first pixel value can be greater than the weight coefficient of the second pixel value.

[0097] For example, see Figure 6 As shown, this is a schematic diagram of fusing the brightness weighted fusion image and the frame difference weighted fusion image. The horizontal axis Y represents the reference pixel value corresponding to the pixel (x, y) in the first intermediate image. That is, the reference pixel value is used as a reference to determine how to determine the target pixel value corresponding to the pixel (x, y). The vertical axis Yout represents the target pixel value corresponding to the pixel (x, y) in the motion region fusion image.

[0098] For each pixel in the motion region fusion image, taking pixel (x, y) as an example: if the reference pixel value Y corresponding to pixel (x, y) is less than the first brightness threshold th3, then the target pixel value Yout corresponding to pixel (x, y) is Yy, where Yy is the first pixel value corresponding to pixel (x, y) in the brightness weight fusion image. If the reference pixel value Y corresponding to pixel (x, y) is greater than the second brightness threshold th4, then the target pixel value Yout corresponding to pixel (x, y) is Ymotion, where Ymotion is the second pixel value corresponding to pixel (x, y) in the frame difference weight fusion image. If the reference pixel value Y corresponding to pixel (x, y) is between the first brightness threshold th3 and the second brightness threshold th4, then the target pixel value Yout corresponding to pixel (x, y) is Yblend, and Yblend can be a weighted pixel value.

[0099] For example, Yblend can be calculated using the following formula: Yblend = w1*a + w2*b. Where a represents the first pixel value of pixel (x, y) in the luminance-weighted fusion image, b represents the second pixel value of pixel (x, y) in the frame difference-weighted fusion image, Yblend represents the target pixel value of pixel (x, y) in the motion region fusion image, w1 represents the weight coefficient of the first pixel value, and w2 represents the weight coefficient of the second pixel value. w1 and w2 can be configured empirically and are not restricted. For example, the sum of w1 and w2 is 1; w1 can be greater than w2, w1 can be equal to w2, and w1 can be less than w2.

[0100] For example, when performing a weighted operation on the first and second pixel values, the fusion weights can be determined based on the ghosting effect. For instance, if the speed of the moving vehicle within the motion area is greater than a speed threshold (e.g., high speed), the weight coefficient w2 of the second pixel value can be greater than the weight coefficient w1 of the first pixel value, thus allowing the weights to be more concentrated on pixel values ​​from the frame difference weighted fusion image. If the speed of the moving vehicle within the motion area is less than a speed threshold (e.g., slow speed), the weight coefficient w1 of the first pixel value can be greater than the weight coefficient w2 of the second pixel value, thus allowing the weights to be more concentrated on pixel values ​​from the brightness weighted fusion image. If the speed of the moving vehicle within the motion area is equal to a speed threshold (e.g., moderate speed), the weight coefficient w1 of the first pixel value can be equal to the weight coefficient w2 of the second pixel value.

[0101] Step 309: Generate the target image to be output based on the motion region fusion image.

[0102] In one possible implementation, the motion region fusion image can be used as the target image. Alternatively, since an alignment operation was performed along the same dimension before the fusion operation (see step 304 for the alignment operation), after obtaining the motion region fusion image, a dynamic range enhancement operation can be performed on the motion region fusion image to obtain the target image. This can be achieved by using methods such as local brightness enhancement to increase the brightness of the motion region fusion image to the brightness corresponding to the first original image. This process is not limited. After obtaining the target image, the target image is the high dynamic range image, which is the final output image.

[0103] As can be seen from the above technical solutions, in this embodiment, combining displacement difference images and motion regions for fusion can fully utilize the information of bright and dark areas in each image, and fully utilize the useful detail information of overexposed and underexposed areas, resulting in a better visual effect of the fused image. For example, displacement difference images are used to determine the displacement difference of license plates in different images, and the fusion weight is determined by the displacement difference, thereby solving the problem of license plate ghosting. Another example is motion region fusion, which achieves the goal of obtaining license plates and headlight halos from short-exposure images, and vehicle bodies and surrounding environments from long-exposure images, solving the ghosting problem of fast-moving vehicles. Without sacrificing the brightness of the surrounding environment, headlight halos are reduced, improving the dynamic range of the image. The first original image undergoes 3D noise reduction, which effectively removes noise and improves the signal-to-noise ratio after wide dynamic range fusion. Simultaneously, the motion region is used for subsequent applications, solving the ghosting problem of wide dynamic range fusion for fast-moving vehicles. Based on true motion region fusion, headlight halos are reduced without sacrificing the brightness of the surrounding environment, improving the dynamic range of the image. By using a fusion method based on the frame difference between long and short frames—that is, the difference between the long and short frames representing the frame difference—the displacement difference of the license plate in the fused long and short frames is determined. The weight of the fusion is determined by the displacement difference, thus solving the problem of license plate ghosting. When the car is moving at high speed, the displacement difference of the license plate is large, and the short frame is selected for the largest displacement difference. The headlight halo also has a displacement difference, and the headlight halo is also selected for the short frame. However, the car body also has a displacement difference. If the short frame is selected for the dark car body, there may be a problem of large car body noise. Therefore, a new fusion based on the actual moving area is added. The effect of brightness fusion is selected for the stationary area, and then fusion is performed separately according to the brightness in the moving area. The result of frame difference fusion is selected for the brighter area in the moving area, and the result of brightness fusion is selected for the darker area in the moving area. In this way, the license plate and headlight halo come from the short frame, the dark car body comes from the long frame, and the surrounding environment comes from the long frame.

[0104] Based on the same concept as the above method, this application proposes a license plate image processing device, see [link]. Figure 7 The diagram shown is a structural schematic of a license plate image processing device, which may include:

[0105] The acquisition module 71 is used to acquire a first intermediate image corresponding to a first original image and a second intermediate image corresponding to a second original image, wherein the exposure of the first original image is greater than the exposure of the second original image; wherein both the first original image and the second original image include the license plate of the moving vehicle.

[0106] Processing module 72 is configured to generate a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image; perform frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image to obtain a frame difference weighted fusion image; and generate a target image to be output based on the frame difference weighted fusion image.

[0107] For example, when the processing module 72 generates a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image, it specifically performs the following: for each pixel in the displacement difference image, based on the first pixel value corresponding to the pixel in the first intermediate image, the second pixel value corresponding to the pixel in the second intermediate image, and the noise level corresponding to the first pixel value, it determines the motion value corresponding to the pixel in the displacement difference image; wherein, the larger the motion value, the larger the displacement difference; the noise level is obtained by querying the mapping relationship through the first pixel value, and the mapping relationship represents the relationship between the pixel value and the noise level.

[0108] For example, when the processing module 72 performs frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image to obtain a frame difference weighted fusion image, it specifically performs the following: for each pixel in the frame difference weighted fusion image, if the target motion value corresponding to the pixel in the displacement difference image is less than a first displacement difference threshold, then the target pixel value corresponding to the pixel in the frame difference weighted fusion image is determined based on the first pixel value corresponding to the pixel in the first intermediate image; if the target motion value is greater than a second displacement difference threshold, then the target pixel value is determined based on the second pixel value corresponding to the pixel in the second intermediate image; if the target motion value is between the first displacement difference threshold and the second displacement difference threshold, then the first pixel value and the second pixel value are weighted to obtain the target pixel value.

[0109] For example, when the acquisition module 71 acquires the first intermediate image corresponding to the first original image and the second intermediate image corresponding to the second original image, it is specifically used to: perform temporal domain denoising on the first original image to obtain a temporally denoised image; determine the exposure difference ratio based on the exposure of the first original image and the exposure of the second original image, and adjust the temporally denoised image based on the exposure difference ratio to obtain an adjusted image; generate the first intermediate image corresponding to the first original image based on the adjusted image; perform spatial domain denoising on the second original image to obtain a spatially denoised image; and generate the second intermediate image corresponding to the second original image based on the spatially denoised image.

[0110] For example, when the processing module 72 generates the target image to be output based on the frame difference weighted fusion image, it is specifically used to: perform brightness fusion on the first intermediate image and the second intermediate image based on the brightness information corresponding to the first intermediate image or the second intermediate image to obtain a brightness weighted fusion image; fuse the brightness weighted fusion image and the frame difference weighted fusion image to obtain a fused image, and generate the target image to be output based on the fused image.

[0111] For example, when the processing module 72 fuses the brightness weighted fusion image and the frame difference weighted fusion image to obtain the fused image, it specifically performs the following steps: obtaining a temporal denoised image corresponding to the first original image; obtaining a motion region corresponding to the moving vehicle in the first original image based on the temporal denoised image; and performing motion region fusion on the brightness weighted fusion image and the frame difference weighted fusion image based on the motion region corresponding to the moving vehicle in the first original image to obtain the fused image.

[0112] For example, when the processing module 72 performs motion region fusion on the brightness weighted fusion image and the frame difference weighted fusion image based on the motion region corresponding to the moving vehicle in the first original image to obtain the fused image, it specifically performs the following: For each pixel in the fused image, if the pixel is not located within the motion region, then the target pixel value corresponding to the pixel in the fused image is determined based on the first pixel value corresponding to the pixel in the brightness weighted fusion image; if the pixel is located within the motion region, and the reference pixel value corresponding to the pixel in the first intermediate image is less than a first brightness threshold, then the target pixel value is determined based on the first pixel value; if the reference pixel value is greater than a second brightness threshold, then the target pixel value is determined based on the second pixel value corresponding to the pixel in the frame difference weighted fusion image; if the reference pixel value is between the first brightness threshold and the second brightness threshold, then a weighted operation is performed on the first pixel value and the second pixel value to obtain the target pixel value; wherein, if the moving speed of the moving vehicle is greater than a speed threshold, then the weight coefficient of the second pixel value is greater than the weight coefficient of the first pixel value, and if the moving speed of the moving vehicle is less than a speed threshold, then the weight coefficient of the first pixel value is greater than the weight coefficient of the second pixel value.

[0113] Based on the same concept as the above method, this application proposes an electronic device, see [link to previous application]. Figure 8 As shown, the electronic device may include a processor 81 and a machine-readable storage medium 82, the machine-readable storage medium 82 storing machine-executable instructions that can be executed by the processor 81; the processor 81 is used to execute the machine-executable instructions to implement the license plate image processing method disclosed in the above example of this application.

[0114] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the license plate image processing method disclosed in the above examples of this application.

[0115] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0116] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer entity or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0117] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for processing license plate images, characterized in that, The method includes: A first intermediate image corresponding to a first original image and a second intermediate image corresponding to a second original image are obtained, wherein the exposure of the first original image is greater than the exposure of the second original image; wherein both the first original image and the second original image include the license plate of the moving vehicle. Based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image, a displacement difference image between the first intermediate image and the second intermediate image is generated. Based on the displacement difference image between the first intermediate image and the second intermediate image, frame difference fusion is performed on the first intermediate image and the second intermediate image to obtain a frame difference weighted fusion image. The target image to be output is generated based on the frame difference weighted fusion image; The step of generating the target image to be output based on the frame difference weighted fusion image includes: performing brightness fusion on the first intermediate image and the second intermediate image based on the brightness information corresponding to the first intermediate image or the second intermediate image to obtain a brightness weighted fusion image; fusing the brightness weighted fusion image and the frame difference weighted fusion image to obtain a fused image; and generating the target image to be output based on the fused image. The step of fusing the brightness weighted fusion image and the frame difference weighted fusion image to obtain the fused image includes: obtaining the temporal denoised image corresponding to the first original image, and obtaining the motion region corresponding to the moving vehicle in the first original image based on the temporal denoised image; Based on the motion region corresponding to the moving vehicle in the first original image, motion region fusion is performed on the brightness weighted fusion image and the frame difference weighted fusion image to obtain the fused image.

2. The method according to claim 1, characterized in that, The step of generating a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image includes: For each pixel in the displacement difference image, based on the first pixel value corresponding to the pixel in the first intermediate image, the second pixel value corresponding to the pixel in the second intermediate image, and the noise level corresponding to the first pixel value, the motion value corresponding to the pixel in the displacement difference image is determined; wherein, the larger the motion value, the larger the displacement difference; the noise level is obtained by querying the mapping relationship through the first pixel value, and the mapping relationship represents the relationship between the pixel value and the noise level.

3. The method according to claim 1, characterized in that, The step of performing frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image to obtain a frame difference weighted fused image includes: For each pixel in the frame difference weighted fusion image, if the target motion value corresponding to the pixel in the displacement difference image is less than a first displacement difference threshold, then the target pixel value corresponding to the pixel in the frame difference weighted fusion image is determined based on the first pixel value corresponding to the pixel in the first intermediate image; if the target motion value is greater than a second displacement difference threshold, then the target pixel value is determined based on the second pixel value corresponding to the pixel in the second intermediate image; if the target motion value is between the first displacement difference threshold and the second displacement difference threshold, the first pixel value and the second pixel value are weighted to obtain the target pixel value.

4. The method according to any one of claims 1-3, characterized in that, The step of obtaining the first intermediate image corresponding to the first original image and the second intermediate image corresponding to the second original image includes: Temporal denoising is performed on the first original image to obtain a temporally denoised image; an exposure difference ratio is determined based on the exposure of the first original image and the exposure of the second original image, and the temporally denoised image is adjusted based on the exposure difference ratio to obtain an adjusted image; a first intermediate image corresponding to the first original image is generated based on the adjusted image. Spatial domain denoising is performed on the second original image to obtain a spatially denoised image; a second intermediate image corresponding to the second original image is generated based on the spatially denoised image.

5. The method according to claim 1, characterized in that, The process involves fusing the motion regions of the moving vehicles in the first original image with the brightness-weighted fused image and the frame difference-weighted fused image to obtain a fused image, including: For each pixel in the fused image, if the pixel is not located in the motion region, the target pixel value corresponding to the pixel in the fused image is determined based on the first pixel value corresponding to the pixel in the brightness weighted fused image. If the pixel is located within the motion region, and the reference pixel value corresponding to the pixel in the first intermediate image is less than the first brightness threshold, then the target pixel value is determined based on the first pixel value; if the reference pixel value is greater than the second brightness threshold, then the target pixel value is determined based on the second pixel value corresponding to the pixel in the frame difference weighted fusion image; if the reference pixel value is between the first brightness threshold and the second brightness threshold, then the first pixel value and the second pixel value are weighted to obtain the target pixel value. Wherein, if the moving speed of the moving vehicle is greater than the speed threshold, the weight coefficient of the second pixel value is greater than the weight coefficient of the first pixel value; if the moving speed of the moving vehicle is less than the speed threshold, the weight coefficient of the first pixel value is greater than the weight coefficient of the second pixel value.

6. A license plate image processing device, characterized in that, The device includes: The acquisition module is used to acquire a first intermediate image corresponding to a first original image and a second intermediate image corresponding to a second original image, wherein the exposure of the first original image is greater than the exposure of the second original image; wherein both the first original image and the second original image include the license plate of the moving vehicle. The processing module is configured to generate a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image; perform frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image to obtain a frame difference weighted fusion image; and generate a target image to be output based on the frame difference weighted fusion image. Specifically, when the processing module generates the target image to be output based on the frame difference weighted fusion image, it performs the following: Based on the brightness information corresponding to the first intermediate image or the second intermediate image, it performs brightness fusion on the first intermediate image and the second intermediate image to obtain a brightness weighted fusion image; it then fuses the brightness weighted fusion image and the frame difference weighted fusion image to obtain a fused image, and generates the target image to be output based on the fused image. Specifically, when the processing module fuses the brightness weighted fusion image and the frame difference weighted fusion image to obtain the fused image, it performs the following steps: obtaining the temporal denoised image corresponding to the first original image; obtaining the motion region corresponding to the moving vehicle in the first original image based on the temporal denoised image; and fusing the motion region of the brightness weighted fusion image and the frame difference weighted fusion image based on the motion region corresponding to the moving vehicle in the first original image to obtain the fused image.

7. The apparatus according to claim 6, characterized in that, in, When the processing module generates a displacement difference image between the first intermediate image and the second intermediate image based on the first intermediate image, the second intermediate image, and the noise level corresponding to the first intermediate image, it specifically performs the following steps: For each pixel in the displacement difference image, based on the first pixel value corresponding to the pixel in the first intermediate image, the second pixel value corresponding to the pixel in the second intermediate image, and the noise level corresponding to the first pixel value, it determines the motion value corresponding to the pixel in the displacement difference image; wherein, the larger the motion value, the larger the displacement difference; the noise level is obtained by querying the mapping relationship through the first pixel value, and the mapping relationship represents the relationship between the pixel value and the noise level; Specifically, when the processing module performs frame difference fusion on the first intermediate image and the second intermediate image based on the displacement difference image between the first intermediate image and the second intermediate image to obtain a frame difference weighted fusion image, it performs the following: for each pixel in the frame difference weighted fusion image, if the target motion value corresponding to the pixel in the displacement difference image is less than a first displacement difference threshold, then the target pixel value corresponding to the pixel in the frame difference weighted fusion image is determined based on the first pixel value corresponding to the pixel in the first intermediate image; if the target motion value is greater than a second displacement difference threshold, then the target pixel value is determined based on the second pixel value corresponding to the pixel in the second intermediate image; if the target motion value is between the first displacement difference threshold and the second displacement difference threshold, then the first pixel value and the second pixel value are weighted to obtain the target pixel value. Specifically, when the acquisition module acquires the first intermediate image corresponding to the first original image and the second intermediate image corresponding to the second original image, it is used to: perform temporal domain denoising on the first original image to obtain a temporally denoised image; determine the exposure difference ratio based on the exposure of the first original image and the exposure of the second original image, and adjust the temporally denoised image based on the exposure difference ratio to obtain an adjusted image; generate the first intermediate image corresponding to the first original image based on the adjusted image; perform spatial domain denoising on the second original image to obtain a spatially denoised image; and generate the second intermediate image corresponding to the second original image based on the spatially denoised image. Specifically, the processing module performs motion region fusion on the brightness weighted fusion image and the frame difference weighted fusion image based on the motion region corresponding to the moving vehicle in the first original image to obtain the fused image. Specifically, for each pixel in the fused image, if the pixel is not located within the motion region, a target pixel value is determined in the fused image based on the first pixel value corresponding to the pixel in the brightness weighted fusion image; if the pixel is located within the motion region, and the reference pixel value corresponding to the pixel in the first intermediate image is less than a first brightness threshold, a target pixel value is determined based on the first pixel value; if the reference pixel value is greater than a second brightness threshold, a target pixel value is determined based on the second pixel value corresponding to the pixel in the frame difference weighted fusion image; if the reference pixel value is between the first brightness threshold and the second brightness threshold, a weighted operation is performed on the first pixel value and the second pixel value to obtain the target pixel value; wherein, if the moving speed of the moving vehicle is greater than a speed threshold, the weight coefficient of the second pixel value is greater than the weight coefficient of the first pixel value; if the moving speed of the moving vehicle is less than a speed threshold, the weight coefficient of the first pixel value is greater than the weight coefficient of the second pixel value.

8. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method described in any one of claims 1-5.

Images