Image processing method and device, electronic equipment, storage medium and chip

Abstract

The invention provides an image processing method and device, electronic equipment, a storage medium and a chip, and the method comprises the steps: carrying out the low-pass filtering of a to-be-processed image, and obtaining a basic image and a detail image of the to-be-processed image; the to-be-processed image is a grey-scale map; performing contrast enhancement processing on the detail image, and determining the detail image after contrast enhancement; and fusing the contrast-enhanced detail image and the basic image to obtain a target image. According to the scheme of the invention, the method comprises the steps: obtaining the detail information of a to-be-processed image under different scales through the low-pass filtering processing of different scales of the to-be-processed image; and the contrast enhancement processing is performed on the detail images under different scales, so that the problem that the details of the to-be-processed image under different scales cannot be processed in related technologies is solved, more accurate image detail information is included, and the image quality is effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device, electronic device, storage medium and chip. Background Art

[0002] As smartphone camera technology continues to update, the software and hardware technologies involved are also developing synchronously. However, due to the technology and cost of display devices, the grayscale range it displays is only 0-255. When the brightness range of a real scene is linearly compressed to such a small range for display, it will cause a large amount of detail information in the image to be seriously lost, resulting in low image quality. Summary of the invention

[0003] The present disclosure provides an image processing method, device, electronic device, storage medium and chip to solve the problems in the related art and improve the quality of images.

[0004] A first aspect of the present disclosure provides an image processing method, the method comprising:

[0005] Performing low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image;

[0006] Performing contrast enhancement processing on the detail image to determine a detail image after contrast enhancement;

[0007] The contrast-enhanced detail image and the base image are fused to obtain a target image.

[0008] In some embodiments, the image processing method further includes:

[0009] The target image is adjusted using the dynamic range adjustment curve to obtain an adjusted target image.

[0010] In some embodiments, the image processing method further includes:

[0011] Get the original image in RGB format;

[0012] Convert the original image into the image to be processed;

[0013] Based on the adjusted target image and the image to be processed, obtaining global gain information;

[0014] Based on the global gain information, each image channel of the original image in the RGB format is enhanced to obtain a processed image in the RGB format.

[0015] In some embodiments, the step of performing low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed includes:

[0016] The image to be processed is subjected to bilateral filtering processing of different scales to obtain a basic image and a detail image of the image to be processed.

[0017] In some embodiments, the performing of bilateral filtering at different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed includes:

[0018] Performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed;

[0019] Performing three-dimensional filtering on the first bilateral grid to obtain a filtered first bilateral grid;

[0020] Performing interpolation processing on the filtered first bilateral grid to obtain a basic image with the same scale as the image to be processed;

[0021] The detail image is obtained based on a difference image between the image to be processed and a basic image.

[0022] In some embodiments, performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed includes:

[0023] Based on a preset spatial sampling rate, a first scale downsampling process is performed on the image to be processed to obtain a first bilateral grid of the image to be processed.

[0024] In some embodiments, the interpolation processing of the filtered first bilateral grid includes:

[0025] Perform trilinear interpolation processing on the filtered first bilateral grid.

[0026] In some embodiments, the step of performing a first scale downsampling process on the image to be processed based on a preset first spatial sampling rate to obtain a first bilateral grid of the image to be processed includes:

[0027] Based on a preset first spatial sampling rate, obtaining a two-dimensional spatial grid of the first bilateral grid;

[0028] Based on the sum of pixel values ​​of all pixel points corresponding to each grid in the two-dimensional space grid in the image to be processed, determine the value of each grid;

[0029] A first bilateral grid of the image to be processed based on the two-dimensional spatial grid and the values ​​of each grid.

[0030] In some embodiments, after obtaining the detail image based on the difference image between the image to be processed and the base image, the method further includes:

[0031] Using the basic image to update the image to be processed, performing low-pass filtering of different scales on the updated image to be processed to obtain an updated detail image;

[0032] After a preset number of iterations, the updated basic image and the detailed image are obtained.

[0033] A second aspect of the present disclosure provides an image processing device, the device comprising:

[0034] A low-pass filtering unit is used to perform low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image;

[0035] A contrast enhancement unit, configured to perform contrast enhancement processing on the detail image to determine a detail image after contrast enhancement;

[0036] The fusion unit is used to fuse the contrast-enhanced detail image and the basic image to obtain a target image.

[0037] A third aspect of the present disclosure provides an electronic device, including:

[0038] at least one processor;

[0039] and a memory communicatively coupled to the at least one processor;

[0040] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the method described in the embodiment of the first aspect of the present disclosure.

[0041] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0042] The fifth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the method described in the first aspect embodiment of the present disclosure.

[0043] In summary, the present disclosure proposes an image processing method, device, electronic device, storage medium and chip. The method includes: performing low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image; performing contrast enhancement on the detail image to determine the detail image after contrast enhancement; fusing the detail image after contrast enhancement and the basic image to obtain a target image; in the scheme of the present disclosure, by performing low-pass filtering of different scales on the image to be processed, the detail information of the image to be processed at different scales is obtained; and by performing contrast enhancement on the detail images at different scales, the problem that the related technology cannot process the details of the image to be processed at different scales is solved. Finally, the target image obtained by the detail image after contrast enhancement contains more accurate image detail information, which effectively improves the image quality.

[0044] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0046] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0047] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0048] Figure 3 A schematic diagram of a tone mapping curve provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a backlit image provided in an embodiment of the present application;

[0050] Figure 5 A flowchart of an image processing method provided for an application example of the present disclosure;

[0051] Figure 6a A schematic diagram of an input image provided for an application example of the present disclosure;

[0052] Figure 6b A schematic diagram of a target image obtained by using Laplace transform provided for an application example of the present disclosure;

[0053] Figure 6cA schematic diagram of a target image obtained by using the image processing method described in the application example of the present disclosure, provided for the application example of the present disclosure;

[0054] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0055] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0056] Fig. 9 A schematic diagram of the structure of a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0057] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0058] As smartphone camera technology continues to update, the software and hardware technologies involved are also developing synchronously. However, due to the technology and cost of display devices, the grayscale range it displays is only 0-255. When the brightness range of the real scene is linearly compressed to such a small range for display, it leads to serious loss of information and details in a large area of ​​the image.

[0059] In related technologies, the global dynamic range and contrast of an image are adjusted mainly through tone mapping technology, and the real scene is displayed in front of people through an image by nonlinearly compressing the dynamic range of the real scene. Tone mapping technology is mainly divided into three types: global mapping, local mapping, and hybrid mapping.

[0060] Global mapping refers to re-adjusting the grayscale range of the image acquired by the sensor by designing a mapping curve. Global mapping includes linear / nonlinear contrast stretching, image histogram equalization, etc. This type of algorithm is relatively simple and convenient for real-time calculation of the algorithm. However, since the entire image is adjusted based on the same mapping curve during global adjustment, it is very likely to cause insufficient local contrast.

[0061] Local mapping performs different processing based on different areas of the image, making up for the problem that global mapping may lose local contrast, such as adaptive histogram equalization with limited contrast, fast bilateral filtering, etc. However, due to the block processing of the image, it may cause halo effect.

[0062] Hybrid mapping combines the global and local mapping schemes, using the global mapping as a foundation and then adjusting the local contrast based on the global mapping. Therefore, hybrid mapping takes into account both the overall contrast and the local contrast of the image, and the image display quality is high.

[0063] In the related art, the widely used hybrid mapping algorithm is the bilateral filtering algorithm. The bilateral filtering algorithm introduces range information, and through the joint action of spatial domain filtering and range filtering, it solves the problem that spatial domain filtering cannot preserve edges. However, due to the high complexity of the algorithm, people have been working hard to accelerate it until the bilateral grid algorithm came out, which can achieve real-time calculation. The basic idea of ​​the bilateral grid algorithm is to transform the nonlinear convolution of a two-dimensional image into a three-dimensional linear convolution operation. Through the dimension-raising operation, the image is raised to three dimensions, and the third dimension is the pixel value, so that the calculation of the two can be unified. Spatial domain filtering still performs convolution on the original two dimensions of the image, while range filtering performs convolution on the third dimension. However, the bilateral filtering algorithm does not take into account the detailed features of the image at different scales, and the details of the image obtained are still lost.

[0064] In the related art, the main algorithm that takes into account the image detail information at different scales is the local Laplacian pyramid algorithm. It achieves the purpose of edge perception by constructing a Laplacian pyramid, and achieves the effect of distinguishing large-scale edges and small-scale details by setting a simple threshold, and distinguishes the details of different scales of the image with the help of different levels of the pyramid. Specifically, the local Laplacian pyramid algorithm first needs to build a Gaussian pyramid based on the original image, and then traverse all pixels in all Gaussian pyramid images. According to the pixel value of each pixel, a new value that is consistent with the original image is calculated by using the original image and a mapping function. Figure 1 The Laplacian pyramid of the image is calculated. At this time, the Laplacian pyramid only needs to be constructed to the layer of the Gaussian pyramid where the corresponding pixel is located, and then the value of the temporary pyramid corresponding to the pixel position is taken as the Laplacian pyramid value of the result image at this position. When the pixels of all layers are calculated, the Laplacian pyramid of the result image is constructed, and the image to be processed can be reconstructed by the Laplacian pyramid. However, since each coefficient of each layer of the pyramid must be constructed once, the local Laplacian pyramid algorithm is very time-consuming, has high requirements on processor performance, and has a large chip area after hardening, high cost, and cannot be used on smart mobile devices.

[0065] The image processing method provided by the present disclosure obtains the detail information of the image to be processed at different scales by performing low-pass filtering processing at different scales on the image to be processed; and solves the problem that the related technology cannot process the details of the image to be processed at different scales by performing contrast enhancement processing on the detail images at different scales. Finally, the target image obtained by the detail image after contrast enhancement contains more accurate image detail information, which effectively improves the image quality.

[0066] Before introducing the detailed solution of the present disclosure, the scenario to which the solution of the present disclosure is applied is described first. Figure 1 FIG. 1 is an application scenario diagram of an image processing method in an embodiment. Figure 1 As shown, the application scenario includes an electronic device 101, in which a camera module and several applications can be installed. The application can initiate an image acquisition instruction to acquire an image, and the camera module acquires an image 102. Among them, the camera module can include a front camera module and / or a rear camera module. Finally, the image 102 is sent to the target application. Among them, the electronic device 101 can be a smart phone, a tablet computer, a personal digital assistant, a wearable device, etc.

[0067] In some optional embodiments, the electronic device 101 may also be a vehicle-mounted device or a vehicle networking device, such as a smart car, etc. In the present disclosure, only a smart phone is taken as an example, but it does not limit the scope of the present disclosure.

[0068] A camera may be installed on the electronic device 101, and an image 102 may be acquired through the installed camera. The camera may be divided into laser cameras, visible light cameras, and other types according to the different acquired images 102. The laser camera may acquire the image 102 formed by irradiating the laser onto the object, and the visible light camera may acquire the image formed by irradiating the visible light onto the object. Several cameras may be installed on the electronic device 101, and the installation positions are not limited. For example, a camera may be installed on the front panel of the electronic device 101, and two cameras may be installed on the back panel. The camera may also be installed inside the electronic device 101 in an embedded manner, and then the camera may be opened by rotating or sliding. Specifically, a front camera and a rear camera may be installed on the electronic device 101. The front camera and the rear camera may acquire the image 102 from different perspectives. Generally, the front camera may acquire the image 102 from the front perspective of the electronic device, and the rear camera may acquire the image 102 from the back perspective of the electronic device.

[0069] It should be understood that in the present disclosure, the front camera or the rear camera is only used as an example to distinguish the shooting angles of different cameras, rather than to limit the functions of multiple cameras. The multiple cameras in the present disclosure can be rear cameras or front cameras at the same time, which is not limited in the present disclosure.

[0070] Several applications can be installed in the electronic device 101. An application refers to software written in an electronic device for a certain application purpose. The electronic device 101 can provide services to users through the application. When the application needs to capture the image 102, an image capture instruction will be initiated. The electronic device 101 will call the camera module according to the image capture instruction to capture the image 102. The image capture instruction refers to an instruction used to trigger the image capture operation.

[0071] The electronic device 101 is also provided with a processor, and the image processing module in the processor can reconstruct the image 102 captured by the camera module, for example, execute the image processing method provided in the present disclosure.

[0072] The embodiments of the present disclosure can be applied to scenes that require image processing, for example, image processing in single-frame shooting mode, image processing in multi-frame shooting mode, image processing in preview mode, image processing in video recording mode, image processing in night scene shooting or video mode, etc. For another example, it can also be applied to the ISP module as a key module that affects the processor. These include but are not limited to the following scenes:

[0073] Image processing in single-frame shooting mode, image processing in multi-frame shooting mode, image processing in preview mode, image processing in video recording mode, night scene shooting, image processing in video mode, etc.

[0074] Figure 2 The following is a flow chart of an image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0075] Step 201, performing low-pass filtering processing of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image;

[0076] In one embodiment, the image to be processed is a grayscale image; the grayscale image may be converted from an image in a different format, or may be a grayscale image directly acquired by a camera sensor. The image in a different format may be a RAW original image in Bayer format, or may be an RGB image obtained by processing the RAW original image, which is not limited in the present disclosure.

[0077] In one embodiment, the low-pass filtering of different scales on the processed image can be performed using different low-pass filtering software, such as OpenCV, MATLAB, etc. A local Laplacian pyramid algorithm, a bilateral filtering algorithm, a bilateral grid algorithm, etc. can also be used, which is not limited in the present disclosure.

[0078] The basic image refers to an image obtained by restoring the original scale of the image to be processed after scale transformation, and the detail image refers to a difference image between the image to be processed and the basic image.

[0079] Step 202, performing contrast enhancement processing on the detail image to determine a detail image after contrast enhancement;

[0080] In one embodiment, the contrast enhancement processing of the detail image includes contrast enhancement processing of each pixel in the detail image. The method of contrast enhancement processing of each pixel can use a method of adjusting the intensity of the lookup table to find the mapping relationship information between the initial brightness value of each pixel and the enhancement degree, or it can be embodied in the form of a brightness enhancement function, which is not limited in the present disclosure. The brightness enhancement lookup table and the brightness enhancement function can be preset.

[0081] Step 203: Fusing the contrast-enhanced detail image and the basic image to obtain a target image.

[0082] In one embodiment, since the scales of the detail image and the basic image obtained by low-pass filtering are the same as the scale of the image to be processed, the difficulty of image fusion is reduced. When fusing the detail image and the basic image, image processing software such as OpenCV and MATLAB may be used, or an image fusion algorithm may be used, which is not limited in the present disclosure.

[0083] The image processing method provided by the disclosed embodiment obtains the detail information of the image to be processed at different scales by performing low-pass filtering processing at different scales on the image to be processed; and solves the problem that the related technology cannot process the details of the image to be processed at different scales by performing contrast enhancement processing on the detail images at different scales. Finally, the target image obtained by the detail image after contrast enhancement contains more accurate image detail information, which effectively improves the image quality.

[0084] Since the target image obtained by low-pass filtering of different scales pays more attention to the contrast adjustment of image details, in order to obtain a target image of higher quality, the target image may be further subjected to global contrast adjustment.

[0085] Based on this, in one embodiment, the image processing method provided by the embodiment of the present disclosure further includes:

[0086] The target image is adjusted using the dynamic range adjustment curve to obtain an adjusted target image.

[0087] In one embodiment, the dynamic range adjustment curve may be: Figure 3 The tone mapping curve tm_curve shown is used to map the target image according to the tm_curve curve to obtain an adjusted target image.

[0088] Since the most commonly used image processing scenario is the image processing scenario of processing the image collected by the camera. Therefore, the image to be processed can be a grayscale image, or a grayscale image converted from an original image in another format. In the related art, the image collected by the camera is usually a RAW original image obtained by a sensor using a Bayer array, and an RGB image after demosaicing, color correction, denoising, etc.

[0089] Based on this, in one embodiment, the image processing method provided by the embodiment of the present disclosure further includes:

[0090] Get the original image in RGB format;

[0091] In one embodiment, the original image in RGB format may be an original image in RGB format obtained after a real object image is captured by a camera and processed, or may be an original image in RGB format crawled from the network.

[0092] Convert the original image into the image to be processed;

[0093] There are many ways to convert an original image in RGB format into a grayscale image, such as using image processing software such as OpenCV and MATLAB, or by multiplying the pixel values ​​of the three channels of each pixel in the original image in RGB format by the corresponding channel coefficients to obtain the grayscale image corresponding to the original image in RGB format, which is not limited in the present disclosure.

[0094] In one embodiment, since the target image is a grayscale image, it needs to be converted into a corresponding original image in RGB format.

[0095] Based on this, in one embodiment, the image processing method provided by the embodiment of the present disclosure further includes:

[0096] Based on the adjusted target image and the image to be processed, obtaining global gain information;

[0097] In one embodiment, based on the adjusted target image and the image to be processed, a gain value of each pixel can be obtained, and based on the gain value of each pixel, the global gain information can be obtained.

[0098] Based on the global gain information, each image channel of the original image in the RGB format is enhanced to obtain a processed image in the RGB format.

[0099] In one embodiment, the step of performing enhancement processing on each image channel of the original image in RGB format based on the global gain information to obtain a processed image in RGB format includes:

[0100] Convert the original image in RGB format into an image of R channel, an image of B channel and an image of G channel;

[0101] The gain value of each pixel point is multiplied by the R pixel value, G pixel value and B pixel value of the R channel image, the B channel image and the G channel image corresponding to each pixel point, and the product is used as the R pixel value, G pixel value and B pixel value of each pixel point of the R channel image, the B channel image and the G channel image after enhancement processing.

[0102] The R channel image, the B channel image and the G channel image are fused to obtain a processed image in RGB format.

[0103] In one embodiment, since the local Laplacian pyramid algorithm can well obtain image detail information at different scales, the local Laplacian pyramid algorithm is often used to perform low-pass filtering processing on the image at different scales. However, since each coefficient of each layer of the pyramid in the local Laplacian pyramid algorithm must be constructed once, the local Laplacian pyramid algorithm is very time-consuming, has high requirements on processor performance, has a large chip area after hardening, and is costly, and cannot be used on smart mobile devices.

[0104] Based on this, in one embodiment, the low-pass filtering of different scales is performed on the image to be processed to obtain a basic image and a detail image of the image to be processed, including:

[0105] The image to be processed is subjected to bilateral filtering adjustment processing of different scales to obtain a basic image and a detail image of the image to be processed.

[0106] In one embodiment, the step of performing bilateral filtering adjustment processing at different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed includes:

[0107] Performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed;

[0108] In one embodiment, when the image to be processed is downsampled at the first scale, the spatial sampling rate corresponding to the first scale downsampling can be preset as needed, for example, the spatial sampling rate is half, one quarter, etc. of the resolution of the image to be processed, which is not limited in the present disclosure.

[0109] Based on this, in one embodiment, performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed includes:

[0110] Based on a preset spatial sampling rate, a first scale downsampling process is performed on the image to be processed to obtain a first bilateral grid of the image to be processed.

[0111] In one embodiment, the bilateral grid is an improved version of the bilateral filter. It performs a dimensionality-up operation based on the two-dimensional spatial grid of the bilateral filter, and uses the pixel values ​​of the pixels in the grid as the value range, that is, the third dimension. Then, filtering is performed through a three-dimensional filter kernel, which greatly speeds up the speed of the bilateral filter.

[0112] Performing three-dimensional filtering on the first bilateral grid to obtain a filtered first bilateral grid;

[0113] In one embodiment, performing three-dimensional filtering on the first bilateral grid includes performing three-dimensional filtering on the first bilateral grid using a three-dimensional filter kernel, wherein the three-dimensional filter kernel can be preset in advance as needed, and the size, step length, etc. of the three-dimensional filter kernel are not limited in the present disclosure.

[0114] Performing interpolation processing on the filtered first bilateral grid to obtain a basic image with the same scale as the image to be processed;

[0115] In one embodiment, the filtered first bilateral grid may be interpolated using a trilinear interpolation method. The trilinear interpolation method may be based on the index of each pixel point in the first bilateral grid information and the eight vertices corresponding to each pixel point in the first bilateral grid information to perform trilinear interpolation to obtain a basic image with the same scale as the image to be processed.

[0116] Based on this, in one embodiment, the interpolation processing of the filtered first bilateral grid includes:

[0117] Perform trilinear interpolation processing on the filtered first bilateral grid.

[0118] The detail image is obtained based on a difference image between the image to be processed and a basic image.

[0119] In one embodiment, the pixel value of each pixel in the detail image is the difference between the corresponding pixel in the image to be processed and the basic image.

[0120] In one embodiment, since the bilateral grid includes a two-dimensional spatial grid based on bilateral filtering and the pixel values ​​of the pixels in the grid as the third dimension of the value range, based on this, in one embodiment, the first scale downsampling processing is performed on the image to be processed based on a preset first spatial sampling rate to obtain the first bilateral grid of the image to be processed, including:

[0121] Based on a preset first spatial sampling rate, obtaining a two-dimensional spatial grid of the first bilateral grid;

[0122] Based on the sum of pixel values ​​of all pixel points corresponding to each grid in the two-dimensional space grid in the image to be processed, determine the value of each grid;

[0123] A first bilateral grid of the image to be processed based on the two-dimensional spatial grid and the values ​​of each grid.

[0124] In one embodiment, since larger-scale detail information can be extracted when the image to be processed is filtered with a larger window, in order to obtain more detailed and comprehensive detail information of the image to be processed at multiple different scales, the image to be processed can be subjected to multiple low-pass filtering processes at different scales.

[0125] Based on this, in one embodiment, after obtaining the detail image based on the difference image between the image to be processed and the basic image, the method further includes:

[0126] Using the basic image to update the image to be processed, performing low-pass filtering of different scales on the updated image to be processed to obtain an updated detail image;

[0127] After a preset number of iterations, the updated basic image and the detailed image are obtained.

[0128] In one embodiment, the preset number of iterations can be set as needed, such as four times, five times, etc., which is not limited in the present disclosure.

[0129] In summary, the image processing method provided by the present disclosure is:

[0130] First, by adjusting the dynamic range of the image to be processed at different scales, the detail information of the image to be processed at different scales is obtained; and by performing contrast enhancement on the detail images at different scales, the problem that the related technology cannot process the details of the image to be processed at different scales is solved. Finally, the target image obtained by the detail image after contrast enhancement contains more accurate image detail information, which effectively improves the image quality.

[0131] Secondly, the target image is adjusted by using a dynamic range adjustment curve to obtain an adjusted target image, thereby ensuring that the adjusted target image has a good global contrast.

[0132] Secondly, by using the bilateral grid to perform low-pass filtering on the processed image at different scales, compared with multi-scale technologies such as the local Laplace pyramid, using the bilateral grid to perform contrast enhancement on the processed image at different scales can speed up the enhancement process of the detail information corresponding to the processed image, thereby improving the image processing efficiency.

[0133] Furthermore, by using the bilateral grid to perform low-pass filtering on the processed image at multiple different scales, more detailed and comprehensive detail information is obtained. The dynamic range of detail-enhanced images of different scales is adjusted to further improve the image quality.

[0134] The image processing method provided by the present disclosure is further illustrated below with an application example.

[0135] The application examples provided in the present disclosure are applied in the fields of taking photos, recording videos, and previewing using smartphones.

[0136] When taking photos with a smartphone, if the image is underexposed, most of the pixels in the image are distributed in the dark area, and the global contrast of the image is insufficient, resulting in a significant loss of image details. If the image is overexposed, most of the pixels in the image are distributed in the bright area, and the global contrast of the image is insufficient, resulting in a significant loss of image details. Figure 4 As shown in the figure, the highlight area outside the window in the image is overexposed, while the portrait indoors is underexposed. Although the global contrast of the image is very good, the local contrast of the image is seriously insufficient, resulting in serious detail loss in both the dark and highlight areas of the image.

[0137] In this case, taking the input image as a three-channel RGB image of R channel, G channel and B channel as an example, the application example of the present disclosure provides an image processing method, such as Figure 5 As shown, the following steps are included:

[0138] Step 501, converting the three-channel RGB image of the R channel, the G channel and the B channel into a target image;

[0139] In the application example of the present disclosure, the target image is a grayscale image y, and the method of converting a three-channel RGB image of an R channel, a G channel, and a B channel into a grayscale image y is:

[0140] y x,y =R x,y *c1+G x,y *c2+B x,y *c3;

[0141] Among them, y x,y represents the pixel value (brightness value) of the pixel point (x, y) in the grayscale image y, R x,y represents the R pixel value of the (x, y) pixel in the R channel image, G x,y represents the G pixel value of the (x, y) pixel in the G channel image, B x,y Represents the B pixel value of the (x, y) pixel in the B channel image. c1, c2 and c3 represent the channel coefficients corresponding to the R channel, G channel and B channel respectively. In the application example of the present disclosure, c1=0.229, c2=0.578, c3=0.114. In one embodiment, c1, c2 and c3 are adjustable values, and the application example of the present disclosure is only an example value and is not a limitation thereto.

[0142] Step 502, presetting a first spatial sampling rate as m1*n1, performing a first scale downsampling process on the image to be processed, and obtaining a small-scale low-pass filtering result of the image to be processed;

[0143] The range of the small-scale low-pass filtering result, that is, the third dimension, is the histogram statistics of k bins in each grid image in the image to be processed and the m*n grids;

[0144] Step 503, using a three-dimensional filter kernel to filter the small-scale low-pass filtering result to obtain a small-scale low-pass filtering result;

[0145] Step 504, performing trilinear interpolation processing on the filtered small-scale low-pass filtering result to obtain a first basic image mask1 having the same scale as the image to be processed;

[0146] Step 505, obtaining a first detail image detail 1 based on the image to be processed and the first basic image mask1;

[0147] Among them, detail_1 = y-mask1;

[0148] Step 506, perform an iterative operation, and preset the i-th spatial sampling rate to be m i *ni , using the (i-1)th basic image mask_(i-1) to update the image to be processed, performing the i-th scale downsampling processing on the mask_(i-1) to obtain the i-th bilateral grid V_Bgridi;

[0149] The m i <m i-1 , n i <n i-1 ;

[0150] Step 507, filtering and interpolating the i-th bilateral grid V_Bgridi to obtain the i-th basic image mask_i and the i-th detail image detail_i;

[0151] Step 508, determining whether the i is equal to the preset number of iterations, if so, proceeding to step 509; if not, proceeding to step 506;

[0152] Step 509, performing contrast enhancement processing on the i-th detail image detail_i, determining a detail image after contrast enhancement, fusing the detail image after contrast enhancement with the i-th basic image mask_i, and obtaining a target image;

[0153] y_i=detail_i*wt_i+mask_i;

[0154] Wherein, y_i is the target image, and wt_i is the gain coefficient of the i-th detail image detail_i;

[0155] Step 510, adjusting the target image y_i using a dynamic range adjustment curve to obtain an adjusted target image;

[0156] Step 511, obtaining a processed image in RGB format based on the adjusted target image and the image to be processed.

[0157] In the application example of the present disclosure, obtaining a processed image in RGB format based on the adjusted target image and the image to be processed includes:

[0158] Based on the adjusted target image and the image to be processed, obtaining global gain information;

[0159] Based on the global gain information, each image channel of the original image in the RGB format is enhanced to obtain a processed image in the RGB format.

[0160] like Figure 6a , 6b As shown in 6c, Figure 6a Input image, Figure 6bTo obtain the target image using Laplace transform, Figure 6c The target image is obtained by using the image processing method described in the application example of the present disclosure. Figure 6b The global contrast is good, but the local contrast is insufficient and details in the image are blurred. Figure 7 In the image, both the global contrast and the local contrast are good, and the image quality is high.

[0161] like Figure 7 As shown, the embodiment of the present disclosure further provides an image processing device, the image processing device 700 comprising:

[0162] A low-pass filtering unit 701 is used to perform low-pass filtering processing of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image;

[0163] A contrast enhancement unit 702 is used to perform contrast enhancement processing on the detail image to determine a detail image after contrast enhancement;

[0164] The fusion unit 703 is used to fuse the contrast-enhanced detail image and the basic image to obtain a target image.

[0165] In one embodiment, the image processing device 700 further includes a curve adjustment unit, configured to adjust the target image using a dynamic range adjustment curve to obtain an adjusted target image.

[0166] In one embodiment, the image processing apparatus 700 further includes an image conversion and global enhancement unit, which is used to:

[0167] Convert the original image into the image to be processed;

[0168] Based on the adjusted target image and the image to be processed, obtaining global gain information;

[0169] Based on the global gain information, each image channel of the original image in the RGB format is enhanced to obtain a processed image in the RGB format.

[0170] In one embodiment, the low-pass filtering unit 701 is specifically configured to:

[0171] Performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed;

[0172] Performing three-dimensional filtering on the first bilateral grid to obtain a filtered first bilateral grid;

[0173] Performing interpolation processing on the filtered first bilateral grid to obtain a basic image with the same scale as the image to be processed;

[0174] The detail image is obtained based on a difference image between the image to be processed and a basic image.

[0175] In one embodiment, the low-pass filtering unit 701 is specifically configured to:

[0176] Perform trilinear interpolation processing on the filtered first bilateral grid.

[0177] In one embodiment, the low-pass filtering unit 701 is specifically configured to:

[0178] Based on a preset spatial sampling rate, a first scale downsampling process is performed on the image to be processed to obtain a first bilateral grid of the image to be processed.

[0179] In one embodiment, the low-pass filtering unit 701 is specifically configured to:

[0180] Based on a preset first spatial sampling rate, obtaining a two-dimensional spatial grid of the first bilateral grid;

[0181] Based on the sum of pixel values ​​of all pixel points corresponding to each grid in the two-dimensional space grid in the image to be processed, determine the value of each grid;

[0182] A first bilateral grid of the image to be processed based on the two-dimensional spatial grid and the values ​​of each grid.

[0183] In one embodiment, the low-pass filtering unit 701 is specifically configured to:

[0184] Using the basic image to update the image to be processed, performing low-pass filtering of different scales on the updated image to be processed to obtain an updated detail image;

[0185] After a preset number of iterations, the updated basic image and the detailed image are obtained.

[0186] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0187] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0188] Figure 8 8 is a block diagram of an electronic device 800 for implementing the above-mentioned image processing method according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0189] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0190] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0191] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0192] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0193] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0194] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0195] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0196] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0197] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0198] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0199] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0200] An embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image processing method described in the above embodiment of the present disclosure.

[0201] The embodiment of the present disclosure also provides a chip, which can be seen in Fig. 9 Schematic diagram of the chip structure shown. Fig. 9 The chip shown includes a processor 901 and an interface 902. The number of processors 901 can be one or more, and the number of interfaces 902 can be multiple. The interface 902 is used to receive a signal from a memory of an electronic device and send the signal to the processor 901, wherein the signal includes a computer instruction stored in the memory.

[0202] Optionally, the chip further includes a memory 903, and the memory 903 is used to store necessary computer instructions and data. When the processor 901 executes the computer instructions, the electronic device executes the image processing method described in the above embodiment of the present disclosure.

[0203] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0204] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0205] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0206] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0207] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0208] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0209] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0210] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Performing low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image; Performing contrast enhancement processing on the detail image to determine a detail image after contrast enhancement; The contrast-enhanced detail image and the base image are fused to obtain a target image.

2. The method according to claim 1, characterized in that Also includes: The target image is adjusted using the dynamic range adjustment curve to obtain an adjusted target image.

3. The method according to claim 2, characterized in that Also includes: Get the original image in RGB format; Convert the original image into the image to be processed; Based on the adjusted target image and the image to be processed, obtaining global gain information; Based on the global gain information, each image channel of the original image in the RGB format is enhanced to obtain a processed image in the RGB format.

4. The method according to any one of claims 1 to 3, characterized in that The step of performing low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed includes: The image to be processed is subjected to bilateral filtering processing of different scales to obtain a basic image and a detail image of the image to be processed.

5. The method according to claim 4, characterized in that The step of performing bilateral filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed includes: Performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed; Performing three-dimensional filtering on the first bilateral grid to obtain a filtered first bilateral grid; Performing interpolation processing on the filtered first bilateral grid to obtain a basic image with the same scale as the image to be processed; The detail image is obtained based on a difference image between the image to be processed and a basic image.

6. The method according to claim 5, characterized in that The step of performing a first scale downsampling process on the image to be processed to obtain a first bilateral grid of the image to be processed includes: Based on a preset spatial sampling rate, a first scale downsampling process is performed on the image to be processed to obtain a first bilateral grid of the image to be processed.

7. The method according to claim 5, characterized in that The interpolation processing of the filtered first bilateral grid includes: Perform trilinear interpolation processing on the filtered first bilateral grid.

8. The method according to claim 6, characterized in that The step of performing a first scale downsampling process on the image to be processed based on a preset first spatial sampling rate to obtain a first bilateral grid of the image to be processed includes: Based on a preset first spatial sampling rate, obtaining a two-dimensional spatial grid of the first bilateral grid; Based on the sum of pixel values ​​of all pixel points corresponding to each grid in the two-dimensional space grid in the image to be processed, determine the value of each grid; A first bilateral grid of the image to be processed based on the two-dimensional spatial grid and the values ​​of each grid.

9. The method according to claim 5, characterized in that After obtaining the detail image based on the difference image between the image to be processed and the basic image, the method further includes: Using the basic image to update the image to be processed, performing low-pass filtering of different scales on the updated image to be processed to obtain an updated detail image; After a preset number of iterations, the updated basic image and the detailed image are obtained.

10. An image processing device, characterized in that: include: A low-pass filtering unit is used to perform low-pass filtering of different scales on the image to be processed to obtain a basic image and a detail image of the image to be processed; the image to be processed is a grayscale image; A contrast enhancement unit, configured to perform contrast enhancement processing on the detail image to determine a detail image after contrast enhancement; The fusion unit is used to fuse the contrast-enhanced detail image and the basic image to obtain a target image.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

13. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal includes a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 9.

Images