Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the local image processing and decomposition processing of a first image, and obtaining illumination image information and reflectivity image information; the illuminance image information comprises one or more first illuminance images, the reflectivity image information comprises reflectivity images corresponding to the one or more first illuminance images, and the image size of the first illuminance image is smaller than that of the first image; performing image brightening on the first illumination image; and generating a second image according to the brightened first illumination image and the reflectivity image information. By implementing the embodiment of the invention, the processing efficiency of image brightening processing can be improved.

Description

Technical Field

[0001] This application relates to the field of imaging technologies, and particularly to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] At night with low light, the images captured by an electronic device usually have low brightness and clarity. To brighten the images, the electronic device needs to consume a certain amount of time, and the larger the size of the image, the more time the electronic device consumes. Therefore, how to improve the processing efficiency of image brightening has become an urgent problem to be solved. Summary of the Invention

[0003] Embodiments of this application disclose an image processing method, apparatus, electronic device, and storage medium, which can improve the processing efficiency of brightening an image.

[0004] Embodiments of this application disclose an image processing method, including:

[0005] Performing local image processing and decomposition processing on a first image to obtain illuminance image information and reflectance image information; the illuminance image information includes one or more first illuminance images, the reflectance image information includes reflectance images corresponding to the one or more first illuminance images respectively, and the image size of the first illuminance image is smaller than the image size of the first image;

[0006] Brightening the first illuminance image;

[0007] Generating a second image according to the brightened first illuminance image and the reflectance image information.

[0008] In one embodiment, the local image processing includes downsampling processing, and the performing local image processing and decomposition processing on the first image to obtain illuminance image information and reflectance image information includes:

[0009] Performing the downsampling processing on the first image to obtain a third image;

[0010] Performing decomposition processing on the third image to obtain the first illuminance image corresponding to the third image and the corresponding reflectance image;

[0011] The generating a second image according to the brightened first illuminance image and the reflectance image information includes: generating a fourth image according to the brightened first illuminance image and the reflectance image corresponding to the third image;

[0012] Performing upsampling processing on the fourth image to obtain a second image.

[0013] In one embodiment, the local image processing includes downsampling processing. The local image processing and decomposition processing of the first image to obtain the illuminance image information and the reflectance image information includes:

[0014] Perform decomposition processing on the first image to obtain the first illuminance image corresponding to the first image and the corresponding reflectance image;

[0015] Perform the downsampling processing on the first illuminance image corresponding to the first image to obtain the downsampled first illuminance image;

[0016] The generating of the second image according to the brightened first illuminance image and the reflectance image information includes:

[0017] Perform upsampling processing on the brightened first illuminance image;

[0018] Generate the second image according to the upsampled first illuminance image and the reflectance image corresponding to the first image.

[0019] In one embodiment, the image brightening of the first illuminance image includes:

[0020] Obtain the preset brightening parameter corresponding to the first illuminance image, and perform image brightening on the first illuminance image according to the preset brightening parameter; or, determine the first brightening weight matrix according to the first image through the trained brightening model, and perform image brightening on the first illuminance image according to the first brightening weight matrix.

[0021] In one embodiment, the determining of the first brightening weight matrix according to the first image through the trained brightening model includes:

[0022] Input the first image into the trained brightening model, and generate the second brightening weight matrix corresponding to the first image through the trained brightening model; the training set of the brightening model includes multiple groups of corresponding sample first brightness images and sample second brightness images, and the brightness of the sample first brightness image is lower than the brightness of the corresponding sample second brightness image;

[0023] Perform downsampling processing on the second brightening weight matrix to obtain the first brightening weight matrix corresponding to the first illuminance image.

[0024] In one embodiment, when the illuminance image information includes multiple first illuminance images, the image brightening of the first illuminance image includes:

[0025] Process each of the first illuminance images in parallel to perform image brightening on each of the first illuminance images.

[0026] In one embodiment, the local image processing includes segmentation processing. The local image processing and decomposition processing of the first image to obtain the illuminance image information and the reflectance image information include:

[0027] Performing segmentation processing on the first image to obtain multiple sub-images;

[0028] Performing decomposition processing on each of the multiple sub-images to obtain a first illuminance image corresponding to each sub-image and a corresponding reflectance image;

[0029] The generating of the second image according to the brightened first illuminance image and the reflectance image information includes:

[0030] Generating a brightened sub-image corresponding to each sub-image according to each brightened first illuminance image and the reflectance image corresponding to each sub-image;

[0031] Stitching the brightened sub-images corresponding to the multiple sub-images respectively to obtain a second image.

[0032] An embodiment of the present application discloses an image processing apparatus, including:

[0033] An information acquisition module, configured to perform local image processing and decomposition processing on a first image to obtain illuminance image information and reflectance image information; the illuminance image information includes one or more first illuminance images, the reflectance image information includes reflectance images respectively corresponding to the one or more first illuminance images, and the image size of the first illuminance image is smaller than the image size of the first image;

[0034] An image brightening module, configured to brighten the first illuminance image;

[0035] An image generation module, configured to generate a second image according to the brightened first illuminance image and the reflectance image information.

[0036] An embodiment of the present application discloses an electronic device, including:

[0037] A memory storing executable program code;

[0038] A processor coupled to the memory;

[0039] The processor calls the executable program code stored in the memory and executes the method according to any one of the above embodiments.

[0040] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the method according to any one of the above embodiments.

[0041] Through the image processing method, device, electronic device, and storage medium disclosed in the embodiments of the present application, the electronic device can perform local image processing and decomposition processing on the first image to obtain illuminance image information and a reflectance image. The illuminance image information may include one or more first illuminance images, and the reflectance image information may include reflectance images corresponding to the one or more first illuminance images respectively. The image size of the first illuminance image is smaller than the image size of the first image. The electronic device can brighten the first illuminance image and generate a second image based on the brightened first illuminance image and the reflectance image. By implementing this embodiment, the electronic device performs local image processing and decomposition processing on the first image, which can make the image size of the first illuminance image smaller than the image size of the first image. The electronic device brightens the first illuminance image with a smaller size, and then generates a second image based on the brightened first illuminance image and the reflectance image, reducing the consumption of computing resources of the electronic device and improving the operating efficiency of the electronic device, that is, improving the processing efficiency of brightening the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a schematic diagram of an application scenario of an image processing method disclosed in the embodiments of the present application;

[0044] Figure 2 It is a schematic flowchart of an image processing method disclosed in the embodiments of the present application;

[0045] Figure 3 It is a schematic flowchart of another image processing method disclosed in the embodiments of the present application;

[0046] Figure 4 It is a schematic flowchart of yet another image processing method disclosed in the embodiments of the present application;

[0047] Figure 5 It is a schematic flowchart of an image processing method disclosed in the embodiments of the present application;

[0048] Figure 6 It is a modular schematic diagram of an image processing device disclosed in the embodiments of the present application;

[0049] Figure 7 It is a structural block diagram of an electronic device disclosed in the embodiments of the present application. Detailed implementation manners

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0051] It should be noted that the terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these process, method, product or device.

[0052] It can be understood that the terms "first", "second", etc. used in the present application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of the present application, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image. Both the first image and the second image are images, but they are not the same image.

[0053] In the related art, based on the RetineX theory, the original image is decomposed into a reflectance image and an illumination image, then the illumination image is brightened, the brightened illumination image is obtained, and then the enhanced image of the original image is obtained by combining with the reflectance image. Since the images captured by current electronic devices are becoming increasingly high-definition and the image size is getting larger, and the image sizes of the illumination image and the original image are the same, brightening the illumination image will consume a large amount of computing resources and memory occupancy, and the efficiency of image brightening is relatively low.

[0054] The embodiments of the present application disclose an image processing method, apparatus, electronic device and storage medium, which can improve the processing efficiency of brightening an image.

[0055] The following will be described in detail with reference to the accompanying drawings.

[0056] As Figure 1 shown Figure 1FIG. 0 is a schematic diagram of an application scenario of an image processing method disclosed in an embodiment of the present application. The application scenario may include an electronic device 110, which may include, but is not limited to, a mobile phone, a tablet computer, a wearable device, a laptop computer, a PC (Personal Computer), etc. Optionally, the electronic device 110 may include a camera device. The electronic device 110 may collect a first image through the camera device. The first image may refer to an image collected by the electronic device 110 through the camera device in a low-light environment. Optionally, the electronic device 110 may also be communicatively connected to other electronic devices. The electronic device 110 may receive the first image sent by other electronic devices. The first image is an image collected by other electronic devices. The embodiment of the present application does not limit the acquisition method of the first image.

[0057] In one embodiment, the electronic device 110 may perform local image processing and decomposition processing on the first image to obtain illumination image information and a reflectance image. The illumination image information may include one or more first illumination images. The reflectance image information includes reflectance images corresponding to the one or more first illumination images respectively. The electronic device 110 may brighten the first illumination image and generate a second image based on the brightened first illumination image and the reflectance image information.

[0058] As Figure 2 shown, Figure 2 FIG. 10 is a schematic flowchart of an image processing method disclosed in an embodiment of the present application. The image processing method may be applied to the electronic device in the above embodiment. The image processing method may include the following steps:

[0059] Step 210, perform local image processing and decomposition processing on the first image to obtain illumination image information and reflectance image information.

[0060] The illumination image information may include one or more first illumination images. The reflectance image information may include reflectance images corresponding to the one or more first illumination images respectively. The first image may include multiple local regions. The electronic device performing local image processing on the first image may refer to processing each local region of the first image. Each local region may include multiple pixels. The number of pixels in each local region may be the same or different, and this is not limited.

[0061] Optionally, this local image processing can be used to make the image size of the obtained first illumination image smaller than that of the first image. Through this local image processing, the electronic device can determine multiple pixels corresponding to each local area as a new pixel, or determine each local area as a new image, so as to achieve the effect of reducing the image size. Specifically, this local image processing can include downsampling processing or segmentation processing. The downsampling processing is used to reduce the size of the first image, and the downsampling processing will not increase the number of obtained first illumination images, that is, one first image obtains one first illumination image. The segmentation processing is used to change the number of the first images, and the segmentation processing can increase the number of obtained first illumination images, that is, one first image can obtain multiple first illumination images.

[0062] The decomposition processing refers to decomposing an image into a corresponding illumination image and a reflectance image. Among them, the illumination image is used to characterize the illumination intensity of the light source on the object surface, and the reflectance image is used to characterize the light reflection characteristics of the object surface. Specifically, as shown in Equation (1), Equation (1) is used to characterize the decomposition processing of the image.

[0063] I(x,y) = T(x,y) * R(x,y) Equation (1);

[0064] Among them, I(x,y) refers to the image information corresponding to the pixel at the pixel coordinate (x,y), T(x,y) refers to the illumination information corresponding to the pixel at the pixel coordinate (x,y), and R(x,y) refers to the reflection information corresponding to the pixel at the pixel coordinate (x,y).

[0065] It can be understood that the electronic device can perform local image processing first and then decomposition processing, or perform decomposition processing first and then local image processing. This application does not limit the execution order of local image processing and decomposition processing.

[0066] Step 220, brighten the first illumination image.

[0067] The electronic device can perform image enhancement processing on the first illuminance image through a preset enhancement method. Optionally, the preset enhancement method can include histogram equalization, Retinex-based optimization methods, and CNN (Convolutional Neural Networks)-based image enhancement methods, and there is no limitation thereto. For example, the Retinex-based optimization method can include the LIME (Low-Light Image Enhancement via Illumination Map Estimation) method. Among them, the electronic device can enhance one or more first illuminance images included in the illuminance image information, and correspondingly obtain one or more enhanced first illuminance images. Optionally, in order to improve the efficiency of image enhancement, when the illuminance image information includes multiple first illuminance images, the electronic device can process each first illuminance image in parallel to enhance the first illuminance image.

[0068] In one embodiment, the electronic device can obtain preset enhancement parameters corresponding to the first illuminance image and perform image enhancement on the first illuminance image according to the preset enhancement parameters; or, determine a first enhancement weight matrix according to the first image through a trained enhancement model, and perform image enhancement on the first illuminance image according to the first enhancement weight matrix.

[0069] The preset enhancement parameters act on all pixels of the first illuminance image. The electronic device can change the brightness of all pixels of the first illuminance image according to the preset enhancement parameters to obtain the enhanced first illuminance image, as shown in Equation (2). Equation (2) is used to represent image enhancement of the first illuminance image according to the preset enhancement parameters.

[0070] T′ = T α Equation (2);

[0071] Among them, T is the first illuminance image, T′ is the enhanced first illuminance image, and α is the preset enhancement parameter. Specifically, this α can be a parameter in the LIME method.

[0072] The multiple enhancement weight values included in the enhancement weight matrix respectively act on each pixel of the first illuminance image, that is, the multiple enhancement weight values correspond one-to-one to the multiple pixels. The electronic device can multiply the enhancement weight matrix by the first illuminance image to obtain the enhanced first illuminance image, as shown in Equation (3). Equation (3) is used to represent image enhancement of the first illuminance image according to the enhancement weight matrix.

[0073] T′(x, y) = T(x, y) * W(x, y) Equation (3);

[0074] Among them, T(x, y) refers to the illumination information corresponding to the pixel at the pixel coordinates (x, y) in the first illumination image, W(x, y) refers to the brightening weight corresponding to the pixel at the pixel coordinates (x, y) in the brightening weight matrix, and T′(x, y) refers to the brightened illumination information corresponding to the pixel at the pixel coordinates (x, y).

[0075] Implementing this embodiment can improve the diversity of image brightening methods and can also improve the diversity of image brightening methods.

[0076] Step 230: Generate a second image according to the brightened first illumination image and the reflectivity image information.

[0077] The electronic device can perform inverse local image processing and synthesis processing according to the brightened first illumination image and the reflectivity image information to obtain a second image. Among them, when the local image processing includes downsampling processing, the inverse local image processing can include upsampling processing. When the local image processing includes segmentation processing, the inverse local image processing can include stitching processing. Optionally, the electronic device can perform synthesis processing as shown in Equation (1), multiply the corresponding pixels of the brightened first illumination image and the corresponding reflectivity image to obtain a synthesized image.

[0078] Optionally, when the illumination image information includes a single first illumination image and the local image processing is downsampling processing, the electronic device can perform synthesis processing of a single brightened first illumination image with the reflectivity image and upsampling processing corresponding to the downsampling processing to obtain a second image. When the illumination image information includes multiple first illumination images and the local image processing is segmentation processing, the electronic device can perform synthesis processing on multiple brightened first illumination images and the corresponding reflectivity images, and then perform stitching processing corresponding to the segmentation processing on multiple synthesized images to obtain a second image. Among them, the second image is the brightened image corresponding to the first image, and the image size of the second image is the same as that of the first image.

[0079] In the embodiment of the present application, the electronic device can perform local image processing and decomposition processing on the first image to obtain illuminance image information and a reflectance image. The illuminance image information may include one or more first illuminance images, and the image size of the first illuminance image is smaller than that of the first image. The electronic device can brighten the first illuminance image and generate a second image according to the brightened first illuminance image and the reflectance image. By implementing this embodiment, the electronic device performs local image processing and decomposition processing on the first image, which can make the image size of the first illuminance image smaller than that of the first image. The electronic device brightens the first illuminance image with a smaller size, and then generates a second image according to the brightened first illuminance image and the reflectance image, reducing the consumption of computing resources of the electronic device and improving the operating efficiency of the electronic device, that is, improving the processing efficiency of brightening the image.

[0080] As Figure 3 shown, Figure 3 FIG. is a schematic flowchart of another image processing method disclosed in the embodiment of the present application. This image processing method can be applied to the electronic device in the above embodiment. This image processing method may include the following steps:

[0081] Step 310, perform downsampling processing on the first image to obtain a third image.

[0082] Since the first image is taken in a low-light environment, the brightness of adjacent pixels in each local area can be considered the same or similar. Therefore, the electronic device can perform downsampling processing on the first image, fuse multiple pixels included in each local area of the first image into one pixel, and obtain a third image. The image size of the third image is smaller than that of the first image. Optionally, the electronic device can use the pixel at the center position of each local area as the fused pixel corresponding to each local area. The electronic device can also calculate the average value of the multiple pixels included in each local area to obtain the fused pixel. The present application does not limit this.

[0083] Optionally, the ratio of the image size of the third image to that of the first image can be a preset ratio, and the preset ratio can be the sampling multiple of the downsampling processing, that is, the number of pixels included in each local area. The sampling multiple can be determined according to one or more of the device calculation speed, device memory, and the detail effect after image brightening. Generally speaking, the faster the device calculation speed, the smaller the sampling multiple can be; the larger the device memory, the smaller the sampling multiple can be; the better the detail effect after image brightening, the smaller the sampling multiple can be. Specifically, the sampling multiple can be a power of 2 to speed up the calculation speed of the electronic device, such as 64 times, 256 times, 1024 times.

[0084] Optionally, the electronic device can obtain the sampling multiple of the downsampling process and the image size of the first image, and calculate the region size of each local region according to the sampling multiple and the image size of the first image. Then, the electronic device divides the first image into multiple local regions according to the region size. Among them, the image size includes the image length size and the image width size. The electronic device can divide the sampling multiple into a region length size and a region width size. The region length size is a factor of the image length size, and the region width size is a factor of the image width size. The electronic device can obtain the region size according to the region length size and the region width size, so as to evenly divide the first image into multiple local regions. For example, the image size of the first image can be 1000*1000 pixels, and the sampling multiple is 16 times. Then the region length size can be 4, and the region width size can also be 4, that is, the region size is 4*4 pixels, so as to divide the first image into 250*250 local regions.

[0085] Step 320: Decompose the third image to obtain the first illuminance image corresponding to the third image and the corresponding reflectance image.

[0086] Among them, the image size of the first illuminance image corresponding to the third image and the image size of the corresponding reflectance image are the same as the image size of the third image, and are both smaller than the image size of the first image. The method for the electronic device to decompose the third image can refer to the content in the above embodiments, and will not be elaborated here. Since the downsampling process does not change the number of images, that is, the third image is one, the electronic device can decompose the one third image to obtain the illuminance image information including one first illuminance image, that is, the first illuminance image corresponding to the third image, and the obtained reflectance image information can include the reflectance image corresponding to one first illuminance image, that is, the reflectance image corresponding to the third image.

[0087] Step 330: Brighten the first illuminance image corresponding to the third image.

[0088] In the embodiment where the electronic device determines the first brightening weight matrix according to the first image through the trained brightening model and brightens the first illuminance image according to the first brightening weight matrix, as an optional implementation manner, the electronic device can input the first image into the trained brightening model, generate the second brightening weight matrix corresponding to the first image through the trained brightening model, the electronic device can perform downsampling processing on the second brightening weight matrix to obtain the first brightening weight matrix corresponding to the first illuminance image, and then brighten the first illuminance image according to the first brightening weight matrix.

[0089] Among them, the training set of the brightening model includes multiple groups of corresponding sample first brightness images and sample second brightness images, and the brightness of the sample first brightness images is lower than that of the corresponding sample second brightness images. Optionally, the sample first brightness images and the sample second brightness images can be images collected by the electronic device at different exposure levels simultaneously. The electronic device can determine the brightening weight matrix corresponding to each sample first brightness image according to each sample first brightness image through the brightening model, and based on the brightening weight matrix corresponding to each sample first brightness image, brighten each sample first brightness image to obtain the brightening image corresponding to each sample first brightness image. By comparing the brightening image corresponding to each sample first brightness image and the corresponding sample second brightness image, the model loss is calculated, so as to update the parameters of the brightening model through the model loss and complete the training of the brightening model.

[0090] Since the number of rows and columns of the second brightening weight matrix output by the brightening model corresponds to the image size of the first image input to the brightening model, that is, the number of rows of the column matrix of the second brightening weight matrix is the same as the image length size of the first image, and this image length size refers to the number of pixels included in the image length direction, and the number of rows of the matrix of the second brightening weight matrix is the same as the image width size of the first image, and this image width size refers to the number of pixels included in the image width direction. And the image size of the first illuminance image is smaller than the image size of the first image, and the second brightening weight matrix output by the brightening model does not match the first illuminance image. Therefore, the electronic device can perform downsampling processing on the brightening weight matrix corresponding to the first image to obtain the first brightening weight matrix corresponding to the first illuminance image, that is, the electronic device can fuse multiple brightening weights into a new brightening weight, and the sampling multiple corresponding to the first brightening weight matrix is the same as the sampling multiple corresponding to the first image. The number of rows and columns of the downsampled first brightening weight matrix can correspond to the image size of the first illuminance image, that is, the number of rows of the column matrix of the downsampled first brightening weight matrix is the same as the image length size of the first image, and the number of rows of the matrix of the downsampled first brightening weight matrix is the same as the image width size of the first image.

[0091] Implementing this implementation method, by performing downsampling processing on the second brightening weight matrix, the number of rows and columns of the second brightening weight matrix obtained after downsampling correspond to the image size of the first illuminance image, ensuring the correct progress of image brightening and improving the accuracy of image brightening.

[0092] Step 340, generate a fourth image according to the brightened first illuminance image and the reflectivity image corresponding to the third image.

[0093] Among them, the first illuminance image after brightening is the same as the reflectance image corresponding to the third image. The electronic device can multiply the first illuminance image after brightening by the reflectance image corresponding to the third image to obtain the fourth image after brightening.

[0094] Step 350: Upsample the fourth image to obtain the second image.

[0095] Since the image size of the fourth image is the same as that of the third image, and both are smaller than the image size of the first image, the electronic device can perform upsampling processing corresponding to the downsampling processing on the fourth image, that is, the sampling multiple of the upsampling processing is the same as that of the downsampling processing, so as to obtain the second image with the same image size as the first image. Among them, the upsampling processing may include, but is not limited to, interpolation method, de-pooling method, transposed convolution method, etc., and there is no limitation on this.

[0096] In the embodiment of the present application, the electronic device can first perform downsampling processing on the first image to obtain the third image, then perform decomposition processing on the third image to obtain the first illuminance image corresponding to the third image and the corresponding reflectance image. The electronic device brightens the first illuminance image corresponding to the third image, and generates the fourth image according to the first illuminance image after brightening and the reflectance image corresponding to the third image, and then performs upsampling processing on the fourth image to obtain the second image. By the processing sequence of first downsampling and then decomposing the first image, the reduction of the image size can be completed in the first step, making the subsequent processing more simple. It not only reduces the memory occupation of the electronic device, but also further improves the efficiency of image brightening. Specifically, based on the LIME method for image brightening, the sampling multiple of the downsampling processing is 256 times, the memory resource occupied by the electronic device is reduced by 4 times, and the processing time of the electronic device is reduced by 4 times.

[0097] As Figure 4 shown, Figure 4 is a schematic flowchart of another image processing method disclosed in the embodiment of the present application. This image processing method can be applied to the electronic device in the above embodiment. This image processing method may include the following steps:

[0098] Step 410: Decompose the first image to obtain the first illuminance image corresponding to the first image and the corresponding reflectance image.

[0099] Compared with the previous embodiment, in this embodiment, the electronic device first decomposes the first image, and can obtain the first illuminance image corresponding to the first image and the corresponding reflectance image. Compared with the first illuminance image corresponding to the third image and the corresponding reflectance image, the first illuminance image corresponding to the first image and the corresponding reflectance image can retain more image information.

[0100] Step 420: Downsample the first illumination image corresponding to the first image to obtain the downsampled first illumination image.

[0101] The image size of the first illumination image is the same as that of the first image. The method for the electronic device to perform downsampling on the first illumination image corresponding to the first image is the same as the method for the electronic device to perform downsampling on the first image in the above embodiment. For reference, please refer to the above embodiment and will not be elaborated here.

[0102] Step 430: Brighten the downsampled first illumination image.

[0103] Step 440: Upsample the brightened first illumination image.

[0104] Step 450: Generate a second image according to the upsampled first illumination image and the reflectance image corresponding to the first image.

[0105] Since the image size of the downsampled first illumination image is smaller than that of the reflectance image corresponding to the first image, the electronic device needs to first upsample the brightened first illumination image so that the image size of the upsampled first illumination image is the same as that of the reflectance image corresponding to the first image. Thus, the upsampled first illumination image can be multiplied by the reflectance image corresponding to the first image to obtain the second image.

[0106] In the embodiment of the present application, the electronic device can first decompose the first image to obtain the first illumination image and the corresponding reflectance image corresponding to the first image, and then brighten the downsampled first illumination image. The electronic device can brighten the downsampled first illumination image and upsample the brightened first illumination image, so as to generate a second image according to the upsampled first illumination image and the reflectance image corresponding to the first image. By the processing sequence of first downsampling and then decomposing the first image, more image information of the first image can be retained in the first illumination image, so as to improve the accuracy and effect of image brightening.

[0107] As Figure 5 shown, Figure 5 is a schematic flowchart of an image processing method disclosed in an embodiment of the present application. The image processing method can be applied to the electronic device in the above embodiment. The image processing method can include the following steps:

[0108] Step 510: Segment the first image to obtain multiple sub-images.

[0109] The electronic device can perform segmentation processing on the first image according to a preset segmentation rule to obtain multiple sub-images. For example, the preset segmentation rule can be to perform segmentation from the diagonal and evenly divide it into multiple sub-images, etc. The embodiments of the present application do not limit this, and it is only used to divide the first image into multiple sub-images.

[0110] As an alternative implementation, after performing segmentation processing on the first image to obtain multiple sub-images, the electronic device can also detect whether there are pixels with a brightness greater than the brightness threshold in each sub-image. If there are pixels with a brightness greater than the brightness threshold in the target sub-image, then divide the number of pixels with a brightness greater than the brightness threshold by the number of pixels in the target sub-image to obtain the over-bright area ratio. If the over-bright area ratio is greater than the ratio threshold, the electronic device can perform further segmentation on the target sub-image to obtain new multiple sub-images. Implementing this implementation can improve the brightness uniformity of each sub-image, thereby improving the effect of image brightening.

[0111] Step 520: Perform decomposition processing on each of the multiple sub-images to obtain a first illuminance image and a corresponding reflectance image corresponding to each sub-image.

[0112] Optionally, the electronic device can perform decomposition processing on each of the multiple sub-images in parallel. The obtained illuminance image information includes the first illuminance images corresponding to each of the multiple sub-images, and the obtained reflectance image information includes the reflectance images corresponding to each of the multiple sub-images.

[0113] Step 530: Process the first illuminance images corresponding to each of the multiple sub-images in parallel to brighten each of the first illuminance images.

[0114] Step 540: Generate a brightened sub-image corresponding to each of the multiple sub-images according to each brightened first illuminance image and the reflectance image corresponding to each sub-image.

[0115] The electronic device can multiply each brightened first illuminance image by the reflectance image corresponding to the sub-image corresponding to the brightened first illuminance image to obtain the brightened sub-image corresponding to each of the multiple sub-images.

[0116] Step 550: Stitch the brightened sub-images corresponding to each of the multiple sub-images to obtain a second image.

[0117] The electronic device can determine the stitching rule corresponding to the multiple brightened sub-images according to the preset segmentation rule corresponding to the first image, and stitch the brightened sub-images corresponding to each of the multiple sub-images according to this stitching rule to obtain a second image.

[0118] In the embodiment of the present application, the electronic device can perform segmentation processing on the first image to obtain multiple sub-images, and perform decomposition processing on the multiple sub-images respectively to obtain the first illuminance image corresponding to each sub-image and the corresponding reflectivity image. The electronic device can parallelize the first illuminance images corresponding to each sub-image to brighten each first illuminance image, and then generate the brightened sub-images corresponding to each sub-image according to each brightened first illuminance image and the reflectivity image corresponding to each sub-image. The electronic device can splice the brightened sub-images corresponding to the multiple sub-images respectively to obtain the second image. By segmenting the first image, then parallelizing the brightening of the first illuminance images corresponding to each sub-image, and finally splicing the brightened first illuminance images, although the memory occupancy of the electronic device is not reduced, the speed of image brightening can be increased, and the efficiency of image brightening is improved.

[0119] As Figure 6 shown, Figure 6 is a modular schematic diagram of an image processing device disclosed in an embodiment of the present application. The image processing device 600 may include an information acquisition module 610, an image brightening module 620, and an image generation module 630, where:

[0120] The information acquisition module 610 is configured to perform local image processing and decomposition processing on the first image to obtain illuminance image information and reflectivity image information; the illuminance image information includes one or more first illuminance images, the reflectivity image information includes the reflectivity images corresponding to one or more first illuminance images respectively, and the image size of the first illuminance image is smaller than the image size of the first image;

[0121] The image brightening module 620 is configured to brighten the first illuminance image;

[0122] The image generation module 630 is configured to generate a second image according to the brightened first illuminance image and the reflectivity image information.

[0123] In one embodiment, the local image processing includes downsampling processing. The information acquisition module 610 is further configured to perform downsampling processing on the first image to obtain a third image; perform decomposition processing on the third image to obtain the first illuminance image corresponding to the third image and the corresponding reflectivity image; the image generation module 630 is further configured to generate a fourth image according to the brightened first illuminance image and the reflectivity image corresponding to the third image; perform upsampling processing on the fourth image to obtain the second image.

[0124] In one embodiment, the local image processing includes downsampling processing. The information acquisition module 610 is further configured to decompose the first image to obtain a first illumination image corresponding to the first image and a corresponding reflectance image; perform downsampling processing on the first illumination image corresponding to the first image to obtain a downsampled first illumination image; the image generation module 630 is further configured to perform upsampling processing on the brightened first illumination image; and generate a second image according to the upsampled first illumination image and the reflectance image corresponding to the first image.

[0125] In one embodiment, the image brightening module 620 is further configured to obtain a preset brightening parameter corresponding to the first illumination image, and perform image brightening on the first illumination image according to the preset brightening parameter; or determine a first brightening weight matrix according to the first image through a trained brightening model, and perform image brightening on the first illumination image according to the first brightening weight matrix.

[0126] In one embodiment, the image brightening module 620 is further configured to input the first image into a trained brightening model, and generate a second brightening weight matrix corresponding to the first image through the trained brightening model; the training set of the brightening model includes multiple groups of corresponding sample first brightness images and sample second brightness images, and the brightness of the sample first brightness images is lower than the brightness of the corresponding sample second brightness images; perform downsampling processing on the second brightening weight matrix to obtain a first brightening weight matrix corresponding to the first illumination image.

[0127] In one embodiment, when the illumination image information includes multiple first illumination images, the image brightening module 620 is further configured to process each of the first illumination images in parallel to perform image brightening on each of the first illumination images.

[0128] In one embodiment, the local image processing includes segmentation processing. The information acquisition module 610 is further configured to perform segmentation processing on the first image to obtain multiple sub-images; perform decomposition processing on each of the multiple sub-images to obtain a first illumination image corresponding to each sub-image and a corresponding reflectance image; the image generation module 630 is further configured to generate a brightened sub-image corresponding to each sub-image according to the brightened first illumination image corresponding to each sub-image and the reflectance image corresponding to each sub-image; and splice the brightened sub-images corresponding to each of the multiple sub-images to obtain a second image.

[0129] In an embodiment of the present application, an electronic device may perform local image processing and decomposition processing on a first image to obtain illuminance image information and a reflectance image. The illuminance image information may include one or more first illuminance images, and the reflectance image information may include reflectance images corresponding to the one or more first illuminance images respectively. The image size of the first illuminance image is smaller than the image size of the first image. The electronic device may brighten the first illuminance image and generate a second image based on the brightened first illuminance image and the reflectance image. By implementing this embodiment, the electronic device performs local image processing and decomposition processing on the first image, which can make the image size of the first illuminance image smaller than the image size of the first image. The electronic device brightens the first illuminance image with a smaller size, and then generates a second image based on the brightened first illuminance image and the reflectance image, reducing the consumption of computing resources of the electronic device and improving the operation efficiency of the electronic device, that is, improving the processing efficiency of brightening the image.

[0130] As Figure 7 shown, in one embodiment, an electronic device is provided, which may include:

[0131] A memory 710 storing executable program code;

[0132] A processor 720 coupled to the memory 710;

[0133] The processor 720 calls the executable program code stored in the memory 710, and can implement the image processing method provided in the above embodiments.

[0134] The memory 710 may include a random access memory (RAM), and may also include a read-only memory (ROM). The memory 710 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 710 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above method embodiments, etc. The data storage area may also store data created during the use of the electronic device.

[0135] The processor 720 may include one or more processing cores. The processor 720 connects various parts within the entire electronic device using various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 710, and by invoking the data stored in the memory 710, it performs various functions of the electronic device and processes data. Optionally, the processor 720 may be implemented in at least one of the hardware forms of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 720 may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the displayed content; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 720 and may be implemented separately through a communication chip.

[0136] It can be understood that the electronic device may include more or fewer structural elements than those shown in the above structural block diagram. For example, it may include a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, sensors, etc., and no further limitation will be made here.

[0137] An embodiment of this application discloses a computer-readable storage medium that stores a computer program, where the computer program causes a computer to execute the methods described in the above various embodiments.

[0138] In addition, an embodiment of this application further discloses a computer program product. When the computer program product runs on a computer, it enables the computer to execute all or part of the steps in any one of the image processing methods described in the above embodiments.

[0139] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0140] The above has introduced in detail an image processing method, device, electronic device, and storage medium disclosed in the embodiments of the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An image processing method, characterized in that, Including: Performing local image processing and decomposition processing on a first image to obtain illumination image information and reflectance image information; the illumination image information includes one or more first illumination images, the reflectance image information includes reflectance images corresponding to the one or more first illumination images respectively, and the image size of the first illumination image is smaller than the image size of the first image; Performing image brightening on the first illumination image; Generating a second image according to the brightened first illumination image and the reflectance image information.

2. The method according to claim 1, characterized in that, The local image processing includes downsampling processing. The performing local image processing and decomposition processing on the first image to obtain illumination image information and reflectance image information includes: Performing the downsampling processing on the first image to obtain a third image; Performing decomposition processing on the third image to obtain a first illumination image corresponding to the third image and a corresponding reflectance image; The generating a second image according to the brightened first illumination image and the reflectance image information includes: Generating a fourth image according to the brightened first illumination image and the reflectance image corresponding to the third image; Performing upsampling processing on the fourth image to obtain a second image.

3. The method according to claim 1, characterized in that, The local image processing includes downsampling processing. The performing local image processing and decomposition processing on the first image to obtain illumination image information and reflectance image information includes: Performing decomposition processing on the first image to obtain a first illumination image corresponding to the first image and a corresponding reflectance image; Performing the downsampling processing on the first illumination image corresponding to the first image to obtain a downsampled first illumination image; The generating a second image according to the brightened first illumination image and the reflectance image information includes: Performing upsampling processing on the brightened first illumination image; Generating a second image according to the upsampled first illumination image and the reflectance image corresponding to the first image.

4. The method according to claim 2 or 3, characterized in that The performing image brightening on the first illumination image includes: Obtaining a preset brightening parameter corresponding to the first illumination image and performing image brightening on the first illumination image according to the preset brightening parameter; or, determining a first brightening weight matrix according to the first image through a trained brightening model and performing image brightening on the first illumination image according to the first brightening weight matrix.

5. The method according to claim 4, wherein The determining a first brightening weight matrix according to the first image through a trained brightening model includes: Inputting the first image into the trained brightening model to generate a second brightening weight matrix corresponding to the first image through the trained brightening model; the training set of the brightening model includes multiple groups of corresponding sample first brightness images and sample second brightness images, and the brightness of the sample first brightness image is lower than the brightness of the corresponding sample second brightness image; Performing downsampling processing on the second brightening weight matrix to obtain a first brightening weight matrix corresponding to the first illumination image.

6. The method according to claim 1, wherein In the case where the illumination image information includes multiple first illumination images, the performing image brightening on the first illumination image includes: Process each of the first illuminance images in parallel to brighten each of the first illuminance images.

7. The method according to claim 6, characterized in that, The local image processing includes a segmentation process. The local image processing and decomposition process of the first image to obtain illuminance image information and reflectance image information includes: Perform a segmentation process on the first image to obtain multiple sub-images; Perform a decomposition process on each of the multiple sub-images to obtain the first illuminance image corresponding to each sub-image and the corresponding reflectance image; The generation of the second image according to the brightened first illuminance image and the reflectance image information includes: Generate a brightened sub-image corresponding to each sub-image according to each brightened first illuminance image and the reflectance image corresponding to each sub-image; Stitch the brightened sub-images corresponding to the multiple sub-images respectively to obtain a second image.

8. An image processing apparatus, characterized in that, Includes: An information acquisition module for performing local image processing and decomposition processing on a first image to obtain illuminance image information and reflectance image information; the illuminance image information includes one or more first illuminance images, and the reflectance image information includes the reflectance images corresponding to the one or more first illuminance images respectively, and the image size of the first illuminance image is smaller than the image size of the first image; An image brightening module for brightening the first illuminance image; An image generation module for generating a second image according to the brightened first illuminance image and the reflectance image information.

9. An electronic device, characterized in that, Includes: A memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images