Image processing method and related device

Abstract

The present application provides an image processing method and related devices, in which the electronic device can obtain M‑1 frames of images collected before the M-th frame of image before adjusting the brightness value of the M-th frame of image, where M is an integer greater than 1. Then, the electronic device determines the target brightness threshold value based on the brightness values ​​corresponding to the M‑1 frames of image. When the brightness value of the M-th frame of image and the target brightness threshold value meet the preset conditions, the electronic device adjusts the brightness value of the M-th frame of image. Thus, on the basis of making the brightness of the adjusted M-th frame of image approach the actual brightness, the brightness "jump" of the M-th frame of image is reduced, thereby improving the user's photo-taking experience.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and related devices. Background Art

[0002] With the continuous development of photography technology in electronic devices (such as smartphones), higher requirements are also placed on the quality of images captured by electronic devices. For example, when there is a deviation between the brightness of an image and the actual brightness of the captured scene, the electronic device can improve the imaging quality of the image by adjusting the brightness value of the image.

[0003] However, if the electronic device adjusts the brightness value of each frame of the image, the brightness of one or several frames of the image may be significantly different from the brightness of adjacent frames of the image, thereby causing a sudden change in brightness and affecting the user's shooting experience. Summary of the invention

[0004] An image processing method and related device provided in an embodiment of the present application enable an electronic device to adjust the brightness value of an image that meets preset conditions, thereby reducing the brightness "jump" of the image and improving the imaging quality.

[0005] In a first aspect, the present application provides an image processing method, the method comprising:

[0006] Acquire the M-1 frame image and the M-th frame image, wherein M is an integer greater than 1;

[0007] Determine a target brightness threshold based on the brightness values ​​corresponding to the M-1 frame images respectively;

[0008] Adjusting the brightness value of the M-th frame image according to the target brightness threshold to obtain a first image;

[0009] The first image is displayed.

[0010] Optionally, the target brightness threshold is used to determine the difference between the brightness value of the Mth frame image and the brightness values ​​corresponding to the previous M-1 frames of image, so as to determine whether the brightness value of the Mth frame image needs to be adjusted based on the above difference.

[0011] In the above method, since the brightness "jump" phenomenon is caused by the difference between the brightness of the frame image before processing and the brightness of the previous frame or the previous frames, if the electronic device adjusts the brightness value of the frame image, the brightness difference may be increased. As a result, the difference between the brightness of the processed frame image and the brightness of the previous frame or the previous multiple frames is large, causing the brightness "jump". Therefore, before the electronic device adjusts the brightness value of the Mth frame image, it can obtain the M-1 frame image collected before the Mth frame image, and then determine the target brightness threshold according to the brightness values ​​corresponding to the M-1 frame images. Based on the target brightness threshold, the difference between the brightness of the Mth frame image and the brightness of the previous M-1 frame image is judged to determine whether to adjust the brightness value of the Mth frame image. Compared with the existing scheme of adjusting the brightness value of each frame image, the embodiment of the present application adjusts the brightness value of the image based on the target brightness threshold, and on the basis of making the brightness of the adjusted image close to the real brightness, the situation of brightness "jump" of the image is reduced, and the user's photo-taking experience is improved.

[0012] In a possible implementation of the first aspect, the target brightness threshold includes a first brightness threshold and a second brightness threshold, and adjusting the brightness value of the M-th frame image according to the target brightness threshold includes:

[0013] When the first brightness average value of the M-th frame image is less than the first brightness threshold, adjusting the brightness value of the M-th frame image, wherein the first brightness average value is determined according to the brightness values ​​corresponding to the pixels included in the first area of ​​the M-th frame image, and the brightness values ​​corresponding to the pixels included in the first area are less than the first preset threshold and greater than the second preset threshold;

[0014] When the second brightness average value of the M-th frame image is greater than the second brightness threshold, adjusting the brightness value of the M-th frame image, wherein the second brightness average value is determined according to the brightness values ​​corresponding to the pixels included in the second area in the M-th frame image, and the brightness values ​​corresponding to the pixels included in the second area are less than or equal to the second preset threshold;

[0015] When the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the brightness value of the Mth frame image is adjusted, wherein the first brightness threshold is greater than the second brightness threshold.

[0016] Optionally, the target brightness threshold is related to the area where the electronic device adjusts the brightness of the image. For example, if the electronic device adjusts the brightness of an area in the image (e.g., the first area), then the target brightness threshold corresponds to the first area, and the target brightness threshold can be determined based on the brightness values ​​corresponding to the pixels contained in the first area. For another example, if the electronic device adjusts the brightness of multiple areas in the image (e.g., the first area and the second area), respectively, then the target brightness threshold corresponds to the multiple areas, and the target brightness threshold can be determined based on the brightness values ​​corresponding to the pixels contained in the multiple areas, respectively, so the number of target brightness thresholds can be multiple. For example, the target brightness threshold includes a first brightness threshold corresponding to the first area, and a second brightness threshold corresponding to the second area.

[0017] In the above method, when the electronic device needs to adjust the brightness of the first area of ​​the M-th frame image, the electronic device compares the first brightness average value with the first brightness threshold value. When the electronic device needs to adjust the brightness of the second area of ​​the M-th frame image, the electronic device compares the second brightness average value with the second brightness threshold value. When the electronic device needs to adjust the brightness of the first area and the second area of ​​the M-th frame image, the electronic device compares the first brightness average value with the first brightness threshold value, and compares the second brightness average value with the second brightness threshold value. Thus, the electronic device can compare the target brightness threshold value with the brightness value of the M-th frame image according to the area to be processed, thereby improving the pertinence of the electronic device in processing the image and reducing the calculation pressure of the electronic device in processing the image.

[0018] In a possible implementation of the first aspect, determining the target brightness threshold based on the brightness values ​​respectively corresponding to the M-1 frames of images includes:

[0019] Respectively obtaining brightness values ​​corresponding to pixels contained in the first area in the M-1 frame image, and brightness values ​​corresponding to pixels contained in the second area in the M-1 frame image;

[0020] Averaging the brightness values ​​corresponding to the pixels in the first area in the M-1 frame image to obtain M-1 third brightness average values;

[0021] Averaging the brightness values ​​corresponding to the pixels in the second area in the M-1 frame image to obtain M-1 fourth brightness average values;

[0022] The target brightness threshold is determined according to M-1 target brightness average values, wherein the target brightness average value includes the third brightness average value and the fourth brightness average value, the third brightness average value is used to determine the first brightness threshold, and the fourth brightness average value is used to determine the second brightness threshold.

[0023] In the above method, since the image contains more pixels and each pixel corresponds to a brightness value, the image contains more brightness values, which is not conducive to the analysis and processing of the electronic device. The electronic device averages the brightness values ​​corresponding to the pixels in the M-1 frame image in order to obtain a value that can characterize the overall brightness of the image. For example, the electronic device averages the brightness values ​​corresponding to the pixels contained in the first area of ​​the image to obtain a third brightness average value, and the brightness of the first area can be represented by the third brightness average value. As a result, the electronic device represents the brightness values ​​corresponding to a large number of pixels by a small number of brightness average values ​​(such as the third brightness average value and the fourth brightness average value), and then analyzes and processes the small number of brightness average values ​​to reduce the computing pressure of the electronic device.

[0024] In a possible implementation of the first aspect, determining the target brightness threshold according to M-1 target brightness averages includes:

[0025] Determine a first function according to the M-1 target brightness averages;

[0026] The target brightness threshold is determined according to the integral values ​​of the brightness values ​​respectively corresponding to the M-1 frame images in the first function.

[0027] Optionally, the electronic device may use a kernel density estimation (KDE) algorithm to perform density estimation on the average brightness values ​​of M-1 targets to obtain a kernel density estimation result, wherein the kernel density estimation result is a probability density function estimate of the sample, that is, the first function of the embodiment of the present application.

[0028] In the above method, a continuous first function is fitted based on the discrete M-1 target brightness average values, and the distribution of brightness values ​​corresponding to the M-1 frames of images can be characterized by the continuous first function. The target brightness threshold determined based on the above distribution conforms to the distribution law of the brightness values ​​corresponding to the M-1 frames of images, so the target brightness threshold based on the above distribution law can be used to determine whether the brightness value of the Mth frame of image also conforms to the above distribution law.

[0029] In a possible implementation of the first aspect, determining the target brightness threshold according to an integral value of brightness values ​​respectively corresponding to the M-1 frame images in the first function includes:

[0030] Performing an integration process on the first function based on a target integral value to determine a target brightness interval, wherein the target brightness interval includes a plurality of brightness values, and the target integral value is used to characterize the probability that the brightness values ​​respectively corresponding to the M-1 frame images are in the target brightness interval;

[0031] The target brightness threshold is determined according to the target brightness interval.

[0032] Optionally, the first function may be a probability density function fitted based on the average values ​​of M-1 target brightnesses. Since the integral value obtained after integrating the probability density function is used to characterize the probability of the sample as a whole (e.g., the brightness values ​​corresponding to the M-1 frames of images) being in the integral interval, the electronic device integrates the first function based on a preset target integral value, i.e., a preset probability, so that the determined target brightness interval (i.e., the integral interval) characterizes the distribution of the sample as a whole (e.g., the brightness values ​​corresponding to the M-1 frames of images) under the preset probability.

[0033] Optionally, when the electronic device needs to increase the brightness value of the first area in the Mth frame image, in order to make the brightness value of the first area in the Mth frame image after the increase not much different from the M-1 third brightness average values, a smaller brightness value can be determined from the first function corresponding to the first area fitted based on the M-1 third brightness average values ​​as the first brightness threshold.

[0034] Optionally, when the brightness value of the first area in the M-th frame image is less than the first brightness threshold, it means that the brightness value of the first area in the M-th frame image is small, and is smaller than most of the values ​​in the M-1 third brightness average values. Therefore, even if the electronic device increases the brightness value of the first area in the M-th frame image, the difference between the increased brightness value of the first area in the M-th frame image and the M-1 third brightness average values ​​is not large, thereby avoiding the phenomenon of brightness "jump" in the M-th frame image.

[0035] Optionally, when the electronic device needs to reduce the brightness value of the second area in the Mth frame image, in order to make the brightness value of the second area in the Mth frame image after reduction not much different from the M-1 fourth brightness average values, a larger brightness value can be determined from the first function corresponding to the second area fitted based on the M-1 fourth brightness average values ​​as the second brightness threshold.

[0036] Optionally, when the brightness value of the second area in the Mth frame image is greater than the second brightness threshold, it means that the brightness value of the second area in the Mth frame image is relatively large, and is greater than most of the values ​​in the M-1 fourth brightness average values. Therefore, even if the electronic device reduces the brightness value of the second area in the Mth frame image, the brightness value of the second area in the Mth frame image after the reduction can be made not much different from the M-1 fourth brightness average values, thereby avoiding the phenomenon of brightness "jump" in the Mth frame image.

[0037] In a possible implementation manner of the first aspect, determining the first function according to the M-1 target brightness average values ​​includes:

[0038] Determine a second function according to the number of the M-1 target brightness averages that are less than or equal to the target brightness threshold, wherein the second function is used to characterize the probability that the M-1 target brightness averages are less than or equal to the target brightness threshold;

[0039] Determine M-1 third functions according to the rate of change of the second function in the width interval, wherein the positions of the center points of the M-1 third functions on the coordinate axis correspond one-to-one to the M-1 target brightness average values, and the width of any third function among the M-1 third functions is determined according to the length of the width interval;

[0040] The first function is determined by superimposing the M-1 third functions.

[0041] Optionally, the kernel density estimation function , that is, the first function can also be expressed according to the kernel function (that is, the third function), specifically as follows:

[0042]

[0043] in, Represents the bandwidth of the kernel function, which is used to determine the width of the kernel function. Indicates when is the average brightness of n targets The function of the average brightness of the target, Used to determine the position of the center point of the kernel function on the coordinate axis. express The sum of kernel functions.

[0044] According to the above expression, it can be known that electronic devices can be respectively indivual Determine indivual , that is, M-1 third functions are determined by using the M-1 target brightness averages obtained by the embodiment of the present application. Then, by superimposing kernel functions (i.e., M-1 third functions) to determine the kernel density estimation function Since the kernel density estimation function It can be approximately equal to the probability density function , so the first function can be determined by superimposing M-1 third functions.

[0045] In the above method, since the result of kernel density estimation (such as the first function) is a continuous density curve, it can provide a smoother and more accurate probability density estimation result. Therefore, the electronic device can fully capture the trend of brightness change of the M-1 frame image through kernel density estimation, which helps to smooth the brightness change and reduce the influence of single frame noise on the estimation result (ie, the first function), thereby improving the accuracy of the estimation result, so that the density estimation results obtained based on different kernel functions are not much different. In addition, the accurate first function provides a reliable basis for whether to perform brightness correction on the M-th frame image in the future, making the image detail enhancement more effective.

[0046] In a possible implementation of the first aspect, adjusting the brightness value of the Mth frame image according to the target brightness threshold includes:

[0047] When the first brightness average value is less than the first brightness threshold, increasing the brightness values ​​corresponding to the pixels included in the first area in the Mth frame image;

[0048] When the second brightness average value is greater than the second brightness threshold, reducing the brightness values ​​corresponding to the pixels included in the second area in the Mth frame image;

[0049] When the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the brightness values ​​corresponding to the pixels contained in the first area in the M-th frame image are increased, and the brightness values ​​corresponding to the pixels contained in the second area in the M-th frame image are decreased.

[0050] Optionally, if the comparison result is that the first brightness average value is greater than or equal to the first brightness threshold, it means that the brightness of the first area in the Mth frame image is significantly different from the brightness of the first area corresponding to the M-1 frame image. The electronic device does not adjust the brightness of the Mth frame image to avoid brightness "jumps" after adjusting the brightness of the Mth frame image.

[0051] Optionally, if the comparison result is that the second brightness average value is less than or equal to the second brightness threshold, it means that the brightness of the second area in the Mth frame image is significantly different from the brightness of the second area corresponding to the M-1 frame image. In this case, the electronic device does not adjust the brightness of the Mth frame image to avoid brightness "jumps" after adjusting the brightness of the Mth frame image.

[0052] In the above method, the electronic device increases the brightness value of the pixels included in the first area of ​​the M-th frame image to enhance the brightness details of the first area of ​​the M-th frame image, so that the M-th frame image is easier for the user to watch. The electronic device reduces the brightness value of the pixels included in the second area of ​​the M-th frame image to enhance the details of the second area of ​​the M-th frame image, so that the M-th frame image can display more details in the shooting scene.

[0053] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the image processing method described in the first aspect or any possible implementation of the first aspect.

[0054] In a third aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the image processing method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0055] In a possible implementation, the chip or chip system described above in the embodiment of the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0056] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. When the computer program is executed by a processor, the computer performs the image processing method described in the first aspect or any possible implementation of the first aspect.

[0057] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a communication device, enables the communication device to perform an image processing method as described in the first aspect or any possible implementation of the first aspect.

[0058] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The following is an introduction to the drawings used in the embodiments of the present application.

[0060] Figure 1A is a scene graph of a captured image provided by an embodiment of the present application;

[0061] Figure 1B It is an image with a sudden brightness change provided in an embodiment of the present application;

[0062] Figure 2A is another scene graph of captured images provided in an embodiment of the present application;

[0063] Figure 2B is another image with sudden brightness change provided in an embodiment of the present application;

[0064] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0065] Figure 4 is a schematic diagram of the software architecture of the electronic device provided in the embodiment of the present application;

[0066] Figure 5 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0067] Figure 6 is a schematic diagram of a first function corresponding to a first region provided in an embodiment of the present application;

[0068] Figure 7 is a schematic diagram of a first function corresponding to a second region provided in an embodiment of the present application;

[0069] Figure 8 It is a flowchart of another image processing method provided in an embodiment of the present application;

[0070] Fig. 9 is a schematic diagram of a first function corresponding to a first region provided in an embodiment of the present application;

[0071] Fig.10 is a schematic diagram of a first function corresponding to a second region provided in an embodiment of the present application;

[0072] Fig.11 is a flow chart of adjusting image brightness provided by an embodiment of the present application;

[0073] Fig. 12A It is a system desktop of an electronic device provided by an embodiment of the present application;

[0074] Fig. 12B , Fig. 12C and Fig.12D It is a preview interface of a group of camera applications provided in an embodiment of the present application;

[0075] Fig.13A It is a system desktop of an electronic device provided by an embodiment of the present application;

[0076] Fig. 13B , Fig. 13C and Fig.13D This is another set of preview interfaces for camera applications provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0078] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0079] In order to facilitate the understanding of the embodiments of the present application, the technical problems that the present application specifically aims to solve are analyzed and proposed below.

[0080] With the continuous development of photography technology in electronic devices (such as smartphones), higher requirements are also placed on the quality of images captured by electronic devices. Usually, electronic devices use automatic exposure (AE) technology to adjust exposure parameters according to the difference between the brightness of the image and the preset brightness, so that the brightness of the image captured based on the adjusted exposure parameters is equal to or close to the preset brightness. However, in the case where the shooting scene contains a large dark area or a large bright area, the electronic device adjusts the overall brightness of the image to the preset brightness, which may cause a deviation between the brightness of the image display and the actual brightness of the shooting scene, affecting the quality of the image.

[0081] Electronic devices usually use 18% neutral gray as the preset brightness because the brightness and color of an object are determined by the reflectivity of the object to light. For example, the reflectivity of a pure black object is 0, the reflectivity of a pure white object is 100%, and the reflectivity of an object in neutral gray is 18%. On the color spectrum, the neutral gray in the center is the average of all gray levels, so it is generally believed that the average reflectivity of objects in nature is 18% neutral gray. Therefore, electronic devices can adjust the brightness of an image to 18% neutral gray to make the image appear averagely bright.

[0082] For example, when there is a large dark area in the shooting scene, the average brightness of the image is less than 18% neutral gray. The electronic device increases the brightness of the image by adjusting the exposure parameters. This may result in the acquisition of an image with excessive brightness based on the adjusted exposure parameters, resulting in poor imaging effect of detailed textures in the dark area of ​​the image.

[0083] See also Figure 1A and Figure 1B , Figure 1A is a scene graph of a captured image provided by an embodiment of the present application, Figure 1B This is an image with a sudden change in brightness provided in an embodiment of the present application.

[0084] like Figure 1AAs shown, the electronic device 100 calls the camera to shoot the shooting scene A, and then displays the captured image in the preview interface of the camera application. The shooting scene A includes a dark table 1011, on which a white cup 1012 and a box 1013 are placed. Exemplarily, the table 1011 has the darkest color and the lowest brightness relative to the cup 1012 and the box 1013, and the brightness of the box 1013 is between the table 1011 and the cup 1012. Optionally, the area where the table 1011 is located can be called a dark area, the area where the box 1013 is located can be called a transition area, and the area where the cup 1012 is located can be called a bright area. It can be seen that the area where the table 1011 is located accounts for a large proportion of the shooting scene A, so the shooting scene A contains a large area of ​​dark areas. Optionally, the average brightness of the picture collected by the electronic device is less than 18% neutral gray. For example, when the average brightness of the captured image is less than 18% neutral gray, the electronic device usually increases the brightness of the image by increasing the light flux, so that the brightness of the image is close to 18% neutral gray. For example, the electronic device obtains exposure parameters by increasing the light flux, captures image 102 based on the exposure parameters, and then Figure 1A The preview interface 103 shown displays an image 102 .

[0085] Exemplarily, the brightness corresponding to the object in the shooting scene A can be regarded as the real brightness, which is equivalent to the brightness corresponding to the accurate exposure. Image 102 is an image captured by the electronic device based on 18% neutral gray. It can be seen that the overall brightness of image 102 is higher than that of the shooting scene A, and there is a difference between the brightness of the image captured by the electronic device and the real brightness. Optionally, the table 1021 shown in image 102 is brighter than the table 1011 shown in the shooting scene A, resulting in the table 1021 shown in image 102 losing some detailed texture compared to the table 1011 shown in the shooting scene A, which is inconsistent with the actual scene, thereby affecting the imaging quality. In view of the problem that there is a difference between the brightness of the image captured by the above-mentioned electronic device and the real brightness, the electronic device can adjust the brightness value of the image after capturing the image corresponding to the shooting scene A, and then display the adjusted image in the preview interface, so as to reduce the difference between the brightness of the image and the real brightness by adjusting the brightness value.

[0086] For example, the electronic device collects image A after image 102, and then adjusts the brightness value of image A based on the difference between the brightness of image 102 and the actual brightness to obtain image 104. Figure 1B The preview interface 105 shown displays an image 104 .

[0087] like Figure 1BAs shown, the overall brightness of image 104 becomes darker than that of image 102, and the degree of change is relatively large. Optionally, table 1041 shown in image 104 is darker than table 1021 shown in image 102. For example, since electronic devices usually capture multiple frames of continuous images at short intervals and then display the captured images in the preview interface. Therefore, the time interval between the electronic device displaying image 102 and then displaying image 104 is relatively short, which may result in the electronic device displaying image 102 and then displaying image 104 with a large brightness difference from image 102 for a short period of time, causing a brightness "jump" phenomenon, thereby affecting the user's photo-taking experience.

[0088] The above reasons for the brightness "jump" can be understood as follows: for shooting scene A, the image collected by the electronic device is usually different from the image captured by the electronic device. Figure 1A The brightness difference between the images 102 in the captured scene A is small, and the electronic device can adjust the brightness of one or more frames of images corresponding to the captured scene A based on the same adjustment method, so that the brightness of the adjusted image approaches the real brightness. However, when the brightness difference between the image captured by the electronic device for the captured scene A and the image 102 is large (for example, the brightness difference between the captured image A and the image 102 is large), if the electronic device adjusts the brightness of the image A according to the above adjustment method, the brightness difference between the adjusted image A and the image 102 may be further increased, resulting in a brightness "jump" phenomenon in the image 104 obtained after the image A is adjusted.

[0089] See also Figure 2A and Figure 2B , Figure 2A is another scene graph for capturing an image provided in an embodiment of the present application. Figure 2B This is another image with a sudden change in brightness provided in an embodiment of the present application.

[0090] like Figure 2AAs shown, the electronic device 100 calls the camera to shoot the shooting scene B, and then displays the captured image in the preview interface of the camera application. The shooting scene B includes a light-colored desktop 2011, a black box 2012 and a notebook 2013 placed on the desktop 2011. Exemplarily, the desktop 2011 has the lightest color and the highest brightness relative to the box 2012 and the notebook 2013, and the brightness of the notebook 2013 is between the brightness of the desktop 2011 and the brightness of the box 2012. Optionally, in the embodiment of the present application, the area where the notebook 2013 is located can be called a transition area, the area where the desktop 2011 is located can be called a bright area, and the area where the box 2012 is located can be called a dark area. Optionally, the area where the box 2012 is located accounts for a small proportion of the shooting scene shown in the shooting scene B, so the area containing the dark area in the shooting scene B is small, and the area containing the transition area and the bright area is large. Optionally, the average brightness of the picture collected by the electronic device is greater than 18% neutral gray. For example, when the average brightness of the captured image is greater than 18% neutral gray, the electronic device usually increases the brightness of the image by reducing the light flux, so that the brightness of the image is close to 18% neutral gray. For example, the electronic device obtains exposure parameters by reducing the light flux, captures image 202 based on the exposure parameters, and then Figure 2A The preview interface 203 shown displays an image 202 .

[0091] Exemplarily, the brightness corresponding to the object in shooting scene B can be regarded as the real brightness, which is equivalent to the brightness corresponding to the accurate exposure. Image 202 is an image captured by the electronic device based on 18% neutral gray. It can be seen that the overall brightness of image 202 is lower than that of shooting scene B, and there is a difference between the brightness of the image captured by the electronic device and the real brightness. Optionally, the notebook 2021 shown in image 202 has a lower brightness than the notebook 2013 shown in shooting scene B, which is inconsistent with the actual scene, thereby affecting the imaging quality. In response to the problem that there is a difference between the brightness of the image captured by the above-mentioned electronic device and the real brightness, the electronic device can adjust the brightness value of the image after capturing the image corresponding to shooting scene B, and then display the adjusted image in the preview interface, so as to reduce the difference between the brightness of the image and the real brightness by adjusting the brightness value.

[0092] For example, the electronic device collects image B after image 202, and then adjusts the brightness value of image B based on the difference between the brightness of image 202 and the actual brightness to obtain image 204. Figure 2B The preview interface 205 shown displays an image 204 .

[0093] like Figure 2BAs shown, the overall brightness of image 204 becomes brighter than that of image 202, and the degree of change is relatively large. Optionally, the notebook 2041 shown in image 204 is brighter than the notebook 2021 shown in image 202. Exemplarily, since electronic devices usually capture multiple frames of continuous images at short intervals and then display the captured images in the preview interface. Therefore, the time interval between the electronic device displaying image 202 and then displaying image 204 is relatively short, which may result in the electronic device displaying image 202 and then displaying image 204 with a large brightness difference from image 202 at a short interval, causing a brightness "jump" phenomenon, thereby affecting the user's photo-taking experience.

[0094] The above reasons for the brightness "jump" can be understood as follows: for shooting scene B, the image collected by the electronic device is usually different from the image captured by the electronic device. Figure 2A The brightness difference between the images 202 in the scene B is small, and the electronic device can adjust the brightness of one or more frames of images corresponding to the shooting scene B based on the same adjustment method, so that the brightness of the adjusted image approaches the real brightness. However, when the brightness difference between the image captured by the electronic device for the shooting scene B and the image 202 is large (for example, the brightness difference between the captured image B and the image 202 is large), if the electronic device adjusts the brightness of the image B according to the above adjustment method, the brightness difference between the adjusted image B and the image 202 may be further increased, resulting in a brightness "jump" phenomenon in the image 204 obtained after the adjustment of the image B.

[0095] During research, the present application discovered that when the brightness of a first image is significantly different from that of a second image (the second image is captured after the first image) (for example, the difference between the brightness of a dark area in the second image and that of a dark area in the first image is greater than a first difference threshold, and / or the difference between the brightness of a transition area in the second image and that of the first image is greater than a second difference threshold), the electronic device adjusts the brightness value of the second image, which may increase the brightness difference between the first image and the adjusted second image, thereby causing a brightness "jump" phenomenon and affecting the user's photo-taking experience.

[0096] In view of this, an embodiment of the present application provides an image processing method. Before adjusting the brightness value of the Mth frame image, the M-1 frame image captured before the Mth frame image can be obtained, where M is an integer greater than 1. Then, the electronic device determines the target brightness threshold value based on the brightness values ​​corresponding to the M-1 frame images. When the brightness value of the Mth frame image and the target brightness threshold value meet the preset conditions, it means that the difference between the brightness of the Mth frame image and the brightness of the previous M-1 frame image is small, and the electronic device can adjust the brightness value of the Mth frame image. On the basis of making the brightness of the adjusted Mth frame image close to the actual brightness, the brightness "jump" of the Mth frame image is reduced, thereby improving the user's photo-taking experience.

[0097] The image processing method of the embodiment of the present application is described below in combination with the hardware structure and software structure of the electronic device.

[0098] In some embodiments, electronic devices include but are not limited to mobile phones, tablet computers (Portable Android Device, PAD), personal digital assistants (Personal Digital Assistant, PDA), handheld devices with wireless communication functions, computing devices, vehicle-mounted devices or wearable devices, virtual reality (Virtual Reality, VR) terminal devices, augmented reality (Augmented Reality, AR) terminal devices, wireless terminals in industrial control (Industrial Control), wireless terminals in self-driving (Self Driving), wireless terminals in remote medical (Remote Medical), wireless terminals in smart homes (Smart Home) and other mobile terminals or fixed terminals. The form of the electronic device in the embodiments of the present application is not specifically limited.

[0099] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 3As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0100] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0101] The processor 110 can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0102] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0103] In some embodiments, the processor 110 determines a first function based on the M-1 frame image, and the first function is used to characterize the brightness distribution of the M-1 frame image. The processor 110 determines a target brightness threshold value based on the first function, and then compares the target brightness threshold value with the brightness value of the M-th frame image. When the comparison result meets the preset condition, the processor 110 adjusts the brightness value of the M-th frame image. When the comparison result does not meet the preset condition, the processor 110 does not adjust the brightness value of the M-th frame image. For example, the target brightness threshold value includes a first brightness threshold value corresponding to the first area and a second brightness threshold value corresponding to the second area. When the first brightness average value corresponding to the first area of ​​the M-th frame image is less than the first brightness threshold value, and the second brightness average value corresponding to the second area of ​​the M-th frame image is greater than the second brightness threshold value, the comparison result meets the preset condition, and the processor 110 can adjust the brightness of the first area and the second area of ​​the M-th frame image.

[0104] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and a subscriber identity module (SIM) interface.

[0105] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160.

[0106] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like.

[0107] In some embodiments, the processor 110 and the display screen 194 communicate via a DSI interface to implement the display function of the electronic device 100 .

[0108] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0109] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0110] The wireless communication module 160 can provide wireless communication solutions applied to the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0111] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0112] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0113] In one embodiment of the present application, the internal memory 121 may store M-1 frames of images captured by the camera 193 , or brightness values ​​corresponding to the M-1 frames of images.

[0114] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0115] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0116] The software system of an electronic device (such as a mobile phone) can adopt a layered architecture, a transaction-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. This application embodiment takes the Android system with a layered architecture as an example to illustrate the software architecture of a mobile phone. Figure 4 , Figure 4 It is a schematic diagram of the software architecture of the electronic device provided in the embodiment of the present application.

[0117] like Figure 4 As shown, the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer (application), the application framework layer (frame), the hardware abstraction layer (HAL), the driver layer, and the hardware layer. Among them:

[0118] The application layer (application) can include a series of program packages. For example, the application can include camera, gallery and other applications. Among them, the camera application may include but is not limited to: UI module, camera module, gallery module, etc. The UI module can be a cameraUI module, which is mainly responsible for the human-computer interaction of the camera application, such as controlling the display of the preview interface and the preview screen therein, receiving and responding to user operations occurring in the preview interface. The camera module provides camera functions, focus functions, etc. The gallery module can be used to store photos taken by the user in the file system or a specific database of the electronic device for retrieval by applications such as the gallery.

[0119] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. It mainly involves the camera framework, which can include camera access interfaces such as camera extension libraries and camera services, and plays a role in connecting the upper and lower levels. It can interact with camera applications through application APIs and can also interact with HAL through HAL interface definition language (HIDL). The application framework layer can also include a window manager. With the support of the window manager, camera applications and gallery applications can display photos taken to users.

[0120] The hardware abstraction layer (HAL) is an interface layer between the application framework layer and the driver layer, which provides a virtual hardware platform for the operating system. Exemplarily, the hardware abstraction layer may include a camera hardware abstraction layer and an image processing module. The camera hardware abstraction layer may provide virtual hardware for camera device 1 (first camera), camera device 2 (second camera) and more camera devices.

[0121] The image processing module is used to determine whether to adjust the brightness value of the M-th frame image, and to adjust the brightness value of the M-th frame image. The image processing module includes a kernel density estimation algorithm, through which a first function can be determined based on the brightness values ​​corresponding to the M-1 frame images, so that the image processing module can determine a target brightness threshold according to the first function.

[0122] In one implementation, the image processing module can determine the first function according to the brightness values ​​corresponding to the pixels contained in the M-1 frame image by a kernel density estimation algorithm, and then determine the target brightness threshold based on the first function. The image processing module compares the brightness value of the M-th frame image with the target brightness threshold, and when the comparison result meets the preset condition, the image processing module adjusts the brightness value of the M-th frame image so that the brightness value of the adjusted M-th frame image approaches the true value.

[0123] The driver layer is a layer between hardware and software, which includes drivers for various hardware. The driver layer may include camera device drivers, digital signal processor drivers, and image processor drivers. Among them, the camera device driver is used to drive the image sensor of one or more cameras in the camera module to collect images and drive the image signal processor to pre-process the image. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0124] The hardware layer may include a camera module, an image signal processor, a digital signal processor, and an image processor.

[0125] The camera module may include one or more camera image sensors (eg, image sensor 1, image sensor 2, Figure 3 Camera 193 shown, etc.).

[0126] The following is an example of the workflow of electronic device software and hardware, using a camera application scenario.

[0127] In the embodiment of the present application, when the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer, and the kernel layer processes the user operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information), and identifies the control corresponding to the input event. Taking the touch operation as an example of a user operation acting on a camera application, the camera application calls the camera access interface of the application framework layer to start the camera application, and then sends an instruction to start the camera capability by calling the camera device in the camera hardware abstraction layer, such as camera device 1. The camera hardware abstraction layer sends the instruction to the camera device driver of the driver layer, and the camera device driver can start the sensor corresponding to the camera device, such as sensor 1, and then collect the image light signal through sensor 1 to obtain the Mth frame image.

[0128] In one implementation, the image signal processor transmits the M-th frame image back to the hardware abstraction layer through the camera device driver, and the hardware abstraction layer sends the M-th frame image to the image processing module. Based on the support of the image signal processor and the digital signal processor, the image processing module in the hardware abstraction layer can combine the camera parameters of the M-1 frame image reported by the camera module and / or other sensors, determine the first function used to characterize the brightness distribution of the M-1 frame image through the kernel density estimation algorithm, and then determine the target brightness threshold based on the first function. The image processing module compares the brightness value of the M-th frame image with the target brightness threshold. When the comparison result meets the preset conditions, the image processing module adjusts the brightness value of the M-th frame image to obtain the first image.

[0129] For example, the target brightness threshold includes a first brightness threshold corresponding to the first area, and a second brightness threshold corresponding to the second area. When the first brightness average value corresponding to the first area of ​​the M-th frame image is less than the first brightness threshold, and the second brightness average value corresponding to the second area of ​​the M-th frame image is greater than the second brightness threshold, the comparison result meets the preset condition.

[0130] The hardware abstraction layer returns the first image data to the camera application through the camera interface. Subsequently, the camera application can display the first image to the user with the support of the window manager, and the brightness display effect of the first image is better than that of the Mth frame image.

[0131] The image processing method provided in the embodiment of the present application can be applied to Figure 3 The hardware structure shown in Figure 4 In an electronic device with the software structure shown in the figure, or with more or fewer components than shown in the figure, or with some components combined, or some components separated, or with different component arrangements, etc., and with similar hardware structures and software structures.

[0132] The following will be combined Figure 5The image processing solution provided by the embodiment of the present application is described in detail. The application scenario of the solution provided by the embodiment of the present application is: after the electronic device turns on the automatic exposure function, the photographic component is called to capture an image, and then the image is displayed on the preview interface.

[0133] See also Figure 5 , Figure 5 It is a flowchart of an image processing method provided in an embodiment of the present application. The method includes the following steps S501-S504.

[0134] Step S501, obtaining the M-1 frame image and the Mth frame image.

[0135] The electronic device can collect multiple frames of images at preset intervals, and then display the collected multiple frames of images in the preview interface of the camera application. If there are images with large brightness changes in the multiple frames of images, it may cause a brightness "jump" phenomenon, which is specifically manifested as the difference between the brightness of the frame image and the brightness of the previous frame or the previous multiple frames of images is greater than the difference threshold, and the user visually feels that it suddenly becomes brighter or suddenly becomes brighter.

[0136] For example, based on Figure 1B , Figure 2B It can be seen from the corresponding embodiments that the brightness "jump" phenomenon occurs because there is a difference between the brightness of the frame image before processing and the brightness of the previous frame or previous frames. If the electronic device adjusts the brightness value of the frame image, the brightness difference may be increased. As a result, the difference between the brightness of the processed frame image and the brightness of the previous frame or previous frames is large, causing the brightness "jump". Therefore, before adjusting the brightness value of the Mth frame image, the electronic device can obtain the M-1 frame image collected before the Mth frame image, and then comprehensively consider the brightness of the M-1 frame image and the brightness of the Mth frame image according to steps S502 and S503 to determine whether to adjust the brightness value of the Mth frame image.

[0137] Among them, M is an integer greater than 1, which can be understood as the electronic device captures at least two frames of images when executing step S501. For example, if M is equal to 2, the electronic device captures the first frame image and the second frame image, and the capture time of the first frame image is earlier than the capture time of the second frame image. Optionally, the first frame image is used to determine whether the electronic device needs to adjust the brightness value of the second frame image, and the second frame image is the image to be processed. In an embodiment of the present application, the M-1 frame image can be an image that has not been processed after being captured by the electronic device, or it can be an image that has been processed after being captured by the electronic device, and there is no limitation here.

[0138] Step S502: determining a target brightness threshold based on the brightness values ​​corresponding to the M-1 frame images.

[0139] Specifically, after acquiring the M-1 frame image, the electronic device can determine the target brightness threshold based on the brightness values ​​corresponding to the M-1 frame image. The target brightness threshold is used to determine the difference between the brightness value of the M-th frame image and the brightness values ​​corresponding to the previous M-1 frame images, so as to determine whether the brightness value of the M-th frame image needs to be adjusted based on the above difference according to step S503.

[0140] Among them, the target brightness threshold is related to the area where the electronic device adjusts the brightness of the image. For example, if the electronic device adjusts the brightness of an area in the image (such as the first area), then the target brightness threshold corresponds to the first area, and the target brightness threshold can be determined based on the brightness values ​​corresponding to the pixels contained in the first area. For another example, if the electronic device adjusts the brightness of multiple areas in the image (such as the first area and the second area) respectively, then the target brightness threshold corresponds to the multiple areas, and the target brightness threshold can be determined based on the brightness values ​​corresponding to the pixels contained in the multiple areas, so the number of target brightness thresholds can be multiple. For example, the target brightness threshold includes a first brightness threshold corresponding to the first area, and a second brightness threshold corresponding to the second area.

[0141] Exemplarily, the target brightness threshold includes a first brightness threshold and a second brightness threshold as an example for explanation, wherein the first brightness threshold corresponds to the first area, and the second brightness threshold corresponds to the second area. The brightness values ​​corresponding to the pixels contained in the first area are all less than the first preset threshold and greater than the second preset threshold, which can be understood as an area in the image where the brightness is at an intermediate value, and the first area can also be called a transition area. The brightness values ​​corresponding to the pixels contained in the second area are all less than or equal to the second preset threshold, which can be understood as an area with lower brightness in the image, and the second area can also be called a dark area. Therefore, the first brightness threshold is determined based on the transition area in the M-1 frame image, and the second brightness threshold is determined based on the dark area in the M-1 frame image.

[0142] In a possible implementation, the electronic device obtains brightness values ​​corresponding to pixels contained in the first area in the M-1 frame image respectively, and then averages the brightness values ​​corresponding to pixels contained in the first area in the M-1 frame image respectively to obtain M-1 third brightness average values, and determines the first brightness threshold value based on the M-1 third brightness average values.

[0143] Among them, the above-mentioned electronic device performs averaging processing, and the brightness average value obtained includes multiple forms. For example, the brightness average value includes but is not limited to the arithmetic mean, geometric mean, square mean, harmonic mean, weighted mean, etc., which is not limited in the embodiments of the present application.

[0144] In a possible implementation, the electronic device determines a first function corresponding to the first area based on M-1 third brightness average values, and then determines a first brightness threshold based on an integral value of brightness values ​​corresponding to the M-1 frames of images in the first function corresponding to the first area.

[0145] Among them, the first function corresponding to the first area can be a probability density function fitted according to the M-1 third brightness average values. Since the integral value obtained after the probability density function is integrated is used to characterize the probability of the sample as a whole (for example, the brightness values ​​corresponding to the first area in the M-1 frame image) in the integral interval, the electronic device integrates the first function corresponding to the first area based on the preset target integral value, that is, the preset probability, so that the determined target brightness interval (that is, the integral interval) characterizes the distribution of the sample as a whole (for example, the brightness values ​​corresponding to the first area in the M-1 frame image) under the preset probability.

[0146] For example, if the pre-set probability is large, then the determined target brightness interval includes a larger portion of the brightness values ​​in the sample as a whole (e.g., the brightness values ​​corresponding to the first region in the M-1 frame image), indicating that the number of brightness values ​​corresponding to the first region in the M-1 frame image distributed in the target brightness interval is large. For another example, if the pre-set probability is small, then the determined target brightness interval includes a smaller portion of the brightness values ​​in the sample as a whole (e.g., the brightness values ​​corresponding to the first region in the M-1 frame image), indicating that the number of brightness values ​​corresponding to the first region in the M-1 frame image distributed in the target brightness interval is small.

[0147] For example, the following will be combined with Figure 6 The process of determining the first brightness threshold value by the electronic device according to the first function corresponding to the first area is described in detail. Figure 6 The first function shown is a possible example. The embodiment of the present application does not limit the shape and type of the first function. The process of the electronic device determining the first function corresponding to the first area can be referred to in Fig. 9 The corresponding embodiments.

[0148] See also Figure 6 , Figure 6 is a schematic diagram of a first function corresponding to a first region provided in an embodiment of the present application. Figure 6 As shown, the horizontal axis of the coordinate axis represents the brightness value, and the vertical axis represents the probability density. The first function corresponding to the first area represents the distribution of M-1 third brightness average values. For example, the larger the vertical coordinate in the first function corresponding to the first area, the greater the density of the M-1 third brightness average values ​​distributed around the brightness value corresponding to the vertical coordinate.

[0149] In the embodiment of the present application, based on the concept of probability density function, the probability density function is in the target brightness range (for example Figure 6 The integral value of the first brightness interval shown in FIG. 1 ) and the area in the target brightness interval can be equivalent to the probability that the sample as a whole is distributed in the target brightness interval. Thus, the electronic device can perform a predetermined probability (i.e., target integral value) on the sample as follows: Figure 6 The first function corresponding to the first region shown in FIG. 1 is integrated, or the first function corresponding to the first region shown in FIG. 1 is obtained as follows: Figure 6 The area enclosed by the first function corresponding to the first region shown and the horizontal axis is used to determine the target brightness interval corresponding to the preset probability, so as to determine the distribution of the brightness value under the preset probability.

[0150] In one implementation, see Figure 6 After determining the target integral value, the electronic device can perform integration processing on the first function corresponding to the first area based on the target integral value, wherein the lower limit of the integration processing is the left limit (point a) of the first function corresponding to the first area, and the determined upper limit (point c) is the first brightness threshold.

[0151] For another implementation, see Figure 6 , the electronic device may calculate the area enclosed by the first function corresponding to the first area and the horizontal axis starting from the left limit (point a) of the first function corresponding to the first area. When the above area is equal to the target integral value, the electronic device may use the interval of the area on the horizontal axis as the first brightness interval, and the maximum value (point c) of the first brightness interval as the first brightness threshold.

[0152] Optionally, other brightness values ​​in the first brightness interval may also be used as the first brightness threshold, which is not limited here.

[0153] It is understandable that electronic devices get Figure 6 The length of the first brightness interval shown depends on the size of the preset probability (that is, the target integral value), and the length of the first brightness interval increases as the target integral value increases. The electronic device can select the target integral value corresponding to the shooting scene based on different shooting scenes to determine the first brightness threshold, so that the obtained first brightness threshold conforms to the current shooting scene. Exemplarily, the target integral value corresponding to the first area includes one or more of the following: 0.01, 0.05, 0.1, etc., which are not limited here.

[0154] For example, if the current shooting scene makes the brightness of the first area in the Mth frame image differ greatly from the actual brightness, for example, greater than the first scene threshold, the electronic device needs to adjust the brightness of the first area in the Mth frame image by a large amplitude. In order to avoid a large difference between the brightness of the adjusted Mth frame image and the brightness of the M-1 frame image, the electronic device may choose to adjust the brightness of the first area in the Mth frame image when the brightness value of the first area in the Mth frame image is small. The electronic device may select a target integral value as small as possible from multiple target integral values ​​(for example, the target integral value is selected as 0.01), so that a first brightness interval with a shorter length is determined based on the smaller target integral value, so that the first brightness threshold obtained based on the first brightness interval with a shorter length is a value as small as possible among the M-1 third brightness average values. When the brightness value of the first area of ​​the Mth frame image is less than the first brightness threshold as small as possible, even if the electronic device adjusts the brightness of the first area of ​​the Mth frame image by a large amplitude, the difference between the brightness of the adjusted Mth frame image and the brightness of the M-1 frame image may be small.

[0155] For another example, if the current shooting scene makes the difference between the brightness of the first area in the Mth frame image and the actual brightness smaller, for example, less than or equal to the first scene threshold, the electronic device needs to adjust the brightness of the first area in the Mth frame image by a smaller amplitude, and the possibility of the adjusted Mth frame image producing a brightness "jump" is smaller, and the target integral value can be larger (for example, a target integral value of 0.1 is selected).

[0156] It should be noted that the above embodiment is for electronic equipment from Figure 6 The electronic device can also start from the left limit of the first function corresponding to the first area shown in FIG. Figure 6 The first region shown corresponds to the right limit of the first function (e.g. Figure 6 If the electronic device starts processing from the right limit, the sum of the target integral value and the integral value corresponding to the above processing from the left limit is 1. For example, if the integral value corresponding to the above processing from the left limit includes but is not limited to 0.01, 0.05, 0.1, etc., the integral value corresponding to the processing from the right limit may include one or more of the following: 0.99, 0.95, 0.9, etc.

[0157] In a possible implementation, the electronic device respectively obtains the brightness values ​​corresponding to the pixels contained in the second area in the M-1 frame image, and then the electronic device respectively averages the brightness values ​​corresponding to the pixels contained in the second area in the M-1 frame image to obtain M-1 fourth brightness average values, and determines the second brightness threshold based on the M-1 fourth brightness average values.

[0158] In a possible implementation, the electronic device determines the first function corresponding to the second area based on M-1 fourth brightness average values, and then determines the second brightness threshold based on the integral value of the brightness values ​​corresponding to the M-1 frame images in the first function corresponding to the second area.

[0159] The following will be combined Figure 7 The process of determining the second brightness threshold value by the electronic device according to the first function corresponding to the second area is described in detail. Figure 7 The first function shown is a possible example. The embodiment of the present application does not limit the shape and type of the first function. The process of the electronic device determining the first function corresponding to the second area can be referred to in Fig.10 The corresponding embodiments.

[0160] See also Figure 7 , Figure 7 is a schematic diagram of a first function corresponding to a second region provided in an embodiment of the present application. Figure 7 As shown, the horizontal axis of the coordinate axis represents the brightness value, and the vertical axis represents the probability density. The first function corresponding to the second area represents the distribution of M-1 fourth brightness average values. For example, the larger the vertical coordinate in the first function corresponding to the second area, the greater the density of the M-1 fourth brightness average values ​​distributed around the brightness value corresponding to the vertical coordinate.

[0161] In one implementation, see Figure 7 After determining the target integral value, the electronic device can perform integration processing on the first function corresponding to the second area based on the target integral value, wherein the lower limit of the integration processing is the left limit (point d) of the first function corresponding to the second area, and the determined upper limit (point f) is the second brightness threshold.

[0162] For another implementation, see Figure 7 , the electronic device may calculate the area enclosed by the first function corresponding to the second area and the horizontal axis starting from the left limit (point d) of the first function corresponding to the second area. When the above area is equal to the target integral value, the electronic device may use the interval of the area on the horizontal axis as the second brightness interval (that is, the target brightness interval in the embodiment of the present application), and use the maximum value (point f) of the second brightness interval as the second brightness threshold.

[0163] Optionally, other brightness values ​​in the second brightness interval may also be used as the second brightness threshold, which is not limited here.

[0164] For example, the electronic device may select a target integral value corresponding to a shooting scene based on different shooting scenes. Figure 7The first function corresponding to the second area shown determines the second brightness threshold so that the obtained second brightness threshold conforms to the current shooting scene. Exemplarily, the target integral value corresponding to the second area includes one or more of the following: 0.99, 0.95, 0.9, etc., which are not limited here.

[0165] For example, if the current shooting scene makes the brightness of the second area in the Mth frame image differ greatly from the actual brightness, for example, greater than the second scene threshold, the electronic device needs to adjust the brightness of the second area in the Mth frame image by a large amplitude. In order to avoid a large difference between the brightness of the adjusted Mth frame image and the brightness of the M-1 frame image, the electronic device may choose to adjust the brightness of the second area in the Mth frame image when the brightness value of the second area in the Mth frame image is large. The electronic device may select a target integral value as large as possible from multiple target integral values ​​(for example, the target integral value is selected as 0.99), so that a second brightness interval with a longer length is determined based on the larger target integral value, so that the second brightness threshold obtained based on the second brightness interval with a longer length is a value as large as possible among the M-1 fourth brightness average values. When the brightness value of the second area of ​​the Mth frame image is greater than the second brightness threshold as large as possible, even if the electronic device adjusts the brightness of the second area of ​​the Mth frame image by a large amplitude, the difference between the brightness of the adjusted Mth frame image and the brightness of the M-1 frame image may be small.

[0166] For another example, if the current shooting scene makes the difference between the brightness of the second area in the Mth frame image and the actual brightness smaller, for example, less than or equal to the second scene threshold, the electronic device needs to adjust the brightness of the second area in the Mth frame image by a smaller amplitude, and the possibility of the adjusted Mth frame image producing a brightness "jump" is smaller, and the target integral value can be smaller (for example, a target integral value of 0.9 is selected).

[0167] It should be noted that the above embodiment is for electronic equipment from Figure 7 The electronic device can also perform integration processing or calculate the area from the left limit of the first function corresponding to the second area shown. Figure 7 The second region shown corresponds to the right limit of the first function (e.g. Figure 7 If the electronic device starts processing from the right limit, the sum of the target integral value and the corresponding integral value when processing from the left limit is 1. For example, if the corresponding integral value when processing from the left limit includes but is not limited to 0.99, 0.95, 0.9, etc., the corresponding integral value when processing from the right limit may include one or more of the following: 0.01, 0.05, 0.1, etc.

[0168] Optionally, when the electronic device adjusts the brightness of an area in the Mth frame image, the target brightness average value includes the average value corresponding to the area, and the target brightness threshold value includes the brightness threshold value corresponding to the area. For example, the electronic device adjusts the brightness of the first area in the Mth frame image, the target brightness average value includes the third brightness average value, and the target brightness threshold value includes the first brightness threshold value. For another example, the electronic device adjusts the brightness of the second area in the Mth frame image, the target brightness average value includes the fourth brightness average value, and the target brightness threshold value includes the second brightness threshold value.

[0169] Optionally, when the electronic device adjusts the brightness of multiple areas (for example, the first area and the second area) in the Mth frame image, the target brightness average value includes a third brightness average value corresponding to the first area and a fourth brightness average value corresponding to the second area, and the target brightness threshold includes a first brightness threshold corresponding to the first area and a second brightness threshold corresponding to the second area.

[0170] It can be understood that, by fitting a continuous first function based on the discrete M-1 target brightness average values, the distribution of brightness values ​​corresponding to the M-1 frames of images can be characterized by the continuous first function. The target brightness threshold determined based on the above distribution conforms to the distribution law of the brightness values ​​corresponding to the M-1 frames of images, so that the target brightness threshold based on the above distribution law can be used to determine whether the brightness value of the Mth frame of image also conforms to the above distribution law. Optionally, the process of the electronic device determining the first function can refer to Fig. 9 and Fig.10 The corresponding embodiments will not be described in detail here.

[0171] Step S503: adjusting the brightness value of the Mth frame image according to the target brightness threshold to obtain a first image.

[0172] Specifically, after the electronic device obtains the target brightness threshold (for example, the first brightness threshold and the second brightness threshold) according to step S502, it can determine whether to adjust the brightness value of the Mth frame image according to the target brightness threshold. If it is determined that the brightness value of the Mth frame image needs to be adjusted, the brightness value of the Mth frame image can be adjusted to obtain the first image.

[0173] In a possible implementation, the electronic device determines a first average brightness value according to brightness values ​​corresponding to pixels included in a first area in the Mth frame image, and determines a second average brightness value according to brightness values ​​corresponding to pixels included in a second area in the Mth frame image. Then, the first average brightness value, the second average brightness value, and the target brightness threshold are combined to determine whether to adjust the brightness of the Mth frame image.

[0174] In a possible implementation, when the first brightness average value is less than the first brightness threshold, the electronic device adjusts the brightness value of the Mth frame image.

[0175] Specifically, when the electronic device adjusts the brightness of the first area of ​​the Mth frame image, the electronic device compares the first brightness average value with the first brightness threshold value. When the comparison result is that the first brightness average value is less than the first brightness threshold value, the electronic device adjusts the brightness value of the first area in the Mth frame image. For example, the electronic device increases the brightness value of the pixels contained in the first area in the Mth frame image to enhance the brightness details of the first area in the Mth frame image, so that the Mth frame image is easier for users to watch. Optionally, when the comparison result is that the first brightness average value is greater than or equal to the first brightness threshold value, the electronic device does not adjust the brightness value of the first area in the Mth frame image.

[0176] Among them, the situation where the electronic device adjusts the brightness of the first area includes but is not limited to the situation where the first area accounts for a large proportion in the Mth frame image (for example, greater than the first proportion threshold). The electronic device adjusts the brightness of the first area to achieve adjustment of the Mth frame image.

[0177] In a possible implementation, when the second brightness average value is greater than the first brightness threshold, the electronic device adjusts the brightness value of the Mth frame image.

[0178] Specifically, when the electronic device adjusts the brightness of the second area of ​​the M-th frame image, the electronic device compares the second brightness average value with the second brightness threshold value. Optionally, when the comparison result is that the second brightness average value is greater than the second brightness threshold value, the electronic device adjusts the brightness value of the second area in the M-th frame image, for example, the electronic device reduces the brightness value of the pixels included in the second area in the M-th frame image to enhance the details of the second area in the M-th frame image, so that the M-th frame image is easier for the user to watch. Optionally, when the comparison result is that the second brightness average value is less than or equal to the second brightness threshold value, the electronic device does not adjust the brightness value of the second area in the M-th frame image.

[0179] Among them, the situation where the electronic device adjusts the brightness of the second area includes but is not limited to the situation where the second area accounts for a large proportion in the Mth frame image (for example, greater than the second proportion threshold). The electronic device adjusts the brightness of the second area to achieve adjustment of the Mth frame image.

[0180] In a possible implementation, when the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the brightness values ​​of the M frames of images are adjusted.

[0181] Specifically, when the electronic device needs to adjust the brightness of the first area and the second area of ​​the M-th frame image, the electronic device compares the first brightness average value with the first brightness threshold, and compares the second brightness average value with the second brightness threshold. When the comparison result is that the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the brightness value of the M-frame image is adjusted. Optionally, when the comparison result is that the first brightness average value is greater than or equal to the first brightness threshold, or the second brightness average value is less than or equal to the second brightness threshold, the brightness value of the M-frame image is not adjusted.

[0182] It should be noted that the first brightness threshold is a relatively small value among the M-1 third brightness average values. When the brightness value of the first area in the M-th frame image is less than the first brightness threshold, it means that the brightness value of the first area in the M-th frame image is relatively small, and is smaller than most of the values ​​in the M-1 third brightness average values. Therefore, even if the electronic device increases the brightness value of the first area in the M-th frame image, the difference between the increased brightness value of the first area in the M-th frame image and the M-1 third brightness average values ​​can be made small, thereby avoiding the phenomenon of brightness "jump" in the M-th frame image.

[0183] The first brightness threshold is a smaller value among the M-1 third brightness average values. Figure 6 , it can be seen that the integral value of the first function corresponding to the first area in the first brightness interval (i.e., the target brightness interval corresponding to the first brightness threshold) is small, so the probability that the M-1 third brightness average values ​​fall in the first brightness interval is small, which can explain that the brightness values ​​of the M-1 third brightness average values ​​distributed in the first brightness interval are small. The minimum value of the above-mentioned first brightness interval is the left limit of the first function corresponding to the first area, and the brightness values ​​in the first brightness interval are arranged from small to large, which means that the brightness value in the first brightness interval is a smaller value among the M-1 third brightness average values. Since the brightness values ​​of the M-1 third brightness average values ​​distributed in the first brightness interval are small, it can be explained that the maximum value of the first brightness interval is smaller than most of the M-1 third brightness average values, that is, the first brightness threshold determined based on the first brightness interval is a smaller value among the M-1 third brightness average values.

[0184] It should be noted that the second brightness threshold is a larger value among the M-1 fourth brightness average values. When the brightness value of the second area in the M-th frame image is greater than the second brightness threshold, it means that the brightness value of the second area in the M-th frame image is larger, and is greater than most of the values ​​in the M-1 fourth brightness average values. Therefore, even if the electronic device reduces the brightness value of the second area in the M-th frame image, the difference between the brightness value of the second area in the M-th frame image after the reduction and the M-1 fourth brightness average values ​​is not large, thereby avoiding the phenomenon of brightness "jump" in the M-th frame image.

[0185] The second brightness threshold is a larger value among the M-1 fourth brightness average values. Figure 7 , it can be seen that the integral value of the first function corresponding to the second area in the second brightness interval (i.e., the target brightness interval corresponding to the second brightness threshold) is relatively large, so the probability that the M-1 fourth brightness average values ​​fall in the second brightness interval is relatively large, which can explain that the brightness values ​​of the M-1 fourth brightness average values ​​distributed in the second brightness interval are relatively large. The minimum value of the above-mentioned second brightness interval is the left limit of the first function corresponding to the second area, and the brightness values ​​in the second brightness interval are arranged from small to large. Since the brightness values ​​of the M-1 fourth brightness average values ​​distributed in the second brightness interval are relatively large, it can be explained that the maximum value of the second brightness interval is greater than most of the M-1 fourth brightness average values, that is, the second brightness threshold determined based on the second brightness interval is a relatively large value among the M-1 fourth brightness average values.

[0186] Step S504: display the first image on the preview interface.

[0187] Specifically, the electronic device may aggregate the results of adjusting the brightness of the pixels of the Mth frame image to obtain a first image, and then display the first image in the preview interface, for example Fig.12D and Fig.13D .

[0188] above Figure 5 The illustrated embodiment includes a variety of possible solutions. For ease of understanding, one possible solution is described below. For example, taking the electronic device adjusting the brightness of the first area and the second area in the Mth frame image as an example, the solution provided by the embodiment of the present application is described. It should be understood that Figure 8 For explanations of some of the terms, logic, etc. in the scheme shown, see Figure 5 The embodiment shown.

[0189] See also Figure 8 , Figure 8 It is a flowchart of another image processing method provided in an embodiment of the present application. Figure 8 The method shown can be applied to Figure 3 The hardware structure shown in Figure 4 The method comprises one or more steps from step S801 to step S810, and each step is specifically as follows:

[0190] S801, obtaining the M-1 frame image and the Mth frame image in the preview stream.

[0191] For example, after the electronic device starts the camera application, it can call the camera to capture a frame of image at a preset interval, and then display the frame of image in the preview interface of the camera application. In the embodiment of the present application, the continuous multiple frames of image captured by the electronic device at a preset interval can be called a preview stream, for example, Figure 8 The preview stream shown includes images ,image ,image To Image A total of M-1 frames of images, and images ,image To Image The electronic device may adopt a first-in, first-out (FIFO) method to update the image of the historical frame. For example, when the electronic device updates the Mth frame image (i.e., the image ) for brightness calibration, the electronic device can obtain the first M-1 frames of the Mth frame (i.e., the image To Image A total of M-1 frames of images). Optionally, when the electronic device receives the M+1th frame of image (i.e., image ) to perform brightness calibration, the electronic device can obtain the M-1 frames of image before the M+1 frame of image (i.e., image To Image In the embodiment of the present application, based on the first-in-first-out method, the shooting scene of the acquired M-1 frame image can be highly similar to the shooting scene of the M-th frame image, which can further improve the accuracy of the target brightness threshold determined based on the M-1 frame image in step S502.

[0192] S802: Determine a data set of a third average brightness value and a data set of a fourth average brightness value.

[0193] Specifically, the electronic device can determine the third brightness average and the fourth brightness average by the brightness values ​​corresponding to the pixels in the image, thereby forming a data set with the M-1 third brightness averages corresponding to the M-1 frames of images, and forming a data set with the M-1 fourth brightness averages corresponding to the M-1 frames of images.

[0194] The brightness value corresponding to the pixel in the image may be any information that can reflect the brightness of the image, which is not limited here. The following describes a method for an electronic device to obtain a brightness value by taking an example.

[0195] In a possible implementation, if the color mode of the M-1 frame image is a color mode with a luminance value component, such as the (Luminance, Y) brightness (Chrominance, U) concentration (Chroma, V) mode, the electronic device can directly obtain the Y value of the pixel in the M-1 frame image, divide the pixels whose Y value is less than the first preset threshold and greater than the second preset threshold into the first area, and divide the pixels whose Y value is less than or equal to the second preset threshold into the second area. Then, the Y values ​​of the pixels in the first area and the second area are averaged respectively to obtain the third luminance average value corresponding to the first area and the fourth luminance average value corresponding to the second area.

[0196] In another possible implementation, if the color mode of the M-1 frame image is not a color mode with a brightness value component, the color mode of the M-1 frame image can be converted into a color mode with a brightness value component, and then the brightness value corresponding to the color mode converted into the color mode with a brightness value component can be processed.

[0197] For example, when the color mode of the M-1 frame image is a red (red, R) green (green, G) blue (blue, B) mode, a hue (hue, H) saturation (saturation, S) brightness (brightness, B) mode, etc., the electronic device can convert the color mode of the M-1 frame image into a YUV mode according to a color mode conversion formula, so that the Y value of the pixel in the converted M-1 frame image corresponds to the brightness value of the pixel, and the electronic device can perform averaging processing based on the Y value obtained after the conversion.

[0198] It is understandable that since the image contains more pixels and each pixel corresponds to a brightness value, the image contains more brightness values, which is not conducive to the analysis and processing of the electronic device. The electronic device averages the brightness values ​​corresponding to the pixels in the M-1 frame image in order to obtain a value that can characterize the overall brightness of the image. For example, the electronic device averages the brightness values ​​corresponding to the pixels contained in the first area of ​​the image to obtain a third brightness average value, and the brightness of the first area can be represented by the third brightness average value. As a result, the electronic device represents the brightness values ​​corresponding to a large number of pixels by a small number of brightness average values ​​(such as the third brightness average value and the fourth brightness average value), and then analyzes and processes the small number of brightness average values ​​to reduce the computing pressure of the electronic device.

[0199] In one implementation, the electronic device may perform averaging processing according to the following expression to determine the third brightness average value:

[0200]

[0201] in, is the third brightness average value corresponding to the i-th frame image, is any value between 1 and M-1. Indicates the number of pixels contained in the transition area in the i-th frame image. Indicates the brightness value of the j-th pixel in the transition area of ​​the i-th frame image (for example, the Y value of the j-th pixel), where j is less than or equal to A positive integer. Indicates that j is changed from 1 to Corresponding The sum of the brightness values ​​of the pixels.

[0202] Therefore, the electronic device can obtain the M-1 third brightness average values ​​corresponding to the M-1 frames of images based on the above expression, and the data set composed of the M-1 third brightness average values ​​is: .

[0203] In one implementation, the electronic device may perform averaging processing according to the following expression to determine the fourth brightness average value:

[0204]

[0205] in, is the fourth average brightness value corresponding to the i-th frame image, is any value between 1 and M-1. Indicates the number of pixels contained in the dark area of ​​the i-th frame image. Indicates the dark area in the i-th frame image contains The brightness value of the pixel (for example Y value of pixels), is less than or equal to A positive integer. Indicates that From 1 to Corresponding The sum of the brightness values ​​of the pixels.

[0206] Therefore, the electronic device can obtain the M-1 fourth brightness average values ​​corresponding to the M-1 frames of images based on the above expression, and the data set composed of the M-1 fourth brightness average values ​​is: .

[0207] S803: Determine a first function corresponding to the first area according to M-1 third brightness average values.

[0208] Specifically, the electronic device may use a kernel density estimation (KDE) algorithm to perform density estimation on the M-1 third brightness average values ​​to obtain a kernel density estimation result, wherein the kernel density estimation result is a probability density function estimate of the sample, that is, the first function of the embodiment of the present application.

[0209] For example, see Fig. 9 , Fig. 9 is a schematic diagram of a first function corresponding to a first region provided in an embodiment of the present application. Fig. 9 As shown, the horizontal axis of the coordinate axis represents the brightness value, and the vertical axis represents the probability density. There are M-1 third brightness average values ​​distributed on the horizontal axis, for example Fig. 9 Shown , , , , to A total of M-1 third brightness average values.

[0210] Based on expression (7), we can know that the kernel density estimation function It is related to one or more of the following factors: the category of the first kernel function, the length of the bandwidth, the average value of the target brightness, etc. For example, the category of the first kernel function includes but is not limited to Gaussian kernel, Epanechnikov kernel and Uniform kernel, etc. The embodiment of the present application does not limit the category of the first kernel function. For the introduction of expression (7), please refer to the following embodiment.

[0211] Exemplarily, first, the electronic device may obtain the category and bandwidth of the preset first kernel function, and then, Fig. 9 On the coordinate axis shown, a first kernel function is applied to each third brightness average value to obtain M-1 first kernel functions. For example, Fig. 9 Shown , , , , to Each corresponds to a first kernel function, and the positions of the center points of the M-1 first kernel functions on the coordinate axis correspond to the M-1 third brightness average values, for example, Fig. 9 The center point of the first kernel function 1 and the first third brightness average ( ). The width of the M-1 first kernel functions is determined according to the length of the width interval (e.g., 2h in expression (3)), that is, the width of the first kernel function is equal to the bandwidth h. Finally, the electronic device sums the M-1 first kernel functions to obtain Fig. 9The first function corresponding to the first region shown is shown in FIG. Wherein, expression (3) and the width interval can refer to the following embodiment.

[0212] Optionally, after obtaining the M-1 first kernel functions, the electronic device may also calculate the probability density contribution at each position, and then sum the probability density contributions to obtain a probability density estimate for the position. Thus, by summing each position, the first function corresponding to the first region may be obtained.

[0213] S804: Determine a first function corresponding to the second area according to M-1 fourth brightness average values.

[0214] Specifically, the electronic device may use a kernel density estimation algorithm to perform density estimation on M-1 fourth brightness average values ​​to obtain a kernel density estimation result, wherein the kernel density estimation result is a probability density function estimation of the sample, that is, the first function of the embodiment of the present application.

[0215] For example, see Fig.10 , Fig.10 is a schematic diagram of a first function corresponding to a second region provided in an embodiment of the present application. Fig.10 As shown, the horizontal axis of the coordinate axis represents the brightness value, and the vertical axis represents the probability density. There are M-1 fourth brightness average values ​​distributed on the horizontal axis, for example Fig.10 Shown , , , , B5 to There are a total of M-1 fourth brightness average values.

[0216] First, the electronic device can obtain the category and bandwidth of the preset second kernel function, and then Fig.10 On the coordinate axis shown, a second kernel function is applied to each fourth brightness average value to obtain M-1 second kernel functions, for example, Fig.10 Shown , , , , B5 to Each corresponds to a second kernel function. And the positions of the center points of the M-1 second kernel functions on the coordinate axis correspond to the M-1 fourth brightness average values ​​one by one, for example, Fig.10 The center point of the second kernel function 1 and the first fourth brightness average ( ). The width of the M-1 second kernel functions is determined according to the length of the width interval (e.g., 2h in expression (3)), that is, the width of the second kernel function is equal to the bandwidth h. Finally, the electronic device sums the M-1 second kernel functions to obtain Fig.10The first function corresponding to the second region shown is shown in FIG. Wherein, expression (3) and the width interval can refer to the following embodiment.

[0217] Optionally, after obtaining M-1 second kernel functions, the electronic device can also calculate the probability density contribution at each position, and then sum the probability density contributions to obtain a probability density estimate of the position. Thus, by summing each position, the first function corresponding to the second area can be obtained.

[0218] It should be noted that the choice of kernel function category has a certain impact on the final density estimation result, but the density estimation results given by different kernel functions are not much different. For example, when M is 2, the probability density estimation of the electronic device based on the first frame image depends on the choice of kernel function category. For example, the Gaussian kernel has smoothness and is suitable for processing continuous data, but may produce large estimation errors at the edges. The uniform kernel gives the same weight to all data points, which is simple and easy to use, but may lead to insufficient smoothness of the estimation result. However, when the number of M-1 frame images is large enough (more than the number threshold), the electronic device can fully capture the trend of brightness change of the M-1 frame image through kernel density estimation, which helps to smooth the brightness change and reduce the influence of single frame noise on the estimation result (ie, the first function), thereby improving the accuracy of the estimation result, so that the density estimation results obtained based on different kernel functions are not much different. In addition, the accurate first function provides a reliable basis for whether to perform brightness correction on the Mth frame image in the future, making the image detail enhancement more effective.

[0219] Among them, the above-mentioned reduction of single-frame noise can be understood as the electronic device can filter out the target brightness average value that may interfere with the result through the kernel function. For example, in the M-1 frame image, the shooting scene corresponding to the Kth frame and the K+1th frame image is the second scene, and the shooting scene corresponding to the other images in the M-1 frame image and the Mth frame image is the first scene. Since the second scene is different from the first scene, the Kth frame and the K+1th frame image are images that may interfere with the result. The electronic device can filter out the target brightness average value corresponding to the Kth frame and the K+1th frame image through the kernel function to reduce the interference of noise and improve the accuracy of the first function.

[0220] It should be noted that the above-mentioned use of the kernel density estimation algorithm to determine the first function is only used as an example. Since the result of the kernel density estimation (such as the first function) is a continuous density curve, it can provide a smoother and more accurate probability density estimation result. Therefore, the embodiment of the present application is explained based on the kernel density estimation algorithm as an example. Other density estimation algorithms can also be used for estimation, which is not limited here.

[0221] S805: Determine a first brightness threshold according to a first function corresponding to the first area.

[0222] Specifically, the electronic device may integrate the first function corresponding to the first area based on the first integral value to obtain a first brightness interval, and then determine a first brightness threshold according to the first brightness interval. The specific process of the electronic device determining the first brightness threshold according to the first function corresponding to the first area can be found in Figure 6 The corresponding embodiments.

[0223] S806: Determine a second brightness threshold according to the first function corresponding to the second area.

[0224] Specifically, the electronic device may integrate the first function corresponding to the second area based on the second integral value to obtain a second brightness interval, and then determine the second brightness threshold according to the second brightness interval. The specific process of the electronic device determining the second brightness threshold according to the first function corresponding to the second area can be referred to. Figure 7 The corresponding embodiments.

[0225] S807, determining a first average brightness value and a second average brightness value.

[0226] Specifically, the electronic device determines the first average brightness value according to the average brightness value corresponding to the pixels contained in the first area in the Mth frame image, and determines the second average brightness value according to the average brightness value corresponding to the pixels contained in the second area. The brightness values ​​corresponding to the pixels contained in the first area are less than the first preset threshold and greater than the second preset threshold, and the brightness values ​​corresponding to the pixels contained in the second area are less than or equal to the second preset threshold. The process of the electronic device determining the first average brightness value and the second average brightness value is consistent with the method of determining the third average brightness value and the fourth average brightness value in step S802, and will not be repeated here.

[0227] S808: Determine whether the first brightness average value is less than a first brightness threshold.

[0228] Specifically, after obtaining the first average brightness value, the electronic device compares the first average brightness value with the first brightness threshold. If the comparison result is that the first average brightness value is less than the first brightness threshold, S809 is executed. If the comparison result is that the first average brightness value is greater than or equal to the first brightness threshold, it means that the brightness of the first area in the M-th frame image is significantly different from the brightness of the first area corresponding to the M-1 frame image. The electronic device does not adjust the brightness of the M-th frame image to avoid brightness "jumps" after adjusting the brightness of the M-th frame image. The electronic device can display the M-th frame image in the preview interface. Optionally, the electronic device can obtain the M+1 frame image and determine whether to adjust the brightness of the M+1 frame image based on the M-1 frame images before the M+1 frame image.

[0229] S809: Determine whether the second brightness average value is greater than the second brightness threshold.

[0230] Specifically, after obtaining the second average brightness value, the electronic device compares the second average brightness value with the second brightness threshold. If the comparison result is that the second average brightness value is greater than the second brightness threshold, and according to S808, it is determined that the first average brightness value is less than the first brightness threshold, it means that the brightness difference between the first area in the M-th frame image and the first area corresponding to the M-1 frame image is small, and the brightness difference between the second area in the M-th frame image and the second area corresponding to the M-1 frame image is small, and the electronic device can execute S810. If the comparison result is that the second average brightness value is less than or equal to the second brightness threshold, it means that the brightness difference between the second area in the M-th frame image and the second area corresponding to the M-1 frame image is large, then the electronic device does not adjust the brightness of the M-th frame image to avoid brightness "jump" caused by adjusting the brightness of the M-th frame image, and the electronic device can display the M-th frame image in the preview interface. Optionally, the electronic device can obtain the M+1 frame image, and determine whether to adjust the brightness of the M+1 frame image based on the M-1 frame image before the M+1 frame image.

[0231] It should be noted that the embodiment of the present application does not limit the order in which the electronic device executes S808 and S809. The electronic device may first execute S808 and then execute S809, or first execute S808 and then execute S809, or execute S808 and S809 at the same time.

[0232] S810, adjusting the brightness of the Mth frame image to obtain a first image.

[0233] Specifically, when the electronic device obtains, based on S808 and S809, that the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the electronic device can respectively adjust the brightness of the first area and the second area of ​​the Mth frame image to enhance the detail brightness of the Mth frame image.

[0234] For example, in order to match the nonlinear response characteristics of the human eye to light intensity, electronic devices usually perform a gamma transformation on the brightness of the image after capturing the image, and transform the linear brightness value of the image into a nonlinear brightness value. Therefore, if the brightness value of the Mth frame image is nonlinear, the electronic device needs to transform the nonlinear brightness value into a linear brightness value through an inverse gamma transformation before adjusting the brightness value of the Mth frame image, and then adjust the linear brightness value. Therefore, the embodiments of the present application are combined with Fig.11 Taking the nonlinear brightness value of the M-th frame image as an example, the process of the electronic device adjusting the brightness of the M-th frame image is described.

[0235] See also Fig.11 , Fig.11 This is a flow chart of adjusting image brightness provided by an embodiment of the present application. Fig.11 As shown, the flowchart includes steps S1101 to S1103, and the specific process is as follows:

[0236] S1101, partitioning pixels of the Mth frame image.

[0237] For example, taking the color mode of the M-th frame image as the (Luminance, Y) brightness (Chrominance, U) density (Chroma, V) mode as an example, the process of transforming the nonlinear brightness value into the linear brightness value through the inverse gamma transformation can be seen in the following expression:

[0238]

[0239] in, is the brightness value corresponding to the i-th pixel in the M-th frame image, for example, the Y value corresponding to the i-th pixel, is the gamma coefficient, For The result of inverse gamma transformation. Thus, the electronic device can obtain the result of inverse gamma transformation of the pixels contained in the M-th frame image according to the above expression.

[0240] Exemplarily, the electronic device partitions the pixels of the Mth frame image according to the strategy for adjusting the brightness of the Mth frame image. For example, the strategy for adjusting the brightness of the electronic device involves a first area (e.g., a transition area), a second area (e.g., a dark area), and a third area (e.g., a bright area). Therefore, the electronic device can divide the pixels contained in the Mth frame image into the first area, the second area, and the third area.

[0241] For example, when the result of the inverse gamma transformation of the pixel is less than or equal to the first transformation threshold, the pixel is divided into the second area. When the result of the inverse gamma transformation of the pixel is greater than the first transformation threshold and less than the second transformation threshold, the pixel is divided into the first area. When the result of the inverse gamma transformation of the pixel is greater than or equal to the second transformation threshold, the pixel is divided into the third area. Thus, the electronic device can obtain the pixels corresponding to the first area, the second area, and the third area respectively.

[0242] S1102, performing brightness fitting on pixels according to different partitioning results.

[0243] Specifically, after the electronic device obtains the pixels corresponding to the first area, the second area, and the third area respectively, the electronic device may perform brightness fitting on the pixels according to adjustment strategies for different areas.

[0244] Exemplarily, the electronic device performs brightness fitting on the pixels of the second area by using a straight line whose fitting function is linear and whose slope is less than 1, so as to reduce the brightness values ​​corresponding to the pixels of the second area and obtain a brightness fitting result for the second area.

[0245] Exemplarily, the electronic device performs brightness fitting on the pixels of the first region by polynomial fitting or table lookup fitting to increase the brightness values ​​corresponding to the pixels of the first region and obtain a brightness fitting result for the first region.

[0246] Exemplarily, the electronic device performs brightness fitting on the pixels of the third region through a straight line with a slope of 1, so that the brightness values ​​corresponding to the pixels of the third region remain unchanged, thereby obtaining a brightness fitting result for the third region.

[0247] S1103, adjusting the brightness value of the Mth frame image according to the brightness fitting result to obtain a first image.

[0248] Specifically, the electronic device adjusts the result of inverse gamma transformation of the pixels in the first area of ​​the M-th frame image according to the brightness fitting result of the first area, so as to enhance the brightness of the first area and increase the detail display effect of the first area. According to the brightness fitting result of the second area, the electronic device adjusts the result of inverse gamma transformation of the pixels in the second area of ​​the M-th frame image, so as to suppress the brightness of the second area and increase the detail display effect of the second area. According to the brightness fitting result of the third area, the electronic device adjusts the result of inverse gamma transformation of the pixels in the third area of ​​the M-th frame image.

[0249] Then, the electronic device performs gamma transformation on the result of inverse gamma transformation on the adjusted pixels, so that the linear brightness value is transformed into a nonlinear brightness value, which is convenient for human eyes to watch.

[0250] For example, the electronic device may perform gamma transformation based on the following expression to obtain the first image:

[0251]

[0252] in, is the result of inverse gamma transformation of the adjusted i-th pixel, is the gamma coefficient, is the brightness value corresponding to the i-th pixel in the first image.

[0253] It should be noted that when the brightness value of the Mth frame image is linear, the electronic device may not perform an inverse gamma transformation on the brightness value corresponding to the pixel contained in the Mth frame image, and adjust the brightness of pixels in different areas of the Mth frame image according to the adjustment strategy in step S1102 to obtain a first image.

[0254] The following describes in detail the process of the electronic device determining the first function by means of kernel density estimation.

[0255] In a possible implementation, the electronic device may obtain a first function based on the M-1 target brightness averages by non-parametric estimation, so as to determine the distribution of brightness values ​​corresponding to the M-1 frames of images according to the first function. Non-parametric estimation, also known as non-parametric testing, refers to a series of methods that directly use prior knowledge of learning samples of known categories (such as the M-1 target brightness averages) to perform statistical testing and judgment analysis without considering the original overall distribution (such as the distribution of brightness values ​​corresponding to the M-1 frames of images) or without making assumptions about parameters.

[0256] Exemplarily, the electronic device determines the second function according to the number of M-1 target brightness average values ​​that are less than or equal to the target brightness threshold. The target brightness threshold here is an unknown quantity, so the target brightness threshold here can be used as the independent variable of the second function. For example, the expression of the second function can refer to the following expression (1):

[0257]

[0258] in, is the average target brightness, is the number of target brightness average values. For example, in the embodiment of the present application, the electronic device captures M-1 frames of images, so n is M-1, that is, n third brightness average values ​​or n fourth brightness average values. is the average brightness of n targets The average brightness of the target.

[0259] In expression (1), is an indicative function, and can also be seen in the following expression (2):

[0260]

[0261] Based on expression (2), it can be seen that when the target brightness average Less than or equal to hour, The value of is 1. When the average brightness of the target Does not satisfy less than or equal to Under this condition, The value of is 0. Therefore, based on It can be determined The average brightness of the targets is less than or equal to The number of times it occurs (i.e. frequency).

[0262] is the empirical distribution function, that is and The divisor of The average brightness of the targets is less than or equal to The probability of , in nonparametric estimation, is approximately equal to the cumulative distribution function of the sample Cumulative distribution function Characterizes the number of brightness values ​​in the sample as a whole (the number of brightness values ​​contained in the sample as a whole is greater than , the embodiment of this application uses The average target brightness is used to fit the overall distribution of the sample) is less than or equal to probability.

[0263] The above cumulative distribution function The derivative of is the probability density function , probability density function The following expression (3):

[0264]

[0265] in, About The differential of About The differential of . represents the cumulative distribution function The increment of the output value, Represents the cumulative distribution function The increment of the independent variable, express In the width range The slope of the second function (or cumulative distribution function) is characterized by the ratio of the increment of the output value to the increment of the independent variable. ) over the width interval. express Approaching 0, This means calculating the limit when the increment of the independent variable approaches 0.

[0266] In one possible implementation, the electronic device can estimate the density of the To approximate the probability density function , see the following expression (4) for details:

[0267]

[0268] Among them, about and The explanation of can be found in the above expression (3). In the kernel density estimation function is the bandwidth of the kernel function (also known as window width). For an explanation of bandwidth, see the following description of kernel functions.

[0269] Since the second function The expression (2) is approximately equal to the cumulative distribution function , in the calculation and When and Substituting into expression (4), we get the following expression (5):

[0270]

[0271] Among them, about , , , , The explanation of can be found in the above expression, the indicative function It can also be expressed as the following expression (6):

[0272]

[0273] In expression (6) is the kernel function, i.e., the third function in the embodiment of the present application. When , the kernel function takes the value of 1. When it is less than -1 or greater than 1, the value of the kernel function is 0.

[0274] Therefore, the kernel density estimation function in expression (5) It can also be expressed based on the kernel function, as shown in the following expression (7):

[0275]

[0276] in, Represents the bandwidth of the kernel function, which is used to determine the width of the kernel function. Indicates when is the average brightness of n targets The function of the average brightness of the target, Used to determine the position of the center point of the kernel function on the coordinate axis. express The sum of kernel functions.

[0277] According to expression (7), the electronic devices can be indivual Determine indivual , that is, M-1 third functions are determined by using the M-1 target brightness average values ​​obtained in the embodiment of the present application.

[0278] Then, by superimposing kernel functions (i.e., M-1 third functions) to determine the kernel density estimation function Since the kernel density estimation function It can be approximately equal to the probability density function , so the first function can be determined by superimposing M-1 third functions.

[0279] The following will be combined Figures 12A-12D The process of adjusting the brightness of an image by an electronic device is described.

[0280] Fig. 12A The system desktop of an electronic device provided in an embodiment of the present application, Fig. 12B , Fig. 12C and Fig.12D It is a preview interface of a group of camera applications provided in an embodiment of the present application.

[0281] Fig. 12A This is a system desktop 1102 of an electronic device 100 provided in an embodiment of the present application. Fig. 12A As shown, the system desktop 1102 may include a status bar 1101, a page indicator 1103, and a plurality of application icons.

[0282] Among them, the status bar 1101 may include one or more signal strength indicators of mobile communication signals (also known as cellular signals), such as the fifth generation mobile communication technology (5th Generation Mobile Communication Technology, 5G), wireless high fidelity (wireless fidelity, Wi-Fi) signal strength indicator, battery status indicator, time indicator (for example, 8:00), etc.

[0283] The page indicator 1103 may be used to indicate the positional relationship between the currently displayed page and other pages.

[0284] Multiple application icons may include a time application icon (e.g., 08:00), a date application icon (e.g., January 1, Friday), a weather application icon (e.g., 5°C), an application market application icon, a memo application icon, a mall application icon, a browser application icon, a phone application icon, a message application icon, a camera application icon, a settings application icon, etc. Not limited to the above icons, the system desktop 1102 may also include other application icons, which are not listed here one by one. Multiple application icons may be distributed on multiple pages. The page indicator 1103 may be used to indicate which page the user is currently browsing among multiple pages that carry multiple applications. The user can browse other pages by sliding left and right.

[0285] Understandably, Fig. 12A The user interface described below illustrates a possible user interface style of an electronic device taking a mobile phone as an example, and should not constitute a limitation on the embodiments of the present application.

[0286] like Fig. 12A As shown, the electronic device 100 may receive a user operation for starting a camera application, such as clicking an operation on a desktop icon of the camera application. In response to the operation, the electronic device may display a Fig. 12B The preview interface of the camera application is shown.

[0287] Fig. 12B The user interface of the shooting and display service provided by the embodiment of the present application is exemplarily shown, also called the preview interface 103. Fig. 12B In the preview interface 103 shown, the electronic device 100 displays an image 102, a function selection area 1031, a mode selection area 1036, a zoom function 1032, a gallery 1033, a shooting button 1034, and a switch button 1035. Among them, the function selection area 1031 includes at least one function setting button. Exemplarily, the function setting button includes but is not limited to: a visual button, a flash button, an artificial intelligence (AI) camera button, a color button, and a setting button. Optionally, if the electronic device 100 detects that the user touches any function setting button in the function selection area 1031, the corresponding function is set according to the touched function setting button. For example, if it is detected that the user touches the flash button, the flash is set. Exemplarily, the mode selection area 1036 includes at least one shooting mode button. The shooting mode button includes but is not limited to: an aperture mode button, a night scene mode button, a portrait mode button, a photo mode button, a video mode button, a professional mode button, a more mode button, etc. Optionally, if the electronic device 100 detects that the user touches any shooting mode button in the mode selection area 1036, it switches to the corresponding shooting mode according to the touched shooting mode button. Exemplarily, the gallery 1033 is used to review the taken pictures. Specifically, the gallery 1033 is used to provide the user with an entrance to review the taken pictures. The user clicks the gallery 1033 to enter the interface for reviewing the taken pictures. Exemplarily, the shooting button 1034 is used to take pictures, and the switch button 1035 is used to switch the front and rear cameras when shooting.

[0288] like Fig. 12B As shown, the electronic device 100 responds to the user operation from Fig. 12A The system desktop 1102 shown switches to Fig. 12BAfter the preview interface 103 is shown, the electronic device 100 may receive a user operation to switch modes, such as swiping left / right in the mode selection area 1036, and change the currently used shooting mode according to the operation. By default, the electronic device first uses the "photographing" mode. In one implementation, if the shooting mode currently used by the electronic device 100 is not the "photographing" mode, when a left swipe operation / right swipe operation of dragging the mode selection area 1036 is received and the float stops at the "photographing" option, the electronic device 100 may switch to the "photographing" mode.

[0289] like Fig. 12B As shown, in the "photographing" mode, the electronic device 100 collects data for Fig. 12B The image corresponding to the shooting scene A shown in the figure is then displayed on the preview interface of the camera application. Figure 1A As shown in the image 102, Fig. 12B The preview interface 103 shown in FIG. 1 shows an image 102. For a description of the shooting scene A, the table 1011, the cup 1012, the box 1013 included in the shooting scene A, and the image 102, please refer to the above description. Figure 1A The corresponding embodiments will not be described in detail here.

[0290] In a possible implementation, after the electronic device acquires the image 102, it determines based on the image 102 that there is a difference between the brightness of the image 102 and the actual brightness of the shooting scene A. Based on the difference between the brightness of the image 102 and the actual brightness, after acquiring the image corresponding to the shooting scene A, the electronic device may first adjust the brightness value of the image, and then display the adjusted image in the preview interface, so as to reduce the difference between the brightness of the image and the actual brightness by adjusting the brightness value.

[0291] In one possible implementation, the electronic device captures image A after image 102. Image A is the Mth frame image in the embodiment of the present application, and the electronic device needs to determine whether to adjust the brightness of image A. First, the electronic device obtains the M-1 frame image before image A, and determines the target brightness threshold according to the brightness values ​​corresponding to the M-1 frame images. Then, the target brightness threshold is compared with the brightness value of image A. For example, based on image A and image 102, the average brightness of the dark area of ​​image A is less than the average brightness of the dark area of ​​image 102, which does not meet the preset conditions. Therefore, the electronic device does not adjust the brightness of image A. Fig. 12C The preview interface 107 shown shows image A (ie Fig. 12C The dark areas of image A and image 102 can be seen in Figure 1A The explanation about the dark area of ​​shooting scene A is not repeated here.

[0292] like Fig. 12C As shown, the preview interface 107 displays an image 106. Fig. 12C The image 106 shown is Fig. 12B The brightness difference between the images 102 shown is a first brightness difference. Figure 1B The brightness difference between the image 104 (the image after brightness adjustment of the image A) and the image 102 is the second brightness difference, and the first brightness difference is smaller than the second brightness difference. Optionally, the brightness difference between the table 1061 in the image 106 and the table 1021 in the image 102 is smaller than the brightness difference between the table 1041 in the image 104 and the table 1021 in the image 102. Therefore, when the difference between the M-th frame image (e.g., image A) and the M-1 frame image (e.g., image 102) is large, if the electronic device does not adjust the brightness of the M-th frame image, the brightness "jump" phenomenon can be reduced.

[0293] In a possible implementation, after the electronic device acquires the image 106, Fig. 12B The shooting scene A shown captures image C. Image C is the Mth frame image in the embodiment of the present application, and the electronic device needs to determine whether to adjust the brightness of image C. First, the electronic device obtains the M-1 frame image before image C, and determines the target brightness threshold (including the first brightness threshold corresponding to the first area and the second brightness threshold corresponding to the second area) according to the brightness values ​​corresponding to the M-1 frame images. Then, the target brightness threshold is compared with the brightness value of image C. If the first brightness average value corresponding to the first area in image C is less than the first brightness threshold, and the second brightness average value corresponding to the second area is greater than the second brightness threshold, the electronic device adjusts the brightness of the Mth frame image. Then, the electronic device displays the adjusted image C (i.e., image 108) as shown in the figure. Fig.12D In the preview interface 109 shown.

[0294] like Fig.12D As shown, the preview interface 109 displays an image 108. It can be seen that the brightness difference between the image 108 and the images 106 and 102 is small, and there is no brightness "jump" phenomenon. In addition, the brightness of the image 108 is close to Fig. 12B The actual brightness of the captured scene A is shown. For example, the brightness of the table 1081 in the image 108 is close to Fig. 12B The table 1011 in the shooting scene A is shown, thereby achieving correction of the image brightness value.

[0295] The following will be combined Figures 13A-13D The process of adjusting the brightness of an image by an electronic device is described.

[0296] Fig.13AThe system desktop of an electronic device provided in an embodiment of the present application, Fig. 13B , Fig. 13C and Fig.13D This is another set of preview interfaces for camera applications provided in an embodiment of the present application.

[0297] Fig.13A This is a system desktop 210 of an electronic device 100 provided in an embodiment of the present application. Fig.13A As shown, the system desktop 210 may include a status bar 1101, a page indicator 1103, and a plurality of application icons. The status bar 1101, the page indicator 1103, and the plurality of application icons included in the system desktop 210 may be referred to in Fig. 12A The relevant introduction will not be repeated here.

[0298] Understandably, Fig.13A The user interface described below illustrates a possible user interface style of an electronic device taking a mobile phone as an example, and should not constitute a limitation on the embodiments of the present application.

[0299] like Fig.13A As shown, the electronic device 100 may receive a user operation for starting a camera application, such as clicking an operation on a desktop icon of the camera application. In response to the operation, the electronic device may display a Fig. 13B The preview interface of the camera application is shown.

[0300] Fig. 13B The user interface of the shooting and display service provided by the embodiment of the present application is exemplarily shown, also called the preview interface 203. Fig. 13B In the preview interface 203 shown, the electronic device 100 displays an image 202, a function selection area 1031, a mode selection area 1036, a zoom function 1032, a gallery 1033, a shooting button 1034, and a switch button 1035. The function selection area 1031, the mode selection area 1036, the zoom function 1032, the gallery 1033, the shooting button 1034, and the switch button 1035 can be seen in FIG. Fig. 12B The relevant introduction will not be repeated here.

[0301] like Fig. 13B As shown, the electronic device 100 responds to the user operation from Fig.13A The system desktop 210 shown switches to Fig. 13B After the preview interface 203 is shown, the electronic device 100 may receive a user operation of switching modes, such as sliding left / right in the mode selection area 1036, and change the currently used shooting mode according to the operation. By default, the electronic device first uses the "photographing" mode.

[0302] like Fig. 13BAs shown, in the "photographing" mode, the image frames captured by the camera for the shooting scene B are displayed in the preview interface 203, for example, Figure 2A The image 202 shown. For the description of the shooting scene B, the desktop 2011, the box 2012 and the notebook 2013 included in the shooting scene B, and the image 202, please refer to the above Figure 2A The corresponding embodiments will not be described in detail here.

[0303] In a possible implementation, after the electronic device acquires the image 202, it determines based on the image 202 that there is a difference between the brightness of the image 202 and the actual brightness of the shooting scene B. The electronic device may adjust the brightness value of the image after acquiring the image corresponding to the shooting scene B based on the difference between the brightness of the image 202 and the actual brightness, and then display the adjusted image in the preview interface, so as to reduce the difference between the brightness of the image and the actual brightness by adjusting the brightness value.

[0304] In one possible implementation, the electronic device captures image B after image 202. Image B is the Mth frame image in the embodiment of the present application, and the electronic device needs to determine whether to adjust the brightness of image B. First, the electronic device obtains the M-1 frame image before image B, and determines the target brightness threshold according to the brightness values ​​corresponding to the M-1 frame images. Then, the target brightness threshold is compared with the brightness value of image B. For example, the average brightness of the transition area of ​​image B is greater than the average brightness of the transition area of ​​image 202, which does not meet the preset conditions. Therefore, the electronic device does not adjust the brightness of image B. Fig. 13C The preview interface 207 shown shows image B (ie Fig. 13C The transition area of ​​image B and the transition area of ​​image 202 can be seen in Figure 2A Explanation of the transition area for shooting scene B.

[0305] like Fig. 13C As shown, the preview interface 207 displays an image 206. Fig. 13C The image 206 shown is similar to Fig. 13B The brightness difference between the images 202 shown is the third brightness difference. Figure 2BThe brightness difference between the image 204 (the image after brightness adjustment of image B) and the image 202 is the fourth brightness difference, and the third brightness difference is smaller than the fourth brightness difference. Optionally, the brightness difference between the notebook 2061 in the image 206 and the notebook 2021 in the image 202 is smaller than the brightness difference between the notebook 2041 in the image 204 and the notebook 2021 in the image 202. Therefore, when the difference between the M-th frame image (e.g., image B) and the M-1 frame image (e.g., image 202) is large, if the electronic device does not adjust the brightness of the M-th frame image, the brightness "jump" phenomenon can be reduced.

[0306] In a possible implementation, after the electronic device acquires the image 206, Fig. 13B The shooting scene B shown captures image D. Image D is the Mth frame image in the embodiment of the present application, and the electronic device needs to determine whether to adjust the brightness of image D. First, the electronic device obtains the M-1 frame image before image D, and determines the target brightness threshold (including the first brightness threshold corresponding to the first area and the second brightness threshold corresponding to the second area) according to the brightness values ​​corresponding to the M-1 frame images. Then, the target brightness threshold is compared with the brightness value of image D. If the first brightness average value corresponding to the first area in image D is less than the first brightness threshold, and the second brightness average value corresponding to the second area is greater than the second brightness threshold, the electronic device adjusts the brightness of the Mth frame image. Then, the electronic device displays the adjusted image D (i.e., image 208) as shown in the figure. Fig.13D In the preview interface 209 shown.

[0307] like Fig.13D As shown, the preview interface 209 displays an image 208. It can be seen that the brightness difference between the image 208 and the images 206 and 202 is small, and there is no brightness "jump" phenomenon. In addition, the brightness of the image 208 is close to Fig. 13B The actual brightness of the captured scene B is shown. For example, the brightness of the notebook 2081 in the image 208 is close to Fig. 13B The notebook 2013 in the photographing scene B is shown, thereby achieving correction of the image brightness value.

[0308] It should be understood that each step in the above method embodiment provided by the present application can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0309] The present application also provides an electronic device, which may include: a memory and a processor, wherein the memory may be used to store a computer program; and the processor may be used to call the computer program in the memory so that the electronic device executes the method in any one of the above embodiments.

[0310] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.

[0311] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0312] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0313] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0314] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0315] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0316] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables a computer to execute the method executed by the electronic device in any of the above embodiments when the computer program is executed.

[0317] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.

[0318] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0319] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk), etc.

[0320] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0321] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire the M-1 frame image and the M-th frame image, wherein M is an integer greater than 1; Determine a first function based on the brightness values ​​respectively corresponding to the M-1 frames of images; Performing an integration process on the first function based on a target integral value to determine a target brightness interval, wherein the target brightness interval includes a plurality of brightness values, and the target integral value is used to characterize the probability that the brightness values ​​respectively corresponding to the M-1 frame images are in the target brightness interval; Determining a target brightness threshold according to the target brightness interval; Adjusting the brightness value of the M-th frame image according to the target brightness threshold to obtain a first image; The first image is displayed on the preview interface.

2. The method according to claim 1, characterized in that The target brightness threshold includes a first brightness threshold and a second brightness threshold, and adjusting the brightness value of the M-th frame image according to the target brightness threshold includes: When the first brightness average value of the M-th frame image is less than the first brightness threshold, adjusting the brightness value of the M-th frame image, wherein the first brightness average value is determined according to the brightness values ​​corresponding to the pixels included in the first area of ​​the M-th frame image, and the brightness values ​​corresponding to the pixels included in the first area are less than the first preset threshold and greater than the second preset threshold; When the second brightness average value of the M-th frame image is greater than the second brightness threshold, adjusting the brightness value of the M-th frame image, wherein the second brightness average value is determined according to the brightness values ​​corresponding to the pixels included in the second area in the M-th frame image, and the brightness values ​​corresponding to the pixels included in the second area are less than or equal to the second preset threshold; When the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the brightness value of the Mth frame image is adjusted, wherein the first brightness threshold is greater than the second brightness threshold.

3. The method according to claim 2, characterized in that The determining of the first function based on the brightness values ​​respectively corresponding to the M-1 frame images includes: Respectively obtaining brightness values ​​corresponding to pixels contained in the first area in the M-1 frame image, and brightness values ​​corresponding to pixels contained in the second area in the M-1 frame image; Averaging the brightness values ​​corresponding to the pixels in the first area in the M-1 frame image to obtain M-1 third brightness average values; Respectively average the brightness values ​​corresponding to the pixels included in the second area in the M-1 frame image to obtain M-1 fourth brightness average values; The first function is determined according to M-1 target brightness average values, wherein the target brightness average value includes the third brightness average value and the fourth brightness average value, the first function includes a first function corresponding to the first area and a first function corresponding to the second area, the third brightness average value is used to determine the first function corresponding to the first area, the fourth brightness average value is used to determine the first function corresponding to the second area, the first function corresponding to the first area is used to determine the first brightness threshold, and the first function corresponding to the second area is used to determine the second brightness threshold.

4. The method according to claim 3, characterized in that The determining the first function according to the average brightness values ​​of M-1 targets includes: Determine a second function according to the number of the M-1 target brightness averages that are less than or equal to the target brightness threshold, wherein the second function is used to characterize the probability that the M-1 target brightness averages are less than or equal to the target brightness threshold; Determine M-1 third functions according to the rate of change of the second function in the width interval, wherein the positions of the center points of the M-1 third functions on the coordinate axis correspond one-to-one to the M-1 target brightness average values, and the width of any third function among the M-1 third functions is determined according to the length of the width interval; The first function is determined by superimposing the M-1 third functions.

5. The method according to claim 2, characterized in that: The adjusting the brightness value of the Mth frame image according to the target brightness threshold comprises: When the first brightness average value is less than the first brightness threshold, increasing the brightness values ​​corresponding to the pixels included in the first area in the Mth frame image; When the second brightness average value is greater than the second brightness threshold, reducing the brightness values ​​corresponding to the pixels included in the second area in the Mth frame image; When the first brightness average value is less than the first brightness threshold and the second brightness average value is greater than the second brightness threshold, the brightness values ​​corresponding to the pixels contained in the first area in the M-th frame image are increased, and the brightness values ​​corresponding to the pixels contained in the second area in the M-th frame image are decreased.

6. An electronic device, characterized in that: The electronic device comprises: one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the electronic device executes the method as described in any one of claims 1 to 5.

7. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes any one of the methods described in claims 1 to 5.

8. A computer program product comprising instructions, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 5.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 5.

Images