Electronic equipment, execution method thereof and computer readable storage medium

Abstract

The embodiment of the invention provides electronic equipment, an execution method thereof and a computer readable storage medium, and relates to the field of artificial intelligence. The method executed by the electronic equipment comprises the following steps: acquiring image brightness information of a first image and scene brightness information of a shooting scene corresponding to the first image; and performing brightness adjustment on the first image based on the image brightness information and the scene brightness information. Optionally, the method may be performed using an artificial intelligence model.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a method executed by an electronic device, the electronic device, and a computer-readable storage medium. Background Art

[0002] As users have higher and higher demands for image quality, better image brightness adjustment technology is needed. Adjustable brightness correction is an advanced image quality improvement function, which can include: manual image brightness adjustment and automatic image brightness adjustment, providing users with higher freedom of image brightness adjustment and better experience. However, the image quality obtained by the current image brightness adjustment solution is still difficult to meet user needs. Summary of the invention

[0003] According to a first aspect of an embodiment of the present disclosure, a method executed by an electronic device is provided, comprising: acquiring image brightness information of a first image and scene brightness information of a shooting scene corresponding to the first image; and performing brightness adjustment on the first image based on the image brightness information and the scene brightness information.

[0004] Optionally, performing brightness adjustment on the first image based on the image brightness information and the scene brightness information includes: determining an adjustment intensity parameter range corresponding to the first image based on the image brightness information and the scene brightness information; and performing brightness adjustment on the first image according to the adjustment intensity parameter range.

[0005] Optionally, determining the adjustment intensity parameter range corresponding to the first image based on the image brightness information and the scene brightness information includes: determining the brightness range of the first image based on image features of the first image; and determining the adjustment intensity parameter range based on the brightness range, the image brightness information and the scene brightness information.

[0006] Optionally, determining the brightness range of the first image based on the image features of the first image includes: based on the image features of the first image, determining the maximum brightness feature and the minimum brightness feature of the first image using maximum pooling and minimum pooling respectively; wherein, determining the adjustment intensity parameter range based on the brightness range, the image brightness information and the scene brightness information includes: determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the maximum brightness feature, the image brightness information and the scene brightness information; determining the minimum brightness adjustment intensity parameter corresponding to the first image based on the minimum brightness feature, the image brightness information and the scene brightness information.

[0007] Optionally, the adjustment intensity parameter range includes: a maximum brightness adjustment intensity parameter and a minimum brightness adjustment intensity parameter; performing brightness adjustment on the first image according to the adjustment intensity parameter range includes: for each local area of ​​the first image, based on the brightness value of the local area, fusing the maximum brightness adjustment intensity parameter and the minimum brightness adjustment intensity parameter corresponding to the local area, and adjusting the brightness of the local area based on the fused adjustment intensity parameter.

[0008] Optionally, based on the brightness value of the local area, the maximum brightness adjustment intensity parameter and the minimum brightness adjustment intensity parameter corresponding to the local area are fused, and the brightness of the local area is adjusted based on the fused adjustment intensity parameter, including: determining the local brightness adjustment intensity parameter corresponding to the local area based on the image features of the local area; determining the first weight of the maximum brightness adjustment intensity parameter, the second weight of the minimum brightness adjustment intensity parameter and the third weight of the local brightness adjustment intensity parameter corresponding to the local area based on the brightness value of the local area; using the first weight, the second weight and the third weight to weightedly fuse the maximum brightness adjustment intensity parameter, the minimum brightness adjustment intensity parameter and the local brightness adjustment intensity parameter to obtain the brightness adjustment intensity parameter corresponding to the local area; adjusting the brightness of the local area based on the brightness adjustment intensity parameter corresponding to the local area.

[0009] Optionally, obtaining the image brightness information of the first image and the scene brightness information of the shooting scene corresponding to the first image includes: performing global average pooling on the first image to obtain the image brightness information; obtaining shooting parameter information of the first image, and obtaining the scene brightness information based on the shooting parameter information.

[0010] Optionally, the acquiring the scene brightness information based on the shooting parameter information includes: determining a camera measured exposure value based on the shooting parameter information; and acquiring the scene brightness information based on the camera measured exposure value.

[0011] Optionally, acquiring the scene brightness information based on the shooting parameter information further includes: determining a normalized exposure value based on the shooting parameter information; wherein acquiring the scene brightness information based on the camera measured exposure value includes: acquiring the scene brightness information based on the camera measured exposure value and the normalized exposure value.

[0012] Optionally, the shooting parameter information includes at least one of the following: aperture value, exposure time, sensitivity, and baseline exposure value.

[0013] Optionally, adjusting the brightness of the local area based on the brightness adjustment intensity parameter corresponding to the local area includes: for the central pixel in the local area, adjusting the brightness of the central pixel using the brightness adjustment intensity parameter corresponding to the central pixel among the brightness adjustment intensity parameters corresponding to the local area; for the non-central pixel in the local area, adjusting the brightness of the non-central pixel based on the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local areas of the local area.

[0014] Optionally, adjusting the brightness of the non-center pixel based on the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area of ​​the local area includes: mapping the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area to the same spatial position; respectively applying the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area to obtain the brightness value of the non-center pixel adjusted using different brightness adjustment intensity parameters; and performing bilinear interpolation on the brightness values ​​adjusted using different brightness adjustment intensity parameters to obtain the final brightness value of the non-center pixel.

[0015] Optionally, the method further includes: detecting a target brightness adjustment intensity input by a user; and performing brightness adjustment on the first image based on the target brightness adjustment intensity.

[0016] Optionally, the target brightness adjustment intensity is the brightness adjustment intensity corresponding to the global area of ​​the first image, and performing brightness adjustment on the first image based on the target brightness adjustment intensity includes: obtaining brightness statistical information of the first image; determining a maximum brightness adjustment intensity parameter corresponding to the first image based on the brightness statistical information; and performing brightness adjustment on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity.

[0017] Optionally, determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the brightness statistical information includes: performing histogram equalization on the first image to obtain a histogram equalized image corresponding to the first image based on the brightness statistical information; and determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the histogram equalized image.

[0018] Optionally, determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the histogram equalized image includes: obtaining the brightness difference between the first image and the histogram equalized image; and determining the maximum brightness adjustment intensity parameter according to the brightness difference and image features of the first image.

[0019] Optionally, the target brightness adjustment intensity is the brightness adjustment intensity corresponding to the target local area in the first image, and performing brightness adjustment on the first image based on the target brightness adjustment intensity includes: adjusting the brightness of the target local area based on the target brightness adjustment intensity; obtaining the correlation between semantic features of each semantic category in the first image; and performing brightness adjustment on other local areas of the first image except the target local area based on the correlation.

[0020] Optionally, adjusting the brightness of the target local area based on the target brightness adjustment intensity includes: determining brightness distribution information of the target local area; and adjusting the brightness of the target local area based on the brightness distribution information of the target local area and the target brightness adjustment intensity.

[0021] Optionally, the brightness adjustment of the target local area based on the brightness distribution information of the target local area and the target brightness adjustment strength includes: determining a brightness enhancement coefficient for each semantic area in the target local area based on the brightness distribution information of the target local area and the target brightness adjustment strength; and adjusting the brightness of the target local area based on the brightness enhancement coefficient.

[0022] Optionally, the brightness adjustment of the target local area based on the brightness enhancement coefficient includes: converting the brightness enhancement coefficient into a convolution kernel; using the convolution kernel, performing mask convolution on the image features containing the brightness distribution information to obtain enhanced image features of the first image, wherein obtaining the correlation between the semantic features of each semantic category in the first image includes: extracting the semantic features based on the semantic map of the first image and the enhanced image features; and obtaining the correlation based on the semantic features.

[0023] Optionally, performing brightness adjustment on other local areas of the first image except the target local area based on the correlation includes: adjusting semantic features corresponding to the other local areas according to the correlation; and performing brightness adjustment on the other local areas according to the adjusted semantic features corresponding to the other local areas.

[0024] Optionally, performing brightness adjustment on the first image based on the target brightness adjustment intensity further includes: for local regions with the same semantic category in the first image, performing brightness adjustment on corresponding local regions based on image features of the local regions.

[0025] Optionally, in response to detecting that the brightness adjustment of the first image based on the target brightness adjustment intensity is completed, a step of performing brightness adjustment on the first image based on the image brightness information and the scene brightness information is performed.

[0026] According to a second aspect of an embodiment of the present disclosure, a method performed by an electronic device is provided, including: detecting a target brightness adjustment intensity input by a user, wherein the target brightness adjustment intensity is a brightness adjustment intensity corresponding to a global area of ​​a first image; obtaining brightness statistical information of the first image; determining a maximum brightness adjustment intensity parameter corresponding to the first image based on the brightness statistical information; and performing brightness adjustment on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity.

[0027] Optionally, determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the brightness statistical information includes: performing histogram equalization on the first image to obtain a histogram equalized image corresponding to the first image based on the brightness statistical information; and determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the histogram equalized image.

[0028] Optionally, determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the histogram equalized image includes: obtaining the brightness difference between the first image and the histogram equalized image; and determining the maximum brightness adjustment intensity parameter according to the brightness difference and image features of the first image.

[0029] According to a third aspect of an embodiment of the present disclosure, a method executed by an electronic device is provided, including: detecting a target brightness adjustment intensity input by a user, wherein the target brightness adjustment intensity is a brightness adjustment intensity corresponding to a target local area in a first image; performing brightness adjustment on the target local area based on the target brightness adjustment intensity; obtaining a correlation between semantic features of each semantic category in the first image; and performing brightness adjustment on other local areas of the first image except the target local area based on the correlation.

[0030] Optionally, adjusting the brightness of the target local area based on the target brightness adjustment intensity includes: determining brightness distribution information of the target local area; and adjusting the brightness of the target local area based on the brightness distribution information of the target local area and the target brightness adjustment intensity.

[0031] Optionally, the brightness adjustment of the target local area based on the brightness distribution information of the target local area and the target brightness adjustment strength includes: determining a brightness enhancement coefficient for each semantic area in the target local area based on the brightness distribution information of the target local area and the target brightness adjustment strength; and adjusting the brightness of the target local area based on the brightness enhancement coefficient.

[0032] Optionally, the brightness adjustment of the target local area based on the brightness enhancement coefficient includes: converting the brightness enhancement coefficient into a convolution kernel; using the convolution kernel, performing mask convolution on the image features containing the brightness distribution information to obtain enhanced image features of the first image, wherein obtaining the correlation between the semantic features of each semantic category in the first image includes: extracting the semantic features based on the semantic map of the first image and the enhanced image features; and obtaining the correlation based on the semantic features.

[0033] Optionally, performing brightness adjustment on other local areas of the first image except the target local area based on the correlation includes: adjusting semantic features corresponding to the other local areas according to the correlation; and performing brightness adjustment on the other local areas according to the adjusted semantic features corresponding to the other local areas.

[0034] Optionally, the method further includes: for local regions with the same semantic category in the first image, performing brightness adjustment on corresponding local regions based on image features of the local regions.

[0035] According to a fourth aspect of an embodiment of the present disclosure, a method performed by an electronic device is provided, including: detecting a target brightness adjustment intensity input by a user; performing brightness adjustment on a first image based on the target brightness adjustment intensity; in response to detecting that the brightness adjustment on the first image based on the target brightness adjustment intensity ends, performing brightness adjustment on the first image based on image brightness information of the first image and scene brightness information of a shooting scene corresponding to the first image.

[0036] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a memory; and a processor coupled to the memory and configured to execute the method as described above.

[0037] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium storing instructions is provided, which, when executed by at least one processor, prompts the at least one processor to execute the method described above.

[0038] According to the technical solution provided by the embodiments of the present disclosure, since the image brightness information of the first image and the scene brightness information of the shooting scene corresponding to the first image are obtained, and the brightness adjustment of the first image is performed based on the image brightness information and the scene brightness information, the image after the brightness adjustment can be made more suitable for the corresponding scene, thereby improving the quality of the image after the brightness adjustment.

[0039] According to the technical solution provided by the embodiments of the present disclosure, since the maximum brightness adjustment intensity parameter corresponding to the first image is determined according to the brightness statistical information of the first image, and brightness adjustment is performed on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity, the brightness adjustment can be performed within the range of the maximum brightness adjustment intensity parameter determined according to the brightness statistical information, so that different brightness areas in the image can obtain appropriate brightness enhancement, thereby improving the image quality after brightness adjustment.

[0040] According to the technical solution provided by the embodiments of the present disclosure, after the brightness of the target local area in the first image is adjusted based on the target brightness adjustment intensity, the correlation between the semantic features of each semantic category in the first image is obtained, and the brightness adjustment is performed on other local areas of the first image except the target local area based on the correlation. Therefore, the inharmonious phenomenon between the target local area and other local areas after the brightness adjustment of the target local area can be reduced, the adjusted image can be made more natural, and the image quality after the brightness adjustment is improved.

[0041] According to the technical solution provided by the embodiments of the present disclosure, since brightness adjustment is first performed on the first image based on the target brightness adjustment intensity, and then in response to detecting that the brightness adjustment on the first image based on the target brightness adjustment intensity is ended, brightness adjustment is performed on the first image based on the image brightness information of the first image and the scene brightness information of the shooting scene corresponding to the first image, thereby making it possible to obtain a higher quality brightness enhanced image with fewer user operations.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate exemplary embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0044] Figure 1 is a flowchart illustrating a method performed by an electronic device according to an embodiment of the present disclosure.

[0045] Figure 2An example of a global brightness adjustment framework according to an embodiment of the present disclosure is shown.

[0046] Figure 3 is a schematic diagram illustrating brightness embedding according to an embodiment of the present disclosure.

[0047] Figure 4 is a schematic diagram illustrating an example of local brightness adjustment according to an embodiment of the present disclosure.

[0048] Figure 5 is a schematic diagram illustrating semantic-aware enhancement according to an embodiment of the present disclosure.

[0049] Figure 6 is a schematic diagram illustrating performing masked convolution according to an embodiment of the present disclosure.

[0050] Figure 7 is a schematic diagram illustrating inter-region harmonization according to an embodiment of the present disclosure.

[0051] Figure 8 is a schematic diagram illustrating extraction of semantic features according to an embodiment of the present disclosure.

[0052] Fig. 9 is a schematic diagram illustrating inter-region harmonization based on semantic features according to an embodiment of the present disclosure.

[0053] Fig.10 is a schematic diagram illustrating intra-region differentiation according to an embodiment of the present disclosure.

[0054] Fig.11 Schematic diagram of the operation of the a priori calculation block according to an embodiment of the present disclosure.

[0055] Fig.12 is a schematic diagram of the operation of the brightness mapping control block according to an embodiment of the present disclosure.

[0056] Fig.13 is a schematic diagram of a prediction adjustment curve according to an embodiment of the present disclosure.

[0057] Fig.14 is a schematic diagram of a fusion adjustment curve according to an embodiment of the present disclosure.

[0058] Fig.15 is a schematic diagram of the operation of the brightness mapping and curve interpolation blocks according to an embodiment of the present disclosure.

[0059] Fig.16 is a schematic diagram of curve interpolation according to an embodiment of the present disclosure.

[0060] Fig.17 It is shown Figure 1 A schematic diagram of an example of a method performed by an electronic device is shown.

[0061] Fig.18 is a flowchart of a method executed by an electronic device according to another exemplary embodiment of the present disclosure.

[0062] Fig.19 is a flowchart of a method executed by an electronic device according to yet another exemplary embodiment of the present disclosure.

[0063] Fig. 20 is a flowchart of a method executed by an electronic device according to yet another exemplary embodiment of the present disclosure.

[0064] Fig.21 is a schematic diagram showing an example of an image brightness adjustment architecture according to an embodiment of the present disclosure.

[0065] Fig. 22 is a schematic diagram showing a detailed structure of an image brightness adjustment architecture according to an embodiment of the present disclosure.

[0066] Fig.23 is a schematic diagram illustrating the local brightness adjustment effect according to an embodiment of the present disclosure.

[0067] Fig.24 An example of the overall process of image brightness adjustment according to an embodiment of the present disclosure is shown.

[0068] Fig.25 is a schematic diagram of using the brightness adjustment solution according to an embodiment of the present disclosure in an album application.

[0069] Fig.26 It is a schematic diagram of the operation of using the brightness adjustment solution according to an embodiment of the present disclosure in an album application.

[0070] Fig. 27 is a schematic diagram of using the brightness adjustment solution according to an embodiment of the present disclosure in camera photography.

[0071] Fig.28 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0072] Fig.29 is a schematic structural diagram showing an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0073] The following description with reference to the accompanying drawings is provided to facilitate a comprehensive understanding of the various embodiments of the present disclosure as defined by the claims and their equivalents. This description includes various specific details to facilitate understanding but should be considered as exemplary only. Therefore, one of ordinary skill in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, for the sake of clarity and conciseness, descriptions of well-known functions and structures may be omitted.

[0074] The terms and expressions used in the following specification and claims are not limited to their dictionary meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Therefore, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purposes only and not for the purpose of limiting the present disclosure as defined in the appended claims and their equivalents.

[0075] It should be understood that the singular forms "a", "an", and "the" may also include plural references unless the context clearly indicates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more such surfaces. When we refer to an element as being "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or the one element and the other element may establish a connection relationship through an intermediate element. In addition, "connected" or "coupled" as used herein may include wireless connection or wireless coupling.

[0076] The term "include" or "may include" refers to the presence of the corresponding disclosed functions, operations or components that can be used in various embodiments of the present disclosure, rather than limiting the presence of one or more additional functions, operations or features. In addition, the term "include" or "have" may be interpreted as indicating certain characteristics, numbers, steps, operations, constituent elements, components or combinations thereof, but should not be interpreted as excluding the possibility of the presence of one or more other characteristics, numbers, steps, operations, constituent elements, components or combinations thereof.

[0077] The term "or" used in various embodiments of the present disclosure includes any of the listed terms and all combinations thereof. For example, "A or B" may include A, may include B, or may include both A and B. When describing multiple (two or more) items, if the relationship between the multiple items is not clearly defined, the multiple items may refer to one, multiple, or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" may be implemented as parameter A including A1 or A2 or A3, or may be implemented as parameter A including at least two of the three items A1, A2, and A3.

[0078] Unless defined differently, all terms (including technical terms or scientific terms) used in the present disclosure have the same meanings as understood by those skilled in the art described in the present disclosure. Common terms as defined in dictionaries are interpreted as having meanings consistent with the context in the relevant technical field, and should not be interpreted ideally or overly formally unless clearly defined in the present disclosure.

[0079] At least some functions of the device or electronic device provided in the embodiments of the present disclosure can be implemented by an AI model, such as at least one module among multiple modules of the device or electronic device can be implemented by an AI model. Functions associated with AI can be performed by non-volatile memory, volatile memory and processor.

[0080] The processor may include one or more processors. In this case, the one or more processors may be general-purpose processors, such as a central processing unit (CPU), an application processor (AP), etc., or pure graphics processing units, such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an AI-specific processor, such as a neural processing unit (NPU).

[0081] The one or more processors control the processing of input data according to predefined operating rules or artificial intelligence (AI) models stored in non-volatile memory and volatile memory. The predefined operating rules or artificial intelligence models are provided by training or learning.

[0082] Here, providing by learning means obtaining a predefined operating rule or an AI model with desired characteristics by applying a learning algorithm to a plurality of learning data. The learning can be performed in the device or electronic device itself in which the AI ​​according to the embodiment is executed, and / or can be implemented by a separate server / system.

[0083] The AI ​​model may include multiple neural network layers. Each layer has multiple weight values, and each layer performs neural network calculations by calculating between the input data of the layer (such as the calculation results of the previous layer and / or the input data of the AI ​​model) and the multiple weight values ​​of the current layer. Examples of neural networks include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), and deep Q networks.

[0084] A learning algorithm is a method of using a plurality of learning data to train a predetermined target device (e.g., a robot) to enable, allow, or control the target device to make a determination or prediction. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0085] According to the present disclosure, at least one step in the method performed by the electronic device, such as determining a maximum brightness adjustment intensity parameter, performing brightness adjustment on the first image based on image brightness information and scene brightness information, etc., can be implemented using an artificial intelligence model. The processor of the electronic device can perform preprocessing operations on the data to convert it into a form suitable for use as an artificial intelligence model input. The artificial intelligence model can be obtained through training. Here, "obtained through training" means obtaining a predefined operating rule or artificial intelligence model configured to perform a desired feature (or purpose) by training a basic artificial intelligence model with multiple training data through a training algorithm.

[0086] The following describes several optional embodiments to illustrate the technical solutions of the embodiments of the present disclosure and the technical effects produced by the technical solutions of the present disclosure. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0087] The current image brightness adjustment scheme has at least one of the following problems: the brightness adjustment result is not suitable for the corresponding scene, the different brightness areas in the image cannot be appropriately enhanced (for example, the extremely dark area in the image cannot be effectively enhanced, and the area with better brightness in the image is over-enhanced during the enhancement process), or the image after brightness adjustment is unnatural, etc., so that the adjusted image quality still cannot meet the user's needs, or the user operation is not simple enough to obtain a satisfactory brightness adjustment result in fewer operations.

[0088] To this end, the present disclosure proposes various methods performed by an electronic device to solve at least one of the above problems.

[0089] Figure 1 A flowchart of a method executed by an electronic device according to an embodiment of the present disclosure is shown.

[0090] like Figure 1 As shown, in step S110, image brightness information of a first image and scene brightness information of a shooting scene corresponding to the first image are acquired.

[0091] As an example, the first image here may be an image directly acquired by the user, or may be an image obtained by performing processing on the image acquired by the user. For example, the first image may be an image obtained by performing manual global brightness adjustment and / or local brightness adjustment on the first image. Therefore, optionally, Figure 1 The method shown may also include performing global brightness adjustment and / or local brightness adjustment on the first image. For example, optionally, Figure 1The method shown may also include: detecting a target brightness adjustment strength input by a user; and performing brightness adjustment on the first image based on the target brightness adjustment strength. Here, the target brightness adjustment strength may be a brightness adjustment strength corresponding to a global area of ​​the first image, in which case global brightness adjustment is performed on the first image. Alternatively, the target brightness adjustment strength may be a brightness adjustment strength corresponding to a target local area in the first image, in which case local brightness adjustment is performed on the first image.

[0092] Next, global brightness adjustment and local brightness adjustment according to an embodiment of the present disclosure are described.

[0093] According to an embodiment, in the case where the target brightness adjustment intensity is the brightness adjustment intensity corresponding to the global area of ​​the first image, performing brightness adjustment on the first image based on the target brightness adjustment intensity may include: acquiring brightness statistical information of the first image; determining a maximum brightness adjustment intensity parameter corresponding to the first image based on the brightness statistical information; and performing brightness adjustment on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity. Since the maximum brightness adjustment intensity parameter corresponding to the first image is determined based on the brightness statistical information of the first image, and the brightness adjustment is performed on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity, the brightness adjustment can be performed within the range of the maximum brightness adjustment intensity parameter determined based on the brightness statistical information, so that different brightness areas in the image can be appropriately enhanced, thereby improving the image quality after brightness adjustment.

[0094] Figure 2 An example of a global brightness adjustment framework according to an embodiment of the present disclosure is shown.

[0095] According to an embodiment, Figure 2 As shown, determining the maximum brightness adjustment intensity parameter corresponding to the first image according to the brightness statistical information may include: performing histogram equalization on the first image according to the brightness statistical information to obtain a histogram equalized image corresponding to the first image; and determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the histogram equalized image. For example, the distribution of the brightness values ​​of the first image may be first counted, and redistributed into a uniform distribution to obtain an equalized histogram, and then the histogram equalized image may be obtained based on the equalized histogram. As an example, the maximum brightness adjustment intensity parameter may be represented in the form of a maximum brightness residual map. The maximum brightness residual map may indicate the maximum brightness residual between the brightness enhanced image and the first image that meets the expected visualization effect.

[0096] Since manual adjustment of image brightness is limited, it may result in insufficient brightness improvement of extremely dark images. In order to enable the image to obtain the maximum brightness adjustment strength, the image pixel values ​​can be redistributed based on the brightness statistical information of the first image to obtain a histogram equalized image, and the maximum brightness adjustment intensity parameter can be determined based on the histogram equalized image. For example, the histogram equalized image is used as prior information to guide the network to obtain the maximum brightness residual map within a range with good visualization effect. The reason why the histogram equalized image is used as the maximum brightness prior to obtain the maximum brightness residual map is that histogram equalization expands the pixel values ​​to the entire dynamic range for redistribution. Since histogram equalization stretches the pixel values ​​in the entire range, it enhances the contrast of the image and increases the brightness of the dark area, so that the histogram equalized image has higher brightness and higher global contrast, which helps the network to better handle overly dark and overly bright areas. However, it is unnatural due to wrong colors and excessive brightness. Therefore, the present disclosure uses the histogram equalized image after histogram equalization as good brightness prior information to help the network estimate the maximum brightness residual map. Under the guidance of this prior information, the image is guided to obtain the largest possible brightness, rather than directly using the histogram equalized image to perform brightness adjustment on the first image, so that the dark area can be enhanced to a sufficient brightness and the bright area is not overexposed, while avoiding the unnatural problem in the histogram equalized image.

[0097] According to an embodiment, based on the histogram equalization image, determining the maximum brightness adjustment intensity parameter corresponding to the first image may include: obtaining the brightness difference between the first image and the histogram equalization image; and determining the maximum brightness adjustment intensity parameter according to the brightness difference and the image features of the first image. Figure 2 As shown, a brightness difference map between the first image and the histogram equalized image can be first obtained; based on the brightness difference map and the image features of the first image, enhanced image features of the first image can be obtained; and based on the enhanced image features, the maximum brightness residual map can be predicted using an artificial intelligence network. Figure 2 As shown, a brightness difference map is obtained by subtracting the first image and the histogram equalized image, and brightness embedding is performed on the brightness difference map, and the features obtained by performing the brightness embedding are connected with the image features of the first image to obtain enhanced image features. The brightness difference map between the histogram equalized image and the input image is used as a priori of the maximum brightness, which can guide the brightness enhancement of the image with maximum brightness and better visualization.

[0098] As an example, Figure 3 As shown, brightness embedding is performed on the brightness difference map. First, the brightness difference map can be smoothed by performing Gaussian filtering to remove noise and details. Then, average pooling is used to obtain regional features to avoid abnormal pixels misleading image features. Next, convolution is performed on the regional features to obtain the feature F a,Then the normalized regional features are obtained according to the ,following equation to transform the brightness difference map into the ,image feature space.

[0099]

[0100]

[0101]

[0102] in, is the i-th region feature, μ b and σ b are the image features F of the first image respectively b The mean and variance of a and σ a F a The mean and variance of H and W are F a The height and width, F a The number of feature channels is C.

[0103] Finally, the image feature F b The normalized features of the α and β maps are combined as enhanced image features.

[0104] like Figure 2 As shown, after acquiring the first image, feature extraction may be performed on the first image to obtain image features of the first image, and then, based on the image features of the first image, the maximum brightness guide block according to the embodiment of the present disclosure may be used to obtain enhanced image features of the first image. After obtaining the enhanced image features of the first image, the maximum brightness residual map may be predicted based on the enhanced image features using an artificial intelligence network. After obtaining the maximum brightness residual map, brightness adjustment may be performed on the first image based on the maximum brightness residual map and the target brightness adjustment strength to obtain an enhanced image of the first image.

[0105] According to an embodiment, Figure 2 As shown, the target brightness adjustment strength α can be detected based on the user's control input. For example, as the user slides a slider for adjusting the brightness, the target brightness adjustment strength α can be detected according to the parameter indicated by the slider. According to an embodiment, performing global brightness adjustment on the first image based on the maximum brightness residual map and the target brightness adjustment strength may include: performing weighted fusion on the first image and the maximum brightness residual map based on the target brightness adjustment strength to obtain an enhanced image of the first image. For example, the enhanced image of the first image may be obtained according to the following equation.

[0106] output=input+α×Residual

[0107] Among them, input is the first image, α is the target brightness adjustment intensity, Residual is the maximum brightness residual map, and output is the enhanced image of the first image.

[0108] In the global brightness adjustment process, in addition to the target brightness adjustment intensity, only the maximum brightness residual map may be needed, so the network for predicting the maximum brightness residual map can be run only once to obtain and store the maximum brightness residual map, and then enhanced images of various brightness levels can be obtained by adjusting the target brightness adjustment intensity, which allows the brightness adjustment process to run in real time.

[0109] The enhanced image of the first image obtained after the global brightness adjustment can be output to the user, or the enhanced image of the first image obtained after the global brightness adjustment can be used as Figure 1 The first image of step 110 is shown, and the execution may continue Figure 1 The step S120 shown is used to further perform brightness adjustment to obtain a better brightness adjustment effect.

[0110] The global brightness adjustment according to the embodiment of the present disclosure has been introduced above, and the local brightness adjustment according to the embodiment of the present disclosure will be described below.

[0111] According to an embodiment of the present disclosure, the target brightness adjustment intensity may be the brightness adjustment intensity corresponding to the target local area in the first image. In this case, the above-mentioned performing brightness adjustment on the first image based on the target brightness adjustment intensity may include: performing brightness adjustment on the target local area based on the target brightness adjustment intensity; obtaining the correlation between semantic features of each semantic category in the first image; and performing brightness adjustment on other local areas of the first image except the target local area based on the correlation.

[0112] Since after the brightness adjustment is performed on the target local area, the adjusted target local area can easily become much brighter than other areas, if other areas are not processed accordingly, especially the areas around the boundaries, the visual effect may be discordant. In order to eliminate this discord between areas, according to an embodiment of the present disclosure, after the brightness adjustment is performed on the target local area of ​​the first image, the correlation between the semantic features of each semantic category in the first image is obtained, and based on the correlation, brightness adjustment is performed on other local areas of the first image except the target local area, thereby reducing the discord between the target local brightness adjustment and other local areas, making the adjusted image more natural and improving the image quality.

[0113] According to an embodiment, adjusting the brightness of a target local area based on the target brightness adjustment strength may include: determining the brightness distribution information of the target local area; adjusting the brightness of the target local area based on the brightness distribution information of the target local area and the target brightness adjustment strength. Since the brightness of the target local area is adjusted based on the brightness distribution information of the target local area and the target brightness adjustment strength, the brightness of the target local area can be adjusted, and different pixels in the target local area can be enhanced to different degrees. Using a local adjustment tool with an adaptive shape for different semantic areas can improve the image quality, for example, using a gradient filter to adjust the darkness of the sky, while using an elliptical filter of an appropriate size to increase the brightness of the face. Therefore, optionally, according to an embodiment, different brightness enhancement coefficients can be applied to different semantic areas to adjust the brightness. For example, the brightness enhancement coefficient for each semantic area in the target local area can be determined based on the brightness distribution information of the target local area and the target brightness adjustment strength, and then the brightness of the target local area can be adjusted based on the brightness enhancement coefficient. The brightness distribution information of the target local area may include the brightness distribution characteristics of each semantic area in the target local area. In the present disclosure, this enhancement method is referred to as "semantic-aware feature enhancement".

[0114] As an example, a semantic map corresponding to the first image may be obtained, and based on the semantic map and the target brightness adjustment strength, the target brightness adjustment strength for each semantic area in the target local area may be determined. For example, after detecting the target brightness adjustment strength input by the user, a target brightness adjustment strength map corresponding to the first image may be obtained based on the semantic map and the target brightness adjustment strength, wherein the target brightness adjustment strength for each semantic area in the target local area is indicated in the target brightness adjustment strength map. Figure 4 As shown in FIG. 1 , in the target brightness adjustment intensity map, the target brightness adjustment intensities of different semantic regions included in the target local region may be different. After obtaining the target brightness adjustment intensity map, as shown in FIG. Figure 4 As shown, semantic-aware feature enhancement can be performed based on the target brightness adjustment intensity map and the image features of the first image containing brightness distribution information to obtain enhanced image features. For example, the brightness enhancement coefficient for each semantic area in the target local area can be determined based on the target brightness adjustment intensity map and the image features containing brightness distribution information, and the brightness of each semantic area in the target local area can be adjusted based on the brightness enhancement coefficient. This feature enhancement containing semantic information can achieve appropriate enhancement of different semantic areas, making the image display clearer.

[0115] According to an embodiment, brightness adjustment is performed on a target local area based on the brightness enhancement coefficient, including: converting the brightness enhancement coefficient into a convolution kernel; using the convolution kernel, performing mask convolution on image features containing brightness distribution information to obtain enhanced image features of the first image.

[0116] Figure 5 FIG. 2 is a schematic diagram showing semantic perception enhancement according to an embodiment of the present disclosure. Figure 5 As shown, the brightness enhancement coefficient of each semantic area in the target local area can be determined based on the image features containing brightness distribution information of the first image and the target brightness adjustment intensity map. Such brightness enhancement coefficients may be referred to as "semantically significant local feature enhancement coefficients" in the present disclosure. Figure 5 As shown in FIG. 1 , after the brightness enhancement coefficient is obtained, the brightness enhancement coefficient can be convolved and resized to convert it into a convolution kernel W. Figure 5 In the example, h, w, k and ch are the height, width, convolution kernel size and number of channels of the feature, respectively. After converting the brightness enhancement coefficient into a convolution kernel, the convolution kernel can be used to perform mask convolution on the image feature to obtain the enhanced image feature of the first image. By performing mask convolution, it is possible to enhance the features of only the target local area in the first image, thereby achieving brightness adjustment of the target local area.

[0117] Figure 6 is a schematic diagram showing masked convolution according to an embodiment of the present disclosure. Figure 6 As shown, unlike ordinary convolution that uses the entire convolution kernel to perform convolution on image features, masked convolution uses mask M and convolution kernel W to convolve only a part of the image features. For example, the position where the mask is 0 is not convolved. Masked convolution can be used to perform convolution on the corresponding semantic area using only the brightness enhancement coefficient of each semantic area in the local area, thereby obtaining an enhanced image feature in which only the features of the target local area of ​​the first image are enhanced, thereby achieving brightness adjustment only for the target local area).

[0118] As mentioned above, after the target local area is adjusted, there may be disharmony with other areas. For this reason, Figure 4 As shown, after performing semantic perception enhancement, inter-region harmonization can be performed to eliminate the disharmony between regions. According to an embodiment, the correlation between the semantic features of each semantic category in the first image can be obtained, and based on the correlation, brightness adjustment is performed on other local regions of the first image except the target local region, so that the image after brightness adjustment looks more harmonious.

[0119] Figure 7 is a schematic diagram showing the inter-regional harmonization according to an embodiment of the present disclosure. Figure 7As shown, for example, after acquiring the enhanced image features of the first image, the semantic features can be extracted based on the semantic graph of the first image and the enhanced image features, and the correlation can be acquired based on the semantic features. Figure 8 As shown in FIG. 1 , semantic features can be extracted by clustering enhanced image features using semantic labels in a semantic graph. For example, semantic features of semantic labels Tree, Car, and People are extracted respectively.

[0120] After the semantic features are extracted, the correlation between the semantic features can be calculated, and brightness adjustment can be performed on other local areas based on the correlation to achieve inter-region harmony. According to an embodiment, performing brightness adjustment on other local areas of the first image other than the target local area based on the correlation may include: adjusting the semantic features corresponding to the other local areas according to the correlation; and performing brightness adjustment on the other local areas according to the adjusted semantic features corresponding to the other local areas.

[0121] For example, Fig. 9 As shown, in step 1, the extracted semantic feature X can be mapped in three ways: V, K, and Q, and the semantic features after K and Q mapping are calculated and multiplied and then passed through the softmax function to obtain an attention map. The attention map can also be called a correlation map between semantic features, which can reflect the correlation between semantic features. Subsequently, by performing an operation on the semantic features after V mapping and the attention map, the correlation between semantic features can be reflected in the semantic features, and the semantic features corresponding to other local areas can be adjusted according to the correlation. Subsequently, by performing brightness adjustment on the other local areas according to the adjusted semantic features corresponding to the other local areas, the inter-regional harmony between the target local area and other local areas can be achieved. Next, in step 2, the semantic feature X' reflecting the above correlation can be further mapped in three ways: V, K, and Q, and the semantic features after K and Q mapping are multiplied and then passed through the softmax function to obtain a corrected attention map, which can reflect the correlation between semantic features after the semantic features corresponding to other local areas are adjusted. Subsequently, by performing operations on the semantic features mapped by V and the corrected attention map, the correlation between the corrected semantic features can be reflected in the semantic feature X', so that the semantic features corresponding to other local regions can be recalibrated, thereby making it possible to further achieve inter-regional harmony between other local regions through step 2 on the basis of achieving harmony between the target local region and other local regions through step 1. Through steps 1 and 2, the overall harmony of all semantic regions in the input image is achieved.

[0122] In such Fig. 9 After calculating the correlation and adjusting the semantic features corresponding to other local regions according to the correlation, the adjusted and harmonized semantic features corresponding to other local regions can be mapped back to the image space according to the semantic map corresponding to the first image to obtain the inter-regional harmonious features. Figure 7 As shown, the semantic features can be first dispersed back to the image space, and then convolution is performed to obtain the inter-regional harmony features. The inter-regional harmony features are the features obtained after the enhanced image features are harmonized between regions.

[0123] After inter-region harmony, the same image may contain different individuals of the same semantic category, which may have different brightness and need to have different brightness adjustment strengths. In order to keep different individuals of the same semantic category different, e.g. Figure 4 As shown, after performing inter-region harmony, intra-region harmony may be further performed, and intra-region harmony may also be referred to as "intra-region differentiation". Optionally, according to an embodiment, brightness adjustment is performed on the first image based on the target brightness adjustment intensity, and the process also includes: for local areas with the same semantic category in the first image, brightness adjustment is performed on the corresponding local areas based on the image features of the local areas. For local areas with the same semantic category in the first image, by performing brightness adjustment on the corresponding local areas based on the image features of the local areas, the diversity of different individuals in the same semantic category may be achieved. For example, the enhanced image features are fused with the inter-region harmony features to achieve the fusion of the original image features into the semantic features, thereby increasing the diversity of different individuals in the same semantic category.

[0124] Fig.10 Schematic diagram showing intra-regional differentiation according to an embodiment of the present disclosure. Fig.10 As shown in FIG. 1 , the enhanced image features of the first image can be connected with the inter-regional harmony features, and then the features of the two channels after the connection are averaged and pooled, and the importance of the two features is obtained by the Sigmoid function. Subsequently, the enhanced image features and the inter-regional harmony features can be fused according to their importance to obtain the fused features, that is, the final harmony features (i.e., Figure 4Enhanced image features can reflect the differences between different individuals of the same semantic category, while inter-regional harmonious features can reflect the similarities between individuals of the same semantic category. The fusion between enhanced image features and inter-regional harmonious features can achieve a balance between differences and similarities. The importance of enhanced image features and inter-regional harmonious features can measure the importance of image features and semantic features. For example, channel attention can be used to adaptively learn the importance of different features, enhance important features and weaken unimportant features to balance the importance between semantic features and image features, so that brightness diversity of different individuals in the same semantic category can be achieved.

[0125] The local brightness adjustment according to the embodiment of the present disclosure has been described above. The enhanced image obtained after the local brightness adjustment can be output to the user, or the enhanced image obtained after the local brightness adjustment can be used as Figure 1 The first image of step 110 is shown, and the execution may continue Figure 1 The step S120 shown is used to perform further brightness adjustment, thereby obtaining a better brightness adjustment effect.

[0126] According to an embodiment, optionally, the first image may also be an image obtained by first performing global brightness adjustment on the acquired image and then performing local brightness adjustment on the globally adjusted image. Alternatively, the first image may be an image obtained by first performing local brightness adjustment on the acquired image and then performing global brightness adjustment on the locally adjusted image.

[0127] Return to reference Figure 1 After obtaining the image brightness information of the first image and the scene brightness information of the shooting scene corresponding to the first image in step 110, in step S120, brightness adjustment can be performed on the first image based on the image brightness information and the scene brightness information. According to an embodiment, the image brightness information and the scene brightness information can be used as prior information to guide the brightness adjustment of the first image.

[0128] It should be noted that Figure 1 The method shown to be performed by the electronic device can be independently executed to directly perform automatic brightness adjustment on the first image, or it can be executed after the global brightness adjustment and / or local brightness adjustment described above to achieve further fine-tuning of the manual brightness adjustment result, and the present disclosure is not limited to this.

[0129] Usually manual adjustment can roughly adjust the brightness of the image in real time, but not all parts of the image are pleasing. Automatic adjustment of the brightness of the image ignores the interaction with the user. In the current solution, the user usually makes manual adjustments first, and then actively selects automatic adjustment to fine-tune the image to obtain a satisfactory result in the end, but such operation is not simple enough because the user cannot directly obtain a pleasing result in one step. If the user manually adjusts the brightness to an appropriate level, and then fine-tunes automatically without the user's selection, it will reduce user operations and improve user experience.

[0130] In this regard, according to an embodiment, optionally, step S120 can be executed in response to detecting that brightness adjustment of the first image based on the target brightness adjustment intensity is completed, thereby enabling a higher quality enhanced image to be obtained with fewer user operations, making the brightness adjustment more user-friendly.

[0131] The operations involved in steps S110 and S120 are described in detail below.

[0132] For the convenience of description, in the present disclosure, the module for obtaining image brightness information and scene brightness information may be referred to as a “prior computing block (PCB)”, and its operation diagram is shown in FIG. Fig.11 shown.

[0133] According to an embodiment, Fig.11 As shown, obtaining image brightness information of a first image and scene brightness information of a shooting scene corresponding to the first image may include: performing global average pooling on the first image to obtain image brightness information; obtaining shooting parameter information of the first image, and obtaining scene brightness information based on the shooting parameter information.

[0134] For example, the image brightness information may include a global brightness value (Image Global Brightness Value, Fig.11 For example, the shooting parameter information may include at least one of the following: Aperture value (in Fig.11 F in the Fig.11 abbreviated as T in Fig.11 ISO is a quantitative representation of sensitivity, which can measure the sensitivity of the film (photosensitive element) to light. ISO can also be called sensitivity), baseline exposure value (Baseline exposure, in Fig.11 BLE in short.

[0135] According to an embodiment, obtaining scene brightness information based on shooting parameter information may include: determining a camera measurement exposure value based on the shooting parameter information; and obtaining scene brightness information based on the camera measurement exposure value. Fig.11 As shown, the camera metering exposure value (Camera Metering Exposure Value, Fig.11 ②CMEV in the figure). Optionally, obtaining scene brightness information based on shooting parameter information further includes: determining a normalized exposure value based on the shooting parameter information. In this case, obtaining scene brightness information based on the camera measured exposure value may include: obtaining scene brightness information based on the camera measured exposure value and the normalized exposure value. For example, Fig.11 As shown, the normalized exposure value (Normalized Exposure Value, Fig.11 ③NEV in it).

[0136] For example, CMEV can be calculated as follows:

[0137] 1) Based on the following exposure formula, the measured exposure value EV can be calculated meteribg :

[0138] EV metering =log 2 (F m 2 / T m )

[0139] Among them, EV metering =log 2 (F n 2 / T m ), F m is the F number of the camera aperture, T m is the exposure time. It should be noted that this formula is derived under the condition of ISO = 100. When taking a photo, the camera gives a recommended EV metering and the corresponding F m , T m It can be seen that: (1) When EV metering When fixed, when we use a larger T m We should use a smaller aperture (larger F m ); (2) The brighter the environment, the higher the T m The smaller the recommendation, the EV metering should be larger; (3) when the exposure time is twice as long as 2T m EVmetering It should be -1.

[0140] 2) When taking photos, although the metering mode will suggest measuring the exposure value EV metering , but users can still adjust the exposure by "minimum / add N stops", called exposure compensation (EC, Exposure Compensation), to obtain the desired exposure value EV expected =EV metering -EC.

[0141] 3) When taking a photo, you now have the desired exposure value and the corresponding aperture value F real and exposure time T real However, due to differences in image sensors, in order to obtain the expected EV expected , we should consider a deviation baseline exposure value that varies with ISO.

[0142] EV expected =EV metering -EC=EV real -BLE

[0143] EV real =log 2 (F real 2 / T real )

[0144] Among them, F real and T real Can be read from the image's metadata.

[0145] 4) Calculate CMEV according to the following equation:

[0146] CMEV=EV metering =EV real -BLE+EC

[0147] For example, NEV can be calculated as follows:

[0148] NEV=EV real -BLC+log 2 (ISO / 100)

[0149] As an example, the brightness adjustment of the first image can be performed based on CMEV, NEV and IGBV. Information such as CMEV, NEV and IGBV can be used as prior information to guide the brightness adjustment of the first image. Optionally, if the first image is an image obtained by performing manual global brightness adjustment and / or local brightness adjustment, the target brightness adjustment intensity detected by the user during the global brightness adjustment and / or local brightness adjustment, for example, brightness level control (Brightness Level Control, BLC) information, can also be obtained here, and CMEV, NEV, IGBV and BLC are used as prior information to guide the brightness adjustment of the first image. For example, Fig.11 As shown, CMEV, NEV, IGBV and BLC can be resized and concatenated to obtain a feature of size 1*1*6, where 1 is the width and height, and 6 is the number of channels.

[0150] After acquiring the image brightness information and scene brightness information of the first image, the brightness adjustment can be performed on the first image based on the image brightness information and the scene brightness information. Since the brightness adjustment is performed on the first image based on the image brightness information and the scene brightness information, the image after brightness adjustment can be made more suitable for the corresponding scene, thereby improving the image quality. For example, an artificial intelligence network can be used to perform brightness adjustment on the first image based on the image brightness information and the scene brightness information. Since images correspond to different appropriate brightnesses in scenes with different ambient lights, the introduction of scene brightness information and image brightness information as prior information can guide the network to predict a brightness that is more suitable for the corresponding scene. For example, the brightness of an image taken in a well-lit environment should be much higher in a dark area than in a well-lit area, and for images taken in a dark light environment, the brightness of the entire image should be greatly improved.

[0151] As an example, performing brightness adjustment on a first image based on image brightness information and scene brightness information may include: determining an adjustment intensity parameter range corresponding to the first image based on the image brightness information and the scene brightness information; and performing brightness adjustment on the first image according to the adjustment intensity parameter range.

[0152] According to an embodiment, determining the adjustment intensity parameter range corresponding to the first image based on the image brightness information and the scene brightness information may include: determining the brightness range of the first image based on the image features of the first image; determining the adjustment intensity parameter range based on the brightness range, the image brightness information and the scene brightness information. For example, determining the brightness range of the first image based on the image features of the first image includes: determining the maximum brightness feature and the minimum brightness feature of the first image using maximum pooling and minimum pooling respectively based on the image features of the first image. In this case, determining the adjustment intensity parameter range based on the brightness range, the image brightness information and the scene brightness information may include: determining the maximum brightness adjustment intensity parameter corresponding to the first image based on the maximum brightness feature, the image brightness information and the scene brightness information; determining the minimum brightness adjustment intensity parameter corresponding to the first image based on the minimum brightness feature, the image brightness information and the scene brightness information. According to an embodiment, the adjustment intensity parameter range may include a maximum brightness adjustment intensity parameter and a minimum brightness adjustment intensity parameter. In this case, performing brightness adjustment on the first image according to the adjustment intensity parameter range may include: for each local area of ​​the first image, based on the brightness value of the local area, fusing the maximum brightness adjustment intensity parameter and the minimum brightness adjustment intensity parameter corresponding to the local area, and adjusting the brightness of the local area based on the fused adjustment intensity parameter. For example, this process may include: determining the local brightness adjustment intensity parameter corresponding to the local area based on the image features of the local area; determining the first weight of the maximum brightness adjustment intensity parameter, the second weight of the minimum brightness adjustment intensity parameter, and the third weight of the local brightness adjustment intensity parameter corresponding to the local area based on the brightness value of the local area; using the first weight, the second weight, and the third weight, weighted fusion of the maximum brightness adjustment intensity parameter, the minimum brightness adjustment intensity parameter, and the local brightness adjustment intensity parameter to obtain the brightness adjustment intensity parameter corresponding to the local area; adjusting the brightness of the local area based on the brightness adjustment intensity parameter corresponding to the local area.

[0153] As an example, the maximum brightness adjustment intensity parameter may be a maximum adjustment curve for image brightness adjustment, wherein the maximum adjustment curve indicates the maximum brightness adjustment intensity of each pixel value in the first image. As an example, the minimum brightness adjustment intensity parameter may be a minimum adjustment curve for image brightness adjustment, wherein the minimum adjustment curve indicates the minimum brightness adjustment intensity of each pixel value in the first image. As an example, the local brightness adjustment intensity parameter may be a local adjustment curve for brightness adjustment of a local area in an image, wherein the local adjustment curve indicates the brightness adjustment intensity of each pixel value in the local area. As an example, the brightness adjustment intensity parameter corresponding to the local area may be an adjustment curve for brightness adjustment of the local area obtained by fusing the maximum adjustment curve, the minimum adjustment curve and the local adjustment curve, wherein the adjustment curve indicates the final brightness adjustment intensity of each pixel in the local area.

[0154] In the present disclosure, for the convenience of description, the module for determining the adjustment intensity parameter range corresponding to the first image based on the image brightness information and the scene brightness information may be referred to as a brightness control mapping block, and its operation schematic diagram may be, for example, as shown in FIG. Fig.12 shown.

[0155] For example, Fig.12 As shown, the maximum adjustment curve and the minimum adjustment curve can be predicted based on the output of the PCB (including image brightness information and scene brightness information) and the image features of the first image. In addition, the local adjustment curves corresponding to the local areas can be obtained based on the image features of the local areas of the first image. Fig.13 FIG. 1 is a schematic diagram showing a prediction adjustment curve according to an embodiment of the present disclosure. Fig.13 As shown, different prediction networks can be used to predict the maximum adjustment curve and the minimum adjustment curve based on the image brightness information, the scene brightness information, and the image features of the first image. In other words, different prediction networks can be used to predict the maximum adjustment curve and the minimum adjustment curve based on the output of the PCB and the image features. Since different prediction networks are used to predict the maximum adjustment curve and the minimum adjustment curve, the prediction result can be less affected by the average attribute of the image, and the problem of insufficient brightness in extremely dark areas caused by small pixel values ​​being easily submerged during the prediction process can be avoided.

[0156] like Fig.13 As shown in the figure, for example, the maximum pooling can be performed on the image features of M*N*C to obtain the maximum brightness feature 1*1*C, which is resized and connected with the output of PCB, and then the maximum adjustment curve is predicted after convolution / full connection / self-attention operations. The maximum adjustment curve is Fig.13For example, we can perform minimum pooling on the image features of M*N*C to obtain the minimum brightness feature 1*1*C, resize and connect it with the output of PCB, and then predict the minimum adjustment curve through convolution / full connection / self-attention operations. Fig.13 In addition, a convolution operation can be performed on the M*N*C image features to obtain a local adjustment curve. Figure 4 It is referred to as "local TM" in the following. Fig.12 As shown, the maximum adjustment curve, the minimum adjustment curve and the local adjustment curve are fused to determine the adjustment curve corresponding to each local area, that is, the adjustment curve corresponding to the image block. During fusion, the maximum adjustment curve should be given a greater weight for extremely dark areas, and the minimum adjustment curve should be given a greater weight for overexposed areas, so as to avoid underexposure of dark areas and overexposure of bright areas during brightness adjustment. For areas with better lighting, the local adjustment curve can be given a greater weight.

[0157] Fig.14 is a schematic diagram of a fusion adjustment curve according to an embodiment of the present disclosure. Fig.14 As shown, the maximum adjustment curve, the minimum adjustment curve and the local adjustment curve are fused to determine the adjustment curve corresponding to each local area, which may include: based on the brightness value of the local area in the first image, predicting the weights of the maximum adjustment curve, the minimum adjustment curve and the local adjustment curve when performing the fusion; and performing weighted fusion of the maximum adjustment curve, the minimum adjustment curve and the local adjustment curve according to the predicted weights to determine the adjustment curve corresponding to the local area. For example, the weight W of the maximum adjustment curve may be predicted using a weight prediction network based on the brightness value of the local area in the first image. Max , the weight of the minimum adjustment curve W MIN and the weight W of the local adjustment curve Local Then, the maximum adjustment curve TM is converted into Max With weight W Max Multiply element by element to get the minimum adjustment curve TM Min With weight W MI

[0158] Multiply element by element and adjust the local adjustment curve TM Local With weight W Local Multiply element by element and add the multiplication results to obtain the adjustment curve corresponding to the image block, that is, the fused adjustment curve TM:

[0159] TM=W Max *TM Max +W Min *TM Min +WLocal *TM Local

[0160] After the adjustment curves corresponding to the local areas in the first image are determined, the brightness of the local areas may be adjusted based on the adjustment curves corresponding to the local areas.

[0161] The study found that the adjustment curve corresponding to each local area is most suitable for the central pixel of the local area, while the pixels at other positions in the local area will be affected by the adjustment curve corresponding to the nearby local area. Therefore, optionally, in order to obtain a better brightness adjustment result, according to an embodiment of the present disclosure, the brightness of the local area is adjusted based on the brightness adjustment intensity parameter corresponding to the local area, including: for the central pixel in the local area, the brightness of the central pixel is adjusted using the brightness adjustment intensity parameter corresponding to the central pixel in the brightness adjustment intensity parameter corresponding to the local area; for the non-central pixel in the local area, the brightness of the non-central pixel is adjusted based on the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area of ​​the local area. For example, for the central pixel in the local area, the brightness of the central pixel is adjusted using the brightness adjustment intensity of the central pixel indicated in the adjustment curve corresponding to the local area; for the non-central pixel in the local area, the brightness of the non-central pixel is adjusted based on the adjustment curve corresponding to the local area and the adjustment curve corresponding to the adjacent local area of ​​the local area.

[0162] According to an embodiment, based on the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area of ​​the local area, the brightness of the non-central pixel is adjusted, including: mapping the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area to the same spatial position; applying the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area to obtain the brightness value of the non-central pixel adjusted by different brightness adjustment intensity parameters; performing bilinear interpolation on the brightness value adjusted by different brightness adjustment intensity parameters to obtain the final brightness value of the non-central pixel. For example, mapping the adjustment curve corresponding to the local area and the adjustment curve corresponding to the adjacent local area to the same spatial position, applying the adjustment curve corresponding to the local area and the adjustment curve corresponding to the adjacent local area in parallel to obtain the brightness value of the non-central pixel adjusted by different adjustment curves, and then performing bilinear interpolation on the brightness value adjusted by different adjustment curves to obtain the final brightness value of the non-central pixel. Through the above operations, the brightness adjustment speed can be improved, which is conducive to performing brightness adjustment on a graphics processing unit (GPU) or a neural network processing unit (NPU) in an end-to-end manner to generate an enhanced image of the first image.

[0163] In the embodiment of the present disclosure, for the convenience of description, the module for adjusting the brightness of each local area according to the brightness adjustment intensity parameter corresponding to each local area can be called a brightness mapping and curve interpolation block, and its operation diagram is as follows: Fig.15 For example, Fig.15 As shown, an alignment operation may be performed first. The alignment operation ( Fig.15 ①) in it is used to spatially shift the adjustment curve of the adjacent local area (also referred to as the adjacent block) associated with the pixel in the local area (also referred to as the "image block"). For example, assuming that the pixel is affected by the adjustment curves of three adjacent blocks, the first adjacent block can be left-shifted convolved, the second adjacent block can be up-shifted convolved, and the third adjacent block can be left-up-shifted convolved according to the positions of the three adjacent blocks relative to the pixel, so that the adjustment curve corresponding to the image block where the pixel is located and the adjustment curve corresponding to the adjacent block are mapped to the same spatial position. In addition, the first image can be converted from the spatial dimension to the depth dimension so that the spatial dimension is reduced and the channel dimension is increased. For example, the size of the first image after being converted from the spatial dimension to the channel dimension can be M*N*T. Subsequently, the adjustment curve corresponding to the image block and the adjustment curve corresponding to the adjacent block are respectively applied in parallel to obtain the brightness value of the non-central pixel adjusted by different adjustment curves. For example, a dictionary mapping operation can be performed first ( Fig.15 ② in the figure, which is used to search the adjustment intensity of the pixel on different adjustment curves in parallel. Next, multiple ratio operations can be performed ( Fig.15 ③) in the figure, this operation is used to apply the adjustment strength of the pixel on different adjustment curves in parallel to adjust the brightness of the pixel, and obtain the brightness enhancement results of the pixel under different curves. Subsequently, the brightness enhancement results adjusted by different curves are converted from the depth dimension back to the spatial dimension, that is, the brightness enhanced pixels are mapped back to the original image space along the channel dimension ( Fig.15 Finally, bilinear interpolation is performed on the brightness values ​​adjusted using different adjustment curves ( Fig.15 For example, bilinear interpolation can be performed on the brightness values ​​enhanced by different adjustment curves according to the distance from the pixel to the adjacent block to obtain the brightness value enhanced by fusing different adjustment curves.

[0164] For example, Fig.16 As shown, for the pixel p in the lower left image block b , the adjustment curve of the image block is t3. In addition, the pixel p b It is also affected by the adjustment curve t4 corresponding to the surrounding adjacent blocks. In this case, the pixel p b The adjusted brightness value of can be obtained based on the adjustment curves t3 and t4. bThe weight is determined by the distance e and f to the adjacent block, and the adjusted brightness value t is respectively adjusted using the adjustment curves t3 and t4 according to the determined weight according to the following equation: 3 (p b ) and t 4 (p b ) performs bilinear interpolation to obtain pixel p b The final brightness value

[0165]

[0166] For another example, for pixel p in the lower left image block m , the adjustment curve of the image block is t3. In addition, the pixel p m It is also affected by the adjustment curves t1, t2 and t4 corresponding to the surrounding adjacent blocks. In this case, the pixel p m The adjusted brightness value of can be obtained based on the adjustment curves t1, t2, t3 and t4. For example, the following equation can be used based on p m The weights determined by the distances a, b, c and d to the adjacent blocks are respectively used to perform the adjusted brightness value t using the adjustment curves t1, t2, t3 and t4. 1 (p m ), t 2 (p m ), t 3 (p m ) and t 4 (p m ) performs bilinear interpolation to obtain pixel p m The final brightness value

[0167]

[0168] For another example, for pixel p c , because it is located in the corner area of ​​the entire image and is far away from the adjacent blocks, the weight of the adjustment curve corresponding to the adjacent blocks is 0, and it can be based only on pixel p c The adjustment curve t3 corresponding to the image block is for pixel p c Perform brightness adjustment to obtain the adjusted brightness value of the pixel

[0169] Fig.17 It is shown Figure 1 Schematic diagram of an example of a method performed by an electronic device. Fig.17As shown, after acquiring the first image, first, the image brightness information is acquired based on the first image using the prior calculation block, and the scene brightness information is acquired based on the shooting parameter information, and then the brightness mapping control block obtains the adjustment curve corresponding to the image block based on the image features of the first image and the output of the prior calculation block. Then, the brightness mapping and curve interpolation block adjusts the brightness of each image block using the adjustment curve corresponding to each image block to obtain an enhanced image of the first image.

[0170] The above has been combined with the accompanying drawings Figure 1 The method shown in FIG. 1 is described, and the global brightness adjustment and local brightness adjustment according to the embodiment of the present disclosure are described above. The global brightness adjustment and local brightness adjustment according to the embodiment of the present disclosure are not limited to only Figure 1 The method is not used in the illustrated method, but can be independently applied to adjust the brightness of the first image to obtain an enhanced image of the first image. Therefore, the present disclosure also provides a method performed by an electronic device as described below.

[0171] Fig.18 is a flowchart of a method executed by an electronic device according to another exemplary embodiment of the present disclosure. Fig.18 The method corresponds to the global brightness adjustment mentioned above. Fig.18 In step S1810, the target brightness adjustment intensity input by the user is detected, wherein the target brightness adjustment intensity is the brightness adjustment intensity corresponding to the global area of ​​the first image. In step S1820, the brightness statistical information of the first image is obtained. In step S1830, the maximum brightness adjustment intensity parameter corresponding to the first image is determined based on the brightness statistical information. In step S1840, the brightness adjustment is performed on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity. The relevant details involved in the above operations have been described above and will not be repeated here. According to Fig.18 The method shown, since the maximum brightness adjustment intensity parameter corresponding to the first image is determined according to the brightness statistical information of the first image, and the brightness adjustment is performed on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity, the brightness adjustment can be performed within the range of the maximum brightness adjustment intensity parameter determined according to the brightness statistical information, so that different brightness areas in the image can obtain appropriate brightness enhancement, thereby improving the image quality after brightness adjustment.

[0172] Fig.19 is a flowchart of a method executed by an electronic device according to yet another exemplary embodiment of the present disclosure. Fig.19 The method corresponds to the local brightness adjustment mentioned above. Fig.19In step S1910, the target brightness adjustment strength input by the user is detected, wherein the target brightness adjustment strength is the brightness adjustment strength corresponding to the target local area in the first image. In step S1920, the brightness of the target local area is adjusted based on the target brightness adjustment strength. In step S1930, the correlation between the semantic features of each semantic category in the first image is obtained. In step S1940, brightness adjustment is performed on other local areas of the first image except the target local area based on the correlation. The relevant details involved in the above operations have been described above and will not be repeated here. According to Fig.19 The method shown, after the brightness of the target local area in the first image is adjusted based on the target brightness adjustment intensity, obtains the correlation between the semantic features of each semantic category in the first image, and performs brightness adjustment on other local areas of the first image except the target local area based on the correlation. Therefore, the inharmonious phenomenon between the target local area and other local areas after the brightness adjustment of the target local area can be reduced, the adjusted image is more natural, and the image quality after the brightness adjustment is improved.

[0173] Fig. 20 is a flowchart of a method executed by an electronic device according to another exemplary embodiment of the present disclosure. Fig. 20 , in step S2010, the target brightness adjustment intensity input by the user is detected. For example, the target brightness adjustment intensity may be the brightness adjustment intensity corresponding to the global area of ​​the first image or may be the brightness adjustment intensity corresponding to the target local area in the first image. In step S2020, brightness adjustment is performed on the first image based on the target brightness adjustment intensity. If the target brightness adjustment intensity may be the brightness adjustment intensity corresponding to the global area of ​​the first image, then in step S2020, brightness adjustment is performed on the global area of ​​the first image. If the target brightness adjustment intensity is the brightness adjustment intensity corresponding to the target local area in the first image, then in step S2020, brightness adjustment is performed on the target local area in the first image. For example, it may be performed according to Fig.18 The global brightness adjustment method shown and Fig.19 In step S2030, in response to detecting that the brightness adjustment of the first image based on the target brightness adjustment intensity is completed, the brightness adjustment is performed on the first image based on the image brightness information of the first image and the scene brightness information of the shooting scene corresponding to the first image. Figure 1 The brightness adjustment method shown performs further automatic brightness adjustment on the first image to obtain a brightness enhanced image of the first image. Fig. 20 The method shown can obtain a higher quality brightness enhanced image with less user operation.

[0174] According to various embodiments of the present disclosure, an input low-brightness image can be restored to a high-brightness image while maintaining the overall contrast and color saturation of the image. A low-brightness image refers to an image taken in a low-brightness environment, where the image brightness is darker and the details of the low-brightness area of ​​the image are unclear. In contrast, a high-brightness image refers to an image with a higher brightness, where the details of the low-brightness area of ​​the image are clear, and the brightness of the high-brightness area of ​​the image is appropriate, without overexposure, and the image as a whole has appropriate contrast and color saturation.

[0175] Fig.21 is a schematic diagram showing an example of an image brightness adjustment architecture according to an embodiment of the present disclosure. Fig.21 As shown, for an input low-brightness image, firstly, a global manual brightness adjustment is performed according to the brightness adjustment strength input by the user. Then, the image after global brightness adjustment can be locally adjusted according to the local area and the brightness adjustment strength of the local area specified by the user. Alternatively, local brightness adjustment can be performed directly. Finally, the image after manual coarse brightness adjustment is automatically adjusted to obtain a high-brightness image with appropriate brightness.

[0176] Fig. 22 FIG. 2 is a schematic diagram showing the detailed structure of the image brightness adjustment architecture according to an embodiment of the present disclosure. Fig. 22 As shown, after inputting a low-brightness image, global manual brightness adjustment can be performed under user control. For example, the global manual brightness adjustment module including the maximum brightness guide block mentioned above can be used to perform global brightness adjustment and output a coarsely adjusted high-brightness image. Alternatively, local brightness adjustment can be performed under user control. For example, the regional brightness harmony module performs local brightness adjustment according to the local brightness adjustment method described above based on the image features obtained by performing feature extraction on the low-brightness image and the semantic map of the input image. By performing the inter-regional harmony and intra-regional harmony described above, a better brightness adjustment effect can be obtained. For example, Fig.23 As shown, you can first choose to adjust the brightness of the car in the original image. After adjusting the brightness of the car, you can perform inter-regional harmonization to adjust the brightness of the road around the car and the brightness of the person. Finally, perform intra-regional harmonization to adjust the brightness of the car, road and person, and finally obtain a more natural brightness adjustment effect after harmonization. Alternatively, you can further perform local brightness adjustment on the image after global brightness adjustment.

[0177] The above manual global and local image brightness adjustment can achieve rough adjustment of the image and quickly obtain an overall suitable brightness enhanced image. On this basis, we can further use the above mentioned Figure 1 According to an embodiment, the method for performing automatic brightness adjustment Figure 1The brightness fine-tuning module of the method shown may include the a priori calculation block, the brightness mapping control block, the brightness mapping and the curve interpolation block mentioned above. The details of the operations performed by the above blocks have been referred to above. Figure 1 It has been introduced in the description of , so it will not be repeated here.

[0178] Fig.24 An example of the overall process of image brightness adjustment according to an embodiment of the present disclosure is shown.

[0179] like Fig.24 As shown, first, an input image is obtained. For example, when a user takes a photo with a mobile phone, or when a user selects an image from a mobile phone album for editing, an input image can be obtained. Subsequently, it can be determined whether the image needs to be adjusted in brightness, and if not, the process ends. If brightness adjustment is required, an image brightness adjustment interface can be provided for the input image. The user can control the brightness adjustment method and the brightness adjustment intensity through the image brightness adjustment interface. For example, the user can touch and move the slider on the brightness adjustment interface to adjust the brightness. When the user slides the brightness adjustment slider, the brightness adjustment scheme of the embodiment of the present disclosure is activated. As the user slides the slider, the input adjustment parameter (target brightness adjustment intensity) can be determined. In addition, it is further determined whether the user selects the desired adjustment area through the image brightness adjustment interface. If no area is selected, the image can be globally adjusted in brightness according to the adjustment parameter determined by the user sliding the slider. If an area is selected, the image is locally adjusted in brightness based on the area selected by the user and the adjustment parameter obtained by the user sliding the slider. When the user selects a target local area for adjustment, the target local area can be selected by clicking the target local area or pointing out the outline of the target local area. According to the embodiment of the present disclosure, both global brightness adjustment and local brightness adjustment can utilize an artificial intelligence network. For example, in global brightness adjustment, an artificial intelligence network can be used to predict the maximum brightness residual map. For another example, in local brightness adjustment, an artificial intelligence network can be used to enhance semantic perception, inter-regional harmony, and intra-regional harmony. Finally, when the user releases the finger touching the slider, after detecting this operation, the automatic brightness adjustment scheme according to the embodiment of the present disclosure is activated, and the coarse-adjusted image after global brightness adjustment or local brightness adjustment can be automatically fine-tuned to generate a brightness-enhanced image of the input image. It should be noted that Fig.24 The process shown is only an example, and the present disclosure is not limited to Fig.24 The process shown in the figure can, for example, perform global brightness adjustment, local brightness adjustment and automatic brightness adjustment in sequence. In fact, the above three brightness adjustments can be performed in any order.

[0180] The brightness adjustment solution according to the embodiment of the present disclosure can be applied to a variety of scenarios, for example, in a photo album application, when taking photos with a mobile phone, and so on.

[0181] Fig.25 1 is a schematic diagram of using the brightness adjustment scheme according to an embodiment of the present disclosure in an album application. First, an image is selected from the album application, and then it is determined whether the image needs to be adjusted in brightness. If not, it ends. If brightness adjustment is required, the user can touch and move the slider, and the adjustment parameter (target brightness adjustment intensity) during brightness adjustment can be determined according to the movement of the slider. Subsequently, if the user does not select an area, global brightness adjustment is performed according to the adjustment parameter. If the user selects an area, local brightness adjustment of the selected area is performed according to the adjustment parameter. For example, the sky, road and people in the image can be adjusted in sequence according to the user's selection, and finally a harmonized local brightness adjustment result is generated. Finally, when the user releases the finger touching the slider, the automatic brightness adjustment scheme mentioned above can be used to automatically perform brightness optimization. If global brightness adjustment was previously performed, global brightness optimization can be automatically performed to obtain a globally optimized brightness enhanced image. If local brightness adjustment was previously performed, global brightness optimization can be automatically performed to finally obtain a locally harmonized and globally optimized brightness enhanced image.

[0182] Fig.26 FIG. 2 is a schematic diagram of the operation of using the brightness adjustment solution according to an embodiment of the present disclosure in a photo album application. Fig.26 As shown, for example, the album application icon on the user interface can be clicked to start the album application, and the image desired to be adjusted can be selected. Subsequently, the user can click the edit button on the image display interface, and then continue to select the brightness adjustment button. After the brightness adjustment button is selected, the global brightness adjustment of the present invention can be applied to obtain the global brightness adjustment image of the selected image. For example, the images before and after the adjustment can be compared and displayed. Alternatively, the user can perform an operation to automatically adjust the brightness of the selected image. For example, the user can further select a menu for automatic brightness adjustment by clicking on the function menu. When the menu is selected, the automatic brightness adjustment scheme of the present invention can be automatically applied to automatically adjust the brightness of the selected image. Finally, for example, the images before and after the adjustment can be compared and displayed.

[0183] Fig. 27 is a schematic diagram of using the brightness adjustment solution according to an embodiment of the present disclosure in camera photography.

[0184] For example, when taking a photo with a camera, after the image is captured by the camera sensor and, for example, de-mosaicing, color space conversion, and software ISP (Image Signal Processor) are performed, the brightness adjustment scheme according to the embodiment of the present disclosure can be applied to perform automatic brightness adjustment, and then the adjusted image is compressed and saved in the album.

[0185] The above only briefly introduces the application examples of the brightness adjustment scheme according to the embodiments of the present disclosure by taking the album application and camera photography as examples. However, the brightness adjustment scheme according to the embodiments of the present disclosure is not limited to the above two scenarios, but can be applied to any scenario requiring brightness adjustment.

[0186] In the above, the method performed by the electronic device according to the embodiment of the present disclosure has been described. Below, the electronic device according to the embodiment of the present disclosure is briefly described.

[0187] Fig.28 is a block diagram showing an electronic device according to an embodiment of the present disclosure. Fig.28 , the electronic device 2800 may include a memory 2801 and a processor 2802, wherein the processor 2802 is coupled to the memory 2801 and is configured to execute any one of the methods described above.

[0188] An electronic device is also provided in an embodiment of the present disclosure, which includes at least one processor and, optionally, may also include at least one transceiver and / or at least one memory coupled to the at least one processor, and the at least one processor is configured to execute the steps of the method provided in any optional embodiment of the present disclosure.

[0189] Fig.29 A schematic diagram of the structure of an electronic device applicable to an embodiment of the present invention is shown in FIG. Fig.29 As shown, Fig.29 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, each of the processor 4001, the memory 4003 and the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present disclosure. Optionally, the electronic device may be a first network node, a second network node or a third network node.

[0190] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0191] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.29 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0192] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0193] The memory 4003 is used to store computer programs or executable instructions for executing the embodiments of the present disclosure, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer programs or executable instructions stored in the memory 4003 to implement the steps shown in the above method embodiments.

[0194] An embodiment of the present disclosure provides a computer-readable storage medium having a computer program or instructions stored thereon. When the computer program or instructions are executed by at least one processor, the steps and corresponding contents of the aforementioned method embodiment can be executed or implemented.

[0195] The embodiments of the present disclosure also provide a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiments when executed by a processor.

[0196] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than that shown or described in the text.

[0197] It should be understood that, although the flowchart of the embodiment of the present disclosure indicates each operation step by arrows, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present disclosure, the implementation steps in each flowchart can be executed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage in these sub-steps or stages can also be executed at different times. In scenarios with different execution times, the execution order of these sub-steps or stages can be flexibly configured according to demand, and the embodiment of the present disclosure does not limit this.

[0198] The above text and drawings are provided only as examples to help readers understand the present disclosure. They are not intended and should not be interpreted as limiting the scope of the present disclosure in any way. Although certain embodiments and examples have been provided, based on the contents disclosed herein, it is obvious to those skilled in the art that the embodiments and examples shown can be changed without departing from the scope of the present disclosure, and other similar implementation means based on the technical ideas of the present disclosure are adopted, which also fall within the protection scope of the embodiments of the present disclosure.

Claims

1. A method performed by an electronic device, include: Acquire image brightness information of the first image and scene brightness information of a shooting scene corresponding to the first image; Based on the image brightness information and the scene brightness information, brightness adjustment is performed on the first image.

2. The method according to claim 1, in, The performing brightness adjustment on the first image based on the image brightness information and the scene brightness information includes: Determining an adjustment intensity parameter range corresponding to the first image based on the image brightness information and the scene brightness information; Brightness adjustment is performed on the first image according to the adjustment intensity parameter range.

3. The method according to claim 2, in, The determining, based on the image brightness information and the scene brightness information, an adjustment intensity parameter range corresponding to the first image includes: determining a brightness range of the first image based on an image feature of the first image; The adjustment intensity parameter range is determined based on the brightness range, the image brightness information, and the scene brightness information.

4. The method according to claim 3, in, The determining, based on the image feature of the first image, a brightness range of the first image includes: Based on the image features of the first image, determining a maximum brightness feature and a minimum brightness feature of the first image by using maximum pooling and minimum pooling respectively; Wherein, determining the adjustment intensity parameter range based on the brightness range, the image brightness information and the scene brightness information includes: Determining a maximum brightness adjustment intensity parameter corresponding to the first image based on the maximum brightness feature, the image brightness information, and the scene brightness information; Based on the minimum brightness feature, the image brightness information and the scene brightness information, a minimum brightness adjustment intensity parameter corresponding to the first image is determined.

5. The method according to claim 2, in, The adjustment intensity parameter range includes: a maximum brightness adjustment intensity parameter and a minimum brightness adjustment intensity parameter; The performing brightness adjustment on the first image according to the adjustment intensity parameter range includes: For each local area of ​​the first image, based on the brightness value of the local area, the maximum brightness adjustment intensity parameter and the minimum brightness adjustment intensity parameter corresponding to the local area are fused, and the brightness of the local area is adjusted based on the fused adjustment intensity parameters.

6. The method according to claim 5, in, The method of fusing the maximum brightness adjustment intensity parameter and the minimum brightness adjustment intensity parameter corresponding to the local area based on the brightness value of the local area, and adjusting the brightness of the local area based on the fused adjustment intensity parameters, includes: Determining a local brightness adjustment intensity parameter corresponding to the local area based on the image features of the local area; Based on the brightness value of the local area, determine a first weight of a maximum brightness adjustment intensity parameter, a second weight of a minimum brightness adjustment intensity parameter, and a third weight of a local brightness adjustment intensity parameter corresponding to the local area; Using the first weight, the second weight, and the third weight, weighted fusion is performed on the maximum brightness adjustment intensity parameter, the minimum brightness adjustment intensity parameter, and the local brightness adjustment intensity parameter to obtain the brightness adjustment intensity parameter corresponding to the local area; The brightness of the local area is adjusted based on the brightness adjustment intensity parameter corresponding to the local area.

7. The method according to claim 1, in, The acquiring the image brightness information of the first image and the scene brightness information of the shooting scene corresponding to the first image includes: Performing global average pooling on the first image to obtain brightness information of the image; Acquire shooting parameter information of the first image, and acquire the scene brightness information based on the shooting parameter information.

8. The method according to claim 7, in, The acquiring the scene brightness information based on the shooting parameter information includes: Based on the shooting parameter information, determining a camera measurement exposure value; The scene brightness information is obtained based on the exposure value measured by the camera.

9. The method according to claim 8, in, The acquiring the scene brightness information based on the shooting parameter information further includes: Based on the shooting parameter information, determining a normalized exposure value; The step of obtaining the scene brightness information based on the exposure value measured by the camera includes: The scene brightness information is acquired based on the camera measured exposure value and the normalized exposure value.

10. The method according to claim 8 or 9, in, The shooting parameter information includes at least one of the following: aperture value, exposure time, sensitivity, and baseline exposure value.

11. The method according to claim 6, in, The adjusting the brightness of the local area based on the brightness adjustment intensity parameter corresponding to the local area includes: For a central pixel in the local area, adjusting the brightness of the central pixel using a brightness adjustment intensity parameter corresponding to the central pixel among the brightness adjustment intensity parameters corresponding to the local area; For a non-central pixel in the local area, the brightness of the non-central pixel is adjusted based on a brightness adjustment intensity parameter corresponding to the local area and a brightness adjustment intensity parameter corresponding to a local area adjacent to the local area.

12. The method according to claim 11, in, The adjusting the brightness of the non-central pixel based on the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area of ​​the local area includes: Mapping the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area to the same spatial position; Applying the brightness adjustment intensity parameter corresponding to the local area and the brightness adjustment intensity parameter corresponding to the adjacent local area respectively to obtain the brightness value of the non-central pixel adjusted by using different brightness adjustment intensity parameters; Bilinear interpolation is performed on the brightness values ​​adjusted by using different brightness adjustment strength parameters to obtain a final brightness value of the non-central pixel.

13. The method according to claim 1, further comprising: include: Detecting the target brightness adjustment intensity input by the user; Based on the target brightness adjustment intensity, brightness adjustment is performed on the first image.

14. The method according to claim 13, in, The target brightness adjustment intensity is the brightness adjustment intensity corresponding to the global area of ​​the first image, The step of performing brightness adjustment on the first image based on the target brightness adjustment intensity includes: Obtaining brightness statistical information of the first image; Determining a maximum brightness adjustment intensity parameter corresponding to the first image according to the brightness statistical information; Brightness adjustment is performed on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity.

15. The method according to claim 14, in, The step of determining the maximum brightness adjustment intensity parameter corresponding to the first image according to the brightness statistical information includes: Performing histogram equalization on the first image according to the brightness statistical information to obtain a histogram equalized image corresponding to the first image; Based on the histogram equalization image, a maximum brightness adjustment intensity parameter corresponding to the first image is determined.

16. A method performed by an electronic device, include: Detecting a target brightness adjustment intensity input by a user, wherein the target brightness adjustment intensity is a brightness adjustment intensity corresponding to a global area of ​​the first image; Obtaining brightness statistical information of the first image; Determining a maximum brightness adjustment intensity parameter corresponding to the first image according to the brightness statistical information; Brightness adjustment is performed on the first image based on the maximum brightness adjustment intensity parameter and the target brightness adjustment intensity.

17. A method performed by an electronic device, include: Detecting a target brightness adjustment intensity input by a user, wherein the target brightness adjustment intensity is a brightness adjustment intensity corresponding to a target local area in the first image; Based on the target brightness adjustment intensity, adjusting the brightness of the target local area; Obtaining correlations between semantic features of each semantic category in the first image; Brightness adjustment is performed on other local areas of the first image except the target local area based on the correlation.

18. A method performed by an electronic device, include: Detecting the target brightness adjustment intensity input by the user; performing brightness adjustment on the first image based on the target brightness adjustment intensity; In response to detecting that brightness adjustment of the first image based on the target brightness adjustment intensity is completed, brightness adjustment is performed on the first image based on image brightness information of the first image and scene brightness information of a shooting scene corresponding to the first image.

19. An electronic device, include: Memory; A processor is coupled to the memory and configured to execute the method according to any one of claims 1 to 18.

20. A computer-readable storage medium storing instructions, which, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 18.

Images