Image display method and related device

Abstract

The present application provides an image display method and a related device. The image display method can be applied to an electronic device, the method comprising: the electronic device detects a first user operation indicating to display a preview interface; in response to the first user operation, displays a first image on the preview interface at a first moment, the first image not comprising a human face, the first image comprising a first area and a second area, and a brightness mean value of the first area being greater than a brightness mean value of the second area; and at a second moment, displays a second image on the preview interface, the second image comprising a face, the second image comprising a third area and a fourth area, the face being located in the fourth area, and a brightness mean value of the third area being greater than a brightness mean value of the fourth area. The brightness mean value of the first area is greater than the brightness mean value of the third area, and the brightness mean value of the fourth area is greater than the brightness mean value of the second area. According to the described method, a high-dynamic-range image can be displayed in an image preview process, and user experience is improved.

Description

Image display method and related equipment Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image display method and related equipment. Background Art

[0002] Dynamic range (DR) is used in many fields to express the ratio of the maximum and minimum values ​​of each variable. In digital images, dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the image display range, that is, the number of grayscale levels between the "brightest" and "darkest" of the image. Since the dynamic range of natural scenes in the real world is between 10 3 to 10 6 The dynamic range is very large, so it is called high dynamic range (HDR). Compared with high dynamic range images, the dynamic range of ordinary images is low dynamic range (LDR), sometimes also called standard dynamic range (SDR).

[0003] The greater the dynamic range of an image, the richer the brightness levels it can represent, showing better contrast and detail. The image also appears more realistic, more closely recreating the real scene and providing a better visual experience for users. However, the dynamic range of images currently displayed by electronic devices during image preview is very limited, resulting in a significant difference between the user experience and the real scene. How to display high dynamic range images during image preview is an issue that the industry urgently needs to discuss.

[0004] Summary of the Invention

[0005] The present application provides an image display method and related devices. The image display method can display high dynamic range images during image preview, thereby improving user experience.

[0006] In a first aspect, the present application provides an image display method, applied to an electronic device, the method comprising: detecting a first user operation indicating displaying a preview interface; in response to the first user operation, displaying a first image on the preview interface at a first moment, the first image not including a face, the first image including a first area and a second area, the average brightness of the first area being greater than the average brightness of the second area; at a second moment, displaying a second image on the preview interface, the second image including a face; the second image including a third area and a fourth area, the face being located in the fourth area; the average brightness of the third area being greater than the average brightness of the fourth area; wherein the average brightness of the first area is greater than the average brightness of the third area, and the average brightness of the fourth area is greater than the average brightness of the second area.

[0007] In an embodiment of the present application, the electronic device displays a preview interface in response to a first user operation instructing to display a preview interface. The preview interface can display images (such as the first image and the second image mentioned above) acquired in real time by the electronic device and obtained by dynamic range expansion. The first moment and the second moment are two moments in the process of the electronic device displaying the preview interface. The first image displayed at the first moment does not include a face, and the second image displayed at the second moment includes a face. The electronic device can perform different dynamic range expansions on the image based on whether the acquired image includes a face. In an embodiment of the present application, the dynamic range of the first image is greater than the dynamic range of the second image.

[0008] As described above, the mean brightness N2 of the first region in the first image is greater than the mean brightness N1 of the second region. The mean brightness M1 of the third region in the second image is greater than the mean brightness M2 of the fourth region. Furthermore, the mean brightness N1 of the first region is greater than the mean brightness M1 of the third region, and the mean brightness M2 of the fourth region is greater than the mean brightness N2 of the second region. In other words, N2 > M1 > M2 > N1. Therefore, the dynamic range of the first image is greater than that of the second image.

[0009] It should be understood that the greater the dynamic range of an image, the more pronounced the lines and details in the image will be. The presentation of facial areas with a large dynamic range often does not meet user needs. This method, by expanding the dynamic range to varying degrees for images that do not include facial areas (i.e., the first image described above) and images that include facial areas (i.e., the second image described above), can achieve a high dynamic range when displaying images that do not include facial areas, and moderately expand the dynamic range when displaying images that include facial areas. This can meet the different display needs of users for images that do and do not include facial areas, thereby improving the user experience.

[0010] In one possible implementation, the first image is composed of the first and second regions, that is, the embodiment of the present application can divide the first image into the first and second regions; the second image is composed of the third and fourth regions, that is, the embodiment of the present application can divide the second image into the third and fourth regions. The first region corresponds to the third region, and the second region corresponds to the fourth region.

[0011] Optionally, the image content corresponding to the first area and the third area is the same, that is, the image content corresponding to the first area and the third area is the content of the same shooting position; the image content corresponding to the second area and the fourth area is different only in the area where the face is located, that is, the image content corresponding to the second area and the fourth area is the content obtained by shooting at the same shooting position, and the shooting position does not include the face at the first moment, and the shooting position includes the face at the second moment, such as the face is not in the frame at the first moment, and the face is in the frame at the shooting positions corresponding to the second area and the fourth area at the second moment.

[0012] In combination with the first aspect, in one possible implementation, the first image is obtained based on metadata information corresponding to the first image; the second image is obtained based on metadata information corresponding to the second image; the metadata information includes at least one of grayscale information and brightness information of the face area.

[0013] In combination with the first aspect, in one possible implementation, the grayscale information includes a first grayscale histogram and a second grayscale histogram; the first grayscale histogram is a grayscale histogram calculated from the grayscale histogram of an image with a first exposure level or the grayscale histogram of an image obtained by fusing multiple frames of images with different exposure levels; the second grayscale histogram is a histogram of an output image or a low dynamic range image synthesized from multiple frames of images with different exposure levels.

[0014] Illustratively, the grayscale histogram of the image at the first exposure level may be a normally exposed raw image, and the multiple frames of images with different exposure levels may be raw images with different exposure levels such as ultra-short exposure frames, ultra-long exposure frames, and normal exposure frames.

[0015] In the embodiment of the present application, the dynamic range of an image can be expanded by using metadata information corresponding to the image.

[0016] In combination with the first aspect, in a possible implementation, the method also includes: based on the grayscale information of the first image, compressing the brightness of the fused image corresponding to the first image to obtain a first image to be processed; the fused image corresponding to the first image is obtained by fusing multiple frames of images with different exposure levels corresponding to the first image, and the first image to be processed includes a first area to be processed and a second area to be processed; the average brightness of the first area to be processed is greater than the average brightness of the second area to be processed; based on the grayscale information of the first image, enhancing the brightness of the first area to be processed to obtain a second image to be processed; performing color gamut mapping on the second image to be processed to obtain a first image, the first area to be processed corresponds to the first area, and the second area to be processed corresponds to the second area.

[0017] In the embodiment of the present application, the brightness of the low-brightness area (i.e., the second area to be processed) and the high-brightness area (i.e., the first area to be processed) in the fused image corresponding to the first image are adjusted respectively, thereby obtaining a first image with a high dynamic range.

[0018] In combination with the first aspect, in a possible implementation, the method also includes: calculating a first expansion coefficient of the first image based on a first grayscale histogram corresponding to the first image and a second grayscale histogram corresponding to the first image; calculating a second expansion coefficient of the first image based on the first grayscale histogram corresponding to the first image; the first expansion coefficient of the first image or the second expansion coefficient of the first image is used to adjust the brightness of the fused image corresponding to the first image.

[0019] In combination with the first aspect, in a possible implementation, the method also includes: performing regularization processing on the second grayscale histogram corresponding to the first image to obtain a regularized histogram corresponding to the first image, and the regularized histogram corresponding to the first image is used to adjust the brightness of the fused image corresponding to the first image when predicting that the first image includes a quantization band.

[0020] In an embodiment of the present application, when predicting that the first image includes quantization bands, the brightness of the fused image corresponding to the first image can be adjusted based on the regularized histogram corresponding to the first image. This method can remove the quantization bands in the first image and improve the display effect.

[0021] In combination with the first aspect, in a possible implementation, the method also includes: based on the metadata information corresponding to the second image, compressing the brightness of the fused image corresponding to the second image to obtain a third image to be processed; the fused image corresponding to the second image is obtained by fusing multiple frames of images with different exposure levels corresponding to the second image; the third image to be processed includes a third area to be processed and a fourth area to be processed; the average brightness of the third area to be processed is greater than the average brightness of the fourth area to be processed; based on the metadata information corresponding to the second image, increasing the brightness of the third area to be processed to obtain a fourth image to be processed; performing color gamut mapping on the fourth image to be processed to obtain a second image, the third area to be processed corresponds to the third area, and the fourth area to be processed corresponds to the fourth area.

[0022] In combination with the first aspect, in a possible implementation, the method also includes: calculating a first expansion coefficient of the second image based on a first grayscale histogram corresponding to the second image and a second grayscale histogram corresponding to the second image; calculating a second expansion coefficient of the second image based on the second grayscale histogram corresponding to the second image; when it is determined that the second image includes a face, calculating a third expansion coefficient of the second image based on the brightness information of the face area corresponding to the second image, the third expansion coefficient being positively correlated with the brightness information of the face area; at least one of the first expansion coefficient of the second image, the second expansion coefficient of the second image, and the third expansion coefficient of the second image is used to perform brightness adjustment on the fused image corresponding to the second image.

[0023] In combination with the first aspect, in a possible implementation, the method also includes: performing regularization processing on the second grayscale histogram corresponding to the second image to obtain a regularized histogram corresponding to the second image, and the regularized histogram corresponding to the second image is used to adjust the brightness of the fused image corresponding to the second image when predicting that the second image includes a quantization band.

[0024] In an embodiment of the present application, when predicting that the second image includes quantization bands, the brightness of the fused image corresponding to the second image can be adjusted based on the regularized histogram corresponding to the second image. This method can remove the quantization bands in the second image and improve the display effect.

[0025] In combination with the first aspect, in a possible implementation, the method further includes: when it is determined that the similarity between the first image and the previous frame image of the first image is greater than a preset threshold, brightness adjustment information obtained from metadata information corresponding to the previous frame image is performed on the fused image corresponding to the first image to obtain the first image; the fused image corresponding to the first image is obtained by fusing multiple frames of images with different exposure levels corresponding to the first image.

[0026] In embodiments of the present application, when the similarity between a first image and a previous image is greater than a preset threshold, brightness adjustment information derived from metadata corresponding to the previous image can be used to expand the dynamic range of the first image, thereby reducing computational complexity and improving image processing efficiency. For example, the brightness expansion information can include an expansion coefficient corresponding to the previous image and an inverse tone mapping curve.

[0027] In combination with the first aspect, in one possible implementation, the first expansion coefficient of the target image is obtained based on a first contrast and a second contrast, the first contrast is obtained based on the expectations of the highlight and low-brightness parts in a first grayscale histogram corresponding to the target image; the second contrast is obtained based on the expectations of the highlight and low-brightness parts in a second grayscale histogram corresponding to the target image, and the target image is the first image or the second image.

[0028] In combination with the first aspect, in one possible implementation, the second expansion coefficient of the target image is obtained based on the screen pixel mean and highlight coefficient of the target image, the highlight coefficient is used to represent the distribution of the highlight area in the target image, the screen pixel mean and highlight coefficient of the target image are obtained based on the second grayscale histogram corresponding to the target image, and the target image is the first image or the second image.

[0029] In a possible implementation, the second expansion coefficient of the target image is obtained based on the first coefficient and the second coefficient, the first coefficient is obtained based on the average value of the pixels in the picture; the second coefficient is obtained based on the highlight coefficient.

[0030] It can be understood that a larger average value of picture pixels indicates that the overall brightness of the target image is brighter. The embodiment of the present application can avoid the picture being too bright after the dynamic range is expanded by reducing the first coefficient when the average value of picture pixels is greater than a preset threshold.

[0031] It is understood that the highlight coefficient is used to represent the distribution of highlight areas in the target image (or the proportion of highlight areas). That is, a larger highlight coefficient indicates a larger distribution area of ​​highlight areas in the image, and a smaller highlight coefficient indicates a smaller distribution area of ​​highlight areas in the image. In this embodiment of the application, by using a smaller second coefficient when the distribution area of ​​highlight areas is large, it is possible to avoid a larger highlight area in the image after dynamic range expansion.

[0032] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code comprises computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the method described in the first aspect and any possible implementation of the first aspect.

[0033] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0035] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0036] It is understandable that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to perform the methods provided in this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] 1A to 1C exemplarily illustrate user interfaces related to enabling the “HDR preview” function for a preview scene;

[0038] FIG2 is a schematic diagram of the overall process of an image display method exemplarily provided in an embodiment of the present application;

[0039] FIG3 is a schematic diagram of the hardware structure of an electronic device exemplarily provided in an embodiment of the present application;

[0040] FIG4 is a schematic diagram of the software and hardware structure of an electronic device exemplarily provided in an embodiment of the present application;

[0041] FIG5 is a schematic diagram of an image display method exemplarily provided in an embodiment of the present application;

[0042] FIG6 is a schematic diagram of an exemplary method of calculating a first expansion coefficient provided in an embodiment of the present application;

[0043] FIG7 is a schematic diagram of an exemplary method of calculating a second expansion coefficient provided in an embodiment of the present application;

[0044] FIG8A is a schematic diagram of Table 1 exemplarily provided in an embodiment of the present application;

[0045] FIG8B is a schematic diagram of Table 2 exemplarily provided in an embodiment of the present application;

[0046] FIG9 is a schematic diagram of an exemplary method for calculating a second ITM gain curve provided in an embodiment of the present application;

[0047] FIG10 is a schematic diagram of a slope calculation method exemplarily provided in an embodiment of the present application;

[0048] FIG11 is a schematic diagram of an exemplary method for calculating a third inverse tone mapping curve provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0050] It should be understood that the terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish between different objects, rather than to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0051] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0052] To better understand the embodiments of the present application, the terms or concepts that may be involved in the embodiments are explained below.

[0053] (1) The backlight brightness of the screen, or screen brightness, is a physical quantity that characterizes the light intensity of the screen of an electronic device. The unit can be nit or candela per square meter (cd / m2).

[0054] (2) Pixels

[0055] A pixel is the smallest imaging unit in an image. It corresponds to a coordinate point on the image. It can correspond to a single parameter (such as grayscale) or a combination of multiple parameters (such as grayscale, brightness, color, etc.).

[0056] Taking color information as an example, usually, a frame of an image has three basic colors, namely red (Red, hereinafter represented by R), green (hereinafter represented by G), and blue (hereinafter represented by B). Other colors can be composed of these three basic colors. Therefore, each pixel on a frame of an image can contain R, G, and B color information, and the values ​​of the R, G, and B color information on each pixel (abbreviated as R value, G value, and B value) can be different. For example, when the values ​​of the R, G, and B color information corresponding to a pixel are all 0, the pixel appears white. When the values ​​of the R, G, and B color information corresponding to a pixel are all 255, the pixel appears black.

[0057] Grayscale, also known as grayscale, is a parameter that characterizes the brightness of an image. The light source behind each pixel can exhibit different brightness levels. Grayscale represents the gradation of brightness from darkest to brightest. For an 8-bit screen, an image can have 256 brightness levels, or 256 grayscales, from 0 to 255.

[0058] In some embodiments of the present application, 0-255 may also be converted into corresponding values ​​within 0-1 to represent grayscale.

[0059] (3) Inverse tone mapping

[0060] The process of recovering HDR information from a normal image (such as an LDR image or an SDR image) is called inverse tone mapping (ITM). The ITM function (also known as the inverse tone mapping curve) can be used to perform inverse tone mapping on the LDR image to obtain the HDR image corresponding to the LDR image. The ITM function can also be called the inversion of the complete camera response function.

[0061] For example, the electronic device can expand a smaller parameter range of the SDR image data to a larger parameter range based on the inverse tone mapping curve. For example, if the brightness range of the SDR image data is 0-200 nits, the brightness range of 0-200 nits can be expanded to a brightness range of 0-1000 nits based on the inverse tone mapping curve.

[0062] The dynamic range of images captured and displayed by traditional imaging display devices is very limited. Therefore, the images viewed by users on traditional imaging display devices differ significantly from the real scene, resulting in a poor user experience. In recent years, with the continuous advancement of software and hardware technologies in the display field, more and more imaging display devices can support the display of HDR images.

[0063] However, the majority of images displayed in preview scenarios on current electronic devices, such as those in camera preview interfaces, are in low dynamic range or standard dynamic range. The lack of HDR images not only wastes display capacity for HDR-capable display devices, but also results in poor image quality, impacting the user experience.

[0064] To address this issue, embodiments of the present application provide an image display method that allows an electronic device to display an image with an expanded dynamic range on a preview interface. This method can provide users with a visual effect closer to the real scene in the preview scene, thereby improving the user experience.

[0065] In this application, an electronic device is a device with a display service, not limited to mobile phones and tablet computers. The electronic device can also be a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiments of this application do not impose any special restrictions on the specific type of the terminal.

[0066] For example, the image display method provided in the embodiment of the present application can be embodied as an “HDR preview” function on an electronic device.

[0067] Among them, the "HDR preview" function can be turned on through a control, such as the preview interface of the camera application displayed by the electronic device includes a first control, and the first control is used to turn on the "HDR preview" function. When the user touches the first control, the electronic device responds to the user operation and displays the image processed by the image display method provided by this application in real time on the preview window of the preview interface. After turning on the function, the dynamic range of the image displayed in the preview window is greater than the dynamic range of the image displayed in the preview window before turning on the function; alternatively, the "HDR preview" function can also be in the default on state all the time, such as when the electronic device displays the preview interface of the camera application, the image processed by the image display method provided by this application is displayed in real time on the preview window of the preview interface.

[0068] Below, taking the “HDR preview” function in the default on state as an example, the “HDR preview” function for the preview scene provided in an embodiment of the present application is described in combination with the user interface (UI) provided on the electronic device.

[0069] Figures 1A to 1C exemplarily illustrate the UI for enabling the "HDR preview" function for a preview scene.

[0070] The user interface 11 shown in FIG1A is an exemplary desktop of an electronic device provided in an embodiment of the present application. The user interface 11 may include an icon for at least one application (e.g., weather, calendar, email, settings, app store, notes, gallery 110, phone, short message, browser, and camera, etc.). The positions of the application icons and corresponding application names may be adjusted according to the user's preferences, and this application does not impose any restrictions thereon.

[0071] It should be noted that the desktop shown in FIG1A is an exemplary display of an embodiment of the present application. The desktop of the electronic device may also be in other styles, and the present application does not limit this.

[0072] In Figure 1A, a user can click on camera icon 110 in user interface 11. Accordingly, the electronic device can detect the user operation on camera icon 110 and, in response to the user operation, display user interface 12 shown in Figure 1B. User interface 12 is an example of a preview interface shown in the embodiment of this application. In this application, the user operation on camera icon 110 can be referred to as the first user operation instructing to display the preview interface.

[0073] The user interface 12 may be a specific control or switch as shown in FIG1B , which is described in detail below: the shooting mode menu 122 may include options for multiple camera modes such as portrait 1221, video 1222, photo 1223, night scene 1224 and more 1225. Different camera modes can achieve different shooting functions. The camera mode pointed to by the "triangle" in the shooting mode menu 122 is used to indicate the initial or user-selected camera mode. As shown in FIG1B , the "triangle" points to "photo", indicating that the current camera is in photo mode.

[0074] Preview area 120 is used to display images that can be previewed by the user. For ease of description, this application may refer to the image displayed in preview area 120 as a preview image. For example, an electronic device can obtain a high dynamic range image using the image display method provided in this application and display the high dynamic range image in preview area 120. For details, please refer to the detailed description below and will not be elaborated here.

[0075] For example, the embodiment of the present application shows that the preview area 120 includes a butterfly-shaped wall lamp (also referred to as a butterfly lamp). When the wall lamp is turned on, the average brightness of the wall lamp area is higher than the average brightness of the non-wall lamp area, that is, the average brightness of the wall lamp area is N1, and the average brightness of the non-wall lamp area is N2, and N2 is greater than N1. In the present application, the user interface 12 shown in Figure 1B can be a preview interface displayed by the electronic device at the first moment, wherein the image displayed in the preview area 120 can be referred to as the first image, the wall lamp area in the preview area 120 can be referred to as the first area, and the non-wall lamp area in the preview area 120 can be referred to as the second area. The average brightness N1 of the wall lamp area can be referred to as the average brightness of the first area, and the average brightness N2 of the non-wall lamp area can be referred to as the average brightness of the second area.

[0076] Small window 123 is used for users to view the pictures and videos they have taken.

[0077] The shooting control 124 is used to enable the electronic device to shoot images or record videos in response to user operations.

[0078] The camera switching control 125 is used to switch the camera for collecting images between the front camera and the rear camera.

[0079] The shooting function switch 121 may include a beauty switch 1211, an AI mode switch 1212, a flash switch 1213, a filter switch 1214, and a settings control 1215. The beauty switch 1211 is used to turn the beauty function on or off. The AI ​​mode switch 1212 is used to turn the automatic recognition mode function on or off. The flash switch 1213 is used to turn the flash on or off. The filter switch 1214 is used to turn the filter on or off. The settings control 1215 is used to set various parameters for image capture.

[0080] In other embodiments of the present application, the "HDR preview" function may be enabled via a control. For example, the user interface 12 may further display an HDR preview mode switch for turning the "HDR preview" function on or off. It should be noted that the present application does not limit the display pattern and position of the HDR preview mode switch; after the user opens the camera and enters the preview interface, the "HDR preview" function may be enabled or disabled.

[0081] For example, assuming that the first user points the camera of the electronic device at the wall lamp, after the user interface 12 shown in Figure 1B is displayed, the second user is allowed to enter the screen, and the electronic device can display the user interface 13 shown in Figure 1C. User interface 13 is another preview interface exemplarily shown in the embodiment of the present application.

[0082] The user interface 13 includes a preview area 130, a shooting function switch 121, and a shooting mode menu 122. The preview area 130 includes a wall lamp area and a non-wall lamp area, the wall lamp in the wall lamp area is in the on state; the non-wall lamp area includes the face of the second user. Assuming that the average brightness of the wall lamp area in the preview area 130 is M1, and the average brightness of the non-wall lamp area is M2, since the wall lamp is in the on state, then M2 is less than M1. In the present application, the user interface 13 shown in Figure 1C can be a preview interface displayed by the electronic device at the second moment, wherein the image displayed in the preview area 130 can be referred to as the second image, the wall lamp area in the preview area 130 can be referred to as the third area, and the non-wall lamp area in the preview area 130 can be referred to as the fourth area; the average brightness M1 of the wall lamp area can be referred to as the average brightness of the third area, and the average brightness M2 of the non-wall lamp area can be referred to as the average brightness of the fourth area.

[0083] The image display method provided in the present application has a higher degree of dynamic range expansion for images that do not include faces, and a lower degree of dynamic range expansion for images that include faces. For example, the screen content displayed in preview area 120 in user interface 12 and preview area 130 in user interface 13 remains unchanged except for the appearance of a face. For example, if the screen content is the same wall lamp and both are turned on, then the average brightness N1 of the wall lamp area in preview area 120 is greater than or equal to the average brightness M1 of the wall lamp area in preview area 130, and the average brightness M2 of the non-wall lamp area in preview area 120 is less than or equal to the average brightness M2 of the non-wall lamp area in preview area 130.

[0084] It should be understood that the larger the dynamic range, the more obvious the lines and other information in the area corresponding to the dynamic range will be. Since the clarity of the face area when the dynamic range is large does not meet the user's conventional aesthetic standards, this application can ensure that the dynamic range of the image is increased without affecting the rendering effect of the face area by moderately expanding the dynamic range of the image including the face. This can meet the different needs of users for images with and without faces, and can improve the user experience.

[0085] The preview images displayed in the preview interface (such as the image displayed in the preview area 120 in FIG. 1B and the image displayed in the preview area 130 in FIG. 1C ) are displayed by the electronic device using the image display method provided in this application.

[0086] Next, the overall process of the image display method provided by the present application is introduced through Figure 2. Exemplarily, the method is described by taking the display of a frame of image in a preview scene as an example.

[0087] First, the electronic device uses a camera to capture the raw image corresponding to the frame. In Figure 2, different patterns represent different exposure levels. For example, the raw image may include multiple frames with different exposure levels, such as ultra-short exposure frames, ultra-long exposure frames, and standard exposure frames. The electronic device then performs fusion processing (such as multi-exposure fusion, white balance correction, and color gamut conversion) on the raw images to obtain a color image corresponding to the raw image (referred to as the first image for ease of description).

[0088] It should be understood that a raw image is the output image of a camera. This refers to the raw data obtained by the camera by converting the light information reflected by an object into a digital image signal. This raw data has not been processed. For example, a raw image can be in raw format. This raw format data can include object information and camera parameters. Camera parameters may include shutter speed, aperture value, white balance, etc.

[0089] Figure 2 exemplifies the image data corresponding to the first image, showing that the R, G, and B values ​​of the diffuse reflection area (e.g., the wall lamp area) are a1, b1, and c1, respectively, where a1, b1, and c1 are all positive numbers; the R, G, and B values ​​of the highlight area (e.g., the non-wall lamp area) are a2, b2, and c2, respectively, where a2, b2, and c2 are all positive numbers; and the current screen backlight brightness is 400 nits. The division of the diffuse reflection area and the highlight area can be found in the relevant description below and will not be expanded here.

[0090] Furthermore, the electronic device can increase the screen backlight brightness; compress the pixel grayscale of the first image to obtain the image data corresponding to the second image, that is, the R value, G value, and B value of the diffuse reflection area and the highlight area in the image data corresponding to the second image are respectively compressed from the R value, G value, and B value of the diffuse reflection area and the highlight area in the image data corresponding to the first image. Taking the compression factor n as an example, n is a positive number, the R value, G value, and B value in the image data corresponding to the second image are 1 / n of a1, b1, and c1, that is, a1 / n, b1 / n, and c1 / n; the R value, G value, and B value of the highlight area in the first image are 1 / n of a2, b2, and c2, that is, a2 / n, b2 / n, and c2 / n; Figure 2 exemplifies that the screen backlight brightness in the image data corresponding to the second image is 1000 nit. This application does not limit the compression factor of the compressed pixel grayscale (that is, the above-mentioned n) and the increase factor of the screen backlight brightness; it can be understood that the screen backlight brightness is increased to a preset maximum value at most.

[0091] Furthermore, the electronic device can keep the brightness of the diffuse reflection area in the second image unchanged / fine-tune it according to the metadata information (matedata), and increase the brightness of the highlight area to obtain a third image; wherein the R value, G value, and B value in the image data corresponding to the third image can be the same as the R value, G value, and B value of the second image, or can deviate from the R value, G value, and B value of the second image within a preset difference; the R value, G value, and B value of the highlight area in the third image are a3, b3, and c3, respectively, wherein a3≥a2; b3≥b2; c3≥c2; the screen backlight brightness in the image data corresponding to the third image is the same as that of the second image, which is still 1000nit.

[0092] Finally, the electronic device may perform color gamut mapping on the third image to obtain a preview image; the electronic device may display the preview image on a preview interface (such as Figures 1B and 1C above). For example, the specific process of the electronic device obtaining the preview image can be found in the embodiment shown in Figure 5 below.

[0093] The following describes the form and hardware and software architecture of the electronic device provided in the embodiments of the present application.

[0094] Electronic equipment can be equipped or portable terminal devices with other operating systems, such as mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices and / or smart city devices, etc.

[0095] FIG3 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0096] As shown in FIG3 , the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 126, a camera 130, a display screen 140, an audio module 150, a speaker 150A, a receiver 150B, a microphone 150C, an earphone interface 150D, and a sensor module 160. The sensor module 160 may include a pressure sensor 160A, a distance sensor 160F, a proximity light sensor 160G, a touch sensor 160K, an ambient light sensor 160L, and the like.

[0097] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0098] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on instruction operation codes and timing signals to complete the control of instruction fetching and execution.

[0099] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0100] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0101] In an embodiment of the present application, the processor 110 may be used to execute the aforementioned embodiments to implement the image display method provided by the present application.

[0102] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0103] The internal memory 126 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0104] The electronic device implements its display functionality through a GPU, display screen 140, and an application processor. The GPU is a microprocessor for image processing that connects display screen 140 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0105] The display screen 140 is used to display images, videos, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD). The display screen panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniLED, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device can include one or N display screens 140, where N is a positive integer greater than one.

[0106] The electronic device can implement a shooting function through an ISP, a camera 130 , a video codec, a GPU, a display screen 140 , and an application processor.

[0107] The ISP is used to process data fed back by camera 130. Camera 130 is used to capture still images or video. In some embodiments, the electronic device may include one or N cameras 130, where N is a positive integer greater than 1. In the embodiment of the present application, camera 130 may be a telephoto camera, a main camera, a wide-angle camera, or the like.

[0108] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency, the DSP performs a Fourier transform on the frequency energy.

[0109] Video codecs are used to compress or decompress digital video. Electronic devices may support one or more video codecs. This allows them to play or record videos in a variety of encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0110] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications in electronic devices, such as image recognition, face recognition, speech recognition, and text comprehension.

[0111] The electronic device can implement audio functions such as music playback and recording through the audio module 150, the speaker 150A, the receiver 150B, the microphone 150C, the headphone jack 150D, and the application processor.

[0112] The structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. For example, the electronic device may further include buttons, motors, indicators, and subscriber identification module (SIM) card interfaces, etc. For another example, the sensor module may further include: a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a fingerprint sensor, a temperature sensor, a bone conduction sensor, and the like.

[0113] In the embodiment of the present application, the electronic device can execute the image display method provided by the present application through the above-mentioned processor 110, and display the preview image obtained by executing the image display method provided by the present application through the display screen 140.

[0114] FIG4 is a schematic diagram of the software and hardware structure of an electronic device provided in an embodiment of the present application.

[0115] A layered architecture divides the system into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the system is divided into five layers: application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer, from top to bottom.

[0116] The application layer can include a series of application packages. As shown in Figure 4, the application layer can include applications such as camera and gallery (also referred to as applications). The application layer can also include applications such as calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. (not shown in Figure 4).

[0117] Among them, the camera application is used to display the preview interface. The process of the user opening the camera application can refer to the relevant interfaces of Figures 1A to 1C above; the gallery application is used to display images or videos stored in the electronic device.

[0118] In an embodiment of the present application, the electronic device can display the preview image obtained by the image display method provided in the present application through the preview interface of the camera application.

[0119] The application framework layer provides an application programming interface (API) and programming framework for the application layer's applications. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, where the camera access interface includes camera management and camera devices. The camera access interface is used to provide an application programming interface and programming framework for camera applications.

[0120] The hardware abstraction layer is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. In the embodiment of the present application, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0121] Among them, the camera hardware abstraction layer can provide virtual hardware of camera device 1, camera device 2 or more camera devices and a camera algorithm library. The camera algorithm library may include running code and data for implementing the image display method provided in the embodiment of the present application.

[0122] Exemplarily, the camera hardware abstraction layer may process the original image to obtain a preview image and a preview video with a dynamic range expanded.

[0123] The driver layer is the layer between hardware and software. It includes drivers for various hardware components. These drivers can include camera device drivers, digital signal processor drivers, and image processor drivers.

[0124] The camera device driver drives the camera's sensor to capture images and the image signal processor to pre-process them. The digital signal processor driver drives the digital signal processor to process images. The image processor driver drives the graphics processor to process images.

[0125] The image display method in the embodiment of the present application is described in detail below in conjunction with the above system structure:

[0126] In response to a user opening a camera application, such as clicking a camera application icon, the camera application invokes the camera access interface of the application framework layer to start the camera application. The camera application then sends a camera start instruction by invoking the camera device (camera device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends this instruction to the camera device driver in the kernel layer. The camera device driver can then activate the corresponding camera sensor and collect image light signals through the sensor. A camera device in the camera hardware abstraction layer corresponds to a camera sensor in the hardware layer.

[0127] Then, the camera sensor can transmit the collected image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the above raw image to the camera hardware abstraction layer through the camera device driver.

[0128] The camera hardware abstraction layer can send the original image to the camera algorithm library. The camera algorithm library stores the program code that implements the image display method provided by the embodiments of this application. Based on the digital signal processor and image processor, the camera algorithm library executes this code to achieve the "HDR preview" and other capabilities described above.

[0129] The camera algorithm library sends the raw image captured by the camera to the camera hardware abstraction layer. The camera hardware abstraction layer then processes the raw image using the camera algorithm library to generate a preview image, which is then sent for display. This allows the camera application to display the preview image in the preview window.

[0130] It should be noted that the software structure diagram of the electronic device shown in FIG4 provided in this application is only an example.

[0131] The specific module division in different layers of the Android operating system is not limited. For details, please refer to the introduction of the Android operating system software structure in conventional technology. In addition, the image display method provided in this application can also be implemented based on other operating systems, and this application will not give examples one by one.

[0132] Based on the user interface displayed on the above-mentioned electronic device and the software and hardware structure of the above-mentioned electronic device, the image display method provided in the embodiment of the present application is introduced below.

[0133] An image display method provided by an embodiment of the present application is described in conjunction with FIG5 . The method comprises the following steps:

[0134] S501: The electronic device obtains a first image corresponding to the current frame image based on an original image corresponding to the current frame image.

[0135] For example, upon detecting a user operation to open a camera application, the electronic device may capture, via the camera, a raw image corresponding to each frame in real time. The raw image may include multiple raw images with varying degrees of exposure, such as ultra-short exposure frames, ultra-long exposure frames, and normal exposure frames. This embodiment of the present application is described using the current frame image as an example. After capturing the raw image corresponding to the current frame image, the electronic device obtains a first image corresponding to the current frame image based on the raw image.

[0136] Regarding the specific implementation of obtaining the first image corresponding to the current frame image, please refer to the relevant content of Figure 2 above, which will not be repeated here.

[0137] S502: The electronic device obtains metadata information of the current frame image, where the metadata information includes a first grayscale histogram, a second grayscale histogram, and brightness information of a face area.

[0138] In one implementation, the electronic device may obtain metadata information of the current frame image based on the original image, the first image, and camera parameters. The metadata information may include:

[0139] (1) a first grayscale histogram (also referred to as AE histogram), which is a grayscale histogram calculated from a grayscale histogram of a normally exposed raw image or a grayscale histogram of an image fused from multiple frames of raw images with different exposures;

[0140] (2) a second grayscale histogram (also referred to as Yhist), which is a histogram of an output image or an SDR image (such as the first image) synthesized from multiple frames of images with different exposures;

[0141] (3)Zoon Ratio;

[0142] (4) Face auto exposure value, also known as the brightness information of the face area (faceaeValue, FaceEV). The larger the FaceEV value, the darker the brightness of the face area, and vice versa.

[0143] S503: The electronic device performs filtering and regularization processing on the second grayscale histogram to obtain a regularized histogram corresponding to the current frame image.

[0144] Optionally, the electronic device may use mean filtering or Gaussian filtering to filter the second grayscale histogram. The embodiment of the present application does not limit the filtering method.

[0145] Furthermore, the electronic device may perform regularization processing on the grayscale histogram obtained by filtering to obtain a regularized histogram, where the regularized histogram includes a regularized value corresponding to each grayscale.

[0146] For example, the regularization process can be expressed as follows: norm(j) =(H (j) -H m ) / H m ,H m =mean(H)

[0147] Wherein, j is any gray level in the gray level histogram obtained by filtering; H (j) is the number of pixels of gray level j; H norm(j) is the regularization value corresponding to grayscale j; H is the histogram obtained by filtering the second grayscale histogram.

[0148] S504: The electronic device determines whether the similarity between the current frame image and the previous frame image is greater than a preset threshold, and if so, executes step S505; otherwise, executes step S506.

[0149] In some embodiments, the electronic device may determine the similarity between the current frame image and the previous frame image based on the regularized histogram corresponding to the current frame image and the regularized histogram corresponding to the previous frame image. The calculation method of the regularized histogram corresponding to the previous frame image can be referred to step S3 and will not be repeated here.

[0150] For example, the calculation formula of the similarity Coh can be as follows:

[0151] Among them, H last is the histogram obtained by processing the regularized histogram corresponding to the previous frame image; H current It is the histogram obtained by processing the regularized histogram corresponding to the current frame image.

[0152] For example, the regularization value corresponding to each grayscale of the regularization histogram corresponding to the previous frame image is processed by the following formula to obtain the above H last ; The above H can be obtained by processing the regularization value corresponding to each grayscale in the regularization histogram corresponding to the current frame image through the following formula: current The formula is as follows: H x2 =maximum(H x1 ,Th1)

[0153] Among them, H x1 is the regularization value corresponding to each grayscale; H x3 The value after regularization processing corresponding to each grayscale;

[0154] Th1 is a preset value. For example, Th1 can be any value from 2 to 4.

[0155] It should be understood that the images in the preview process are continuous. For example, when there is no scene switching, the video content changes slightly. By using the ITM curve calculated for the previous frame of image, the amount of calculation can be saved.

[0156] S505: The electronic device determines the inverse tone mapping curve corresponding to the previous frame image as the target inverse tone mapping curve corresponding to the current frame image.

[0157] In some embodiments, when the electronic device determines that the similarity between the current frame image and the previous frame image is greater than a preset threshold, the electronic device determines the inverse tone mapping curve corresponding to the previous frame image as the target inverse tone mapping curve corresponding to the current frame image.

[0158] S506: The electronic device calculates a first expansion coefficient of the current frame image based on the first grayscale histogram and the second grayscale histogram corresponding to the current frame image.

[0159] In some embodiments, when the electronic device determines that the similarity between the current frame image and the previous frame image is not greater than a preset threshold, the electronic device calculates a first expansion coefficient of the current frame image based on the first grayscale histogram and the second grayscale histogram corresponding to the current frame image.

[0160] Exemplarily, in conjunction with FIG6 , an implementation of calculating the first expansion coefficient of the current frame image is introduced through the following S5061 to S5063.

[0161] S5061: The electronic device may convert the first grayscale histogram (Yhist) and the second grayscale histogram (AEhist) from a nonlinear domain to a linear domain to obtain a first linear domain histogram (LYhist) and a second linear domain histogram (LAEhist).

[0162] Please refer to FIG6 , which exemplarily shows the histograms of Yhist and AEhist before and after the conversion from the nonlinear domain to the linear domain.

[0163] S5062: The electronic device calculates the expectations corresponding to the low-brightness area and the high-brightness area in LYhist and LAEhist respectively, and obtains the expectation corresponding to the low-brightness area in LYhist (Y_meanL), the expectation corresponding to the high-brightness area in Yhist (Y_meanH), the expectation corresponding to the low-brightness area in LAEhist (AE_meanL), and the expectation corresponding to the high-brightness area in LAEhist (AE_meanH).

[0164] For example, the first linear domain histogram (LYhist) is used as an example for explanation:

[0165] First, the electronic device can accumulate the probability of the number of pixels corresponding to each grayscale in the first linear domain histogram starting from low grayscale to high grayscale (such as starting from grayscale 0); when the accumulated probability is greater than a preset probability Prob (such as 0.95), the current grayscale is determined as the grayscale threshold (also denoted as Y_grayValue), wherein the probability of the number of pixels corresponding to each grayscale can be the proportion of the number of pixels corresponding to the grayscale to the total number of pixels in the image; then, based on the grayscale threshold, the first linear domain histogram is divided into histograms corresponding to low-brightness areas (LYh ist[0-Y_grayValue]) and the histogram corresponding to the highlight area (LYhist[Y_grayValue-255]), where the histogram corresponding to the low-light area is the histogram area corresponding to the grayscale below the grayscale threshold, and the histogram corresponding to the highlight area is the histogram area corresponding to the grayscale not lower than the grayscale threshold; calculate the expectation corresponding to the histogram corresponding to the low-light area to obtain the (Y_meanL) corresponding to the low-light area; calculate the expectation corresponding to the histogram corresponding to the highlight area to obtain the expectation corresponding to the highlight area (Y_meanH).

[0166] Similarly, the electronic device can calculate AE_grayValue and AE_meanL, AE_meanH in LAEHist. The preset probability used to calculate AE_grayValue in LAEHist can be the same as the preset probability used to calculate Y_grayValue in LYhist.

[0167] Please refer to FIG6 , which exemplarily shows the positions of Y_grayValue and AE_grayValue.

[0168] S5063: The electronic device obtains a first expansion coefficient based on the expectations corresponding to the low-brightness area and the high-brightness area in LYhist and LAEhist.

[0169] In some embodiments, the electronic device can calculate the expected ratio corresponding to the low-brightness area and the high-brightness area in LYhist to obtain a first contrast; calculate the expected ratio corresponding to the low-brightness area and the high-brightness area in LAEhist to obtain a second contrast; and determine the product of the first contrast and the second contrast as a first expansion coefficient.

[0170] Exemplarily, the electronic device may calculate the first expansion coefficient (Err1) using the following formula:

[0171] Among them, the first contrast in FIG6 can be the above The second contrast ratio in FIG. 6 can be the above

[0172] S507: The electronic device calculates a second expansion coefficient of the current frame image based on a second grayscale histogram corresponding to the current frame image.

[0173] Exemplarily, in conjunction with FIG7 , an implementation of calculating the second expansion coefficient of the current frame image is introduced through the following S5071 to S5073.

[0174] S5071: Determine a first coefficient based on a picture pixel mean value of the current frame image, wherein the picture pixel mean value is obtained based on a second grayscale histogram.

[0175] In one implementation, the electronic device may calculate the picture pixel average (APL) of the current frame image based on the second grayscale histogram; when the picture pixel average is less than a preset threshold, the first preset coefficient is determined as the first coefficient; when the picture pixel average is not less than the preset threshold, the second preset coefficient corresponding to the picture pixel average is determined as the first coefficient (erra), wherein the picture pixel average and its corresponding second preset coefficient are negatively correlated; the first preset coefficient is greater than the second preset coefficient corresponding to the picture pixel average.

[0176] For example, the process of determining the first coefficient based on the average value of the pixels of the current frame image can be determined by the following piecewise function:

[0177] Wherein, erra is the first coefficient, apl is the average value of pixels in the picture, ErrMaxTh is the preset maximum expansion coefficient (ie, the above-mentioned first preset coefficient), Th_apl is the above-mentioned preset threshold, k<0, b>0.

[0178] It should be noted that the above piecewise function can also be presented in the form of a table, as shown in Table 1 in Figure 8A, where the horizontal axis is apl and the vertical axis is erra; after calculating apl, erra can be queried in Table 1 based on apl.

[0179] It can be understood that a larger average value of picture pixels indicates that the overall brightness of the current frame image is brighter. The embodiment of the present application can avoid the picture being too bright after the dynamic range is expanded by reducing the first coefficient when the average value of picture pixels is greater than a preset threshold.

[0180] S5072: Determine a second coefficient based on the distribution of highlight areas in the current frame image.

[0181] In one implementation, the electronic device may first determine the area in the second grayscale histogram corresponding to the current frame image where the grayscale is greater than a preset value (such as grayscale>220) as the histogram corresponding to the highlight area; then, perform weighted summation on the number of pixels corresponding to the grayscale in the histogram (such as the higher the grayscale, the higher the weight corresponding to the grayscale) to obtain a highlight coefficient, wherein the highlight coefficient is used to represent the distribution of the highlight area in the current frame image; and the second coefficient may be determined based on the highlight coefficient.

[0182] Exemplarily, the electronic device may determine the second coefficient by the following piecewise function based on the highlight coefficient.

[0183] Wherein, errb is the second coefficient mentioned above; ErrMaxTh, ErrMinTh, ThL, ThH are preset parameters; ErrMaxTh>ErrMinTh; a*H factor 2 +b*H factor The maximum value of +c is less than ErrMaxTh, a*H factor 2 +b*H factor The minimum value of +c is greater than ErrMinTh.

[0184] It should be noted that the above piecewise function can also be presented in the form of a table, as shown in Table 2 in FIG8B , where the horizontal axis is H factor , the vertical axis is errb; when calculating H factor After that, you can query errb in Table 2 based on apl.

[0185] It is understandable that the highlight coefficient is used to represent the distribution of the highlight area in the current frame image (or the proportion of the highlight area). That is, the larger the highlight coefficient, the larger the distribution area of ​​the highlight area in the image, and the smaller the highlight coefficient, the smaller the distribution area of ​​the highlight area in the image. The embodiment of the present application uses a smaller second coefficient when the distribution area of ​​the highlight area is large, which can avoid the highlight area of ​​the picture being large after the dynamic range is expanded.

[0186] S5073: Determine a second expansion coefficient based on the first coefficient and the second coefficient. Err2 = erra*a1 + errb*b1

[0187] Among them, a1 and b1 are preset coefficients.

[0188] In the embodiment of the present application, the second expansion coefficient can be used to avoid the situation where the picture is too bright or the highlight area of ​​the picture is large, which causes a poor user experience.

[0189] S508: The electronic device determines whether the current frame image includes a human face, and if so, executes S509; otherwise, executes S511.

[0190] S509: The electronic device calculates a first inverse tone mapping curve based on the brightness information of the face area.

[0191] In one implementation, when the electronic device determines that the current preview scene is a face scene, it can determine the expansion coefficient corresponding to the face based on FaceEV in the metadata, and FaceEV and the expansion coefficient corresponding to the face are positively correlated; based on the expansion coefficient corresponding to the face, the ITM gain curve corresponding to the face scene (referred to as the first ITM gain curve) is determined; based on the first ITM gain curve, a first inverse tone mapping curve is obtained.

[0192] For example, the electronic device may determine the expansion coefficient corresponding to the face using the following formula:

[0193] Among them, Err face is the expansion coefficient corresponding to the face; ev is the above FaceEV; a p =Err person -ErrMinTh; b_p, c_p, d_p, ErrMinTh and Err person is the preset coefficient; Err person It is the maximum value of the preset face scene expansion coefficient.

[0194] It should be understood that the larger the FaceEV value, the darker the brightness of the face area. face is a positive correlation, then the darker the face area, the higher the Err face The larger the value, the brighter the face area. face The embodiment of the present application sets ErrMinT<Err face <Err person And ev and Err face The dynamic range of the image can be flexibly adjusted according to the brightness of the face area, and the adjustment range is guaranteed to be within a reasonable range.

[0195] For example, the ITM gain curve corresponding to the face scene (referred to as the first ITM gain curve) is as follows: gain1 =pa*e x*x+pb*x

[0196] Wherein, x is the normalized grayscale (0-1) of the first image; pa and pb are preset coefficients.

[0197] Exemplarily, the first inverse tone mapping curve f itm1 It can be as follows: itm1 =x*(f gain1 +1)*CC=1 / Err 1 / 2.2 Err′=min(Err1, Err2, Err face ,Err_res) Err=max(Err′,1)

[0198] Among them, Err_res is the screen clearance. For example, the maximum screen brightness is 1000 nit and the current screen brightness is 200 nit, Err_res = 1000 / 200.

[0199] In the embodiment of the present application, when the current frame image includes a face, a corresponding expansion coefficient is preset for the face, which can avoid the face area not meeting the user's requirements under too high a dynamic range and contrast.

[0200] S510: The electronic device determines the first inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image.

[0201] In some embodiments, when the electronic device determines that the current frame image includes a face, it calculates a first inverse tone mapping curve based on the brightness information of the face area; determines the calculated first inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image; and then processes the first image through the target inverse tone mapping curve. For details, please refer to the following steps S516 to S518.

[0202] S511: The electronic device calculates a second ITM gain curve and a second inverse tone mapping curve based on a first grayscale histogram corresponding to the current frame image.

[0203] In one implementation, when the current frame image does not include a human face, the electronic device divides the current frame image into multiple areas, and the multiple areas correspond to different brightness ranges; different coefficients are set for the multiple areas, wherein the brightness of the area and the coefficient are inversely proportional, and the coefficient corresponding to the area represents the size of the dynamic range expansion of the area; based on the coefficients of different areas, a second inverse tone mapping curve is determined.

[0204] Exemplarily, an implementation of calculating the second inverse tone mapping curve is introduced through the following S5101 to S5105.

[0205] S5101: Calculate the first grayscale and the second grayscale based on Yhist of the current frame image.

[0206] Exemplarily, a method for an electronic device to divide different areas may be: calculating the average value of the picture pixels based on the Yhist of the current frame image; determining a first grayscale based on the average value of the picture pixels; then, using the OSTU algorithm to calculate the part with a grayscale higher than the first grayscale to obtain OSTU; then, determining a second grayscale based on OSTU.

[0207] Exemplarily, the first grayscale and the second grayscale can be determined by the following formula: X1 = max(APL, Thx1), X2 = max(OSTU, Thx2)

[0208] Wherein, X1 is the first grayscale, X2 is the second grayscale, and Thx1 and Thx2 are preset parameters.

[0209] Please refer to FIG. 9 , which exemplarily shows the positions of the first grayscale X1 and the second grayscale X2 in Yhist of the current frame image.

[0210] S5102: Divide parts of different brightness according to the first grayscale and the second grayscale.

[0211] Furthermore, the electronic device can divide the Yhist curve into three parts: low brightness, medium brightness and high brightness according to X1 and X2, that is, x≤X1 is the low brightness part, X1<x<X2 is the medium brightness part, and x≥X is the high brightness part.

[0212] S5103: Determine coefficients corresponding to different brightness parts.

[0213] Furthermore, the electronic device can calculate a coefficient curve based on X1 and X2.

[0214] The coefficient curve f is shown below as an example xs The calculation formula is:

[0215] Wherein, v1 and v2 are preset parameters, and x is the pixel grayscale of the first image.

[0216] Exemplarily, the coefficient solution method in the coefficient curve may include the following steps: Step S1: Calculate gain according to Err (Err = min (Err1, Err2, Err_res), gain = Err 1 / 2.2 -1; Step S2: k1 can be solved by v1+k1 / 2*(x2-x1)+k1*(1-x2)=gain; Step S3: According to the continuity of the function and the continuity of the slope, a2, b2, c2, and d2 can be solved.

[0217] S5104: Determine a second ITM gain curve based on coefficients corresponding to different brightness parts.

[0218] For example, the electronic device can use the following formula based on the coefficient curve f xs , calculate the second ITM gain curve f gain2 : f gain2 =f xs *x 2.2

[0219] Please refer to FIG9, which shows an example of the coefficient curve f xs and the second ITM gain curve f gain2 Schematic diagram of .

[0220] S5105: Determine a second inverse tone mapping curve based on the second ITM gain curve.

[0221] For example, the formula of the second inverse tone mapping curve may be as follows: itm2 =x*C*(f gain2 +1) C=1 / Err 1 / 2.2 Err′=min(Err1,Err2,Err_res) Err=max(Err′,1)

[0222] S512: The electronic device predicts whether the preview image corresponding to the current frame image includes a quantization band based on the regularized histogram corresponding to the current frame image. If so, step S513 is executed; otherwise, step S514 is executed.

[0223] In some embodiments, when the electronic device determines that the current frame image does not include a face, it determines whether quantization bands may appear in the preview image corresponding to the current frame image; if so, it executes step S513, i.e., determines the second inverse tone mapping curve calculated in S511 as the target inverse tone mapping curve corresponding to the current frame image; if not, it executes S514, i.e., calculates the third inverse tone mapping curve corresponding to the case where the quantization bands are included.

[0224] In one implementation, the electronic device may predict that a quantization band will appear in the preview image corresponding to the current frame image when the proportion of the number of pixels of a certain grayscale in the highlight area exceeds a preset number, and vice versa. For example, FIG10 exemplarily shows that in the regularized histogram corresponding to the current frame image (i.e., the regularized histogram of Yhist), the number of pixels corresponding to the grayscale in the area above the first grayscale X1 (i.e., the above-mentioned highlight area) is higher than a first threshold value (i.e., the proportion exceeds a preset number), then the electronic device may predict that a quantization band will appear in the preview image corresponding to the current frame image.

[0225] S513: The electronic device calculates a third inverse tone mapping curve based on the second ITM gain curve.

[0226] In some embodiments, when the electronic device determines that the current frame image does not include a face but includes a quantization band, the electronic device may calculate the value of the second ITM gain curve f gain2 and Yhist; detect peaks and troughs in the regularized histogram of the current frame image Yhist that are higher than the first grayscale X1; segment the regularized histogram based on the peaks and troughs; and generate a third inverse tone mapping curve using a piecewise function. Then, step S515 is executed to determine the third inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image.

[0227] For example, the process of generating the slope is described in conjunction with FIG10. As shown in FIG10, the electronic device can generate the second ITM gain curve f gain2 Calculate its first-order derivative; use the regularized histogram of Yhist to calculate the weight; multiply the first-order derivative and the weight to get the slope.

[0228] As an example, the process of generating the third inverse tone mapping curve is described with reference to FIG11. As shown in FIG11, the electronic device can detect peaks and valleys in the regularized histogram of the current frame image Yhist that are higher than the first grayscale X1 to obtain x1, x2, and x3; the electronic device divides the regularized histogram into three intervals based on x1, x2, and x3; calculates the curve corresponding to each interval to obtain a third ITM gain curve; and further, based on the third ITM gain curve, obtains a third inverse tone mapping curve.

[0229] The following example takes the interval [x1, x2] as an example to introduce the process of calculating the ITM gain curve corresponding to this interval:

[0230] First, find the slopes corresponding to x1 and x2. Assuming that the mapped value of x1 is y1, calculate the mapped value of x2 is y2. Then, calculate the mapping curve for this interval. The method for calculating the y2 value at the endpoint is as follows:

[0231] Where f′(x) is the slope function of all grayscales, x1 and x2 are points on the line, so f′(x1) and f′(x1) are the slopes at x1 and x2.

[0232] Then, according to the slopes at x1 and x2 and the corresponding y values, the third-order ITM gain curve of the dequantization band in this interval can be calculated: y2=m*x 3 +n*x 2 +p*x+q

[0233] Wherein, x is the grayscale value of the pixel of the first image; m, n, p, and q are obtained by solving the equations of y1 and y2 corresponding to x1 and x2 and their corresponding slopes (derivatives).

[0234] For example, the third ITM gain curve corresponding to the current frame image shown in FIG11 may be as follows:

[0235] Furthermore, the third ITM gain curve corresponding to the current frame image shown in FIG11 can be shown as follows: itm3 =x*(f gain3 +1)*CC=1 / Err 1 / 2.2 Err′=min(Err1,Err2,Err_res) Err=max(Err′,1)

[0236] In the embodiment of the present application, a regularized histogram is obtained by regularizing the grayscale histogram, and a slope is calculated based on the regularized histogram, which can make the obtained slope more accurate and have a smaller error.

[0237] S514: The electronic device determines the second inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image.

[0238] In some embodiments, when the electronic device determines that the current frame image does not include a face and does not include a quantization band, it determines the above-mentioned second inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image; then, the first image is processed by the target inverse tone mapping curve. For details, please refer to the following steps S516 to S518.

[0239] S515: The electronic device determines the third inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image.

[0240] In some embodiments, when the electronic device determines that the current frame image does not include a face but includes a quantization band, the electronic device determines the third inverse tone mapping curve as the target inverse tone mapping curve corresponding to the current frame image; then, the first image is processed by the target inverse tone mapping curve. For details, please refer to the following steps S516 to S518.

[0241] S516: The electronic device performs time-domain filtering on the target inverse tone mapping curve corresponding to the current frame image based on the inverse tone mapping curve corresponding to the previous frame image.

[0242] In some embodiments, the electronic device may perform time-domain filtering on the target inverse tone mapping curve corresponding to the current frame image using the following formula: f = ρ * f itm +(1-ρ)*fitm_current

[0243] Where f is the inverse tone mapping curve obtained by time domain filtering; ρ is the preset weight; f itmis the target inverse tone mapping curve of the previous frame image after time domain filtering; fitm_current is the target inverse tone mapping curve corresponding to the current frame image.

[0244] In the first implementation, the current frame image includes a face, and the target inverse tone mapping curve corresponding to the current frame image is the first inverse tone mapping curve f itm1 .

[0245] In the second implementation, the current frame image does not include a face and a quantization band, and the target inverse tone mapping curve corresponding to the current frame image is the second inverse tone mapping curve f itm2 .

[0246] In the third implementation, when the current frame image does not include a face but includes a quantization band, the target inverse tone mapping curve corresponding to the current frame image is the third inverse tone mapping curve f itm3 .

[0247] In the fourth implementation, when the similarity between the current frame image and the previous frame image is greater than a preset threshold, the target inverse tone mapping curve fitm_current corresponding to the current frame image is the target inverse tone mapping curve f corresponding to the previous frame image. itm_last .

[0248] The embodiment of the present application can prevent the video presented on the preview interface from flickering by performing a time domain filtering operation on the target inverse tone mapping curve of the inverse tone mapping result.

[0249] S517: The electronic device processes the first image corresponding to the current frame image based on the inverse tone mapping curve obtained by time domain filtering to obtain a preview image corresponding to the current frame image.

[0250] In some embodiments, the electronic device may perform inverse tone mapping on the first image based on the inverse tone mapping curve obtained by time-domain filtering to obtain a third image. The inverse tone mapping process may include: tone mapping the R, G, and B values ​​of each pixel in the first image using the expansion coefficients in the inverse tone mapping curve obtained by time-domain filtering. Furthermore, color gamut mapping is performed on the third image to obtain a preview image.

[0251] For example, the second inverse tone mapping curve f itm2 =x*C*(f gain2 +1) as an example, where x is the grayscale of the first image. First, the second image is obtained by x*C; then, the second image*(f gain2 +1) to get the third image.

[0252] Exemplarily, the process of color gamut mapping may be: applying a color gamut mapping method to the R, G, and B values ​​of each pixel point in the third image, converting the color gamut of the third image to the color gamut corresponding to the screen (such as Rec709 to P3 color gamut), thereby obtaining the R, G, and B values ​​of the pixel points corresponding to each pixel point in the third image.

[0253] In other embodiments of the present application, the electronic device may set a maximum expansion coefficient (Err_Tele) for telephoto scenes (e.g., zoom > 10). In step S514, Err in the inverse tone mapping curve may be changed to Err3, where: Err3 = min(Err, Err_Tele). It should be understood that due to the high noise level in telephoto scenes, performing a high dynamic range expansion on images of telephoto scenes will result in significant image noise, poor image quality, and a poor user experience.

[0254] S518: The electronic device displays a preview image corresponding to the current frame image on the preview interface.

[0255] In some embodiments, after obtaining a preview image corresponding to the current frame image, the electronic device may display the preview image in a preview interface. For example, the preview interface may be as shown in user interface 12 of FIG1B and user interface 13 of FIG1C , where the images displayed in preview area 120 of user interface 12 and preview area 130 of user interface 13 are preview images.

[0256] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image display method, applied to an electronic device, characterized in that: The method comprises: detecting a first user operation instructing to display a preview interface; In response to the first user operation, at a first moment, displaying a first image on the preview interface, the first image not including a face, the first image including a first area and a second area, the average brightness of the first area being greater than the average brightness of the second area; At a second moment, a second image is displayed on the preview interface, wherein the second image includes a human face; the second image includes a third area and a fourth area, and the human face is located in the fourth area; and the average brightness of the third area is greater than the average brightness of the fourth area; The average brightness of the first region is greater than the average brightness of the third region, and the average brightness of the fourth region is greater than the average brightness of the second region.

2. The method according to claim 1, characterized in that The first image is obtained based on metadata information corresponding to the first image; the second image is obtained based on metadata information corresponding to the second image; the metadata information includes at least one of grayscale information and brightness information of the face area.

3. The method according to claim 2, characterized in that The grayscale information includes a first grayscale histogram and a second grayscale histogram; the first grayscale histogram is a grayscale histogram calculated from a grayscale histogram of an image at a first exposure level or a grayscale histogram of an image obtained by fusing multiple frames of images at different exposure levels; The second grayscale histogram is a histogram of an output image or a low dynamic range image synthesized from multiple frames of images with different exposure levels.

4. The method according to claim 3, characterized in that The method further comprises: Based on the grayscale information of the first image, compressing the brightness of a fused image corresponding to the first image to obtain a first image to be processed; the fused image corresponding to the first image is obtained by fusing multiple frames of images corresponding to the first image with different exposure levels, the first image to be processed including a first area to be processed and a second area to be processed; the average brightness of the first area to be processed is greater than the average brightness of the second area to be processed; Based on the grayscale information of the first image, the brightness of the first area to be processed is increased to obtain a second image to be processed; Color gamut mapping is performed on the second image to be processed to obtain the first image, wherein the first area to be processed corresponds to the first area, and the second area to be processed corresponds to the second area.

5. The method according to claim 4, characterized in that The method further comprises: Calculating a first expansion coefficient of the first image based on a first grayscale histogram corresponding to the first image and a second grayscale histogram corresponding to the first image; Based on a first grayscale histogram corresponding to the first image, a second expansion coefficient of the first image is calculated; the first expansion coefficient of the first image or the second expansion coefficient of the first image is used to adjust the brightness of the fused image corresponding to the first image.

6. The method according to claim 4 or 5, characterized in that The method further comprises: Regularization processing is performed on the second grayscale histogram corresponding to the first image to obtain a regularized histogram corresponding to the first image, and the regularized histogram corresponding to the first image is used to adjust the brightness of the fused image corresponding to the first image when predicting that the first image includes a quantization band.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Based on metadata information corresponding to the second image, compressing the brightness of a fused image corresponding to the second image to obtain a third image to be processed; the fused image corresponding to the second image is obtained by fusing multiple frames of images corresponding to the second image at different exposure levels; the third image to be processed includes a third area to be processed and a fourth area to be processed; the average brightness of the third area to be processed is greater than the average brightness of the fourth area to be processed; Based on the metadata information corresponding to the second image, the brightness of the third area to be processed is increased to obtain a fourth image to be processed; The fourth image to be processed is subjected to color gamut mapping to obtain the second image, the third area to be processed corresponds to the third area, and the fourth area to be processed corresponds to the fourth area.

8. The method according to claim 7, characterized in that The method further comprises: calculating a first expansion coefficient of the second image based on a first grayscale histogram corresponding to the second image and a second grayscale histogram corresponding to the second image; calculating a second expansion coefficient of the second image based on a second grayscale histogram corresponding to the second image; When it is determined that the second image includes a face, calculating a third expansion coefficient of the second image based on brightness information of a face region corresponding to the second image, where the third expansion coefficient is positively correlated with the brightness information of the face region; At least one of the first expansion coefficient of the second image, the second expansion coefficient of the second image, and the third expansion coefficient of the second image is used to perform brightness adjustment on a fused image corresponding to the second image.

9. The method according to claim 7 or 8, characterized in that The method further comprises: Regularization processing is performed on a second grayscale histogram corresponding to the second image to obtain a regularized histogram corresponding to the second image, and the regularized histogram corresponding to the second image is used to adjust the brightness of a fused image corresponding to the second image when predicting that the second image includes a quantization band.

10. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When it is determined that the similarity between the first image and the previous frame image of the first image is greater than a preset threshold, brightness adjustment is performed on the fused image corresponding to the first image based on brightness adjustment information obtained from metadata information corresponding to the previous frame image to obtain the first image; the fused image corresponding to the first image is obtained by fusing multiple frames of images corresponding to the first image with different exposure levels.

11. The method according to any one of claims 1 to 10, characterized in that A first expansion coefficient of the target image is obtained based on a first contrast and a second contrast, wherein the first contrast is obtained based on expectations of a highlight portion and a low-brightness portion in a first grayscale histogram corresponding to the target image; The second contrast is obtained based on expectations of a highlight portion and a low-brightness portion in a second grayscale histogram corresponding to the target image, where the target image is the first image or the second image.

12. The method according to any one of claims 1 to 11, characterized in that The second expansion coefficient of the target image is obtained based on the screen pixel mean and highlight coefficient of the target image. The highlight coefficient is used to represent the distribution of the highlight area in the target image. The screen pixel mean and highlight coefficient of the target image are obtained based on the second grayscale histogram corresponding to the target image. The target image is the first image or the second image.

13. An electronic device comprising a camera, a display screen, a memory, and one or more processors, characterized in that: The memory is used to store a computer program; the processor is used to call the computer program, so that the electronic device executes the method according to any one of claims 1 to 12.

14. A computer storage medium, characterized in that include: Computer instructions; when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 12.

Images