Image processing method and related equipment

Abstract

The embodiment of the invention provides an image processing method and related equipment. The method comprises the following steps: performing tone mapping on an SDR image to obtain an HDR image; acquiring a pixel value of each pixel point in the HDR image; pixel points in the HDR image are grouped according to a preset pixel point grouping strategy, and brightness information of each group of pixel points is acquired; obtaining depth information of each pixel point in the HDR image; dividing the HDR image into a plurality of image areas, wherein the minimum image area is a pixel point; determining target display screen brightness based on the current display screen brightness and the maximum display screen brightness; determining brightness adjustment coefficients and depth adjustment coefficients of pixel values of pixel points in the plurality of image areas, and adjusting the pixel values of the pixel points in the plurality of image areas to obtain a target HDR image; and adjusting the display screen brightness to the target display screen brightness, and displaying the target HDR image. According to the embodiment of the invention, the image can be subjected to high-dynamic range expansion according to the depth information, and the image main body is effectively highlighted.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, which belongs to image processing technology, and in particular to an image processing method and related equipment. Background Art

[0002] With the development of terminal technology, smart terminal devices such as smartphones and personal computers all support high dynamic range (HDR) image display. By performing tone mapping on standard dynamic range (SDR) images, HDR images can be generated to expand the dynamic range of the image. However, in the tone mapping process, usually only the pixel information and brightness information of the image are converted, and the depth information of the image cannot be converted, and the main part of the image cannot be highlighted, resulting in poor display effect of HDR images. Summary of the invention

[0003] In view of the above, it is necessary to provide an image processing method and related equipment to solve the problem that the depth information of the image cannot be converted during the tone mapping process and the main part of the image cannot be highlighted, resulting in poor display effect of the HDR image.

[0004] In a first aspect, the present application provides an image processing method, which is applied to an electronic device, the method comprising: obtaining an SDR image, performing tone mapping on the SDR image, and obtaining an HDR image; obtaining a pixel value of each pixel in the HDR image; grouping the pixels in the HDR image according to a preset pixel grouping strategy, and obtaining brightness information of each group of pixels; obtaining depth information of each pixel in the HDR image; determining a brightness mean according to the brightness information, and dividing the HDR image into multiple image areas according to the brightness mean; determining a target display screen brightness based on a current display screen brightness and a maximum display screen brightness; determining a brightness adjustment coefficient of the pixel values ​​of the pixels in the multiple image areas, determining a depth adjustment coefficient of the pixel value of each pixel according to the depth information of each pixel in the multiple image areas, and adjusting the pixel values ​​of the pixels in the multiple image areas according to the brightness adjustment coefficient and the depth adjustment coefficient to obtain a target HDR image; adjusting the display screen brightness of the electronic device to the target display screen brightness, and displaying the target HDR image.

[0005] Through the above technical solution, in the process of displaying HDR images, the pixel value of the highlight area of ​​the HDR image can be increased or maintained, and the pixel value of the dark area can be reduced according to the actual brightness of the HDR image and the brightness capability of the terminal device, so that the highlight area of ​​the HDR image is brighter, and the dark area of ​​the HDR image is darker or close to the original brightness of the dark area of ​​the HDR image, so that the highlight area and the dark area have a higher contrast, thereby expanding the dynamic range of the HDR image, making the HDR image displayed by the electronic device closer to the real scene, and at the same time highlighting the main part of the image, so that the image has a better sense of layering, thereby further improving the display effect of the HDR image.

[0006] In a possible implementation, the method further includes: encoding the brightness information, the depth information, and the pixel value into the HDR image, respectively, to obtain an image file; and in response to a user's operation of viewing the image file, decoding the image file to obtain the brightness information, the depth information, and the pixel value of the HDR image.

[0007] Through the above technical solution, the brightness information, depth information and pixel information of the image are encoded into the HDR image, so that when HDR is displayed on the same device or different devices, the brightness information, depth information and pixel information can be obtained by decoding, and the high dynamic range of the HDR image is expanded and displayed according to the brightness information, depth information and pixel information, thereby optimizing the display effect of the HDR image.

[0008] In a possible implementation, respectively encoding the brightness information, the depth information and the pixel value into the HDR image to obtain an image file includes: encoding the pixel value of each pixel to obtain first encoded data; encoding the brightness information to obtain second encoded data; encoding the depth information to obtain third encoded data; and writing the second encoded data and the third encoded data into the first encoded data to obtain the image file.

[0009] Through the above technical solution, the encoded data corresponding to the brightness information and depth information of the image are written into the encoded data corresponding to the pixel information, so that when the HDR image is displayed, the brightness information, depth information and pixel information of the HDR image can be obtained by decoding.

[0010] In a possible implementation, obtaining brightness information of each group of pixels includes: obtaining a target exposure image corresponding to a target exposure parameter, wherein if the target exposure parameter is any exposure parameter among multiple exposure parameters, the target exposure image is an image corresponding to the any exposure parameter; if the target exposure parameter is a parameter determined according to at least two exposure parameters among the multiple exposure parameters, the target exposure image is an image obtained by fusing the images corresponding to the at least two exposure parameters; obtaining a brightness value of each pixel in each group of pixels in the target exposure image; and taking an average of the brightness values ​​of all pixels in each group of pixels in the target exposure image as the brightness information of the group of pixels under the target exposure parameter.

[0011] Through the above technical solution, the brightness information of the image can be accurately determined according to the exposure parameters of the image.

[0012] In a possible implementation, the depth information of each pixel in the HDR image is the distance between the point corresponding to each pixel in the scene and the camera of the electronic device, and obtaining the depth information of each pixel in the HDR image includes: inputting the HDR image into a preset deep learning model, and outputting a disparity map corresponding to the HDR image through the preset deep learning model; calculating the disparity of each pixel according to the disparity map, and obtaining the depth information of each pixel according to the disparity of each pixel.

[0013] Through the above technical solution, the depth information of the HDR image can be accurately determined by monocular vision estimation, and the calculation efficiency of the depth information is improved.

[0014] In a possible implementation, determining a brightness mean according to the brightness information, and dividing the HDR image into multiple image areas according to the brightness mean, includes: determining the brightness mean according to the brightness information of all pixels in the HDR image; determining a first threshold according to the brightness mean, the first threshold being greater than or equal to the brightness mean; dividing the pixels whose brightness information is less than or equal to the first threshold into the first area of ​​the HDR image; and dividing the pixels whose brightness information is greater than the first threshold into the second area of ​​the HDR image.

[0015] Through the above technical scheme, the HDR image is divided into a highlight image area and a dark image area according to the brightness information of the image, and the high dynamic range expansion display of the HDR image can be performed according to the highlight image area and the dark image area, thereby effectively improving the efficiency of the high dynamic range expansion display of the HDR image and optimizing the high dynamic range expansion display effect of the HDR image.

[0016] In a possible implementation, determining a brightness mean according to the brightness information, and dividing the HDR image into multiple image areas according to the brightness mean, further includes: determining a second threshold and a third threshold according to the brightness mean, the second threshold being less than the brightness mean, and the third threshold being greater than or equal to the brightness mean; dividing pixels whose brightness information is less than the second threshold into the first area of ​​the HDR image; dividing pixels whose brightness information is greater than or equal to the second threshold and less than the third threshold into the third area of ​​the HDR image; and dividing pixels whose brightness information is greater than or equal to the third threshold into the second area of ​​the HDR image.

[0017] Through the above technical scheme, the HDR image is divided into a highlight image area, a transition image area and a dark image area according to the brightness information of the image, and the high dynamic range expansion display of the HDR image can be performed according to the highlight image area, the transition image area and the dark image area, thereby effectively improving the efficiency of the high dynamic range expansion display of the HDR image and optimizing the high dynamic range expansion display effect of the HDR image.

[0018] In a possible implementation, the smallest image area is a pixel. Through the above technical solution, each pixel in the HDR image is regarded as an image area, so that the high dynamic range of the HDR image is extended and displayed according to the pixel information, brightness information and depth information of each pixel, which can optimize the high dynamic range of the HDR image.

[0019] In a possible implementation, the maximum display screen brightness is the maximum brightness supported by the display screen of the electronic device, and determining the target display screen brightness based on the current display screen brightness and the maximum display screen brightness includes: taking a preset multiple of the current display screen brightness as a brightness threshold; if the brightness threshold is less than or equal to the maximum display screen brightness, determining the brightness threshold as the target display screen brightness; or if the brightness threshold is greater than the maximum display screen brightness, determining the maximum display screen brightness as the target display screen brightness.

[0020] Through the above technical solution, the actual maximum display screen brightness of the electronic device can be determined, so as to perform high dynamic range expansion display of the HDR image according to the maximum display screen brightness, and effectively optimize the high dynamic range expansion display effect of the HDR image.

[0021] In a possible implementation, determining the brightness adjustment coefficients of the pixel values ​​of the pixel points in the multiple image areas includes: determining the first brightness adjustment coefficient of the pixel points in the first area as the ratio of the current display brightness to the brightness threshold; and determining the second brightness adjustment coefficient of the pixel points in the second area as the sum of 1 and the normalized brightness value of the pixel points.

[0022] Through the above technical solution, the brightness adjustment coefficients of different image areas can be set, so as to perform high dynamic range expansion display of HDR images according to the brightness adjustment coefficients, and effectively optimize the high dynamic range expansion display effect of HDR images.

[0023] In a possible implementation, determining the depth adjustment coefficient of the pixel value of each pixel point based on the depth information of each pixel point in multiple image areas includes: normalizing the depth information of each pixel point in the multiple image areas; when the depth information of the pixel point is less than or equal to a preset depth threshold, determining that the depth adjustment coefficient of the pixel value of the pixel point is a first depth adjustment coefficient; when the depth information of the pixel point is greater than the preset depth threshold, determining that the depth adjustment coefficient of the pixel value of the pixel point is a second depth adjustment coefficient, wherein the first depth adjustment coefficient and the second depth adjustment coefficient are both constants, and the first depth adjustment coefficient is greater than the second depth adjustment coefficient.

[0024] Through the above technical scheme, the corresponding depth adjustment coefficient can be set according to the depth information of the pixel points in the HDR image and the preset depth threshold, and the high dynamic range expansion display of the HDR image can be performed according to the depth adjustment coefficient. The brightness of the pixel points with a smaller depth of field can be relatively large, and the brightness of the pixel points with a larger depth of field can be relatively small, thereby highlighting the main part of the HDR image and effectively optimizing the high dynamic range expansion display effect of the HDR image.

[0025] In a possible implementation, determining the depth adjustment coefficient of the pixel value of each pixel point based on the depth information of each pixel point in multiple image areas includes: normalizing the depth information of each pixel point in the multiple image areas; and converting the depth information of each pixel point into the depth adjustment coefficient of the pixel point according to a preset inverse proportional mapping function.

[0026] Through the above technical scheme, the corresponding depth adjustment coefficient can be set according to the depth information of the pixel points in the HDR image and the inverse proportional mapping function, and the high dynamic range expansion display of the HDR image can be performed according to the depth adjustment coefficient. The brightness of the pixel points with a smaller depth of field can be relatively large, and the brightness of the pixel points with a larger depth of field can be relatively small, thereby highlighting the main part of the HDR image and effectively optimizing the high dynamic range expansion display effect of the HDR image.

[0027] In a possible implementation, adjusting the pixel values ​​of pixels in the multiple image areas according to the brightness adjustment coefficient and the depth adjustment coefficient to obtain a target HDR image includes: multiplying the brightness adjustment coefficient and the depth adjustment coefficient of each pixel in the multiple image areas with the pixel value to obtain an updated pixel value for each pixel in the multiple image areas.

[0028] Through the above technical solution, the pixel value of the pixel is adjusted according to the brightness adjustment coefficient and the depth adjustment coefficient of the pixel, which effectively increases the dynamic range of the HDR image when displayed, and highlights the subject in the HDR image, realizes the sense of light and shadow in depth, and effectively optimizes the high dynamic range expansion display effect of the HDR image.

[0029] In a second aspect, the present application provides an electronic device, comprising a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device performs the above-mentioned image processing method.

[0030] In a third aspect, the present application provides a chip coupled to a memory in an electronic device, wherein the chip is used to control a processor of the electronic device to execute the above-mentioned image processing method.

[0031] In a fourth aspect, the present application provides a computer storage medium storing program instructions, which, when executed on an electronic device, enables a processor of the electronic device to execute the above-mentioned image processing method.

[0032] In addition, the technical effects brought about by the second to fourth aspects can be found in the descriptions related to the methods of each design in the above method part, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a schematic diagram of a tone mapping curve provided in an embodiment of the present application.

[0034] Figure 2 It is a software architecture diagram of an electronic device provided in one embodiment of the present application.

[0035] Figure 3 It is a flowchart of an image processing method provided by an embodiment of the present application.

[0036] Figure 4 It is a schematic diagram of a shooting preview interface provided by an embodiment of the present application.

[0037] Figure 5 It is a schematic diagram of an image provided by an embodiment of the present application.

[0038] Figure 6 It is a schematic diagram of a pixel grouping strategy provided in an embodiment of the present application.

[0039] Figure 7 It is a schematic diagram of another pixel grouping strategy provided in an embodiment of the present application.

[0040] Figure 8 It is a schematic diagram of the process of converting an SDR image into an HDR image provided by an embodiment of the present application.

[0041] Fig. 9 is a schematic diagram of another image provided by an embodiment of the present application.

[0042] Fig.10 is a schematic diagram of another image provided by an embodiment of the present application.

[0043] Fig.11 is a schematic diagram of another image provided by an embodiment of the present application.

[0044] Fig.12 It is a flowchart of an image processing method provided by another embodiment of the present application.

[0045] Fig.13 This is a hardware architecture diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The terms "first" and "second" involved in an embodiment of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in an embodiment of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" refers to one or more. "Multiple" refers to two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c. In the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0048] In order to make the description of each embodiment of the present application clear and concise, a brief introduction to the relevant concepts or technologies is first given:

[0049] Dynamic Range (DR): is the ratio of the maximum and minimum values ​​of a variable signal (such as sound or light). In the field of imaging, dynamic range refers to the range of brightness in a scene that a camera can capture, which can be expressed as the ratio between the highest and lowest brightness that a camera can record in a single frame, that is, the ratio of the maximum brightness value to the minimum brightness value of a pixel in the image. High Dynamic Range (HDR) imaging, compared to Low Dynamic Range (LDR) imaging or Standard Dynamic Range (SDR) imaging, has a larger dynamic range (i.e., a greater difference between light and dark), can more accurately reflect the range of brightness changes in the real world from direct sunlight to the darkest shadows, has a wider color range and richer image details, and can better reflect the visual effects in a real environment.

[0050] HDR: Indicates that the ratio between the highest and lowest values ​​of the signal (i.e., the dynamic range) is greater than or equal to the preset first threshold. HDR is reflected in the image, which can be understood as the presence of highlight areas and dark areas in the image, and the ratio between the brightness value of the highlight area and the brightness value of the dark area is greater than or equal to the first threshold. Such an image is an HDR image.

[0051] SDR: Indicates that the ratio between the highest and lowest values ​​of the signal (i.e., the dynamic range) is less than the preset first threshold and greater than or equal to the preset second threshold, where the second threshold is less than the first threshold. SDR is reflected in the image, which can be understood as the presence of highlight areas and dark areas in the image. The ratio between the brightness value of the highlight area and the brightness value of the dark area is less than the preset first threshold and greater than or equal to the preset second threshold. Such an image is an SDR image.

[0052] Exposure Value (EV) value: reflects the exposure level of the image. By adjusting the exposure parameters of the camera, the image can have different EVs. Among them, the exposure parameters refer to the parameters used to affect the exposure amount of the camera, and the exposure parameters include but are not limited to: aperture, shutter, exposure time and sensitivity. In an embodiment of the present application, the exposure value of normal exposure is expressed as EV0, the exposure value of underexposure is expressed as EV-, and the exposure value of overexposure is expressed as EV+. The exposure value EV0×2n is expressed as EVn. For example, EV-1 means that the exposure value is half of EV0, and EV-2 means that the exposure value is half of EV-1; for another example, EV1 means that the exposure value is twice that of EV0, and EV2 means that the exposure value is twice that of EV1.

[0053] Electronic devices can use cameras to capture multiple images of the same scene with different exposure values, fuse the multiple images to generate a single image, and perform tone mapping on the single image to generate an HDR image. However, when such HDR images are displayed on a display device, they are still displayed in a standard dynamic range, because such HDR images are generally 8-bit images with a dynamic range of approximately 255:1, while current display devices can generally provide a higher dynamic range. In other words, such HDR images are actually still SDR images.

[0054] See also Figure 1 FIG. 1 is a schematic diagram of a tone mapping curve provided in an embodiment of the present application. After acquiring an SDR image, the electronic device can convert the SDR image into an HDR image by tone mapping. For example, by Figure 1 The S-shaped tone mapping curve (i.e., Sigmoid nonlinear function) shown in FIG. 1 converts the pixel value of each pixel in the SDR image. The calculation formula for the pixel value conversion is:

[0055]

[0056] Among them, x is the initial pixel value of the pixel, and y is the pixel value after HDR conversion.

[0057] In the prior art, the HDR image can be displayed with high dynamic range expansion according to the brightness information of the HDR image. For example, a brightness adjustment coefficient is determined according to the brightness information of the HDR image, and the pixel value of the pixel point in the HDR image is multiplied by the corresponding brightness adjustment coefficient to obtain the updated pixel value of each pixel point. However, in the process of high dynamic range expansion display, if the pixel value and brightness information are converted only according to the brightness information, the main part of the image cannot be highlighted, the image has no sense of hierarchy, and the display effect of the HDR image is poor.

[0058] In order to solve the above problems, the embodiment of the present application provides an image processing method. In the process of displaying an HDR image, the pixel value of the highlight area of ​​the HDR image can be increased or maintained, and the pixel value of the dark area can be reduced according to the actual brightness of the HDR image and the brightness capability of the terminal device, so that the highlight area of ​​the HDR image is brighter, and the dark area of ​​the HDR image is darker or close to the original brightness of the dark area of ​​the HDR image, so that the highlight area and the dark area have a higher contrast, thereby expanding the dynamic range of the HDR image, making the HDR image displayed by the electronic device closer to the real scene, and at the same time highlighting the main part of the image, so that the image has a better sense of layering, thereby further improving the display effect of the HDR image.

[0059] See also Figure 2 As shown, it is a software architecture diagram of the electronic device provided in the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. For example, the Android system is from top to bottom an application layer 101, a framework layer 102, an Android runtime (Android runtime) and a system library 103, a hardware abstraction layer 104, a kernel layer 105, and a hardware layer 106.

[0060] The application layer 101 may include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, device control service and other applications.

[0061] The framework layer 102 provides an application programming interface (API) and a programming framework for the application programs of the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0062] Among them, the window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to the application. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloads, message reminders, etc. The notification manager can also be a notification that appears in the top status bar of the system in the form of an icon or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0063] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library contains two parts: one is the function that the Java language needs to call, and the other is the Android core library.

[0064] The application layer 101 and the framework layer 102 run in a virtual machine. The virtual machine executes the Java files of the application layer and the framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0065] The system library 103 may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (eg, OpenGL ES), a 2D graphics engine (eg, SGL), and the like.

[0066] Among them, the surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0067] The hardware abstraction layer 104 runs in the user space, encapsulates the kernel layer driver, and provides a calling interface to the upper layer.

[0068] The kernel layer 105 is a layer between hardware and software, and includes at least a display driver, a touch driver, an audio driver, and a sensor driver.

[0069] The kernel layer 105 is the core of the operating system of the electronic device. It is the first layer of software expansion based on hardware, provides the most basic functions of the operating system, is the basis of the operating system, is responsible for managing the system's processes, memory, device drivers, files and network systems, and determines the performance and stability of the system. For example, the kernel layer can determine the operating time of an application on a certain part of the hardware.

[0070] The kernel layer 105 includes programs closely related to the hardware, such as interrupt handlers, device drivers, etc., and also includes basic, common, and high-frequency modules, such as clock management modules, process scheduling modules, etc., and also includes key data structures. The kernel layer can be set in the processor or solidified in the internal memory.

[0071] The hardware layer 106 includes the hardware of the electronic device, such as a display screen, buttons, a camera, etc.

[0072] See also Figure 3 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present application. The method is applied in an electronic device, and the image processing method includes:

[0073] S101, acquiring an SDR image, and performing tone mapping on the SDR image to obtain an HDR image.

[0074] In one embodiment of the present application, the electronic device can acquire multiple images and fuse the acquired multiple images to generate an SDR image. The multiple images are images captured by the camera of the electronic device under different exposure parameters in the same scene, and the multiple images correspond to the multiple exposure parameters one by one, that is, each image corresponds to one exposure parameter. Figure 4As shown, it is a schematic diagram of a camera application interface provided by an embodiment of the present application. After the user opens the camera application, the electronic device can display a shooting interface 301, and the shooting interface 301 includes a shooting control 302. In response to the user clicking the shooting control 302, the camera captures multiple images. In one embodiment of the present application, the electronic device may include multiple cameras, and the multiple cameras may respectively shoot with different exposure parameters to obtain multiple images. In other embodiments of the present application, the electronic device may also use a single camera to continuously shoot multiple images according to different exposure parameters, and the shooting time of each frame of the image remains synchronized or nearly synchronized.

[0075] See also Figure 5 As shown, it is a schematic diagram of multiple images provided by an embodiment of the present application. For example, the multiple images may include Figure 5 Image 1 and Image 2 in the figure. Image 1 is taken by the camera based on exposure parameter 1, and Image 2 is taken by the camera based on exposure parameter 2. The EV value corresponding to Image 1 is greater than the EV value corresponding to Image 2. Image 1 is overexposed, and the highlight area in Image 1 is distorted due to overexposure, while the details and contours of the dark area can be clearly seen due to the compensation of the exposure value. Image 2 is underexposed, and the highlight area in Image 2 has a clearer contour, but the dark area is blurred due to lack of exposure.

[0076] Figure 5 Taking images under exposure parameters EV- and EV+ as an example, in fact, the electronic device can also obtain images under more exposure conditions, such as images under EV0, EV-1, EV-2, EV+1, EV+2, etc., and the embodiments of the present application do not make specific limitations here.

[0077] By fusing multiple images to generate an SDR image, the quality of the SDR image can be improved. In another embodiment of the present application, the SDR image can also be a single image captured by a camera of an electronic device under an exposure parameter.

[0078] In one embodiment of the present application, tone mapping may be global tone mapping or local tone mapping. Global tone mapping may be performed by histogram mapping, gamma correction mapping, sigmoid nonlinear function mapping, etc. to globally process the image. Global tone mapping may well preserve global contrast. Local tone mapping may divide the HDR image into regions, and then process each divided region in a manner similar to global tone mapping, which may preserve the relative contrast between adjacent regions, thereby preserving local details.

[0079] For example, the HDR image obtained by tone mapping the SDR image can be found in Figure 5As shown. HDR images are more accurate than single images (e.g. Figure 5 For images 1 and 2 in the figure, both the dark and highlight areas have more obvious details and contours, and have a higher dynamic range.

[0080] S102: Obtain the pixel value of each pixel in the HDR image.

[0081] In one embodiment of the present application, according to the type of image sensor of the camera that captures the image, the pixel value of the pixel point can be an RGB value, an RGGB value, an RGBW value, an RYYB value, etc. Among them, RGB means that the color of each pixel includes three color components: red (red, R), green (green, G) and blue (blue, B). RGGB means that the color of each pixel includes four color components: red, green, green and blue. RGBW means that the color of each pixel includes four components: red, green, blue and white (white, W). RYYB means that the color of each pixel includes four components: red, yellow (yellow, Y), yellow and blue. Among them, the pixel value of each pixel point can be obtained by an image sensor.

[0082] S103: grouping pixels of the HDR image according to a preset pixel grouping strategy, and obtaining brightness information of each group of pixels.

[0083] In one embodiment of the present application, when an SDR image is generated by fusion of multiple images, the target exposure parameter may be one of multiple exposure parameters corresponding to the multiple images, or a parameter obtained by combining at least two of the multiple exposure parameters, and may be set specifically according to actual shooting requirements. For example, if the details of the dark area of ​​the image are required to be clearer, the target exposure parameter may be larger; if the highlight area of ​​the image is required to be clearer and not overexposed, the target exposure parameter may be smaller; if the image is required to have a clearer light-dark contrast and not overexposed or underexposed, the target exposure parameter may be moderate. The target exposure parameter in the embodiment of the present application is a parameter obtained by combining multiple exposure parameters, and the target exposure parameter can be used as the exposure parameter corresponding to the HDR image. When the SDR image is a single image taken by a camera, the target exposure parameter is the exposure parameter corresponding to the single image.

[0084] For example, multiple exposure parameters include EV-1, EV0, EV+1, and EV+2. The electronic device can directly take any one of EV-1, EV0, EV+1, and EV+2 as the target exposure parameter. Alternatively, if in a certain scene, in order to obtain the details of the dark area in the image, the EV needs to be between EV+1 and EV+2, then the electronic device can change the exposure time, sensitivity and other parameters to make the EV between EV+1 and EV+2, and obtain the corresponding target exposure parameter.

[0085] In other embodiments of the present application, multiple exposure parameters may also be input into a pre-trained neural network model to obtain target exposure parameters.

[0086] The image corresponding to the target exposure parameter is the target exposure image. The target exposure image can be understood as the exposure image that should be obtained when the electronic device shoots with the target exposure parameter. If the target exposure parameter is one of multiple exposure parameters, the target exposure image may be the image shot by the electronic device under the target exposure parameter. If the target exposure parameter is a parameter obtained by combining at least two exposure parameters among multiple exposure parameters, the target exposure image may be an image obtained by fusing the images corresponding to the at least two exposure parameters. For example, the multiple exposure parameters include EV0, EV-, and EV+, and correspond to EV0 image, EV- image, and EV+ image, respectively. If the target exposure parameter is EV0, the EV0 image is used as the target exposure image; if the target exposure parameter is a parameter obtained by combining EV- and EV+, the image obtained by fusing the EV- image and the EV+ image is used as the target exposure image. For another example, if the target exposure parameter is a parameter obtained by combining the above multiple exposure parameters, the SDR image obtained by fusing the multiple images corresponding to the multiple exposure parameters is used as the target exposure image.

[0087] The brightness information of each group of pixels under the target exposure parameters can be the brightness information of each group of pixels in the target exposure image. For example, the camera captures EV0 images, EV- images, and EV+ images, and generates an SDR image based on the fusion of the EV0 images, EV- images, and EV+ images, and obtains an HDR image after tone mapping the SDR image. If EV0 is used as the target exposure parameter, the brightness information of a group of pixels in the HDR image under the target exposure parameters is the brightness information of the group of pixels in the EV0 image.

[0088] In an embodiment of the present application, the preset pixel grouping strategy can be set according to actual needs. For example, the preset pixel grouping strategy can divide a pixel into a group, or can divide a pixel matrix composed of n×m pixels into a group, for example, n×m is 2×2 or 3×2, that is, a pixel matrix composed of 2×2 or 3×2 pixels can be divided into a group, and the embodiment of the present application is not specifically limited here.

[0089] If a pixel is divided into a group, the brightness information of each group of pixels under the target exposure parameters is the brightness value of the pixel in the target exposure image. If a pixel matrix consisting of n×m pixels is divided into a group, the brightness information of the group of pixels under the target exposure parameters can be the average brightness value of all pixels in the group of pixels in the target exposure image, or the average brightness value remaining after removing the outliers of the brightness values ​​of all pixels, which is not specifically limited in the embodiments of the present application.

[0090] See also Figure 6 FIG. 1 is a schematic diagram of a pixel grouping strategy provided by an embodiment of the present application. Figure 6 In , each pixel is regarded as a pixel group. Among them, the pixel value of pixel A1 in the target exposure image is R1G1B1, the pixel value of pixel A2 in the HDR image is R2G2B2, and the coordinates of pixel A1 and pixel A2 in the two images are the same. The brightness information of pixel A2 under the target exposure parameters is the brightness information of pixel A2 in the target exposure image, that is, the brightness information of pixel A1. Therefore, the brightness information L1 of pixel A1 can be calculated by the pixel values ​​R1, G1, and B1 of pixel A1, and the brightness information L1 is used as the brightness information of pixel A2 under the target exposure parameters.

[0091] In one embodiment of the present application, the brightness value L and the pixel value RGB can satisfy the calculation formula: L = α*R+β*G+γ*B. For example, α is 0.299, β is 0.587, γ is 0.114, and the pixel value of pixel point A1 is (R100, G100, B100). It can be determined that the brightness information L1 of pixel point A1 is 0.299R+0.587G+0.114B=29.9+58.7+11.4=100. Therefore, the brightness information of pixel point A2 under the target exposure parameters is 100.

[0092] See also Figure 7 FIG. 1 is a schematic diagram of another pixel grouping strategy provided by an embodiment of the present application. Figure 7In , both the HDR image and the target exposure image include 8×6 pixels, which are divided into 16 groups, each of which includes 3×1 pixels. Among them, pixel group 1 in the target exposure image includes pixel M1, pixel M2, and pixel M3, and pixel group 2 in the HDR image includes pixel N1, pixel N2, and pixel N3. The coordinates of pixel M1 and pixel N1 are the same in the two images, the coordinates of pixel M2 and pixel N2 are the same in the two images, and the coordinates of pixel M3 and pixel N3 are the same in the two images. The pixel values ​​of pixel M1 are R3, G3, B3, and W3, the pixel values ​​of pixel M2 are R4, G4, B4, and W4, and the pixel values ​​of pixel M3 are R5, G5, B5, and W5. The pixel values ​​of pixel point N1 are R6, G6, B6, and W6, the pixel values ​​of pixel point N2 are R7, G7, B7, and W7, and the pixel values ​​of pixel point N3 are R8, G8, B8, and W8. The brightness information of pixel group 2 under the target exposure parameters is the brightness information of pixel group 2 in the target exposure image, that is, the brightness information of pixel group 1. The electronic device can calculate the brightness value L3 according to the pixel value (R3, G3, B3, and W3) of pixel point M1, calculate the brightness value L4 according to the pixel value (R4, G4, B4, and W4) of pixel point M2, and calculate the brightness value L5 according to the pixel value (R5, G5, B5, and W5) of pixel point M3, and then use the average value L6 of the brightness value L3, the brightness value L4, and the brightness value L5 as the brightness information of pixel group 2 under the target exposure parameters.

[0093] S104: Obtain depth information of each pixel in the HDR image.

[0094] In one embodiment of the present application, the depth information of each pixel in the HDR image is the distance between the point in the scene corresponding to each pixel and the camera. The depth information of each pixel in the HDR image can be calculated by a monocular depth estimation method or a binocular depth estimation method.

[0095] In one embodiment of the present application, a monocular depth estimation method includes inputting an HDR image into a preset deep learning model, outputting a disparity map corresponding to the HDR image through the preset deep learning model, obtaining the disparity D of each pixel according to the disparity map, and calculating the depth Z of each pixel according to the disparity D of each pixel, and the calculation formula is: Z = f*b / D, where f is the focal length of the camera and b is the baseline distance of the camera. The preset deep learning model can be an autoencoder or other suitable neural network model.

[0096] In one embodiment of the present application, a binocular depth estimation method includes capturing two images of a scene corresponding to an HDR image through two cameras, performing feature matching on the two images, determining multiple groups of matching pixel points in the two images, calculating the distances between points in the scene corresponding to each group of matching pixel points, obtaining a disparity D for each pixel point, and calculating a depth Z for each pixel point based on the disparity D for each pixel point, wherein the calculation formula is: Z=f*b / D, where f is the focal length of the camera and b is the baseline distance of the camera (i.e., the distance between the two cameras).

[0097] S105, determining a brightness mean according to the brightness information, and dividing the HDR image into a plurality of image regions according to the brightness mean.

[0098] In one embodiment of the present application, the brightness mean is the average of the brightness values ​​(i.e., brightness information) of all pixels, which is used to reflect the true brightness level of the HDR image. For example, if the bit width of the brightness information is 8 bits, after the brightness information is digitized, the brightness value ranges from 0 to 255. The brightness mean can be obtained by digitizing all the brightness values ​​and taking the average value.

[0099] In one embodiment of the present application, the plurality of image regions at least include a first region and a second region. The first region is a dark region in the HDR image, and the second region is a highlight region in the HDR image.

[0100] In one embodiment of the present application, a first threshold value can be determined according to the brightness mean, and the HDR image can be divided into a first area and a second area according to the first threshold value. The first threshold value is greater than or equal to the brightness mean value, and the greater the brightness mean value, the greater the first threshold value. The first threshold value can be linearly or nonlinearly related to the brightness mean value, and is not specifically limited here. Pixels with brightness values ​​less than or equal to the first threshold value can be divided into the first area, and pixels with brightness values ​​greater than the first threshold value can be divided into the second area. In this way, the first area can contain pixels in the darker area of ​​the HDR image, and the second area can contain pixels in the brighter area of ​​the HDR image.

[0101] For example, the first threshold may be 165. If the brightness value of pixel 1 is determined to be 100 according to the brightness information, since 100<165, the pixel 1 is divided into the first area. If the brightness value of pixel 2 is determined to be 188 according to the brightness information, since 188>165, the pixel 2 is divided into the second area.

[0102] In another embodiment of the present application, the HDR image may be divided into a first area, a second area and a third area, wherein the third area is a transition area between the first area and the second area.

[0103] In another embodiment of the present application, the second threshold and the third threshold can be determined according to the brightness mean, and the HDR image can be divided into the first area, the second area and the third area based on the second threshold and the third threshold. Among them, the second threshold is less than the brightness mean, the third threshold is greater than or equal to the brightness mean, and the greater the brightness mean, the greater the second threshold and the third threshold. The second threshold and the third threshold can be linearly or nonlinearly related to the brightness mean, and are not specifically limited here. Pixels with brightness values ​​less than the second threshold can be divided into the first area, pixels with brightness values ​​greater than or equal to the second threshold and less than the third threshold can be divided into the third area, and pixels with brightness values ​​greater than or equal to the third threshold can be divided into the second area. In this way, the first area is a darker area in the HDR image, the second area is a brighter area in the HDR image, and the third area is a transition area between the first area and the second area, with moderate brightness.

[0104] The above only shows two ways of dividing the image area. The embodiment of the present application can also use a more sophisticated partitioning method to divide the HDR image into more image areas, where the smallest image area can be a pixel point, that is, each pixel point in the HDR image can be divided into an image area.

[0105] S106: Determine a target display screen brightness based on the current display screen brightness and the maximum display screen brightness.

[0106] In one embodiment of the present application, in order to make the HDR image look brighter, the display brightness can be increased. The maximum display brightness is the maximum brightness supported by the display of the electronic device. The brightness threshold can be first determined based on the current display brightness. For example, a preset multiple of the current display brightness can be used as the brightness threshold. The preset multiple can be any value greater than 1, such as 3. Then, the brightness threshold is compared with the maximum display brightness. If the brightness threshold is less than or equal to the maximum display brightness, the brightness threshold is determined to be the target display brightness. If the brightness threshold is greater than the maximum display brightness, the maximum display brightness is determined to be the target display brightness.

[0107] In another embodiment of the present application, the target display screen brightness may also be fixed as the maximum display screen brightness, that is, no matter what the current display screen brightness is, the maximum display screen brightness is always used as the target display screen brightness.

[0108] S107, determining a brightness adjustment coefficient of the pixel values ​​of the pixel points in the multiple image areas, determining a depth adjustment coefficient of the pixel value of each pixel point according to the depth information of each pixel point in the multiple image areas, and adjusting the pixel values ​​of the pixel points in the multiple image areas according to the brightness adjustment coefficient and the depth adjustment coefficient to obtain a target HDR image.

[0109] In one embodiment of the present application, when multiple image areas include a first area and a second area, the brightness adjustment coefficient of the pixel value of the pixel point in the first area is the first brightness adjustment coefficient, and the brightness adjustment coefficient of the pixel value of the pixel point in the second area is the second brightness adjustment coefficient.

[0110] In an embodiment of the present application, when the first area is a dark area and the second area is a highlight area, the first adjustment coefficient may be less than 1 and greater than 0, and the second adjustment coefficient may be greater than or equal to 1.

[0111] In another embodiment of the present application, when the multiple image areas include a first area, a second area, and a third area, the brightness adjustment coefficient of the pixel value of the pixel point in the first area is the first brightness adjustment coefficient, the brightness adjustment coefficient of the pixel value of the pixel point in the second area is the second brightness adjustment coefficient, and the brightness adjustment coefficient of the pixel value of the pixel point in the third area is the third brightness adjustment coefficient. When the first area is a dark area, the second area is a highlight area, and the third area is a transition area, the first adjustment coefficient may be less than 1 and greater than 0, the second adjustment coefficient may be greater than or equal to 1, and the third adjustment coefficient may be equal to 1.

[0112] In another embodiment of the present application, when each image area is a pixel, each pixel in the HDR image corresponds to a brightness adjustment coefficient and a depth adjustment coefficient.

[0113] In one embodiment of the present application, the depth information of each pixel in multiple image areas is normalized, and it is determined whether the depth information of each pixel after normalization is less than or equal to a preset depth threshold. When the depth information of the pixel is less than or equal to the preset depth threshold, the depth adjustment coefficient of the pixel value of the pixel is determined to be the first depth adjustment coefficient; when the depth information of the pixel is greater than the preset depth threshold, the depth adjustment coefficient of the pixel value of the pixel is determined to be the second depth adjustment coefficient. The first depth adjustment coefficient and the second depth adjustment coefficient are both constants, and the first depth adjustment coefficient is greater than the second depth adjustment coefficient. The normalization of the depth information includes normalizing the depth information in the range of 8bit 0-255 to data between 0-1. For example, the preset depth threshold is 0.1, 0.2, 0.3 or other suitable values.

[0114] In another embodiment of the present application, the depth information of each pixel in multiple image regions is normalized, and the depth information of each pixel is converted into a depth adjustment coefficient of the pixel according to a preset inverse proportional mapping function. The preset inverse proportional mapping function is f(d)=k / d, d is the depth information of the pixel, and k is a constant, such as 1.

[0115] In one embodiment of the present application, the brightness adjustment coefficient and the depth adjustment coefficient are multiplied with the pixel value of each pixel in the multiple image areas to obtain the updated pixel value of each pixel in the multiple image areas. When the multiple image areas include a first area and a second area, the first brightness adjustment coefficient corresponding to the first area is Q1, the second brightness adjustment coefficient corresponding to the second area is Q2, and the depth adjustment coefficient is f(d), where d is the depth information of the pixel. According to the above embodiment, the depth adjustment coefficient f(d) is the first depth adjustment coefficient or the second depth adjustment coefficient, or is a value determined according to the depth information of the pixel and a preset inverse proportional function. That is, the updated pixel value p1 of each pixel in the first area is Q1*f(d)*p0, and the updated pixel value p2 of each pixel in the second area is Q2*f(d)*p0, where p0 is the original pixel value of the pixel in the HDR image.

[0116] In one embodiment of the present application, when the target display brightness is determined by comparing the brightness threshold and the maximum display brightness, if the brightness threshold is less than the maximum display brightness, the brightness threshold is determined as the target display brightness, and the first brightness adjustment coefficient is the ratio of the current display brightness to the brightness threshold, that is, the reciprocal of the preset multiple; if the brightness threshold is greater than or equal to the maximum display brightness, the maximum display brightness is determined as the target display brightness, and the first brightness adjustment coefficient may be the ratio of the current display brightness to the maximum display brightness. When the maximum display brightness is fixed as the target display brightness, the first brightness adjustment coefficient may be the ratio of the current display brightness to the maximum display brightness. In other words, the first brightness adjustment coefficient is the ratio of the current display brightness to the target display brightness.

[0117] When the first brightness adjustment coefficient Q1 is the inverse of the preset multiple, since the preset multiple is greater than 1, it can be obtained that Q1<1. In addition, when the first brightness adjustment coefficient Q1 is the ratio of the current display brightness to the maximum display brightness, ANit represents the current display brightness of the electronic device, and B Nit represents the maximum display brightness of the electronic device, it can be obtained that the first brightness adjustment coefficient Q1 is A / B.

[0118] In one embodiment of the present application, the second brightness adjustment coefficient is a variable value and is related to the brightness information of each pixel in the second area. For example, the brightness information of each pixel in the second area can be digitized (for example, normalized) to obtain a brightness value, and then the second brightness adjustment coefficient of the pixel is calculated based on the brightness value. If the second area includes N pixels, the second brightness adjustment coefficient of the i-th pixel satisfies the calculation formula: Q2i=1+Li / Lmax. Among them, Q2i represents the second brightness adjustment coefficient of the i-th pixel, Li represents the brightness value of the i-th pixel, and Lmax represents the maximum brightness value of the pixel. Taking the brightness information as an 8-bit value as an example, the brightness information can be digitized as 0 to 255. If the brightness value of a pixel is C, the second adjustment coefficient Q2 corresponding to the pixel is Q2=1+C / 255. In other words, the second brightness adjustment coefficient of each pixel in the second area is the sum of 1 and the normalized brightness value of the pixel.

[0119] The above method can make different pixel values ​​have different second brightness adjustment coefficients, and can retain the light and dark relationship between different pixel points in the second area as much as possible.

[0120] In another embodiment of the present application, the second brightness adjustment coefficient may also be 1. When the second brightness adjustment coefficient is 1, the pixel values ​​of the pixels in the second area remain unchanged.

[0121] Since the first brightness adjustment coefficient is less than 1, the updated pixel value corresponding to each pixel in the first area is less than the original pixel value corresponding to the pixel. Since the second brightness adjustment coefficient is greater than or equal to 1, the updated pixel value corresponding to each pixel in the second area is greater than or equal to the original pixel value corresponding to the pixel.

[0122] According to the above embodiment of the present application, when the depth information of the pixel is larger, that is, the depth of field corresponding to the pixel is larger, the depth adjustment coefficient is smaller, so that the gain of the pixel value is reduced; when the depth information of the pixel is smaller, that is, the depth of field corresponding to the pixel is smaller, the depth adjustment coefficient is larger, so that the gain of the pixel value is increased. In this way, the brightness of the main part of the image closer to the camera can be increased, thereby highlighting the main part of the image, so that the image has better layers.

[0123] See also Figure 8As shown, it is a schematic diagram of the process of converting an SDR image into an HDR image provided by an embodiment of the present application. The SDR image is converted into an HDR image by tone mapping to perform pixel value conversion, and the pixel value of each pixel point in the HDR image, the brightness adjustment coefficient determined according to the brightness information, and the depth adjustment coefficient determined according to the depth information are input into the mapping function f as independent variables of the mapping function f, and the pixel value of each pixel point of the target HDR is output by the mapping function f, so that the dynamic range of the HDR image can be expanded according to the brightness of the display screen, and the dynamic range expansion can reflect the depth of field of the pixel points in the image, highlight the main part of the image with a smaller depth of field, thereby enhancing the layering of the image.

[0124] S108, adjusting the display screen brightness of the electronic device to the target display screen brightness, and displaying the target HDR image.

[0125] In one embodiment of the present application, the relationship between the observed brightness of a pixel and the brightness of a display screen can satisfy the calculation formula: Lg = Gray * Lp, where Lg is the observed brightness, Lp is the brightness of the display screen, and Gray is the pixel grayscale. The observed brightness of a pixel can indicate the brightness of the pixel on the display screen. The pixel grayscale of a pixel is related to the pixel value of the pixel, where the larger the pixel value, the larger the pixel grayscale. For example, the pixel grayscale and the pixel value can satisfy the calculation formula: Gray = α * R + β * G + γ * B. That is, the pixel grayscale can be the brightness value of the pixel.

[0126] When an image is displayed on a display screen of an electronic device, the brightness of the display screen of the electronic device and the pixel value of the pixel point will affect the brightness of the pixel point on the screen. Among them, when the pixel value of the pixel point is constant, the greater the brightness of the display screen of the electronic device, the greater the observed brightness of the pixel point, that is, the brighter the pixel point appears; when the brightness of the display screen of the electronic device is constant, the greater the pixel value of the pixel point, the greater the observed brightness of the pixel point, that is, the brighter the pixel point appears. Conversely, if the observed brightness of a certain pixel point is to be kept unchanged while increasing the brightness of the display screen, the pixel value of the pixel point can be reduced.

[0127] It can be understood that the process of displaying the target HDR image in the embodiment of the present application is actually: when the brightness of the display screen of the electronic device is increased, the pixel value of the pixel point in the first area on the target HDR image is reduced so that the observed brightness of the pixel point in the first area on the target HDR image is close to the observed brightness before the display screen brightness is not adjusted; at the same time, the pixel value of the pixel point in the second area on the target HDR image is maintained or increased so that the observed brightness of the pixel point in the second area on the second image is greater than the observed brightness before the display screen brightness is not adjusted. In other words, the brightness of the first area on the target HDR image is kept unchanged, and the brightness of the second area on the target HDR image is increased.

[0128] For example, an electronic device adjusts the display brightness from the current display brightness A to the target display brightness C, the first brightness adjustment coefficient is A / C, and the second brightness adjustment coefficient is 1, to illustrate the process of displaying an HDR image. The electronic device can first generate a target HDR image according to the first brightness adjustment coefficient, the second brightness adjustment coefficient and the corresponding depth adjustment coefficient. In the process of generating the target HDR image, the pixel value of the pixel point on the first area of ​​the HDR image is reduced by A / C*f(d)1 times, where f(d)1 is the first depth adjustment coefficient, A / C*f(d)1 is less than 1, so that the pixel grayscale of the pixel point on the first area is reduced by A / C*f(d) times; the updated pixel value of the pixel point on the second area of ​​the HDR image is f(d)2 times the original pixel value, where f(d)2 is the second depth adjustment coefficient, and f(d)2 is greater than or equal to 1. When the electronic device adjusts the display brightness from the current display brightness A to the target display brightness C, the display brightness of all pixels on the HDR image is enlarged by C / A times. It can be seen that after displaying the HDR image, the observed brightness of the pixels in the first area is first reduced by A / C*f(d)1 times, and then expanded by C / A times, and the observed brightness of the pixels is only affected by the depth information; the observed brightness of the pixels in the second area is expanded by C / A*f(d)2 times, and the brightness of the pixels is higher than the original observed brightness, and is also affected by the depth information. This can increase the light and dark contrast between the first and second areas on the HDR image, expand the dynamic range of the HDR image, and increase the brightness of the main part with a smaller depth of field in the HDR image, highlight the main part in the HDR image, and better reflect the layering of the HDR image.

[0129] For example, an HDR image can be Fig. 9 As shown, the HDR image 800 includes a first area 801 and a second area 802. If the electronic device does not obtain the target HDR image of the HDR image in advance, and directly increases the brightness of the display screen and displays the HDR image, the following may be obtained: Fig.10 In the image 900 shown in FIG. 1 , the observed brightness of the first area 901 and the second area 902 are both increased, but the light-dark contrast of the image 900 does not change and is the same as that of the image 900. After the electronic device obtains the target HDR image of the HDR image, it increases the brightness of the display screen and displays the target HDR image, and the following can be obtained: Fig.11As shown in the image 1000. It can be seen that the observed brightness of the first area 1001 in the image 1000 is close to the observed brightness of the first area 801 in the HDR image 800, and the observed brightness of the second area 1002 in the image 1000 is significantly higher than the observed brightness of the second area 802 in the HDR image 800. Compared with the HDR image 800, the first area 1001 and the second area 1002 in the image 1000 have a higher light-dark contrast, and the image 1000 has a larger dynamic range. In addition, as Fig. 9 The HDR image 800 shown also includes a third area 803. The third area 803 is a main part with a smaller depth of field (ie, closer to the camera). If the pixel value conversion is not performed according to the depth information of the pixel point, the following is obtained: Fig.10 In the image 900 shown in FIG. 1 , the portrait corresponding to the third area 903 cannot be highlighted. If the pixel value is converted according to the depth information of the pixel point, the following is obtained: Fig.11 In the image 1000 shown, the portrait corresponding to the third area 1003 in the image 1000 can be highlighted to show better hierarchy.

[0130] It should be noted that the HDR image displayed by the electronic device may be created by other electronic devices and then transmitted to the electronic device, or may be created by the electronic device. In other words, the process of creating the HDR image and the process of displaying the HDR image may be completed by the same device or by different devices, and the embodiments of the present application do not make specific limitations here.

[0131] The embodiment of the present application increases the observed brightness of each pixel by increasing the brightness of the display screen of the electronic device during the display of the HDR image; at the same time, by reducing the pixel value of the pixel in the first area to reduce the observed brightness of the pixel in the first area, and increasing or maintaining the pixel value of the pixel in the second area to increase or maintain the observed brightness of the pixel in the second area, so as to achieve the effect of not changing the observed brightness of the pixel in the second area and increasing the observed brightness of the pixel in the second area. That is to say, during the display of the HDR image, the bright area in the HDR image can be made brighter, and the dark area can be made darker or remain unchanged, thereby increasing the light and dark contrast of the HDR image and expanding the dynamic range of the HDR image. At the same time, by converting the pixel value according to the depth information of the pixel, the brightness of the main part of the image close to the camera can be increased, thereby highlighting the main part of the image and making the image have better layers.

[0132] In addition, the HDR image display method provided in the embodiment of the present application can adapt to the brightness capability supported by the display screen of the electronic device and can fully utilize the brightness capability of the electronic device.

[0133] See also Fig.12FIG. 1 is a flowchart of an image processing method provided by another embodiment of the present application. The method is applied in an electronic device, and the image processing method includes:

[0134] S201, acquiring an SDR image, and performing tone mapping on the SDR image to obtain an HDR image.

[0135] S202: Obtain the pixel value of each pixel in the HDR image.

[0136] S203: Group the pixels of the HDR image according to a preset pixel grouping strategy to obtain brightness information of each group of pixels.

[0137] S204: Obtain depth information of each pixel in the HDR image.

[0138] S205 , respectively encoding the brightness information, the depth information and the pixel value into the HDR image to obtain an image file.

[0139] In one embodiment of the present application, the pixel value can be encoded using the Joint Photographic Experts Group (JPEG) standard, high efficiency video coding (HEVC), portable network graphics (PNG), run-length encoding (RLE), etc. to obtain the first encoded data.

[0140] In one embodiment of the present application, the format of the first encoded data includes but is not limited to: JPEG, high efficiency image file format (HEIF), HEIC, PNG, bitmap (BMP), graphics interchange format (GIF), and no specific limitation is given here.

[0141] In one embodiment of the present application, the brightness information may be encoded using linear encoding or nonlinear encoding to obtain second encoded data. Each encoded brightness information may have a bit width of 8 bits, 10 bits or higher. Among them, nonlinear encoding may include encoding using a gamma function, etc., which is not specifically limited here. When encoding the brightness information, the grouping method of the pixel group may also be encoded into the second encoded data.

[0142] In one embodiment of the present application, the depth information may be encoded using linear encoding or nonlinear encoding to obtain third encoded data. Each encoded depth information may have a bit width of 8 bits. Among them, nonlinear encoding may include encoding using a gamma function, etc., which is not specifically limited here.

[0143] In one embodiment of the present application, the second encoded data and the third encoded data may also be written into a specific field of the first encoded data to generate an image file. Taking the format of the first encoded data as PNG as an example, a standard PNG file includes a PNG file signature field and a plurality of PNG data blocks. The electronic device may write the second encoded data and the third encoded data into the PNG data block to generate an image file. In another embodiment of the present application, the second encoded data and the third encoded data may also be written into the metadata of the HDR image to generate an image file. The metadata may be an Exchangeable Image File Format (EXIF).

[0144] If the format of the first encoded data is different, the fields in which the second encoded data and the third encoded data are written are not necessarily the same. It is only necessary to write the second encoded data and the third encoded data into the first encoded data without affecting the fields of the information carried by the first encoded data itself. No specific restrictions are made here.

[0145] In another embodiment of the present application, the pixel value, brightness information and depth information of the HDR image can be encoded by video encoding. The encoding format is YUV or YCbCr, etc. The encoder can be H.264 or H.265. The brightness information and depth information are encoded as 8-bit data in the range of 0-255.

[0146] In an embodiment of the present application, the image file created by the electronic device includes both the pixel values ​​of the HDR image and the brightness information and depth information of the SDR image before tone mapping is performed. The brightness information can better reflect the actual brightness of the HDR image, making it easier to adjust the brightness of the HDR image during the subsequent display of the HDR image.

[0147] S206 , in response to the user's operation of viewing the image file, decoding the image file to obtain brightness information, depth information and pixel values ​​of the HDR image.

[0148] In one embodiment of the present application, the image file is decoded according to a decoding method corresponding to the encoding method to obtain brightness information, depth information and pixel values ​​of the HDR image.

[0149] S207, determining a brightness mean according to the brightness information, and dividing the HDR image into a plurality of image regions according to the brightness mean.

[0150] S208: Determine a target display screen brightness based on the current display screen brightness and the maximum display screen brightness.

[0151] S209, determining a brightness adjustment coefficient of the pixel values ​​of the pixel points in the multiple image areas, determining a depth adjustment coefficient of the pixel value of each pixel point according to the depth information of each pixel point in the multiple image areas, and adjusting the pixel values ​​of the pixel points in the multiple image areas according to the brightness adjustment coefficient and the depth adjustment coefficient to obtain a target HDR image.

[0152] S210, adjusting the display screen brightness of the electronic device to a target display screen brightness, and displaying a target HDR image.

[0153] The specific implementations of S201-S204 and S207-S210 are the same as those of S101-S108 and will not be elaborated here.

[0154] In other embodiments of the present application, before tone mapping the SDR image, pixel information (i.e., the pixel value of each pixel), brightness information (i.e., the brightness value of each group of pixels) and depth information (i.e., the depth information of each pixel) of the SDR image may be obtained and encoded into the SDR image to generate an image file. When the electronic device receives an operation from the user to view the image file, the pixel information, brightness information and depth information of the encoded SDR image are decoded, the pixel information, brightness information and depth information of the SDR image are obtained, tone mapping is performed on the SDR image, the SDR image is converted into an HDR image, the HDR image is converted into a target HDR image according to the brightness information and the depth information, and the target HDR image is displayed according to the target display brightness, thereby expanding the display of the HDR image in a high dynamic range.

[0155] The present application also provides an electronic device 100. Fig.13 As shown, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device 100.

[0156] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0157] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0158] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0159] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0160] The processor 110 may also be provided with a memory for storing instructions and data. In one embodiment of the present application, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use instructions or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0161] In one embodiment of the present application, the processor 110 may include one or more interfaces. The interface may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0162] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (Serial Data Line, SDA) and a serial clock line (Serial Clock Line, SCL). In one embodiment of the present application, the processor 110 may include multiple groups of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 can be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0163] The I2S interface can be used for audio communication. In one embodiment of the present application, the processor 110 may include multiple I2S buses. The processor 110 may be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In one embodiment of the present application, the audio module 170 may transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call via a Bluetooth headset.

[0164] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment of the present application, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In one embodiment of the present application, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering a call via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0165] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment of the present application, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In one embodiment of the present application, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0166] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In one embodiment of the present application, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 100.

[0167] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In one embodiment of the present application, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0168] The USB interface 130 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices 100, such as AR devices, etc.

[0169] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0170] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device 100 through the power management module 141.

[0171] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0172] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0173] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0174] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (Low Noise Amplifier, LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0175] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In one embodiment of the present application, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be arranged in the same device as the mobile communication module 150 or other functional modules.

[0176] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0177] In one embodiment of the present application, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include Global System For Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a Global Positioning System (GPS), a Global Navigation Satellite System (GLONASS), a Beidou Navigation Satellite System (BDS), a Quasi-Zenith Satellite System (QZSS) and / or a Satellite Based Augmentation System (SBAS).

[0178] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0179] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, Microled, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In one embodiment of the present application, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0180] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0181] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In one embodiment of the present application, ISP can be set in camera 193.

[0182] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In one embodiment of the present application, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0183] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0184] The video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0185] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0186] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0187] Random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.;

[0188] Non-volatile memory may include disk storage devices and flash memory.

[0189] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle. It can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level. It can be divided into universal flash storage (UFS), embedded Multi Media Card (eMMC), etc. according to the storage specification.

[0190] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0191] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0192] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0193] The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include multiple instructions. When the multiple instructions are executed by the processor 110, the screen display detection method executed on the electronic device 100 in the above embodiment can be implemented to implement the screen display detection function of the electronic device 100.

[0194] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0195] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In one embodiment of the present application, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0196] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0197] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0198] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0199] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm Open Mobile Terminal Platform (OMTP) standard interface or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0200] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0201] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0202] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.

[0203] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In one embodiment of the present application, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100. An embodiment of the present application further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the electronic device 100, the electronic device 100 executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0204] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0205] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the image processing method in the above-mentioned method embodiments.

[0206] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0207] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0211] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program code.

[0212] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. An image processing method, applied to an electronic device, characterized in that: The method comprises: Acquire an SDR image, and perform tone mapping on the SDR image to obtain an HDR image; Obtaining a pixel value of each pixel in the HDR image; The pixels in the HDR image are grouped according to a preset pixel grouping strategy to obtain brightness information of each group of pixels; Obtaining depth information of each pixel in the HDR image; Determine a brightness mean value according to the brightness information, and divide the HDR image into a plurality of image regions according to the brightness mean value; Determine the target display brightness based on the current display brightness and the maximum display brightness; Determine a brightness adjustment coefficient of a pixel value of a pixel point in the multiple image areas, determine a depth adjustment coefficient of a pixel value of each pixel point according to depth information of each pixel point in the multiple image areas, and adjust the pixel values ​​of the pixel points in the multiple image areas according to the brightness adjustment coefficient and the depth adjustment coefficient to obtain a target HDR image; The display screen brightness of the electronic device is adjusted to the target display screen brightness, and the target HDR image is displayed.

2. The image processing method according to claim 1, characterized in that: The method further comprises: Encoding the brightness information, the depth information and the pixel value into the HDR image respectively to obtain an image file; In response to a user's operation of viewing the image file, the image file is decoded to obtain the brightness information, the depth information, and the pixel value of the HDR image.

3. The image processing method according to claim 2, characterized in that: The step of respectively encoding the brightness information, the depth information and the pixel value into the HDR image to obtain an image file comprises: Encoding the pixel value of each pixel to obtain first encoded data; Encoding the brightness information to obtain second encoded data; Encoding the depth information to obtain third encoded data; The second coded data and the third coded data are written into the first coded data to obtain the image file.

4. The image processing method according to claim 1, wherein: The step of obtaining the brightness information of each group of pixels comprises: Acquire a target exposure image corresponding to a target exposure parameter, wherein if the target exposure parameter is any exposure parameter among a plurality of exposure parameters, the target exposure image is an image corresponding to the any exposure parameter; if the target exposure parameter is a parameter determined according to at least two exposure parameters among the plurality of exposure parameters, the target exposure image is an image obtained by fusing images corresponding to the at least two exposure parameters; Obtaining a brightness value of each pixel in each group of pixels in the target exposure image; The average of the brightness values ​​of all the pixels in each group of pixels in the target exposure image is used as the brightness information of the group of pixels under the target exposure parameters.

5. The image processing method according to claim 1, wherein: The depth information of each pixel in the HDR image is the distance between the point in the scene corresponding to each pixel and the camera of the electronic device, and obtaining the depth information of each pixel in the HDR image includes: Inputting the HDR image into a preset deep learning model, and outputting a disparity map corresponding to the HDR image through the preset deep learning model; The disparity of each pixel is obtained according to the disparity map, and the depth information of each pixel is obtained according to the disparity calculation of each pixel.

6. The image processing method according to claim 1, wherein: The step of determining a brightness mean value according to the brightness information, and dividing the HDR image into a plurality of image regions according to the brightness mean value, comprises: Determine the brightness mean value according to the brightness information of all pixels in the HDR image; Determine a first threshold value according to the brightness mean value, where the first threshold value is greater than or equal to the brightness mean value; Dividing the pixel points whose brightness information is less than or equal to the first threshold into a first area of ​​the HDR image; The pixel points whose brightness information is greater than the first threshold are divided into a second area of ​​the HDR image.

7. The image processing method according to claim 6, characterized in that: The method of determining a brightness mean value according to the brightness information, and dividing the HDR image into a plurality of image regions according to the brightness mean value, further includes: Determine a second threshold and a third threshold according to the brightness mean, wherein the second threshold is less than the brightness mean, and the third threshold is greater than or equal to the brightness mean; Dividing the pixel points whose brightness information is less than the second threshold into a first area of ​​the HDR image; Divide the pixel points whose brightness information is greater than or equal to the second threshold and less than the third threshold into a third area of ​​the HDR image; The pixel points whose brightness information is greater than or equal to the third threshold are divided into a second area of ​​the HDR image.

8. The image processing method according to claim 1, characterized in that: The smallest image area is a pixel.

9. The image processing method according to claim 6, characterized in that: The maximum display screen brightness is the maximum brightness supported by the display screen of the electronic device, and determining the target display screen brightness based on the current display screen brightness and the maximum display screen brightness includes: Using a preset multiple of the current display screen brightness as a brightness threshold; If the brightness threshold is less than or equal to the maximum display screen brightness, determine the brightness threshold as the target display screen brightness; or If the brightness threshold is greater than the maximum display screen brightness, the maximum display screen brightness is determined to be the target display screen brightness.

10. The image processing method according to claim 9, characterized in that: The step of determining the brightness adjustment coefficients of the pixel values ​​of the pixel points in the plurality of image areas comprises: Determine a first brightness adjustment coefficient of pixels within the first area as a ratio of the current display screen brightness to the brightness threshold; The second brightness adjustment coefficient of the pixel in the second area is determined as the sum of 1 and the normalized brightness value of the pixel.

11. The image processing method according to claim 1, wherein: The step of determining the depth adjustment coefficient of the pixel value of each pixel point according to the depth information of each pixel point in the multiple image areas includes: Normalizing the depth information of each pixel in the multiple image areas; When the depth information of the pixel point is less than or equal to a preset depth threshold, determining the depth adjustment coefficient of the pixel value of the pixel point to be a first depth adjustment coefficient; When the depth information of the pixel point is greater than the preset depth threshold, the depth adjustment coefficient of the pixel value of the pixel point is determined to be a second depth adjustment coefficient, wherein the first depth adjustment coefficient and the second depth adjustment coefficient are both constants, and the first depth adjustment coefficient is greater than the second depth adjustment coefficient.

12. The image processing method according to claim 1, characterized in that: The step of determining the depth adjustment coefficient of the pixel value of each pixel point according to the depth information of each pixel point in the multiple image areas includes: Normalizing the depth information of each pixel in the multiple image areas; The depth information of each pixel is converted into a depth adjustment coefficient of the pixel according to a preset inverse proportional mapping function.

13. The image processing method according to claim 1, characterized in that: The step of adjusting the pixel values ​​of the pixel points in the plurality of image areas according to the brightness adjustment coefficient and the depth adjustment coefficient to obtain a target HDR image comprises: The brightness adjustment coefficient and the depth adjustment coefficient of each pixel point in the multiple image areas are multiplied by the pixel value to obtain an updated pixel value of each pixel point in the multiple image areas.

14. An electronic device, characterized in that: The electronic device comprises a memory and a processor: Wherein, the memory is used to store program instructions; The processor is used to read and execute the program instructions stored in the memory. When the program instructions are executed by the processor, the electronic device executes the image processing method as described in any one of claims 1 to 13.

15. A chip coupled to a memory in an electronic device, characterized in that: The chip is used to control the electronic device to execute the image processing method according to any one of claims 1 to 13.

16. A computer storage medium, characterized in that: The computer storage medium stores program instructions, and when the program instructions are executed on an electronic device, a processor of the electronic device executes the image processing method according to any one of claims 1 to 13.

Images