Image processing method and electronic device

Abstract

The application discloses an image processing method and an electronic device, relates to the field of image processing, and can improve the display effect of a preview image and improve user experience. The method comprises the following steps: acquiring a brightness histogram of an image frame; the brightness histogram comprises a highlight area; acquiring a first brightness value and a second brightness value; the first brightness value is used for representing the brightness of the highlight area; the second brightness value is used for representing the ambient brightness of the image frame; obtaining a first brightness compensation value based on a first target brightness value and the first brightness value; the first target brightness value is obtained based on the second brightness value; obtaining a second target brightness value based on a preset brightness value and the first brightness compensation value; and adjusting the brightness of the image frame based on the second target brightness value.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image processing method and an electronic device. Background Technology

[0002] Currently, when previewing images on mobile phones and other electronic devices, the process of suppressing overexposure in highlights and brightening dark areas in the auto exposure (AE) algorithm is difficult to control, resulting in overexposed highlights or completely black dark areas in the preview image. Usually, the dynamic preview effect can only be restored through post-processing algorithms (such as sending a variable exposure sequence of short / long frames). This leads to poor display quality of the preview image and affects the user experience. Summary of the Invention

[0003] This application provides an image processing method and an electronic device that can improve the display effect of preview images and enhance the user experience.

[0004] To achieve the objective, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, an image processing method is provided, the method comprising: acquiring a brightness histogram of an image frame; the brightness histogram including a bright region; acquiring a first brightness value and a second brightness value; the first brightness value being used to characterize the brightness of the bright region; the second brightness value being used to characterize the ambient brightness of the image frame; obtaining a first brightness compensation value based on a first target brightness value and the first brightness value; the first target brightness value being obtained based on the second brightness value; obtaining a second target brightness value based on a preset brightness value and the first brightness compensation value; and adjusting the brightness of the image frame based on the second target brightness value.

[0006] The image processing method described above obtains a first target brightness value by acquiring the ambient brightness (second brightness value) of the image frame and using a right-side metering algorithm to acquire the first brightness value of the highlighted area. Next, a first brightness compensation value is obtained based on the first target brightness value and the first brightness value. Then, a second target brightness value is obtained based on a preset brightness value and the first brightness compensation value. Finally, the brightness of the image frame is adjusted based on the second target brightness value. This method can solve the problem of overexposure in highlighted areas, thus improving the display effect of the preview image and enhancing the user experience.

[0007] In one possible implementation of the first aspect, the luminance histogram further includes dark regions; the method further includes: obtaining a third luminance value and a fourth luminance value; the third luminance value is used to characterize the luminance of the dark regions; the fourth luminance value is used to characterize the ambient luminance of the dark regions of the image frame; obtaining a contrast ratio based on the third luminance value and the first luminance value; obtaining a third target luminance value based on the contrast ratio and the fourth luminance value; obtaining a second luminance compensation value based on the third target luminance value and the third luminance value; obtaining a fourth target luminance value based on a preset luminance value and the second luminance compensation value; and adjusting the luminance of the image frame based on the fourth target luminance value.

[0008] In this implementation, firstly, the ambient brightness (fourth brightness value) of the dark areas of the image frame and the third brightness value of the dark areas are obtained. Secondly, the contrast ratio is obtained based on the third brightness value and the first brightness value. Next, after obtaining the third target brightness value based on the contrast ratio and the fourth brightness value, a second brightness compensation value is obtained based on the third target brightness value and the third brightness value. Then, the fourth target brightness value is obtained based on the preset brightness value and the second brightness compensation value. Finally, the brightness of the image frame is adjusted based on the fourth target brightness value. This method can solve the problem of completely black dark areas, thus improving the display effect of the preview image and enhancing the user experience.

[0009] In one possible implementation of the first aspect, the image frame includes a current image frame and a previous image frame; the first brightness value of the current image frame is obtained based on the second target brightness value of the previous image frame, the first brightness value of the previous image frame, and the statistical brightness value of the current image frame; the preset brightness value of the current image frame is obtained based on the preset brightness value of the previous image frame and the first brightness compensation value of the current image frame; the statistical brightness value of the current image frame is used to characterize the average brightness of the brightness histogram of the current image frame.

[0010] In this implementation, the first brightness value of the current image frame can be obtained based on the second target brightness value of the previous image frame, the first brightness value of the previous image frame, and the statistical brightness value of the current image frame. The preset brightness value of the current image frame is obtained based on the preset brightness value of the previous image frame and the first brightness compensation value of the current image frame. This ensures that the first brightness compensation value of the current image frame and the first brightness compensation value of the previous image frame do not have an inverse relationship, thus guaranteeing that the first brightness compensation value is not affected by the convergence of the AE algorithm.

[0011] In one possible implementation of the first aspect, the method further includes: when the fifth target brightness value is greater than a brightness threshold, increasing the brightness of the bright area based on the brightness threshold; wherein the brightness threshold is greater than the first target brightness value and less than or equal to 255; the fifth target brightness value is the sum of the second target brightness value and the brightness difference, and the brightness difference is the difference between the fourth target brightness value and the third target brightness value; when the fifth target brightness value is less than or equal to the brightness threshold, increasing the brightness of the bright area based on the fifth target brightness value.

[0012] In this implementation, when the brightness value of the fifth target is greater than the brightness threshold, the brightness of the highlighted area is increased based on the brightness threshold; when the brightness value of the fifth target is less than or equal to the brightness threshold, the brightness of the highlighted area is increased based on the brightness value of the fifth target. This avoids overexposure of the highlighted area, improves the display effect of the preview image, and enhances the user experience.

[0013] In one possible implementation of the first aspect, the brightness histogram further includes an intermediate region; the method further includes: when the fifth target brightness value is greater than a brightness threshold, increasing the brightness of a low-to-medium brightness region based on a sixth target brightness value; the low-to-medium brightness region includes a dark region and an intermediate region; the sixth target brightness value is greater than or equal to the third target brightness value and less than or equal to the fourth target brightness value; when the fifth target brightness value is less than or equal to a brightness threshold, increasing the brightness of a low-brightness region based on the fourth target brightness value.

[0014] In this implementation, when the fifth target brightness value is greater than the brightness threshold, the brightness of the medium and low brightness areas is increased based on the sixth target brightness value. When the fifth target brightness value is less than or equal to the brightness threshold, the brightness of the low brightness areas is increased based on the fourth target brightness value. This avoids completely black dark areas, improves the display effect of the preview image, and enhances the user experience.

[0015] In one possible implementation of the first aspect, the dark region is the darkest region in terms of brightness, where the number of pixels in the brightness histogram accounts for the second largest proportion.

[0016] In this implementation, by taking the darkest region with the second-highest pixel count in the brightness histogram as the dark region, the average brightness of the dark region can be accurately obtained, which facilitates the subsequent acquisition of the fourth target brightness value of the dark region.

[0017] In one possible implementation of the first aspect, the highlighted area is the area with the highest brightness in the brightness histogram, where the number of pixels accounts for a first proportion.

[0018] In this implementation, by taking the region with the highest brightness, which accounts for the first proportion of the number of pixels in the brightness histogram, as the bright region, the average brightness of the bright region can be accurately obtained, which facilitates the subsequent acquisition of the second target brightness value of the bright region.

[0019] In one possible implementation of the first aspect, the first brightness value is obtained based on the maximum brightness value among the red, green, and blue RGB brightness values ​​in the highlighted region.

[0020] In this implementation, the average brightness is calculated by using the maximum value of the three single channels: R, G, and B, which ensures the accuracy of the average brightness value calculated for the high-brightness area.

[0021] In a second aspect, an electronic device is provided, including a memory and one or more processors. The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device performs an image processing method as described in the first aspect and any embodiment thereof.

[0022] Thirdly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform an image processing method as described in the first aspect and any embodiment thereof.

[0023] Fourthly, a computer program product is provided that, when run on an electronic device, causes the electronic device to perform an image processing method as described in the first aspect and any embodiment thereof.

[0024] The technical effects of the design methods in the second, third, and fourth aspects can be found in the technical effects of the different design methods in the first aspect, and will not be repeated here. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a possible hardware structure of an electronic device provided in an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of a possible software structure of an electronic device provided in an embodiment of this application;

[0027] Figure 3 A brightness diagram of a preview image frame provided for related technologies;

[0028] Figure 4 A flowchart illustrating an image processing method for first brightness adjustment provided in an embodiment of this application;

[0029] Figure 5 A schematic diagram of a brightness histogram provided in an embodiment of this application;

[0030] Figure 6 A schematic diagram illustrating the calculation of the first luminance value using four channels (Y, R, G, and B) provided in an embodiment of this application;

[0031] Figure 7 A flowchart illustrating a second brightness adjustment image processing method provided in this application embodiment;

[0032] Figure 8 This is a schematic diagram illustrating the adjustment of brightness in a highlight area, provided as an embodiment of this application.

[0033] Figure 9 This is a schematic diagram illustrating brightness adjustment in a low-to-medium brightness region, provided as an embodiment of this application.

[0034] Figure 10 This is a brightness diagram of a preview image frame provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding. The terms "coupling" and "connection" involved in the embodiments of this application should be interpreted broadly. For example, it can refer to a physical direct connection or an indirect connection implemented through electronic devices, such as a connection implemented through resistors, inductors, capacitors, or other electronic devices.

[0036] An image signal processor (ISP) is used to process the output data of an image sensor, such as automatic exposure control (AEC), automatic gain control (AGC), automatic white balance (AWB), color correction, sharpening, bad pixel removal, noise reduction, and other functions.

[0037] The auto exposure (AE) algorithm adjusts the exposure to achieve the ideal exposure state by monitoring the image histogram, preventing underexposure or overexposure.

[0038] 18-degree gray refers to a shade of gray with a light reflectance of 18%. When the reflectance of a gray panel changes from 3.6% (which is essentially 0) to 90% (which is essentially 100%), the change in brightness observed by the human eye is centered at 18-degree gray, so it is used as the standard.

[0039] This application provides an electronic device with image processing capabilities. The electronic device can be mobile or fixed. It can be deployed on land (e.g., indoors or outdoors, handheld or vehicle-mounted), on water (e.g., on ships), or in the air (e.g., airplanes, balloons, and satellites). This electronic device can be referred to as user equipment (UE), access terminal, terminal unit, subscriber unit, terminal station, mobile station (MS), mobile station, terminal agent, or terminal device. For example, it can be a mobile phone, tablet computer, laptop computer, smart bracelet, smart screen, smartwatch, virtual reality (VR) device, augmented reality (AR) device, terminal in industrial control, terminal in self-driving, terminal in remote medical care, terminal in smart grid, terminal in transportation safety, terminal in smart city, terminal in smart home, etc. This application does not limit the specific type and structure of the electronic device. The following describes one possible structure of the electronic device.

[0040] Taking mobile phones as an example, the attached document... Figure 1 A possible structure of an electronic device 100 is shown. The electronic device 100 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a power management module 240, a battery 241, a wireless charging coil 242, a mobile communication module 250, a wireless communication module 260, antennas 251 and 261, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. Optionally, in some embodiments, it may also include an audio digital signal processor (ADSP) 243.

[0041] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0042] Processor 210 may include one or more processing units, such as: a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a central processing unit (CPU), an application processor (AP), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, and a neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. For example, processor 210 may be an application processor (AP). Alternatively, processor 210 may be integrated into a system-on-chip (SoC). Or, processor 210 may be integrated into an integrated circuit (IC) chip. The processor 210 may include an analog front end (AFE) and a micro-controller unit (MCU) in an IC chip.

[0043] The processor 210 may also include a memory for storing computer instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store computer instructions or data that the processor 210 has just used or that are being used repeatedly. If the processor 210 needs to use the same computer instructions or data again, it can retrieve them directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves system efficiency.

[0044] In some embodiments, the processor 210 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, etc.

[0045] In some embodiments, the processor may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor described above may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0046] The ADSP 243 can be coupled to the audio module 270 and the sensor module 280. The ADSP 243 can process audio signals and sensor data. Even when the processor 210 is in sleep mode, the ADSP 243 can remain operational, thereby reducing the power consumption of the electronic device.

[0047] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or a combination of multiple interface connection methods.

[0048] The external storage interface 220 can be used to connect an external memory card, such as a micro SanDisk (Micro SD) card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 210 through the external storage interface 220 to perform data storage functions. For example, music, video, and other files can be stored on the external memory card.

[0049] Internal memory 221 can be used to store computer executable program code, which includes computer instructions. Processor 210 executes various functional applications and data processing of electronic device 100 by running the computer instructions stored in internal memory 221. For example, internal memory 221 and processor 210 can be coupled together via a bus. This bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Such buses can be categorized as address buses, data buses, control buses, etc.

[0050] In this embodiment of the application, when computer instructions are executed by processor 210, electronic device 100 performs the image processing method of this embodiment of the application.

[0051] In addition, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0052] The memory involved in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0053] Electronic device 100 can implement audio functions such as music playback and recording through audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, and application processor.

[0054] Buttons 290 include power buttons, volume buttons, etc. Buttons 290 can be mechanical buttons or touch buttons. Electronic device 100 can receive button inputs and generate key signal inputs related to user settings and function control of electronic device 100. Motor 291 can generate vibration alerts. Motor 291 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 292 can be an indicator light, used to indicate charging status, battery level changes, or to indicate messages, missed calls, notifications, etc. SIM card interface 295 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 295 to achieve contact and separation with electronic device 100. Electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. In some embodiments, the electronic device 100 employs an embedded SIM (eSIM) card, which can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0055] The electronic device 100 can implement its shooting function through an ISP, a camera 293, a video codec, a GPU, a display 294, and an application processor. The ISP is used to process data fed back from the camera 293. In some embodiments, the ISP can be located within the camera 293. The camera 293 is used to capture still images or videos. In some embodiments, the electronic device 100 may include one or N cameras 293, where N is a positive integer greater than 1.

[0056] In some embodiments of this application, image frames can be captured by camera 293. In some embodiments, the image processing method can be implemented by an ISP or by processor 210.

[0057] Electronic device 100 can implement display functions through a GPU, display screen 294, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute computer instructions to generate or modify display information.

[0058] In this embodiment of the application, the image frame after image processing can be displayed on the display screen 294.

[0059] The power management module 240 is used to receive charging input from a charger. The charger can be a wireless charger, such as a wireless charging dock, or other electronic device 100 with reverse wireless charging capability. The power management module 240 can receive wireless charging input via the wireless charging coil 242 of the electronic device. The charger can also be a wired charger; for example, the power management module 240 can receive charging input from a wired charger via a USB interface 230. The power management module 240 is also referred to as a charging chip.

[0060] The power management module 240 is used to connect to the battery 241. The power management module 240 receives input from the battery 241 and supplies power to the processor 210, internal memory 221, display screen 294, camera 293, and wireless communication module 260, etc. The power management module 240 can also be used to monitor parameters such as battery 241 capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 240 may also be located within the processor 210.

[0061] The wireless communication function of the electronic device 100 can be realized through antenna 251, antenna 261, mobile communication module 250, wireless communication module 260, modem processor, etc.

[0062] Mobile communication module 250 can provide wireless communication solutions including 2G / 3G / 4G / 5G for use on electronic device 100. Wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) for use on electronic device 100.

[0063] As attached Figure 2 As shown, taking the Android operating system running on the electronic device 100 as an example, the software architecture running on the processor 210 includes the application layer, framework layer, system runtime library layer, hardware abstraction layer (HAL) layer, and kernel layer.

[0064] The kernel layer is the layer between hardware and software. For example, the kernel layer includes display drivers, camera drivers, etc. Display drivers are used to drive the display screen to show images or receive user touch operations, while camera drivers are used to drive the camera to capture images.

[0065] In this embodiment, the camera driver can drive the camera 293 to capture image frames, and the display driver can receive the user's shooting touch operation and drive the display screen 294 to display the preview image frames and the image frames after image processing.

[0066] The Hardware Abstraction Layer (HAL) is used to abstract hardware. The HAL hides the hardware interface details of a specific platform, providing the operating system with a virtual hardware platform and exhibiting hardware independence. For example, the HAL includes a display module and a camera module. The display module is used for a virtual display screen, and the camera module is used for a virtual camera.

[0067] The system runtime library layer includes C / C++ libraries and runtime libraries. Many core components and services of the Android operating system are built from native code and require C / C++ libraries. When an application is first installed, it is pre-compiled into machine code form as a runtime library; this process is called pre-compilation. This allows for faster startup and execution of the application by running the machine code.

[0068] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes predefined implementation methods. For example, it includes a window manager, content providers, a view system, and a notification manager.

[0069] The application layer can include a series of application packages, such as photo, camera, and other applications.

[0070] In related technologies, the CMOS sensors and ISPs in mobile phones and other electronic devices have relatively limited dynamic range processing capabilities. Therefore, in high dynamic range scenes, the dynamic range of the image during preview is much smaller than that of the actual shooting scene. Because the highlight overexposure suppression and shadow brightening processes in the auto exposure (AE) algorithm are difficult to control, the preview image may show overexposed highlights or completely black shadows. For example, see the appendix... Figure 3Before a user clicks / touches the shutter 302 to capture multiple frames of images, the electronic device 100 displays a shooting preview interface 301. In the preview image in the shooting preview interface 301, the bright areas (sun) are overexposed and the dark areas (bottom sea level) are completely black. Usually, the dynamic range can only be restored through post-processing algorithms (e.g., sending a variable exposure sequence of short / long frames). This results in a poor display effect of the preview image, affecting the user's experience.

[0071] Although different AE algorithms use different algorithms, they all set a target brightness value for bright and dark areas, and then apply the same weight to the same area to adjust the exposure parameters. However, this method is not flexible enough, which leads to unstable effects in dynamic scenes.

[0072] Therefore, this application provides an image processing method. Based on the image processing method provided in this application, metering can be applied to the right in bright areas to reduce the brightness of image frames, solving the problem of overexposure in bright areas, and the brightness of image frames in dark areas can be increased to solve the problem of completely black areas. Therefore, the display effect of the preview image can be improved, the user experience enhanced, and the effect is stable.

[0073] The image processing method provided in this application can be implemented on an electronic device 100 and displayed on a display screen 294 in the electronic device 100. This application uses an electronic device 100 that includes image processing functionality as an example to specifically describe the image processing method of this application.

[0074] For example, attached Figure 4 A flowchart illustrating an image processing method provided in an embodiment of this application is attached. Figure 4 As shown, the method may include steps S401-S405:

[0075] Step S401: Obtain the brightness histogram of the image frame; the brightness histogram includes the bright areas.

[0076] The brightness histogram of the image frame is a brightness histogram of one of the image frames in the dynamic preview image captured by the user. The X-axis of the brightness histogram represents the brightness of the statistically analyzed pixels, and the Y-axis represents the number of pixels corresponding to that brightness.

[0077] Step S402: Obtain a first brightness value L1 and a second brightness value L2; the first brightness value L1 is used to characterize the brightness of the bright area; the second brightness value L2 is used to characterize the ambient brightness of the image frame.

[0078] In one possible implementation, the highlighted area is the region with the highest brightness, representing a first proportion of the pixel count in the brightness histogram. For example, see the appendix. Figure 5 The first proportion is 5%, and the bright area is the area with the highest brightness in the brightness histogram that accounts for 5% of the number of pixels, that is, the area on the right side of the brightness histogram that accounts for 5%.

[0079] In one possible implementation, the first ratio can be 5% or 10%, and the embodiments of this application do not limit the size of the first ratio.

[0080] The first brightness value L1 is used to characterize the brightness of the highlighted area. The first brightness value L1 is obtained by averaging the brightness of the highlighted area in the statistical brightness histogram. In one possible implementation, the average brightness of the highlighted area can be calculated using an arithmetic average algorithm or a geometric average algorithm. This application does not limit the algorithm for averaging the brightness of the highlighted area.

[0081] The second brightness value L2 is used to characterize the ambient brightness of the image frame, and is related to the average brightness L of the entire image frame. r Related, while the average brightness L of the entire image frame r It is obtained by averaging the brightness of the entire brightness histogram. In one possible implementation, the average brightness of the brightness histogram can be calculated using either an arithmetic average algorithm or a geometric average algorithm. This application does not limit the algorithm used for averaging the brightness of the brightness histogram.

[0082] The second brightness value L2 and the average brightness L of the image frame r The relationship between them can be expressed by the following formula:

[0083]

[0084] Where K is a constant, t e For exposure time, ISO is the sensitivity, or gain.

[0085] In formula (1), K is a constant, and when the camera is fixed, the exposure time t is... e Since the ISO gain is constant, the second luminance value L2 is only related to the average luminance L of the image frame. r This is relevant. Formula (1) shows that the average brightness L of the image frame is related. r For every doubling of the original value, the second brightness value L2 increases by 10, meaning the ambient brightness of the image frame increases by 10.

[0086] Since in formula (1), the average brightness L of the entire image frame r With exposure H v (exposure H) v = Exposure time t e The gain (ISO) is linear, therefore, the second brightness value L2 is fixed and is used to characterize the brightness of the environment of the image frame.

[0087] Step S403, based on the first target brightness value L 1t The first brightness compensation value E1 is obtained by combining the first brightness value L1 with the first brightness value L; the first target brightness value L 1t This is based on the second brightness value L2.

[0088] In one possible implementation, the first target brightness value L 1t The second brightness value L2 is obtained by looking up a table. In one possible implementation, the first target brightness value L... 1t The second brightness value L2 can have a linear relationship (for example, if the second brightness value L2 is 30, the first target brightness value L in the table is 30). 1t The first target brightness value is 140; the second brightness value L2 is 80, and the first target brightness value L in the table is... 1t The first target brightness value is 170; the second brightness value L2 is 130, and the first target brightness value L in the table is... 1t (200), or it can be a non-linear relationship. In this embodiment of the application, the first target brightness value L 1t The relationship to the second brightness value L2 is not limited.

[0089] First target brightness value L 1t The first target brightness value L2 is obtained based on the second brightness value L2, indicating that the average brightness of the bright area can be obtained from the ambient brightness of the entire image frame. 1t Used to characterize the target brightness of the highlighted area.

[0090] Based on the first target brightness value L 1t The first brightness compensation value E1 is obtained by combining the first brightness value L1 with the first brightness value, which can be expressed by the following formula:

[0091]

[0092] The brightness compensation value of the bright area can be obtained based on the target brightness and the average brightness of the bright area. According to formula (2), the first brightness compensation value E1 is based on the first target brightness value L. 1t The result of taking the logarithm of the ratio of the first brightness value L1 to the first brightness value L2.

[0093] Step S404, based on the preset brightness value L p The second target brightness value L is obtained by combining the first brightness compensation value E1. 2t .

[0094] Among them, the preset brightness value L pIt can be set according to actual needs. For example, it can be obtained by statistically analyzing multiple intermediate brightness values ​​in the process of increasing reflectivity, such as the average or median of multiple intermediate brightness values ​​when the reflectivity increases from low to high; it can also be set based on empirical values, such as the brightness value of 18 degrees gray in nature.

[0095] In one possible implementation, a preset brightness value L is used. p The brightness value can be either the 18-degree gray level found in nature or the 128-degree gray level in the luminance histogram. This application embodiment uses a preset luminance value L. p The value is not limited.

[0096] In this embodiment of the application, the preset brightness value L p The brightness value chosen to characterize 18-degree gray in nature is as follows: 18-degree gray refers to a gray with a light reflectance of 18%. When the reflectance of a gray board changes from 3.6% (essentially 0) to 90% (essentially 100%), the change in brightness observed by the human eye is centered at 18-degree gray, hence it is used as the standard. Examples are shown in Table 1:

[0097] Table 1. Brightness values ​​under different reflectivities

[0098] Reflectivity (brightness) 3.6％ 4.5％ 9％ 18％ 36％ 72％ 90％ The logarithm of reflectance (base 2) -4.8 -4.47 -3.47 -2.47 -1.47 -0.47 -0.15 Logarithm of reflectance based on 18% -2.3 -2.0 -1.0 0 1.0 2.0 2.3 Brightness levels for observing human figures - Ⅲ Ⅳ Ⅴ Ⅵ Ⅶ -

[0099] As shown in Table 1, the reflectivity is 18% gray, which is exactly in the center. Taking 18% gray as the benchmark, the logarithm of the reflectivity is symmetrical about the center of 18% gray. The brightness level observed by the human eye is also centered on the brightness level of 18% gray, gradually becoming darker and brighter.

[0100] Based on the preset brightness value L p The second target brightness value L is obtained by combining the first brightness compensation value E1. 2t It can be expressed using the following formula:

[0101]

[0102] With preset brightness value L p Using the reference brightness value, the preset brightness value L is adjusted according to the first brightness compensation value E1. p By performing compensation, the actual target brightness value, i.e., the second target brightness value L, can be obtained. 2t According to formula (3), since the first brightness compensation value E1 obtained by formula (2) is the logarithm with base 2, in formula (3), the exponent of the first brightness compensation value E1 with base 2 is used as the preset brightness value L. p The compensation coefficient is used to obtain the actual target brightness value, i.e., the second target brightness value L. 2t .

[0103] Step S405, based on the second target brightness value L 2t Adjust the brightness of the image frame.

[0104] After obtaining the second target brightness value L 2t Then, based on the second target brightness value L 2t The brightness of the image frames is adjusted to address the overexposure issue in bright areas. Since the focus is on suppressing overexposure in bright areas, the second target brightness value L... 2t It is less than the first brightness value L1.

[0105] The image processing method described in steps S401-S405 above obtains the first target brightness value L by acquiring the ambient brightness of the image frame, i.e., the second brightness value L2. 1t Furthermore, a right-side metering algorithm is used to obtain the first brightness value L1 of the bright area. Then, based on the first target brightness value L1... 1t The first brightness compensation value E1 is obtained by combining the first brightness value L1 with the first brightness value L2. Next, based on the preset brightness value L1... p The second target brightness value L is obtained by combining the first brightness compensation value E1. 2t Finally, based on the second target brightness value L 2t This method adjusts the brightness of image frames. It can resolve the issue of overexposure in bright areas, thus improving the display quality of the preview image and enhancing the user experience.

[0106] In this embodiment of the application, the image processing method described in steps S401-S405 above, due to the first target brightness value L 1t Since the second brightness value L2 is obtained by looking up a table, the first target brightness value L is obtained when the second brightness value L2 remains unchanged. 1t It also remains unchanged. However, in the previous image frame, the second target brightness value L was obtained based on the first brightness compensation value E1. 2t After adjusting the brightness of the image frame, the average brightness of the highlighted area, i.e., the first brightness value L1, will decrease in the current image frame. With the second brightness value L2 remaining unchanged, according to formula (2), the first target brightness value L... 1t The ratio of the first brightness value L1 will increase, so the first brightness compensation value E1 of the current image frame will increase. This will cause the adjustment of the current image frame to be opposite to the adjustment of the previous image frame, resulting in little adjustment effect of the current image frame. Therefore, it is necessary to ensure that the first brightness compensation value E1 is not affected by the convergence of the AE algorithm.

[0107] To ensure that the first brightness compensation value E1 is not affected by the convergence of the AE algorithm, the first brightness value current_L1 of the current image frame can be based on the second target brightness value last_L of the previous image frame. 2tThe first brightness value of the previous image frame, last_L1, and the statistical brightness value of the current image frame, current_L. r In one possible implementation, the image frame includes the current image frame and the previous image frame; the first brightness value current_L1 of the current image frame is the second target brightness value last_L based on the previous image frame. 2t The first brightness value of the previous image frame, last_L1, and the statistical brightness value of the current image frame, current_L. r Get the statistical brightness value current_L of the current image frame. r This is used to characterize the average brightness of the brightness histogram of the current image frame. In other words, it's the average brightness of the first brightness value (current_L1) of the current image frame and the second target brightness value (last_L) of the previous image frame. 2t The first brightness value of the previous image frame, last_L1, and the statistical brightness value of the current image frame, current_L. r related.

[0108] The statistical brightness value of the current image frame and the average brightness current_L of the entire current image frame. r Related to the average brightness of the entire current image frame, current_L r It is obtained by averaging the brightness of the entire brightness histogram. In one possible implementation, the average brightness of the brightness histogram can be calculated using either an arithmetic average algorithm or a geometric average algorithm. This application does not limit the algorithm used for averaging the brightness of the brightness histogram.

[0109] In this embodiment of the application, the average brightness of the current entire image frame is current_L. r It is obtained by using a weighted average algorithm to calculate the average brightness of the entire brightness histogram of the entire image frame.

[0110] The first brightness value current_L1 of the current image frame and the second target brightness value last_L of the previous image frame 2t The first brightness value of the previous image frame, last_L1, and the statistical brightness value of the current image frame, current_L. r The relationship can be expressed by the following formula:

[0111]

[0112] The first brightness value current_L1 of each current image frame is based on the second target brightness value last_L of the previous image frame. 2t The first brightness value of the previous image frame, last_L1, and the statistical brightness value of the current image frame, current_L. rThe first brightness value current_L1 of each current image frame is obtained, and the second target brightness value current_L of the current image frame is obtained accordingly. 2t Until the second target brightness value current_L of the current image frame. 2t The second target brightness value last_L in the previous image frame 2t Until they are equal, at this point, the second target brightness value current_L of the current image frame. 2t It has reached convergence.

[0113] Synchronized, the preset brightness value current_L of the current image frame. p Based on the preset brightness value last_L of the previous image frame p The preset brightness value current_L of the current image frame is obtained by combining the first brightness compensation value current_E1 of the current image frame. p The preset brightness value last_L of the previous image frame p The relationship between the current_E1 value and the first brightness compensation value of the current image frame can be expressed by the following formula:

[0114]

[0115] According to formula (6), the preset brightness value last_L of the previous image frame is... p The baseline brightness value is used as the base brightness value, and the preset brightness value last_L of the previous image frame is calculated based on the first brightness compensation value current_E1 of the current image frame. p By performing compensation, the actual preset brightness value current_L of the current image frame can be obtained. p That is, the preset brightness value current_L of the current image frame. p .

[0116] In the process of calculating pixel brightness using the aforementioned brightness histogram, the AE algorithm needs to obtain the brightness information of each pixel from the CMOS sensor. Typically, CMOS sensors capture RGB images. To simplify calculations, the AE algorithm converts the RGB values ​​into a single grayscale value Y. The relationship between the grayscale value Y and the RGB values ​​can be expressed by the following formula:

[0117] Y=0.299R+0.587G+0.114B Formula (6);

[0118] Because in the bright areas, one of the values ​​in the R, G, and B channels will be very large, not much different from the actual brightness, and can usually represent the actual brightness. As shown in formula (6), when using Y to calculate the average brightness, each value in the R, G, and B channels is multiplied by a coefficient less than 1, causing each value in the R, G, and B channels to be less than its actual brightness value. Therefore, for example, see Appendix... Figure 6 Instead of using the Y value to calculate the average brightness, the maximum value among the four single channels Y, R, G, and B can be used. In other words, the brightness of a pixel is L = max(Y, R, G, B). This ensures the accuracy of the average brightness value calculated for the bright areas.

[0119] In one possible implementation, the first brightness value L1 can be calculated by the maximum value among the four single channels Y, R, G, and B, or by the maximum value among the three single channels R, G, and B. The present application embodiment does not limit the calculation method of the first brightness value L1.

[0120] In one possible implementation, the first brightness value L1 is obtained based on the maximum brightness value among the red, green and blue RGB brightness values ​​in the bright area. That is, the largest value among the three single channels is selected to calculate the first brightness value current_L1 of the current image frame. This can further optimize the right-side metering algorithm and avoid brightness overflow in bright scenes.

[0121] In this embodiment of the application, a right-side metering algorithm is used to obtain the second target brightness value L for the bright area. 2t After adjusting the brightness of the image frames and solving the problem of overexposure in the bright areas, the next step is to brighten the dark areas to solve the problem of completely black dark areas.

[0122] In one possible implementation, the brightness histogram further includes dark regions; the method further includes: obtaining a third brightness value L3 and a fourth brightness value L4; the third brightness value L3 is used to characterize the brightness of the dark regions; the fourth brightness value L4 is used to characterize the ambient brightness of the dark regions of the image frame; a contrast ratio (contrast) is obtained based on the third brightness value L3 and the first brightness value L1; and a third target brightness value L is obtained based on the contrast ratio (contrast) and the fourth brightness value L4. 3t Based on the third target brightness value L 3t The second brightness compensation value E2 is obtained by combining the third brightness value L3; based on the preset brightness value L... p The second brightness compensation value E2 is used to obtain the fourth target brightness value L. 4t Based on the fourth target brightness value L 4t Adjust the brightness of the image frame.

[0123] For example, attached Figure 7 A flowchart illustrating an image processing method provided in an embodiment of this application is attached. Figure 7 As shown, the method may further include steps S701-S706:

[0124] Step S701: Obtain the third brightness value L3 and the fourth brightness value L4; the third brightness value L3 is used to characterize the brightness of the dark area; the fourth brightness value L4 is used to characterize the ambient brightness of the dark area of ​​the image frame.

[0125] In one possible implementation, the dark area is the darkest region in the brightness histogram, representing the second largest proportion of pixels. For example, see the appendix. Figure 5 The second proportion is 20%, and the dark area is the darkest area in the brightness histogram, which accounts for 20% of the number of pixels, that is, the area on the left side of the brightness histogram that accounts for 20%.

[0126] In one possible implementation, the second ratio can be 20% or 25%, and the embodiments of this application do not limit the size of the second ratio.

[0127] The third luminance value L3 is used to characterize the luminance of the dark area. The third luminance value L3 is obtained by statistically analyzing the average luminance of the dark area in the luminance histogram. In one possible implementation, the average luminance of the dark area can be calculated using an arithmetic average algorithm or a geometric average algorithm. This application does not limit the algorithm used for the average luminance of the dark area in its embodiments.

[0128] The fourth luminance value L4 is used to characterize the ambient luminance of the dark areas of the image frame, and is related to the average luminance L of the dark areas of the image frame. d Correlation, while the average brightness L of the dark areas of the image frame d It is obtained by averaging the brightness of the dark areas in the brightness histogram. In one possible implementation, the dark areas of the brightness histogram can be calculated using either an arithmetic average algorithm or a geometric average algorithm. This application does not limit the algorithm used to calculate the average brightness of the dark areas of the brightness histogram.

[0129] Similarly, the fourth luminance value L4 and the average luminance L of the dark areas of the image frame d The relationship between them can be expressed by the following formula:

[0130]

[0131] Similar to formula (1), K is a constant, t e The exposure time is denoted by ISO, which is the sensitivity, or gain.

[0132] Similarly, formula (7) shows that the average brightness L of the dark areas of the image frame dFor every doubling of the original value, the fourth luminance value L4 increases by 10, meaning the ambient luminance of the dark areas in the image frame increases by 10. The average luminance L of the dark areas in the image frame... d With exposure H v (exposure H) v = Exposure time t e The gain (ISO) is linear. Similarly, the fourth luminance value L4 is fixed and is used to characterize the ambient luminance of dark areas in an image frame.

[0133] Step S702: Based on the third brightness value L3 and the first brightness value L1, the contrast ratio is obtained.

[0134] The relationship between contrast and the third luminance value L3 and the first luminance value L1 can be expressed by the following formula:

[0135]

[0136] In formula (8), contrast is used to characterize the ratio of the average brightness of the bright area and the dark area in the brightness histogram, that is, the dynamic range of the current scene.

[0137] Step S703: Based on the contrast ratio (contrast) and the fourth luminance value (L4), the third target luminance value (L) is obtained. 3t .

[0138] In one possible implementation, the third target brightness value L 3t The third target brightness value L can be obtained by looking up a table using the contrast ratio (contrast) and the fourth brightness value L4. In the embodiments of this application, different contrast ratios and fourth brightness values ​​L4 result in different third target brightness values ​​L. 3t They are different. For example, as shown in Table 2 (rows are used to represent the change of contrast from low to high, columns are used to represent the change of the fourth luminance value L4 from low to high, and cells are used to represent the third target luminance value L corresponding to the contrast and the fourth luminance value L4). 3t ):

[0139] Table 2 shows the different contrast values ​​(contrast) and fourth luminance values ​​(L4) corresponding to different third target luminance values ​​(L). 3t

[0140]

[0141] Table 2 shows that when the fourth luminance value L4 remains constant and the contrast value (contrast) changes from low to high, the third target luminance value L... 3tThere is a reduction; for example, in the first row, when the fourth luminance value L4 is 30 and the contrast is 1.8, the third target luminance value L... 3t With a brightness value of 6.0, and the fourth brightness value L4 being 30 and the contrast value being 2.0, the third target brightness value L... 3t The value is 6.0; while in the second row, when the fourth brightness value L4 is 35 and the contrast is 1.8, the third target brightness value L... 3t The value is 9.0. When the fourth luminance value L4 is 35 and the contrast is 2.0, the third target luminance value L... 3t The value is 6.0. With the contrast ratio remaining constant, as the fourth luminance value L4 increases, the third target luminance value L... 3t The brightness varies from low to high. For example, in the fourth column, when the fourth brightness value L4 is 30 and the contrast is 1.8, the third target brightness value L... 3t The value is 6.0. When the fourth brightness value L4 is 35 and the contrast is 1.8, the third target brightness value L... 3t The value is 9.0. When the fourth luminance value L4 is 40 and the contrast is 1.8, the third target luminance value L... 3t The value is 12.0; while in the tenth column, when the fourth brightness value L4 is 30 and the contrast is 3.2, the third target brightness value L... 3t With a brightness value of 4.0, and a fourth brightness value L4 of 35 and a contrast ratio of 3.2, the third target brightness value L... 3t With a brightness value of 6.0, and the fourth brightness value L4 being 40 and the contrast value being 3.2, the third target brightness value L... 3t It is 9.0.

[0142] By referring to Table 2, the corresponding third target brightness value L can be obtained based on the contrast (contrast) and the fourth brightness value L4. 3t Thus, the third target brightness value L is obtained. 3t Next, the calculation of the second brightness compensation value E2 in step S704 is performed. The fourth brightness value L4 is used to characterize the actual ambient brightness of the dark area, and contrast is used to characterize the dynamic range of the current scene. The corresponding third target brightness value L is obtained according to Table 2. 3t This method can flexibly adjust the brightness value L of the third target according to the dynamic range of the current scene. 3t Furthermore, the brightness of dark areas can be dynamically adjusted according to user preferences. For example, if a user prefers a textured style in scenes with a large dynamic range—that is, a style with high brightness and darker shadows—then the user can select the third target brightness value L corresponding to a higher contrast ratio.3t It has a relatively small value. Furthermore, it ensures good display performance in high dynamic range scenarios and offers greater flexibility in debugging.

[0143] Step S704, based on the third target brightness value L 3t The second brightness compensation value E2 is obtained by combining the third brightness value L3 with the third brightness value L3.

[0144] Similarly, as in step S403, the third target brightness value L 3t The second brightness compensation value E2, obtained from the third brightness value L3, can be expressed by the following formula:

[0145]

[0146] The brightness compensation value of the dark area can be obtained based on the target brightness and the average brightness of the dark area. According to formula (9), the second brightness compensation value E2 is based on 2, and the third target brightness value L is calculated accordingly. 3t The result of taking the logarithm of the ratio of the third brightness value L3 to the third brightness value L4.

[0147] Step S705, based on the preset brightness value L p The second brightness compensation value E2 is used to obtain the fourth target brightness value L. 4t .

[0148] Similarly, the preset brightness value L p And the preset brightness value L in step S403 p Similarly, the embodiments of this application will not be described again here.

[0149] Similarly, based on the preset brightness value L p The second brightness compensation value E2 is used to obtain the fourth target brightness value L. 4t It can be expressed using the following formula:

[0150]

[0151] Similarly, using the preset brightness value L p Using the reference brightness value, the preset brightness value L is adjusted according to the second brightness compensation value E2. p By performing compensation, the actual target brightness value, i.e., the fourth target brightness value L, can be obtained. 4t In formula (10), the second brightness compensation value E2 is exponentially calculated with 2 as the base, and this exponent is used as the preset brightness value L. p The compensation coefficient is used to obtain the actual target brightness value, i.e., the fourth target brightness value L. 4t .

[0152] Step S706, based on the fourth target brightness value L 4t Adjust the brightness of the image frame.

[0153] Similarly, after obtaining the fourth target brightness value L 4t Then, based on the fourth target brightness value L 4t Adjusting the brightness of image frames to resolve the issue of completely black areas. Since the goal is to brighten dark areas, the fourth target brightness value L... 4t It is greater than the third brightness value L3.

[0154] The image processing method described in steps S701-S706 above involves the following steps: First, the ambient brightness (i.e., the fourth brightness value L4) of the dark area of ​​the image frame and the third brightness value L3 of the dark area are obtained. Second, based on the third brightness value L3 and the first brightness value L1, the contrast is obtained. Next, based on the contrast and the fourth brightness value L4, the third target brightness value L is obtained. 3t Then, based on the third target brightness value L 3t The second brightness compensation value E2 is obtained by combining the third brightness value L3 with the third brightness value L3. Then, based on the preset brightness value L... p The second brightness compensation value E2 is used to obtain the fourth target brightness value L. 4t Finally, based on the fourth target brightness value L 4t This method adjusts the brightness of image frames. It can solve the problem of completely black areas in dark regions, thus improving the display effect of the preview image and enhancing the user experience.

[0155] Similarly, in the embodiments of this application, the image processing method described in steps S701-S706 above, in which each previous image frame is based on the fourth target brightness value L obtained through the second brightness compensation value E2. 4t After adjusting the brightness of an image frame, the second brightness compensation value E2 of the current image frame will decrease. This will cause the adjustment of each current image frame to be opposite to the adjustment of the previous image frame, resulting in little effect of the adjustment of the current image frame. Therefore, it is necessary to ensure that the second brightness compensation value E2 is not affected by the convergence of the AE algorithm.

[0156] Similarly, to ensure that the second brightness compensation value E2 is not affected by the convergence of the AE algorithm, the third brightness value current_L3 of the current image frame and the fourth target brightness value last_L of the previous image frame are compared. 4t The third brightness value of the previous image frame, last_L3, and the statistical brightness value of the current image frame, current_L. r The relationship can be expressed by the following formula:

[0157]

[0158] Up to the fourth target brightness value current_L in the current image frame 4t The fourth target brightness value last_L in the previous image frame 4tUntil they are equal, at this point, the fourth target brightness value current_L of the current image frame. 4t It has reached convergence.

[0159] Synchronized, the preset brightness value current_L of the current image frame. p The preset brightness value last_L of the previous image frame p The relationship between the current_E2 value and the second brightness compensation value of the current image frame can be expressed by the following formula:

[0160]

[0161] In this embodiment of the application, firstly, for the bright area, based on the second target brightness value L 2t After adjusting the brightness of the image frame, the overall brightness of the image frame will decrease relative to the original brightness. Secondly, for the low-to-medium brightness areas, including the central and dark regions, based on the fourth target brightness value L... 4t After adjusting the brightness of an image frame, the overall brightness of the image frame will increase compared to the previous adjustment targeting the highlight area. After two brightness adjustments to the image frames, the brightness of the highlight area, i.e., the fifth target brightness value L, will be... 5t The second target brightness value L after the first brightness adjustment 2t The sum of the brightness differences from the second brightness adjustment, where the sum of the brightness differences from the second brightness adjustment is the fourth target brightness value L. 4t The difference between the fifth target brightness value L and the third brightness value L3 is the fifth target brightness value L. 5t =Second target brightness value L 2t +(Fourth target brightness value L) 4t - Third brightness value L3). At this time, the fifth target brightness value L 5t The brightness may be too high, resulting in an overall bright image with little detail in the highlights.

[0162] Therefore, to avoid the fifth target brightness value L in the bright area 5t To ensure the display effect of the highlighted areas, a brightness threshold L can be set in the highlighted areas. th The high-brightness area and the medium-low brightness area are analyzed separately. In the following embodiments of this application, a brightness threshold L is set for the high-brightness area. th Taking an example, the image processing method of this application will be explained in detail.

[0163] For the highlighted areas, in one possible implementation, at the fifth target brightness value L 5t Greater than the brightness threshold L th In the case of a brightness threshold L th Increase the brightness of the highlighted areas; where the brightness threshold L thGreater than the first target brightness value L 1t And less than or equal to 255; the fifth target brightness value L 5t The second target brightness value L 2t The sum of the brightness differences is the fourth target brightness value L. 4t The difference between the third brightness value L3 and the fifth target brightness value L. 5t Less than or equal to the brightness threshold L th In the case of the fifth target brightness value L 5t Increase the brightness of the highlighted areas.

[0164] In other words, after two brightness adjustments, at the fifth target brightness value L 5t (Fifth target brightness value L) 5t =Second target brightness value L 2t +(Fourth target brightness value L) 4t - The third brightness value L3) is greater than the brightness threshold L th In the case of a brightness threshold L th Increase the brightness of the highlighted areas. At the fifth target brightness value L 5t Less than or equal to the brightness threshold L th In the case of the fifth target brightness value L 5t Increase the brightness of the highlighted areas.

[0165] Wherein, brightness = gain ISO × aperture N × exposure time t e It can be expressed using the following formula:

[0166] L = N × t e ×ISO=(N×ISO)×t e Formula (13);

[0167] According to formula (13), the brightness is a slope of (N×ISO) and the exposure time t. e Since N is fixed, the slope of the relevant straight line is only related to the gain ISO.

[0168] For example, see Appendix Figure 8 Highlighted areas, at the fifth target brightness value L 5t Less than or equal to the brightness threshold L th In this case, based on line l1, the brightness of the highlighted area is increased. At the fifth target brightness value L... 5t Greater than the brightness threshold L th In this case, based on the straight line l2, the brightness of the highlighted area is increased.

[0169] For low to medium brightness areas, in one possible implementation, the method further includes: setting a fifth target brightness value L. 5t Greater than the brightness threshold Lth In the case of the sixth target brightness value L 6t Increase the brightness of low and medium brightness areas; sixth target brightness value L 6t Greater than or equal to the third target brightness value L 3t And less than or equal to the fourth target brightness value L 4t ; at the fifth target brightness value L 5t Less than or equal to the brightness threshold L th In the case of the fourth target brightness value L 4t This increases the brightness of areas with low to medium brightness.

[0170] At the fifth target brightness value L 5t Greater than the brightness threshold L th In this case, it indicates that the low to medium brightness area is based on the fourth target brightness value L. 4t When increasing the brightness of the low-to-medium brightness areas, the high-brightness areas are already overexposed. Therefore, in this embodiment, a fourth target brightness value L can be selected. 4t The straight line l1 is used as a reference for the overexposure line. Since the third brightness value L3 is the brightness of the dark area after the bright area has been overexposed, the straight line l2 of the third brightness value L3 can be used as a reference for the underexposure line.

[0171] In one possible implementation, based on the sixth target brightness value L 6t To increase the brightness of low-to-medium brightness areas, a curve can be used. To ensure the brightness of dark areas, the curve can be adjusted between the straight line l2 of the third brightness value L3 and the fourth target brightness value L. 4t A curve l3 is fitted between the straight line l1 and the curve l3 as the sixth target brightness value L. 6t Therefore, based on the sixth target brightness value L 6t This increases the brightness of the low-to-medium brightness areas. Due to the third target brightness value L... 3t It is based on the contrast ratio (contrast) and the fourth luminance value (L4), and the third target luminance value (L). 3t Greater than the third brightness value L3 and less than the fourth target brightness value L 4t Therefore, the third target brightness value L can be selected. 3t The slope of the straight line is used as the sixth target brightness value L. 6t The minimum slope of curve l3.

[0172] For example, see Appendix Figure 9 In the low to medium brightness region, at the fifth target brightness value L 5t Greater than the brightness threshold L th In this case, based on curve l3, the brightness of the low-to-medium brightness region is increased, where the slope of curve l3 is between the straight line l2 of the third brightness value L3 and the fourth target brightness value L. 4tThe slope of the straight line l1, that is, the gain ISO of curve l3 between the third brightness value L3 and the fourth target brightness value L... 4t Between the gain ISO, for example, a third target brightness value L can be selected. 3t The slope of the straight line is taken as the minimum value of the slope of curve l3, that is, the third target brightness value L is selected. 3t The sixth target brightness value L 6t The minimum brightness value. At the fifth target brightness value L 5t Less than or equal to the brightness threshold L th In the case of the fourth target brightness value L 4t The straight line l1 increases the brightness of the low-to-medium brightness area.

[0173] The above-described methods for increasing brightness in both high-brightness and low-to-medium-brightness areas can brighten dark areas while avoiding overexposure in high-brightness areas. For examples, see the appendix. Figure 10 Before a user clicks / touches the shutter 302 to capture multiple frames of images, the electronic device 100 displays a shooting preview interface 301. In the preview image in the shooting preview interface 301, the bright areas (sun) are highlighted and underexposed, and the dark areas (the sea level at the bottom of the image) are brightened, which improves the display effect of the preview image and thus improves the user experience.

[0174] The image processing method provided in this application embodiment can meter the light to the right in bright areas, reducing the brightness of the image frame and solving the problem of overexposure in bright areas. It can also increase the brightness of image frames in dark areas, solving the problem of completely black areas. Therefore, it can improve the display effect of the preview image and enhance the user experience.

[0175] It is understood that, in order to achieve the above functions, the electronic device includes hardware and / or software modules that perform the respective functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application. This embodiment can divide the electronic device into functional modules based on the above method examples. For example, each function can be divided into separate functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0176] This application also provides a computer-readable storage medium storing computer program code. When the processor executes the computer program code, the electronic device executes the relevant method steps in the above method embodiments.

[0177] This application also provides a computer program product that, when run on a computer, causes the computer to execute the relevant method steps described in the above method embodiments.

[0178] The electronic devices, computer storage media, or computer program products provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0181] The units described above as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected based on actual needs to achieve the purpose of this embodiment. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The functions of the integrated unit can be implemented either in hardware or as software functional units.

[0182] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the contributing parts, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0183] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the brightness histogram of the image frame; the brightness histogram includes bright areas; Obtain a first brightness value and a second brightness value; the first brightness value is used to characterize the brightness of the highlighted area; the second brightness value is used to characterize the ambient brightness of the image frame; A first brightness compensation value is obtained based on the first target brightness value and the first brightness value; the first target brightness value is obtained based on the second brightness value. The second target brightness value is obtained based on the preset brightness value and the first brightness compensation value; The brightness of the image frame is adjusted based on the second target brightness value.

2. The image processing method according to claim 1, characterized in that, The brightness histogram further includes dark areas; the method further includes: A third brightness value and a fourth brightness value are obtained; the third brightness value is used to characterize the brightness of the dark area; the fourth brightness value is used to characterize the ambient brightness of the dark area of ​​the image frame. The contrast ratio is obtained based on the third brightness value and the first brightness value; Based on the contrast ratio and the fourth brightness value, a third target brightness value is obtained; Based on the third target brightness value and the third brightness value, a second brightness compensation value is obtained; Based on the preset brightness value and the second brightness compensation value, the fourth target brightness value is obtained; The brightness of the image frame is adjusted based on the fourth target brightness value.

3. The image processing method according to claim 2, characterized in that, The image frame includes the current image frame and the previous image frame; the first brightness value of the current image frame is obtained based on the second target brightness value of the previous image frame, the first brightness value of the previous image frame, and the statistical brightness value of the current image frame; the preset brightness value of the current image frame is obtained based on the preset brightness value of the previous image frame and the first brightness compensation value of the current image frame. The statistical brightness value of the current image frame is used to characterize the average brightness of the brightness histogram of the current image frame.

4. The image processing method according to claim 2 or 3, characterized in that, The method further includes: If the fifth target brightness value is greater than the brightness threshold, the brightness of the bright area is increased based on the brightness threshold; wherein the brightness threshold is greater than the first target brightness value and less than or equal to 255; the fifth target brightness value is the sum of the second target brightness value and the brightness difference, and the brightness difference is the difference between the fourth target brightness value and the third target brightness value; If the fifth target brightness value is less than or equal to the brightness threshold, the brightness of the high-brightness area is increased based on the fifth target brightness value.

5. The image processing method according to claim 4, characterized in that, The brightness histogram further includes an intermediate region; the method further includes: If the fifth target brightness value is greater than the brightness threshold, the brightness of the low-to-medium brightness region is increased based on the sixth target brightness value; the low-to-medium brightness region includes the dark region and the intermediate region; the sixth target brightness value is greater than or equal to the third target brightness value and less than or equal to the fourth target brightness value. If the fifth target brightness value is less than or equal to the brightness threshold, the brightness of the low-to-medium brightness region is increased based on the fourth target brightness value.

6. The image processing method according to any one of claims 2-5, characterized in that, The dark area is the darkest region in the brightness histogram, where the number of pixels accounts for the second largest proportion.

7. The image processing method according to any one of claims 1-6, characterized in that, The highlighted area is the area with the highest brightness in the brightness histogram, where the number of pixels accounts for the first proportion.

8. The image processing method according to any one of claims 1-7, characterized in that, The first brightness value is obtained based on the maximum brightness value among the red, green, and blue RGB brightness values ​​in the bright region.

9. An electronic device, characterized in that, The device includes a memory and one or more processors, wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the image processing method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the image processing method as described in any one of claims 1-8.

11. A computer program product, characterized in that, When the computer program product is run on an electronic device, the electronic device performs the image processing method as described in any one of claims 1-8.

Images