Image processing method and related device

Abstract

This application provides an image processing method and related equipment. The method includes: in response to a user's shooting operation, acquiring brightness information of a preview image using the shooting device of the terminal device; determining a first dynamic range value of the preview image based on the brightness information; determining a channel brightness value corresponding to each channel of each pixel in the preview image; determining color parameters of the preview image based on the channel brightness values; updating the first dynamic range value based on the color parameters to obtain a second dynamic range value of the preview image; and optimizing the preview image using a high dynamic range algorithm if the second dynamic range value indicates that the preview image is in a high dynamic range scene. This application can avoid partial single-channel color overflow in the captured image, improving the imaging effect of the captured image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, and belongs to the technical field of image processing, in particular to an image processing method and related equipment. BACKGROUND

[0002] In the process of photographing an image by using a photographing device (for example, a camera of a mobile phone) of a terminal device, a high dynamic range algorithm is usually used to suppress a highlight overexposure region in the image when it is determined that the current scene is a high dynamic range scene, so that the photographed image can have more dynamic range and image details.

[0003] However, in some color-rich scenes, the system in the terminal device may not call the high dynamic range algorithm to process the image due to the failure to correctly identify the high dynamic range scene, resulting in poor processing effect of the image. SUMMARY

[0004] In view of the above, it is necessary to provide an image processing method and related equipment, which can solve the problem that the high dynamic range scene cannot be correctly identified due to the failure to consider the single-channel color overexposure scene in the related art, so that part of the single-channel color overflow appears in the photographed image.

[0005] In a first aspect, the present application provides an image processing method applied to a terminal device, the method comprising: in response to a photographing operation of a user, acquiring luminance information of a preview image by using a photographing device of the terminal device; determining a first dynamic range value of the preview image according to the luminance information; determining a channel luminance value corresponding to each channel of each pixel point in the preview image; determining a color parameter of the preview image according to the channel luminance value; updating the first dynamic range value based on the color parameter to obtain a second dynamic range value of the preview image; and using a high dynamic range algorithm to optimize the preview image if it is determined that the preview image is in a high dynamic range scene according to the second dynamic range value.

[0006] Through the above technical solution, in the process of photographing an image by using a photographing device, the initial first dynamic range value of the preview image can be determined according to the luminance information of the preview image; then the color parameter corresponding to the exposure state of the single-channel color in the preview image is determined through the channel luminance value corresponding to each channel of each pixel point in the preview image; then the first dynamic range value is updated by using the color parameter, so that the obtained second dynamic range value can be applicable to the single-channel color overexposure scene; if it is determined that the preview image is in a high dynamic range scene based on the second dynamic range value, the preview image is optimized by using the high dynamic range algorithm, which can avoid the single-channel color overexposure in the photographed image and improve the presentation effect of the photographed image.

[0007] In one possible implementation, the brightness information includes: the average brightness of the preview image, the first exposure factor of the preview image, and the second exposure factor of the preview image.

[0008] In one possible implementation, the method for determining the brightness information includes: determining the maximum brightness value of the preview image based on the grayscale image of the preview image; determining a first brightness range, a second brightness range, and a third brightness range based on the maximum brightness value; determining the average brightness value based on a first average brightness value of pixels within the first brightness range; determining the first exposure factor based on a second average brightness value of pixels within the second brightness range; and determining the second exposure factor based on a third average brightness value of pixels within the third brightness range.

[0009] Through the above technical solution, the maximum brightness value of the preview image can be determined based on the grayscale image of the preview image. Then, the first brightness range, the second brightness range and the third brightness range can be determined based on the maximum brightness value, thereby determining the average brightness value, the first exposure factor and the second exposure factor of the preview image, which facilitates the calculation of the first dynamic range value.

[0010] In one possible implementation, determining the first dynamic range value of the preview image based on the brightness information includes: determining the first dynamic range value based on the average brightness value, the first exposure factor, and the second exposure factor.

[0011] Using the above technical solution, based on the average brightness value, the first product of the first exposure factor and the second exposure factor, the first dynamic range value of the preview image, excluding single-channel color overexposure scenarios, can be determined.

[0012] In one possible implementation, determining the color parameters of the preview image based on the channel brightness values ​​includes: determining a first exposure ratio corresponding to each channel and a second exposure ratio of the preview image based on the channel brightness values; determining a maximum exposure ratio of the preview image based on the maximum value among the first exposure ratios; and determining the color parameters based on a first ratio of the maximum exposure ratio to the second exposure ratio.

[0013] The above technical solution allows us to determine the first exposure ratio for each channel based on its channel brightness value, and the second exposure ratio for the white exposure of the preview image, which is determined by comprehensively considering the channel brightness values ​​of all channels. By comparing the maximum value in the first exposure ratio with the second exposure ratio, we can determine whether there is overexposure in a single channel in the preview and determine the corresponding color parameters.

[0014] In one possible implementation, the method for determining the first exposure ratio includes: if the channel brightness value of any channel of any pixel is greater than a preset first threshold, the pixel is designated as the first overexposed pixel of the channel; a first number of first overexposed pixels for each channel is determined; and the first exposure ratio is determined based on a second ratio of the first number to a second number of pixels in the preview image.

[0015] By using the above technical solution, the first number of first overexposed pixels in the preview image whose single-channel brightness value exceeds the first threshold is counted, thereby determining the number of pixels that may be overexposed in a single channel; based on the ratio of the first number to the second number of all pixels in the preview image, the proportion of single-channel overexposed pixels in the preview image can be accurately determined.

[0016] In one possible implementation, the method for determining the second exposure ratio includes: if the channel brightness values ​​of all channels of any pixel are greater than a preset first threshold, the pixel is designated as the second overexposed pixel of the preview image; a third number of the second overexposed pixels is determined, and the second exposure ratio is determined based on a third ratio of the third number to the second number of pixels in the preview image.

[0017] Using the above technical solution, pixels whose channel brightness values ​​are all greater than the first threshold are designated as second overexposed pixels. By counting the third number of second overexposed pixels, the number of white overexposed pixels in the preview image can be determined. Then, based on the ratio of the third number to the second number, the proportion of white overexposed pixels in the preview image can be accurately determined.

[0018] In one possible implementation, determining the color parameter based on a first ratio of the maximum exposure ratio to the second exposure ratio includes: if the first ratio is less than a preset second threshold, using a preset first value as the color parameter; or if the first ratio is greater than a preset third threshold, using a preset second value as the color parameter; or if the first ratio is greater than or equal to the second threshold and less than or equal to the third threshold, determining a ratio difference between the maximum exposure ratio and the second exposure ratio, determining the product of the third threshold and the second exposure ratio, and determining the color parameter based on a fourth ratio of the ratio difference and the product.

[0019] The above technical solution can determine the degree of overexposure of a single channel color in the preview image based on the size of the first ratio, and set the corresponding color parameters according to the degree, so as to make it easier to adjust and update the first dynamic range value according to the color parameters.

[0020] In one possible implementation, updating the first dynamic range value based on the color parameters to obtain a second dynamic range value for the preview image includes: determining the second dynamic range value based on the color parameters and the first dynamic range value.

[0021] In one possible implementation, determining whether the preview image is in a high dynamic range scene based on the second dynamic range value includes: if the second dynamic range value is greater than a preset fourth threshold, determining that the preview image is in a high dynamic range scene; or if the second dynamic range value is less than or equal to the fourth threshold, determining that the preview image is not in a high dynamic range scene.

[0022] The above technical solution allows us to determine whether a preview image is a high dynamic range (HMR) scene based on the magnitude of the second dynamic range value, facilitating the optimization of the preview image using HMR algorithms. Specifically, if the second dynamic range value is greater than or equal to the first dynamic range value, then a single-channel overexposed scene can be identified as a HMR scene.

[0023] Secondly, this application provides a terminal device, the terminal device including a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the terminal device performs the above-described image processing method.

[0024] Thirdly, this application provides a chip coupled to a memory in a terminal device, the chip being used to control the processor of the terminal device to execute the above-described image processing method.

[0025] Fourthly, this application provides a computer storage medium storing program instructions that, when executed on a terminal device, cause the processor of the terminal device to perform the aforementioned image processing method.

[0026] Furthermore, the technical effects brought about by the second to fourth aspects can be found in the descriptions of the methods in the above-mentioned method section, and will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a preview image provided in one embodiment of this application.

[0028] Figure 2 This is a software architecture diagram of a terminal device provided in an embodiment of this application.

[0029] Figure 3 This is a flowchart of an image processing method provided in an embodiment of this application.

[0030] Figure 4 This is an example diagram showing the abbreviations and corresponding meanings of parameters provided in an embodiment of this application.

[0031] Figure 5 This is a flowchart of a method for determining brightness information provided in an embodiment of this application.

[0032] Figure 6 This is a flowchart of a method for determining color parameters provided in an embodiment of this application.

[0033] Figure 7 This is a flowchart of a method for determining the first exposure ratio provided in an embodiment of this application.

[0034] Figure 8 This is a flowchart of a method for determining the second exposure ratio provided in an embodiment of this application.

[0035] Figure 9 This is a flowchart of an image processing method provided in another embodiment of this application.

[0036] Figure 10 This is a hardware architecture diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0037] In one embodiment of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in one embodiment of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to limit the application. It should be understood that, unless otherwise stated, " / " in this application means "or". For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. "At least one" refers to one or more. "More than one" refers to two or more. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, and a, b, and c. Where there is no conflict, the following embodiments and features described herein can be combined with each other.

[0039] To make the description of the embodiments of this application clear and concise, a brief introduction to the relevant concepts or technologies is given first:

[0040] Preview Image: When using the camera (e.g., a mobile phone, tablet, etc.) of a terminal device to capture any object or scene, a preview image can be viewed on the electronic device's display screen. The terminal device can respond to the user's shooting operation by activating the camera to take a picture. For example, when a user opens the camera application on their mobile phone and points the camera at an object to take a picture or record a video, the scene containing that object, which is being pointed at by the phone's camera, can be considered the shooting scene.

[0041] Dynamic Range (DR) value: This value indicates the magnitude of variation in a variable signal (such as sound or light). In the field of imaging, the dynamic range value refers to the range of light intensity in a scene that a camera can capture, and it can be determined based on the brightness values ​​of pixels in the image. A larger dynamic range value indicates a greater difference between the maximum and minimum brightness values ​​in the image, meaning the image has greater contrast between light and dark areas, a wider color gamut, and richer image detail, better reflecting the visual effects of the real environment.

[0042] Multichannel images: These represent images where each pixel has multiple color channels, such as RGB images, RGGB images, RGBW images, and RYYB images. Specifically, an RGB image means each pixel has three color channels: red (R), green (G), and blue (B). An RGGB image means each pixel has four color channels: red, green, green, and blue. An RGBW image means each pixel has four color channels: red, green, blue, and white (W). An RYYB image means each pixel has four color channels: red, yellow (Y), and blue.

[0043] Channel brightness value: This represents the brightness value corresponding to each channel of each pixel in a multi-channel image. For example, if the pixel value of a certain pixel in an RGB image is (200, 100, 150), then the channel brightness value of that pixel includes: a brightness value of 200 for the red channel, a brightness value of 100 for the green channel, and a brightness value of 150 for the blue channel.

[0044] High Dynamic Range (HDR) scene: This refers to a scene where the dynamic range value of an image is greater than or equal to a preset dynamic range (DR) threshold. That is, there are highlight areas and shadow areas in the image, and the difference between the brightness values ​​of the highlight areas and the brightness values ​​of the shadow areas is greater than or equal to the preset DR threshold.

[0045] HDR Algorithm: If the preview image is determined to be a high dynamic range scene, the terminal device can use a high dynamic range algorithm to capture multiple images of the same scene with different exposure values ​​through the camera, fuse the multiple images to generate a single image, and perform tone mapping on the single image to generate an HDR image. The HDR image is the image obtained after optimizing the preview image.

[0046] Exposure factor: This indicates the exposure level of the scene being captured in the preview image. A preview image can include multiple exposure factors, such as overexposure factor and underexposure factor. The overexposure factor characterizes the overexposure of pixels in the preview image, where overexposure pixels are brighter pixels. The underexposure factor characterizes the underexposure of pixels in the preview image, where underexposure pixels are darker pixels.

[0047] Single-channel color overexposure refers to a situation where, in a multi-channel preview image of a scene with multiple colors, the brightness value of a certain color channel exceeds the range that the channel can represent, resulting in loss or compression of detail in that channel. In this case, the color information of that channel will appear abnormal, exhibiting an overly bright or oversaturated effect. For example, the maximum range of brightness values ​​for each color channel in an RGB image is [0, 255]. If the brightness value of the red channel of a pixel in an RGB image is greater than or equal to 255, then the red channel of that pixel is considered to be single-channel overexposed, and this pixel can be regarded as a single-channel overexposed pixel corresponding to the red channel.

[0048] See Figure 1 The diagram shown is a schematic representation of a preview image provided in an embodiment of this application. After acquiring the preview image, the terminal device can determine whether the preview image is an HDR scene based on its brightness information. If it determines that the preview image is not an HDR scene, it directly outputs the preview image; or, if it determines that the preview image is an HDR scene, it uses an HDR algorithm to optimize the preview image and outputs the optimized image. For example... Figure 1 As shown, when photographing green plants and flowers, the image on the left is a preview image that was directly output without HDR algorithm processing, which is prone to overexposure of red flowers when rendering colors; the image on the right is an optimized image output after HDR algorithm processing, which can suppress overexposure of red flowers and more closely resemble the effect of real flowers.

[0049] In some embodiments, in colorful shooting scenes (e.g., scenes containing greenery and flowers), the high dynamic range scene determination methods used in related technologies do not consider single-channel color overexposure scenes. This results in single-channel color overexposure scenes not being identified as HDR scenes, thus preventing the application of HDR algorithms for image processing and causing some single-channel color overflow in the captured image. For example... Figure 1 As shown, the flower colors in the left image are overexposed; the right image uses an HDR algorithm to suppress the overexposure of the flower colors, making them closer to the true colors of the flowers.

[0050] To address the aforementioned problems, this application provides an image processing method. During the image capture process using a shooting device, an initial first dynamic range value for the preview image can be determined based on the brightness information of the preview image. Then, by using the channel brightness value corresponding to each channel of each pixel in the preview image, color parameters corresponding to the exposure state of a single-channel color in the preview image are determined. The first dynamic range value is then updated using these color parameters, making the resulting second dynamic range value applicable to single-channel color overexposure scenarios. If the preview image is determined to be a high dynamic range scenario based on the second dynamic range value, a high dynamic range algorithm is used to optimize the preview image, which can prevent single-channel color overexposure in the captured image and improve the image's presentation. The specific flow of the image processing method will be combined with... Figure 3 The process shown is explained in detail below.

[0051] See Figure 2 The diagram shown is a software architecture diagram of a terminal device provided in an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example, the Android system, from top to bottom, consists of the application layer 101, framework layer 102, Android runtime and system libraries 103, hardware abstraction layer 104, kernel layer 105, and hardware layer 106.

[0052] Application layer 101 may include a series of application packages. For example, application packages may include applications such as camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, SMS, device control services, etc.

[0053] The framework layer 102 provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions. For example, it may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0054] The window manager manages window programs. It can obtain screen size, determine the presence of a status bar, lock the screen, and capture screenshots. The content provider stores and retrieves data, making it accessible to applications. This data can include videos, images, audio, made and received calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. The display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images. The phone manager provides communication functionality for terminal devices, such as managing call status (including connection, hang-up, etc.). The resource manager provides applications with various resources, such as localized strings, icons, images, layout files, and video files. The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short pause without user interaction. For example, the notification manager is used to notify of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the system's top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, causing the device to vibrate, and flashing indicator lights.

[0055] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries.

[0056] Application layer 101 and framework layer 102 run in a virtual machine. The virtual machine executes the Java files of the application layer and framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0057] System library 103 may include multiple functional modules. For example, a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0058] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0059] Hardware Abstraction Layer 104 runs in user space, encapsulates kernel-level drivers, and provides calling interfaces to the upper layers.

[0060] Kernel layer 105 is the layer between hardware and software. Kernel layer 105 contains at least the display driver, touch driver, audio driver, and sensor driver.

[0061] Kernel layer 105 is the core of the operating system for terminal devices. It is the first layer of software extension based on the hardware, providing the most basic functions of the operating system. It is the foundation for the operation of the operating system, responsible for managing system processes, memory, device drivers, files, and network systems, and determining the system's performance and stability. For example, the kernel layer can determine the timing of an application's operation on certain parts of the hardware.

[0062] Kernel layer 105 includes hardware-dependent programs such as interrupt handlers and device drivers, as well as basic, common, and frequently running modules such as clock management and process scheduling modules, and critical data structures. The kernel layer can be located within the processor or embedded in internal memory.

[0063] The hardware layer 106 includes the hardware of the terminal device, such as the display screen, buttons, camera, etc.

[0064] Since the following embodiments involve the processing of different parameters, in order to make the description of each embodiment clear and concise, the following embodiments will be combined with Figure 4 The parameters shown are explained below. Figure 4 Several parameter abbreviations and their corresponding meanings are provided, and the relevant content is also explained in detail in the embodiments below.

[0065] See Figure 3 The diagram shown is a flowchart of an image processing method provided in an embodiment of this application. The image processing method is applied in a terminal device and includes the following steps.

[0066] S101, in response to the user's shooting operation, the brightness information of the preview image is obtained using the shooting device of the terminal device.

[0067] In one embodiment of this application, the user's shooting operation can be either opening a camera application or clicking the shooting control after the camera application is opened. The terminal device can acquire a preview image in response to the user's shooting operation. For example, when a user opens the camera application and points the camera at an object to take a picture or record a video, the terminal device responds to the user's operation and displays a preview image containing the object on the screen. In another embodiment, the user's shooting operation can also be performed when using an application (APP) and using the application's camera function to call the camera application to acquire a preview image. For example, using an instant messaging application to call the camera application to take a picture of any object.

[0068] In one embodiment of this application, the brightness information of the preview image can be determined by converting the preview image into a grayscale image and statistically analyzing the brightness value of each pixel in the grayscale image. The method for determining the brightness information can also be found in the following description. Figure 5 Description of the illustrated embodiment.

[0069] In one embodiment of this application, the grayscale image is an image obtained by equalizing the brightness value of each pixel in the preview image. The method for obtaining the grayscale image of the preview image may include, but is not limited to: (1) averaging method: taking the average value of the brightness values ​​of multiple channels of each pixel in the preview image to obtain the grayscale value of each pixel and generating a grayscale image. For example, if the pixel value of a certain pixel in the RGB image is (200, 100, 150), then the grayscale value of that pixel = (200 + 100 + 150) / 3 = 15 0; (2) Weighted average method: The brightness values ​​of multiple channels of each pixel in the preview image are weighted and averaged to obtain the gray value of each pixel and generate a grayscale image. For example, if the pixel value of a certain pixel in the RGB image is (200, 100, 150), and the weight of the R channel is 0.299, the weight of the G channel is 0.587, and the weight of the B channel is 0.114, then the gray value of the pixel is 0.299*200+0.587*100+0.114*150≈136.

[0070] In one embodiment of this application, the brightness information may include at least: the average brightness of the preview image, a first exposure factor of the preview image, and a second exposure factor of the preview image. The first exposure factor may represent the overexposure factor of the preview image, and the second exposure factor may represent the underexposure factor of the preview image.

[0071] In other embodiments of this application, the brightness information may also include: the maximum brightness value of the preview image, the minimum brightness value of the preview image, the number of pixels with the same brightness value in the preview image, etc., and this application does not impose specific limitations on this.

[0072] S102, determine the first dynamic range value of the preview image based on the brightness information.

[0073] In one embodiment of this application, the first dynamic range value can be determined based on the average brightness, the product of the first exposure factor and the second exposure factor (hereinafter referred to as the first product). For example, the first DR value can be set as value_zone * overexpo * underexpo, where value_zone represents the average brightness of the preview image, overexpo represents the first exposure factor of the preview image, and underexpo represents the second exposure factor of the preview image.

[0074] In other embodiments of this application, the first dynamic range can also be determined based on the ratio of the maximum brightness value to the minimum brightness value of the preview image, or based on the difference between the maximum brightness value and the minimum brightness value of the preview image. This application does not impose specific limitations on this.

[0075] In one embodiment of this application, the brightness information determined from the grayscale image is obtained by equalizing the brightness values ​​of multiple color channels. Therefore, the first dynamic range value of the preview image determined based on the brightness information can be used to measure the overall brightness difference of the preview image, but it cannot be applied to single-channel color overexposure scenarios. Therefore, in subsequent embodiments, the first dynamic range value can be updated using color parameters determined based on the channel brightness values ​​of the preview image to obtain a second dynamic range value suitable for single-channel color overexposure scenarios.

[0076] S103, determine the channel brightness value corresponding to each channel of each pixel in the preview image.

[0077] In one embodiment of this application, the method used to determine the channel brightness value includes, but is not limited to: (1) using a programming language (e.g., Python) and related image processing libraries (e.g., OpenCV, PIL (Python Imaging Library, Python image processing library) to read the pixel value of each pixel in the preview image and split the pixel into different channels, and obtain the corresponding channel brightness value by accessing the pixel value of each channel; (2) converting the preview image into a grayscale image corresponding to each channel, and the grayscale value of each pixel in the grayscale image corresponding to each channel can be used as the channel brightness value. For example, when converting an RGB image into a grayscale image corresponding to the R channel, the weight of the R channel can be 1, the weight of the G channel can be 0, and the weight of the B channel can be 0. Then the grayscale value of each pixel in the grayscale image is the channel brightness value of the R channel.

[0078] S104, determine the color parameters of the preview image based on the channel brightness value.

[0079] In one embodiment of this application, in order to update the first dynamic range value and obtain a second dynamic range value that is applicable to single-channel color overexposure scenarios, color parameters for updating the first dynamic range value can be determined.

[0080] In one embodiment of this application, the color parameter can be used to indicate the degree of overexposure of a single channel color in the preview image. The larger the color parameter, the more severe the overexposure of the single channel color in the preview image. In this embodiment, the color parameter can be represented by a value greater than or equal to 1.

[0081] In one embodiment of this application, the magnitude of the color parameter is related to the number of overexposed pixels in each color channel of the preview image. Furthermore, the magnitude of the color parameter is also related to the number of white overexposed pixels in the preview image. In this embodiment, the value of the color parameter can be determined based on the ratio of the number of overexposed pixels in a single channel to the number of white overexposed pixels. Here, white overexposed pixels refer to pixels whose brightness value in each channel exceeds the range that the channel can represent; white overexposed pixels are considered normal overexposed pixels.

[0082] In one embodiment of this application, the larger the ratio of the number of overexposed pixels in a single channel to the number of overexposed white pixels, the higher the degree of overexposure of the single channel color in the preview image; therefore, the color parameter value can be set to a larger value. The method for determining the color parameter can also be referred to below. Figure 6 Description of the illustrated embodiment.

[0083] S105, update the first dynamic range value based on the color parameters to obtain the second dynamic range value of the preview image.

[0084] In one embodiment of this application, a second dynamic range value is determined based on the color parameter and the first dynamic range value. This second dynamic range value can be determined by multiplying the color parameter and the first dynamic range value; for example, let the second dynamic range value = color parameter × first dynamic range. For instance, if the first dynamic range value is 180 and the color parameter is 2, then the second dynamic range value = 180 × 2 = 360. The method for determining the second dynamic range value is not limited to the implementation method exemplified above; it can also be other mathematical methods or adjusted accordingly based on relevant reference ratios or values.

[0085] S106, determine whether the preview image is in a high dynamic range scene based on the second dynamic range value. If the preview image is in a high dynamic range scene based on the second dynamic range value, execute S107; if the preview image is not in a high dynamic range scene based on the second dynamic range value, execute S108.

[0086] In one embodiment of this application, a preset threshold can be used to determine whether the second dynamic range value is in a high dynamic range scene. For example, assuming the preset threshold is expressed as a fourth threshold, if the second dynamic range value is greater than the preset fourth threshold, the preview image is determined to be in a high dynamic range scene; or if the second dynamic range value is less than or equal to the fourth threshold, the preview image is determined not to be in a high dynamic range scene.

[0087] In one embodiment of this application, the fourth threshold can be customized based on the maximum brightness value that each color channel of the preview image can accommodate, and the fourth threshold will not exceed the maximum brightness value that each color channel of the preview image can accommodate. For example, if the preview image is an RGB image, and the maximum brightness value that an RGB image can accommodate is 255, the fourth threshold can be 255 × 0.9 ≈ 230. The method of setting the fourth threshold is only for illustrative purposes and is not limited in actual applications. For example, the fourth threshold can also be a value such as 240.

[0088] In one embodiment of this application, updating the first dynamic range value based on color parameters yields a second dynamic range value that is greater than or equal to the first dynamic range value. This allows HDR algorithms to be invoked to optimize images even in single-channel color overexposure scenes. For example, if the first dynamic range value of a preview image is 180, which is less than a fourth threshold, the HDR algorithm cannot be invoked for image optimization if only the first dynamic range value is used as the criterion. However, if the method described in the above embodiment is used, and the color parameter of the preview image is determined to be 2, the second dynamic range value can be obtained by updating the first dynamic range value. For example, if the second dynamic range value = 180 × 2 = 360, then the second dynamic range value is greater than the fourth threshold. Therefore, the HDR algorithm can be invoked for image optimization based on the second dynamic range value.

[0089] S107 uses a high dynamic range algorithm to optimize the preview image.

[0090] In one embodiment of this application, the HDR algorithm includes, but is not limited to, a combination of one or more of the following methods: exposure fusion-based methods, image alignment-based methods, and tone mapping-based methods.

[0091] Specifically, optimizing the preview image using the HDR algorithm includes: capturing multiple images at different exposure levels using a camera, including the preview image; performing image alignment on the multiple images to align them at the pixel level, eliminating pixel shifts between images caused by camera positional offset during capture; performing exposure correction on the images based on the exposure time and exposure compensation value of each image to ensure all images have the same exposure level; selecting a short-exposure image from the multiple images, which represents the image with optimal detail in low-brightness areas; and analyzing the multiple images... The system detects the brightness information in each image, identifying the bright areas (areas prone to overexposure). It then fuses the low-brightness details in the short-exposure image with the non-brightness pixels in other images to suppress the brightness of overexposed areas. The fused image undergoes tone mapping to convert it into a displayable image with a wider brightness range and richer details. Post-processing operations, such as noise reduction, sharpening, or color correction, are performed on the displayable image as needed. Finally, the post-processed displayable image is used as the optimized preview image.

[0092] In one embodiment of this application, after the user performs the shooting operation, the image obtained after optimization in the above embodiment is used as the captured image, and then the process proceeds to step S108.

[0093] S108 stores captured images.

[0094] In one embodiment of this application, if the preview image is in a high dynamic range scene, the captured image can be the image obtained after HDR algorithm optimization in S107; if the preview image is not in a high dynamic range scene, the object in the preview image can be captured directly. The captured image can be stored in the storage device of the terminal device, and the user can view the stored image through the gallery application of the terminal device. In another embodiment, the captured image can also be displayed on the screen simultaneously for the user to view while being stored.

[0095] The image processing method provided in this application embodiment can determine the initial first dynamic range value of the preview image based on the brightness information of the preview image during the process of capturing an image using a camera. Then, by using the channel brightness value corresponding to each channel of each pixel in the preview image, the color parameters corresponding to the exposure state of a single channel color in the preview image are determined. The first dynamic range value is then updated using the color parameters, so that the obtained second dynamic range value can be applied to single channel color overexposure scenarios. If the preview image is determined to be in a high dynamic range scenario based on the second dynamic range value, the high dynamic range algorithm is used to optimize the preview image, which can avoid single channel color overexposure in the captured image and improve the presentation effect of the captured image.

[0096] See Figure 5 The diagram shown is a flowchart of a method for determining brightness information according to an embodiment of this application. The method is applied in a terminal device, and the method for determining brightness information includes:

[0097] S201, Determine the maximum brightness value of the preview image.

[0098] In one embodiment of this application, the maximum brightness value of the preview image can be obtained by converting the preview image into a grayscale image, counting the brightness value of each pixel in the grayscale image, and using this maximum brightness value as the maximum brightness value of the preview image. For example, if the preview image is an RGB image, the maximum brightness value can be 255.

[0099] S202, determine the first brightness range, the second brightness range and the third brightness range based on the maximum brightness value.

[0100] In one embodiment of this application, the left boundary of the first brightness range can be determined by multiplying the maximum brightness value by a preset first percentage; the right boundary of the first brightness range can be determined by multiplying the maximum brightness value by a preset second percentage. The first and second percentages can be customized in advance according to actual needs, and this application does not impose any restrictions on them.

[0101] For example, if the maximum brightness value is 255, the first percentage is 5%, and the second percentage is 95%, then the value of the left boundary of the first brightness range is 255 × 5% ≈ 13, the value of the right boundary of the first brightness range is 255 × 95% ≈ 242, and the first brightness range is [13, 242].

[0102] In one embodiment of this application, the left boundary of the second brightness range can be determined by multiplying the maximum brightness value by a preset third percentage; the right boundary of the second brightness range can be determined by multiplying the maximum brightness value by a preset fourth percentage. The third and fourth percentages can be customized as needed, and this application does not impose any restrictions on them.

[0103] For example, if the maximum brightness value is 255, the third percentage is 85%, and the fourth percentage is 100%, then the value of the left boundary of the second brightness range is 255 × 85% ≈ 212, the value of the right boundary of the second brightness range is 255 × 100% ≈ 255, and the second brightness range is [212, 255].

[0104] In one embodiment of this application, the left boundary of the third brightness range can be determined by multiplying the maximum brightness value by a preset fifth percentage; the right boundary of the third brightness range can be determined by multiplying the maximum brightness value by a preset sixth percentage. The fifth and sixth percentages can be set according to actual needs, and this application does not impose any restrictions on them.

[0105] For example, if the maximum brightness value is 255, the fifth percentage is 0%, and the sixth percentage is 15%, then the value of the left boundary of the third brightness range = 255 × 0% ≈ 0, the value of the right boundary of the third brightness range = 255 × 15% ≈ 38, and the third brightness range is [0, 38].

[0106] S203, determine the average brightness value based on the first average brightness value of the pixels within the first brightness range.

[0107] In one embodiment of this application, the number of first pixels in the preview image whose brightness values ​​fall within a first brightness range (hereinafter referred to as the fourth number) and the sum of the brightness values ​​of all first pixels (hereinafter referred to as the first sum) are determined. A first average brightness value is determined based on the ratio of the first sum to the fourth number, and a brightness mean is determined based on the first average brightness value. For example, let the brightness mean = the first average brightness value.

[0108] S204, determine the first exposure factor based on the second average brightness value of the pixels within the second brightness range.

[0109] In one embodiment of this application, the number of second pixels in the preview image whose brightness values ​​are within the second brightness range (hereinafter referred to as the fifth number) and the sum of the brightness values ​​of all second pixels (hereinafter referred to as the second sum) are determined, and a second average brightness value is determined based on the ratio of the second sum to the fifth number.

[0110] In one embodiment of this application, the number of pixels in the preview image whose brightness value is greater than or equal to a second average brightness value (hereinafter referred to as the sixth number) is determined, and a first exposure factor is determined based on the sixth number and the total number of pixels in the preview image (hereinafter referred to as the second number). For example, let the first exposure factor = sixth number / second number. The first exposure factor is used to indicate the overexposure factor of the preview image.

[0111] S205, determine the second exposure factor based on the third average brightness value of the pixels within the third brightness range.

[0112] In one embodiment of this application, the number of third pixels in the preview image whose brightness values ​​are within the third brightness range (hereinafter referred to as the seventh number) and the sum of the brightness values ​​of all third pixels (hereinafter referred to as the third sum) are determined, and a third average brightness value is determined based on the ratio of the third sum to the seventh number.

[0113] In one embodiment of this application, the number of pixels in the preview image whose brightness value is greater than or equal to a third average brightness value (hereinafter referred to as the eighth number) is determined. A second exposure factor is determined based on the eighth number and a second number, wherein the second number represents the total number of pixels in the preview image. For example, let the second exposure factor = eighth number / second number. The second exposure factor is used to indicate the underexposure factor of the preview image.

[0114] Through the above embodiments, the maximum brightness value of the preview image can be determined based on the grayscale image of the preview image. Then, the first brightness range, the second brightness range, and the third brightness range can be determined based on the maximum brightness value, thereby determining the average brightness value, overexposure factor, and underexposure factor of the preview image, and thus determining the first dynamic range value.

[0115] See Figure 6 The diagram shown is a flowchart of a method for determining color parameters according to an embodiment of this application. The method is applied in a terminal device, and the method for determining color parameters includes:

[0116] S301, based on the channel brightness value, determine the first exposure ratio corresponding to each channel and the second exposure ratio of the preview image.

[0117] In one embodiment of this application, the first exposure ratio can be used to indicate the degree of overexposure of a single channel color in each channel of the preview image. The first exposure ratio can be determined based on the ratio of the number of overexposed pixels in a single channel to the total number of pixels in the preview image. The first exposure ratio can also be referred to as the first overexposure ratio. For example, if the preview image is an RGB image, the first exposure ratio for each channel includes: 5% for the R channel, 10% for the G channel, and 15% for the B channel. The method for determining the first exposure ratio can also refer to... Figure 7 The illustrated embodiment.

[0118] In one embodiment of this application, the second exposure ratio is used to indicate the degree of white overexposure in the preview image. The second exposure ratio (e.g., 2%) can be determined based on the ratio of the number of overexposed white pixels to the total number of pixels in the preview image. The second exposure ratio can also be referred to as the second overexposure ratio. The method for determining the second exposure ratio can also refer to...Figure 8 The illustrated embodiment.

[0119] S302, determine the maximum exposure ratio of the preview image based on the maximum value in the first exposure ratio.

[0120] In one embodiment of this application, the maximum value among the first exposure ratios can be used as the maximum exposure ratio. For example, if the preview image is an RGB image, the first exposure ratios corresponding to each channel include: 5% for the R channel, 10% for the G channel, and 15% for the B channel. Then the maximum exposure ratio is the 15% first exposure ratio of the B channel.

[0121] S303 determines the color parameters based on the first ratio of the maximum exposure ratio to the second exposure ratio.

[0122] In one embodiment of this application, the larger the first ratio, the higher the degree of overexposure of a single channel color in the preview image; therefore, the value of the color parameter can be set to a larger value. In this embodiment, the value range of the color parameter can be preset, for example, the value range of the color parameter can be set to [1,2].

[0123] In one embodiment of this application, color parameters can be determined for the following three cases:

[0124] (1) If the first ratio is less than the preset second threshold, the preset first value is used as the color parameter.

[0125] In one embodiment of this application, the second threshold can be customized according to actual needs or based on prior data; this application does not impose any restrictions on this. For example, the second threshold can be set to 2.5.

[0126] In one embodiment of this application, if the first ratio is less than the second threshold, it can be considered that there is no single-channel color overexposure in the preview image or the degree of single-channel color overexposure is very small, and the first DR value can be kept unchanged. Therefore, the first value can be used as a color parameter.

[0127] For example, the first value can be the minimum value of 1 in the range of color parameters. In this way, the second DR value obtained by updating the first DR value with the first value is the same as the first DR value, and the first DR value can remain unchanged.

[0128] (2) If the first ratio is greater than the preset third threshold, the preset second value is used as the color.

[0129] In one embodiment of this application, the third threshold can be customized according to actual needs, or it can be customized based on prior data or empirical values; this application does not impose any restrictions on this. For example, the third threshold can be set to 6.

[0130] In one embodiment of this application, if the first ratio is greater than the third threshold, it can be considered that there is single-channel color overexposure in the preview image and the degree of single-channel color overexposure is very large, so the first DR value needs to be updated to a very large value. Therefore, a larger second value can be used as the color parameter.

[0131] For example, the second value can be the maximum value in the range of color parameters, such as 2. In this case, updating the first DR value with the second value will result in a second DR value that is twice the first DR value, allowing the first DR value to be updated to a larger value. For example, if the preview image is an RGB image and the first DR value is 200, the second DR value = first DR value × color parameter 2 = 200 × 2 = 400.

[0132] (3) If the first ratio is greater than or equal to the second threshold and less than or equal to the third threshold, determine the ratio difference between the maximum exposure ratio and the second exposure ratio, determine the product of the third threshold and the second exposure ratio (hereinafter referred to as the second product), and determine the color parameters based on the fourth ratio of the ratio difference and the second product.

[0133] In one embodiment of this application, if the first ratio is greater than or equal to the second threshold and less than or equal to the third threshold, it can be considered that there is single-channel color overexposure or a large degree of single-channel color overexposure in the preview image, and the first DR value needs to be updated to a larger value. The specific value of the color parameter can be determined based on the fourth ratio of the product of the proportional difference and the second value. Since the value of the fourth ratio is less than 1, the color parameter can be set to the third product of the fourth ratio and the second value, making the range of the color parameter (1, 2).

[0134] For example, if the maximum exposure ratio = 11%, the second exposure ratio = 2%, the second threshold is 2.5, the third threshold is 6, and the first ratio = 11% / 2% = 5.5, then the second threshold < the first ratio < the third threshold; the ratio difference = maximum exposure ratio - second exposure ratio = 11% - 2% = 9%; the second product = 6 × 2% = 12%; the fourth ratio = ratio difference / second product = 9% / 12% = 0.75 < 1. Therefore, we can set the color parameter = fourth ratio × second value = 0.75 × 2 = 1.5.

[0135] The above embodiments can determine the degree of overexposure of a single channel color in the preview image based on the magnitude of the first ratio, and set the corresponding color parameters according to the degree, so as to facilitate the adjustment and updating of the first dynamic range value based on the color parameters.

[0136] See Figure 7The diagram shown is a flowchart of a method for determining a first exposure ratio according to an embodiment of this application. The method is applied in a terminal device, and the method for determining the first exposure ratio includes:

[0137] S401, if the channel brightness value of any channel of any pixel is greater than the preset first threshold, then any pixel is designated as the first overexposed pixel of any channel.

[0138] In one embodiment of this application, the first threshold represents the maximum brightness value that each color channel of the preview image can represent. For example, the maximum range of brightness values ​​for each color channel in an RGB image is [0, 255], and the first threshold can be 255; the first overexposed pixel may include: the first overexposed pixel corresponding to the R channel, the first overexposed pixel corresponding to the G channel, and the first overexposed pixel corresponding to the B channel.

[0139] S402, determine the first number of first overexposed pixels for each channel, and determine the first exposure ratio based on the second ratio of the first number to the second number of pixels in the preview image.

[0140] Through the above embodiments, the first number of overexposed pixels with a single-channel brightness value greater than a first threshold in the preview image can be counted, thereby determining the number of pixels that may have single-channel overexposure. For example, if the preview image is an RGB image, the first number may include: the first number corresponding to the R channel, the first number corresponding to the G channel, and the first number corresponding to the B channel. Based on the ratio of the first number corresponding to each channel to the second number of all pixels in the preview image, the first exposure ratio of the single-channel overexposed pixels in the preview image can be accurately determined.

[0141] See Figure 8 The diagram shown is a flowchart of a method for determining a second exposure ratio according to an embodiment of this application. The method is applied in a terminal device, and the method for determining the second exposure ratio includes:

[0142] S501, if the channel brightness values ​​of all channels of any pixel are greater than the preset first threshold, then any pixel is designated as the second overexposed pixel of the preview image.

[0143] S502, determine the third number of the second overexposed pixels, and determine the second exposure ratio based on the third ratio of the third number to the second number of pixels in the preview image.

[0144] Through the above embodiments, pixels whose channel brightness values ​​are all greater than the first threshold are designated as second overexposed pixels, and the third number of second overexposed pixels is counted to determine the number of white overexposed pixels in the preview image. Then, based on the ratio of the third number to the second number, the proportion of white overexposed pixels in the preview image can be accurately determined.

[0145] See Figure 9 The diagram shown is a flowchart of an image processing method provided in another embodiment of this application. The method is applied in a terminal device, and includes: if the preview image is an RGB image, calculating the R / G / B brightness information of the three color channels R / G / B in the RGB image, where the R / G / B brightness information includes the channel brightness value of the R / G / B channel for each pixel, where " / " represents "or"; and determining whether the R / G / B brightness information of each pixel is greater than or equal to 255.

[0146] Based on the judgment result that the R / G / B brightness information is greater than or equal to 255, the overexposure ratio and white overexposure ratio of the R / G / B channels are calculated respectively, where rRatio represents the overexposure ratio of the R channel, bRatio represents the overexposure ratio of the B channel, gRatio represents the overexposure ratio of the G channel, and whiteRatio represents the white overexposure ratio of the pixels where the R / G / B channels are all greater than or equal to 255; let the maximum exposure ratio maxRatio = max(rRatio / gRatio / bRatio).

[0147] If it is determined that maxRatio is less than or equal to 2.5 × whiteRatio, then the overexposure in the RGB image is determined to be white overexposure, and the color parameter colorRatio = 1. If it is determined that maxRatio is greater than the product of a preset third threshold (e.g., 6) and the white overexposure ratio whiteRatio, then the color parameter colorRatio = 2. If it is determined that maxRatio is less than or equal to a preset second threshold (e.g., 2.5) × white overexposure ratio whiteRatio and less than or equal to a preset third threshold (e.g., 6) × white overexposure ratio whiteRatio, then colorRatio is calculated based on the difference between maxRatio and whiteRatio, and the value range of colorRatio is (1, 2). The DR value is updated using the product of colorRatio and DR value.

[0148] Through the above embodiments, the color parameters corresponding to the exposure state of a single-channel color in a multi-channel image can be determined by the channel brightness value corresponding to each channel of each pixel in the multi-channel image; then, the dynamic range value is updated using the color parameters, so that the updated dynamic range value can be applied to single-channel color overexposure scenarios.

[0149] This application also provides a terminal device 100, see below. Figure 10 As shown, the terminal device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The specific type of terminal device 100 is not specifically limited in the embodiments of this application.

[0150] Terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0151] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0152] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0153] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0154] The processor 110 may also include a memory for storing instructions and data. In one embodiment of this application, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instructions or data again, it can directly retrieve them from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0155] In one embodiment of this application, the processor 110 may include one or more interfaces. These interfaces may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0156] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In one embodiment of this application, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0157] The I2S interface can be used for audio communication. In one embodiment of this application, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to realize communication between the processor 110 and the audio module 170. In one embodiment of this application, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to realize the function of answering phone calls through a Bluetooth headset.

[0158] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In one embodiment of this application, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In another embodiment of this application, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0159] The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In one embodiment of this application, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In one embodiment of this application, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback via Bluetooth headphones.

[0160] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). In one embodiment of this application, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the terminal device 100.

[0161] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In one embodiment of this application, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0162] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge terminal device 100, and can also be used for data transfer between terminal device 100 and peripheral devices. It can also be used to connect headphones for audio playback. Furthermore, the interface can be used to connect other terminal devices 100, such as AR devices.

[0163] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0164] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also supply power to the terminal device 100 via the power management module 141.

[0165] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0166] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0167] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0168] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use on the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In one embodiment of this application, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In another embodiment of this application, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0169] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In one embodiment of this application, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and housed within the same device as the mobile communication module 150 or other functional modules.

[0170] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 100, including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0171] In one embodiment of this application, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the Beidou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0172] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0173] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic Light-Emitting Diode (AMOLED), a Flexible Light-Emitting Diode (FLED), a Minied, Microled, Micro-OLED, or a Quantum Dot Light-Emitting Diode (QLED), etc. In one embodiment of this application, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0174] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0175] The ISP is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In one embodiment of this application, the ISP can be set in the camera 193.

[0176] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In one embodiment of this application, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0177] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0178] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0179] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0180] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0181] Random access memory can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and double data rate synchronous dynamic random-access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM).

[0182] Non-volatile memory can include disk storage devices and flash memory.

[0183] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multi-media card (eMMC), etc.

[0184] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0185] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0186] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the terminal device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0187] Internal memory 121 or external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by processor 110. The one or more computer programs include multiple instructions, which, when executed by processor 110, can implement the screen display detection method executed on terminal device 100 in the above embodiments, so as to realize the screen display detection function of terminal device 100.

[0188] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0189] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In one embodiment of this application, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0190] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music or make hands-free calls through the speaker 170A.

[0191] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 100 answers a phone call or voice message, the receiver 170B can be brought close to the listener's ear to hear the voice.

[0192] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal device 100 may be equipped with at least one microphone 170C. In some embodiments, terminal device 100 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 100 may be equipped with three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0193] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0194] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Terminal device 100 can receive button input and generate key signal inputs related to user settings and function control of terminal device 100.

[0195] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0196] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0197] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal device 100. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In one embodiment of this application, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100. This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on the terminal device 100, the terminal device 100 performs the above-mentioned related method steps to implement the image processing method in the above embodiments.

[0198] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the image processing method described above.

[0199] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the image processing methods in the above-described method embodiments.

[0200] In this embodiment, the terminal device, computer storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0201] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0202] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0203] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0205] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. An image processing method applied to a terminal device, characterized in that, The method includes: In response to the user's shooting operation, the brightness information of the preview image is obtained using the shooting device of the terminal device; The first dynamic range value of the preview image is determined based on the brightness information; Determine the channel brightness value corresponding to each channel of each pixel in the preview image; Determining the color parameters of the preview image based on the channel brightness values ​​includes: determining a first exposure ratio corresponding to each channel and a second exposure ratio of the preview image based on the channel brightness values; determining a maximum exposure ratio of the preview image based on the maximum value among the first exposure ratios; and determining the color parameters based on a first ratio of the maximum exposure ratio to the second exposure ratio. The method for determining the first exposure ratio includes: if the channel brightness value of any channel of any pixel is greater than a preset first threshold, designating the pixel as a first overexposed pixel of that channel; determining a first number of first overexposed pixels for each channel; and determining the first exposure ratio based on a second ratio of the first number to a second number of pixels in the preview image. The method for determining the second exposure ratio includes: if the channel brightness values ​​of all channels of any pixel are greater than a preset first threshold, designating the pixel as a second overexposed pixel of the preview image; determining a third number of second overexposed pixels; and determining the second exposure ratio based on a third ratio of the third number to the second number of pixels in the preview image. The first dynamic range value is updated based on the color parameters to obtain the second dynamic range value of the preview image; If the preview image is determined to be in a high dynamic range scene based on the second dynamic range value, the high dynamic range algorithm is used to optimize the preview image.

2. The image processing method according to claim 1, characterized in that, The brightness information includes: the average brightness of the preview image, the first exposure factor of the preview image, and the second exposure factor of the preview image, wherein the first exposure factor represents the overexposure factor of the preview image, and the second exposure factor represents the underexposure factor of the preview image.

3. The image processing method according to claim 2, characterized in that, The method for determining the brightness information includes: Determine the maximum brightness value of the preview image; The first brightness range, the second brightness range, and the third brightness range are determined based on the maximum brightness value; The average brightness value is determined based on the first average brightness value of the pixels within the first brightness range; The first exposure factor is determined based on the second average brightness value of the pixels within the second brightness range; The second exposure factor is determined based on the third average brightness value of the pixels within the third brightness range.

4. The image processing method according to claim 2 or 3, characterized in that, Determining the first dynamic range value of the preview image based on the brightness information includes: The first dynamic range value is determined based on the average brightness, the first exposure factor, and the second exposure factor.

5. The image processing method according to claim 1, characterized in that, Determining the color parameters based on a first ratio of the maximum exposure ratio to the second exposure ratio includes: If the first ratio is less than a preset second threshold, the preset first value is used as the color parameter; or If the first ratio is greater than a preset third threshold, a preset second value is used as the color parameter; or If the first ratio is greater than or equal to the second threshold and less than or equal to the third threshold, determine the ratio difference between the maximum exposure ratio and the second exposure ratio, determine the product of the third threshold and the second exposure ratio, and determine the color parameter based on the fourth ratio of the ratio difference and the product.

6. The image processing method according to claim 1, characterized in that, Determining whether the preview image is in a high dynamic range scene based on the second dynamic range value includes: If the second dynamic range value is greater than the preset fourth threshold, the preview image is determined to be in a high dynamic range scene; or If the second dynamic range value is less than or equal to the fourth threshold, it is determined that the preview image is not in a high dynamic range scene.

7. A terminal device, characterized in that, The terminal device includes a memory and a processor: The memory is used to store program instructions; The processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the terminal device performs the image processing method as described in any one of claims 1 to 6.

8. A computer storage medium, characterized in that, The computer storage medium stores program instructions that, when executed on a terminal device, cause the processor of the terminal device to perform the image processing method as described in any one of claims 1 to 6.

Images