Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The application relates to an image processing method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a to-be-processed image, the to-be-processed image being generated based on a first image, the brightness of the to-be-processed image being different from the brightness of the first image; determining a first region in the first image and determining a second region corresponding to the first region in the to-be-processed image; determining a tone mapping relationship between the first region and the second region according to each pixel in the first region and each pixel in the second region; and performing global tone mapping on the to-be-processed image based on the tone mapping relationship to obtain a target image. The brightness of the image can be accurately adjusted by using the method.

Description

Technical Field

[0001] This application relates to imaging technology, and in particular to an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of imaging technology, tone mapping algorithms have emerged. Tone mapping algorithms adjust the brightness of an image by mapping and transforming its colors, making the processed image look more comfortable. However, traditional tone mapping can easily lead to over- or under-brightening of certain areas of the image. Summary of the Invention

[0003] This application provides an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can avoid the problem of uneven color tone in different areas of an image.

[0004] An image processing method, the method comprising:

[0005] Obtain an image to be processed, wherein the image to be processed is generated based on a first image, and the brightness of the image to be processed is different from the brightness of the first image;

[0006] A first region in the first image is determined, and a second region in the image to be processed corresponding to the first region is determined;

[0007] Based on each pixel in the first region and each pixel in the second region, determine the tone mapping relationship between the first region and the second region;

[0008] Based on the tone mapping relationship, global tone mapping is performed on the image to be processed to obtain the target image.

[0009] An image processing apparatus, the apparatus comprising:

[0010] The acquisition module is used to acquire an image to be processed, which is generated based on a first image, and the image brightness of the image to be processed is different from that of the first image;

[0011] A region determination module is used to determine a first region in the first image and to determine a second region in the image to be processed that corresponds to the first region;

[0012] The relationship determination module is used to determine the tone mapping relationship between the first region and the second region based on each pixel in the first region and each pixel in the second region;

[0013] The mapping module is used to perform global tone mapping on the image to be processed based on the tone mapping relationship to obtain the target image.

[0014] An electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the following steps:

[0015] A process is to acquire an image to be processed, which is generated based on a first image and has a brightness different from that of the first image; a first region in the first image is determined, and a second region in the image to be processed corresponding to the first region is determined; a tone mapping relationship between the first region and the second region is determined based on each pixel in the first region and each pixel in the second region; and a global tone mapping is performed on the image to be processed based on the tone mapping relationship to obtain a target image.

[0016] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0017] A process is to acquire an image to be processed, which is generated based on a first image and has a brightness different from that of the first image; a first region in the first image is determined, and a second region in the image to be processed corresponding to the first region is determined; a tone mapping relationship between the first region and the second region is determined based on each pixel in the first region and each pixel in the second region; and a global tone mapping is performed on the image to be processed based on the tone mapping relationship to obtain a target image.

[0018] A computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0019] A process is to acquire an image to be processed, which is generated based on a first image and has a brightness different from that of the first image; a first region in the first image is determined, and a second region in the image to be processed corresponding to the first region is determined; a tone mapping relationship between the first region and the second region is determined based on each pixel in the first region and each pixel in the second region; and a global tone mapping is performed on the image to be processed based on the tone mapping relationship to obtain a target image.

[0020] The aforementioned image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product acquire a target image generated based on a first image, wherein the brightness of the target image differs from that of the first image. They then determine a first region in the first image and a second region in the target image corresponding to the first region. Based on pixels in the first and second regions, they determine a tone mapping relationship between the first and second regions, thereby reflecting the tonal difference between the corresponding regions. Based on this tone mapping relationship, they perform global tone mapping on the target image to compensate for its tone, ensuring that the brightness of the target image matches that of the first image. This reduces brightness differences between local regions in the image, avoiding the problem of excessively dark or bright local areas in the target image, thus obtaining a target image with uniform tone across all regions. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an application environment diagram of an image processing method in one embodiment;

[0023] Figure 2 This is a flowchart of an image processing method in one embodiment;

[0024] Figure 3 A flowchart of an image processing method in another embodiment;

[0025] Figure 4 This is a schematic diagram of the interface for global tone mapping in one embodiment;

[0026] Figure 5 This is a flowchart illustrating the calculation of the average first brightness value corresponding to the first region in one embodiment;

[0027] Figure 6 This is a schematic diagram of the interface of a face rectangle in a first grayscale image in one embodiment;

[0028] Figure 7 This is a schematic diagram of a mask image in one embodiment;

[0029] Figure 8 This is a flowchart illustrating the calculation of the average second brightness value corresponding to the second region in one embodiment;

[0030] Figure 9 This is a flowchart illustrating the process of calculating the parameters of a tone mapping function in one embodiment;

[0031] Figure 10 This is a schematic diagram of the tone mapping function in one embodiment;

[0032] Figure 11 This is a structural block diagram of an image processing device in one embodiment;

[0033] Figure 12 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It is understood that the terms "first," "second," etc., used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image. Both the first image and the second image are images, but they are not the same image.

[0036] The image processing method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1As shown, the application environment includes an electronic device 102 and a server 104. The electronic device 102 communicates with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104 or placed in the cloud or on another network server. In one embodiment, both the electronic device 102 and the server 104 can execute the image processing method independently, or they can execute it collaboratively. When the electronic device 102 and the server 104 collaboratively execute the image processing method, the electronic device 102 acquires an image to be processed and sends it to the server 104. This image to be processed is generated based on a first image, but its brightness differs from that of the first image. The server 104 determines a first region in the first image and a second region in the image to be processed corresponding to the first region. The server 104 determines the tone mapping relationship between the first region and the second region based on the pixels in the first region and the pixels in the second region. Based on the tone mapping relationship, the server 104 performs global tone mapping on the image to be processed to obtain the target image. The electronic device 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0037] In one embodiment, such as Figure 2 As shown, an image processing method is provided, which is applied to... Figure 1 Taking an electronic device as an example, the explanation includes the following steps:

[0038] Step 202: Obtain the image to be processed. The image to be processed is generated based on the first image, and the brightness of the image to be processed is different from the brightness of the first image.

[0039] The image to be processed can be a High-Dynamic Range (HDR) image, specifically any one of the following: RGB (Red, Green, Blue) image, RAW image, grayscale image, depth image, or the image corresponding to the Y component of a YUV image. A RAW image is the raw data from which the image sensor converts the captured light source signal into a digital signal. In a YUV image, "Y" represents luminance (or luminance), which is the grayscale value, while "U" and "V" represent chrominance (or chroma), describing the color and saturation of the image and specifying the color of a pixel. The first image can be any one of the following: RGB image, RAW image, grayscale image, depth image, or YUV image.

[0040] The first image can be an image captured from any scene, such as images of people, animals, landscapes, or industrial equipment, but is not limited to these. The first image and the image to be processed are images of the same scene, and the image to be processed is generated based on the first image.

[0041] In an optional embodiment, the first image may be a preview image, and the image to be processed is an image generated based on the preview image.

[0042] It is understandable that the image to be processed and the first image can be a complete image or a partial image region within a complete image.

[0043] The brightness of the image to be processed is different from that of the first image. This means that the brightness value of a pixel in the image to be processed is different from the brightness value of a pixel in the first image. Specifically, it can mean that the brightness value of at least one pixel in the image to be processed is different from the brightness value of the corresponding pixel in the first image.

[0044] Specifically, the electronic device captures images of any scene using a camera to obtain a first image, and then generates a process image based on the first image. The brightness of the process image is different from the brightness of the first image.

[0045] Step 204: Determine the first region in the first image and determine the second region in the image to be processed that corresponds to the first region.

[0046] The first region can be a region of interest in the first image, referred to as the first region of interest. The second region refers to the region in the image to be processed that corresponds to the first region of interest, i.e., the second region of interest in the image to be processed that corresponds to the first region of interest.

[0047] The region of interest (ROI) can be a facial region, a non-facial region, a limb region, a foreground region, a background region, etc., but is not limited to these.

[0048] Specifically, the electronic device can determine the region of interest in the first image and use it as the first region. The electronic device can then determine a region in the image to be processed that matches the first region, thus obtaining a second region in the image to be processed. Further, the electronic device performs matching processing on the pixels in the first region with each pixel in the image to be processed; the region formed by the pixels in the image to be processed that match the pixels included in the second region is the second region.

[0049] In this embodiment, the electronic device can determine any region in the first image as the first region, and determine the region in the image to be processed that matches the first region as the second region.

[0050] In this embodiment, the electronic device can determine a first region in the first image based on the image type of the first image. The first image type can be a facial image type or a non-facial image type. A facial image type refers to an image that contains a facial image, while a non-facial image type refers to an image that does not contain a facial image.

[0051] In this embodiment, determining a first region in a first image and determining a second region in the image to be processed corresponding to the first region includes: determining the first region in the first image and the mask image corresponding to the first region, and determining the second region in the image to be processed corresponding to the first region based on the mask image.

[0052] The mask image, also known as a mask template, is an image filter template used to identify objects in an image. It can obscure other parts of the image and filter out the objects. In this embodiment, the mask image corresponding to the first region can represent the position of the first region in the image to be processed. The second region corresponding to the first region can be filtered out from the image to be processed using the mask image.

[0053] In this embodiment, determining a first region in a first image and determining a second region in an image to be processed that corresponds to the first region includes: determining a first grayscale image corresponding to the first image and determining a first region in the first grayscale image; determining a second grayscale image corresponding to the image to be processed and determining a second region in the second grayscale image that corresponds to the first region.

[0054] Step 206: Determine the tone mapping relationship between the first region and the second region based on each pixel in the first region and each pixel in the second region.

[0055] The tone mapping relationship represents the brightness conversion relationship between the first region and the second region. Through the tone mapping relationship, the brightness of the first region and the brightness of the second region can be mapped to each other, so that the brightness of the first region is consistent with the brightness of the second region, or vice versa.

[0056] The tone mapping relationship between the first region and the second region can be used to characterize the tone mapping relationship between the first image and the image to be processed.

[0057] Specifically, the electronic device determines the brightness value corresponding to each pixel in the first region and the brightness value corresponding to each pixel in the second region, and determines the tone mapping relationship between the first region and the second region based on the brightness values ​​of each pixel in the first region and the brightness values ​​of each pixel in the second region.

[0058] Step 208: Based on the tone mapping relationship, perform global tone mapping on the image to be processed to obtain the target image.

[0059] Specifically, the electronic device determines the brightness value corresponding to each pixel in the image to be processed, and performs tone mapping on the brightness value of each pixel based on the tone mapping relationship, so as to perform global tone mapping on the image to be processed and obtain the target image.

[0060] In this embodiment, the tone mapping relationship can be represented by a target tone mapping function. The electronic device substitutes the brightness values ​​of each pixel in the image to be processed into the target tone mapping function to obtain the target brightness value corresponding to each pixel. Each pixel and its corresponding target brightness value form the target image.

[0061] In this embodiment, the minimum and maximum brightness values ​​of pixels in the image to be processed are determined. The minimum and maximum brightness values ​​and the brightness values ​​of each pixel are substituted into the target tone mapping function to obtain the target brightness value corresponding to each pixel, so as to form the target image.

[0062] In other embodiments, the electronic device determines the luminance value corresponding to each pixel in the luminance channel of the image to be processed, and performs luminance mapping on the luminance value of each pixel in the luminance channel based on the hue mapping relationship, so as to perform global hue mapping on the image to be processed and obtain the target image. The luminance channel may include the "Y" channel in a YUV image.

[0063] In the above image processing method, a target image generated based on a first image is obtained, the brightness of which differs from that of the first image. A first region in the first image is determined, and a second region in the target image corresponding to the first region is also determined. Based on the pixels in the first and second regions, a tone mapping relationship is determined between the first and second regions, thus reflecting the tonal difference between the two corresponding regions. Based on this tone mapping relationship, a global tone mapping is performed on the target image to compensate for its tone, ensuring that the brightness of the target image matches that of the first image. This reduces brightness differences between local regions in the image, avoiding the problem of excessively dark or bright local areas in the target image, thereby obtaining a target image with uniform tone across all regions.

[0064] In one embodiment, determining a first region in a first image and determining a second region in the image to be processed corresponding to the first region includes:

[0065] In the case where the first image includes a facial image, a first facial region in the first image is determined; the first facial region represents a first region in the first image; a second facial region in the image to be processed corresponding to the first facial region is determined, the second facial region representing a second region in the image to be processed.

[0066] Specifically, the electronic device can perform facial recognition on the first image, and if the first image includes a facial image, determine a first facial region in the first image. The electronic device can then determine a facial region in the image to be processed that matches the first facial region, as a second facial region of the image to be processed.

[0067] The first region in the first image is represented by the first facial region, and the second region in the image to be processed is represented by the second facial region.

[0068] Furthermore, after determining the first facial region in the first image, the electronic device determines the mask image corresponding to the first facial region, and determines the second facial region corresponding to the first facial region in the image to be processed based on the mask image.

[0069] In this embodiment, when both the first image and the image to be processed are face images, the first facial region is designated as the first face region, and the second facial region is designated as the second face region. That is, when the first image is a face image, the first face region in the first image is determined, and the second face region in the image to be processed corresponding to the first face region is determined. The first face region is designated as the first region in the first image, and the second face region is designated as the second region in the image to be processed.

[0070] In this embodiment, when the first image includes a facial image, a first facial region in the first image is determined, and a second facial region in the image to be processed corresponding to the first facial region is determined. The first facial region and the second facial region are the main body in the image, so that the tone mapping relationship can be calculated based on the pixels in the first facial region and the pixels in the second facial region. The obtained tone mapping relationship is more representative, so the tone mapping relationship calculated by the main body region is used to perform more accurate global tone mapping on the image to be processed, and a target image with uniform tone can be obtained.

[0071] In one embodiment, determining a first region in a first image and determining a second region in the image to be processed corresponding to the first region includes:

[0072] If the first image does not include a facial image, a first region is determined by each pixel in the first image that is within a preset brightness range; a second region in the image to be processed that corresponds to the first region is determined.

[0073] The preset brightness range is a pre-set brightness range that includes a preset maximum value and a preset minimum value.

[0074] Specifically, the electronic device can perform facial recognition on the first image, and if the first image does not include a facial image, obtain a preset brightness range. The electronic device determines the brightness value corresponding to each pixel in the first image, compares each brightness value with the preset brightness range, filters out the pixels corresponding to each brightness value within the preset brightness range, and uses the area formed by the pixels within the preset brightness range as the first region.

[0075] Furthermore, after determining the first region in the first image, the electronic device determines the mask image corresponding to the first region, and determines the second region corresponding to the first region in the image to be processed based on the mask image.

[0076] In this embodiment, the preset brightness range is used to filter out the intermediate brightness area in the first image. The first area is the intermediate brightness area in the first image, so as to eliminate the interference of overly dark and overexposed areas in the first image.

[0077] In this embodiment, when the first image does not include a facial image, a first region formed by each pixel in the first image within a preset brightness range is determined, and a second region in the image to be processed corresponding to the first region is determined. This allows for the screening of regions with specific brightness, and the accurate calculation of tone mapping relationships based on these regions. Consequently, a more accurate global tone mapping is performed on the image to be processed according to the tone mapping relationships to obtain a target image with uniform tone.

[0078] When the preset brightness range is used to filter out the intermediate brightness area in the first image, the interference of overly dark and overexposed areas in the first image is excluded. The tone mapping relationship calculated using the intermediate brightness area and the corresponding second area is more representative, and global tone mapping can be performed more accurately through this tone mapping relationship.

[0079] In one embodiment, determining the tone mapping relationship between the first region and the second region based on each pixel in the first region and each pixel in the second region includes:

[0080] Based on the brightness values ​​corresponding to each pixel in the first region, a first brightness value corresponding to the first region is determined; based on the brightness values ​​corresponding to each pixel in the second region, a second brightness value corresponding to the second region is determined; based on the first brightness value and the second brightness value, a tone mapping relationship between the first region and the second region is determined.

[0081] The first brightness value can be the sum of the brightness of each pixel in the first region, the average brightness of each pixel, the average brightness of some pixels, the maximum brightness value, the minimum brightness value, or an intermediate brightness value. The second brightness value can be the sum of the brightness of each pixel in the second region, the average brightness of each pixel, the average brightness of some pixels, the maximum brightness value, the minimum brightness value, or an intermediate brightness value.

[0082] Specifically, the electronic device determines each pixel within a first region and determines the brightness value corresponding to each pixel. Based on the brightness values ​​of each pixel, it determines at least one of the following: the sum of brightness values, the average brightness value, the maximum brightness value, the minimum brightness value, or the intermediate brightness value. This sum of brightness values, the average brightness value, the maximum brightness value, the minimum brightness value, or the intermediate brightness value is then used as the first brightness value corresponding to the first region. Here, each pixel within the first region can be all pixels within the first region, a preset number of brightness values ​​selected from all pixels, or pixels corresponding to key locations. Key locations include, for example, locations representing contours within the first region, the center location of the first region, etc. When the first region is a facial region, the key locations can also be the locations of facial key points.

[0083] The electronic device determines each pixel within a second region and its corresponding brightness value. Based on the brightness values ​​of each pixel, it determines at least one of the following: the sum of brightness values, the average brightness value, the maximum brightness value, the minimum brightness value, or the median brightness value. This sum of brightness values, the average brightness value, the maximum brightness value, the minimum brightness value, or the median brightness value is then used as the second brightness value corresponding to the second region. The pixels within the second region can be all pixels within the second region, a preset number of brightness values ​​selected from all pixels, or pixels corresponding to key locations. Key locations include, for example, locations representing contours within the second region, the center location of the second region, etc. When the second region is a facial region, the key locations can also be the locations of facial key points.

[0084] In this embodiment, determining the first brightness value corresponding to the first region based on the brightness value corresponding to each pixel in the first region includes: determining the sum of the brightness values ​​of each pixel in the first region and the first number of each pixel in the first region; and determining the first average brightness value of each pixel in the first region based on the sum of the brightness values ​​and the first number.

[0085] Based on the brightness values ​​corresponding to each pixel in the second region, determine the second brightness value corresponding to the second region, including: determining the sum of the brightness values ​​of each pixel in the second region, and the second number of each pixel in the second region; and determining the second average brightness value of each pixel in the second region based on the sum of the brightness values ​​and the second number of pixels.

[0086] In this embodiment, a first brightness value corresponding to the first region is determined based on the brightness value corresponding to each pixel in the first region, and a second brightness value corresponding to the second region is determined based on the brightness value corresponding to each pixel in the second region. Based on the first brightness value and the second brightness value, the tone mapping relationship between the first region and the second region can be accurately determined, thereby accurately representing the relationship between the first region and the second region in terms of brightness through the tone mapping relationship.

[0087] In one embodiment, determining the tone mapping relationship between the first region and the second region based on the first luminance value and the second luminance value includes:

[0088] Obtain the tone mapping loss function, and iterate the tone mapping loss function based on the first luminance value and the second luminance value to obtain the target tone mapping coefficient; obtain the preset tone mapping function, and determine the tone mapping relationship between the first region and the second region according to the target tone mapping coefficient and the preset tone mapping function.

[0089] Specifically, the electronic device acquires a tone mapping loss function, which is a pre-set function used to calculate the tone loss between different regions or different images. This tone loss can specifically be a brightness loss.

[0090] The tone mapping loss function includes tone mapping coefficients to be solved. Based on the first luminance value and the second luminance value, the tone mapping loss function is iteratively processed to obtain the target candidate tone mapping coefficients.

[0091] Furthermore, the electronic device acquires multiple candidate tone mapping coefficients corresponding to the tone mapping coefficients to be solved, and performs iterative processing based on the multiple candidate tone mapping coefficients, the first brightness value, the second brightness value, and the tone mapping loss function to select the target candidate tone mapping coefficients from the multiple candidate tone mapping coefficients.

[0092] The preset tone mapping function includes tone mapping coefficients to be solved. By replacing the tone mapping coefficients to be solved in the preset tone mapping function with the target tone mapping coefficients, the target tone mapping function can be obtained. The target tone mapping function is the tone mapping relationship between the first region and the second region.

[0093] In this embodiment, the minimum and maximum brightness values ​​of pixels in the image to be processed are determined. Based on the minimum brightness value, the maximum brightness value, the first brightness value, and the second brightness value, the tone mapping loss function is iteratively processed to obtain the target tone mapping coefficients.

[0094] In this embodiment, a tone mapping loss function is obtained. Based on a first brightness value and a second brightness value, the tone mapping loss function is iteratively processed to accurately calculate the target tone mapping coefficient. The target tone mapping coefficient is determined based on the brightness difference between the first region and the second region. A preset tone mapping function is obtained. Based on the target tone mapping coefficient and the preset tone mapping function, a tone mapping relationship that can be used to map the brightness between the first region and the second region can be accurately determined. Thus, the brightness of the image to be processed can be accurately compensated through the tone mapping relationship.

[0095] In one embodiment, the tone mapping loss function is iteratively processed based on a first luminance value and a second luminance value to obtain target tone mapping coefficients, including:

[0096] Based on the minimum and maximum brightness values ​​of pixels in the image to be processed, multiple candidate tone mapping coefficients are determined. For each candidate tone mapping coefficient, the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and candidate tone mapping coefficient are substituted into the tone mapping loss function to calculate the loss value between the first region and the second region, thus obtaining the loss value corresponding to the candidate tone mapping coefficient. The candidate tone mapping coefficients corresponding to the loss values ​​that satisfy the loss conditions are taken as the target tone mapping coefficients.

[0097] The loss value characterizes the difference in brightness between the first and second regions. A smaller loss value indicates a smaller difference, while a larger loss value indicates a larger difference.

[0098] Specifically, the electronic device determines the brightness value of each pixel in the image to be processed, and determines the minimum brightness value and the maximum brightness value. Based on the minimum brightness value, the maximum brightness value, and the brightness values ​​between the minimum and maximum brightness values, multiple candidate tone mapping coefficients are determined. For example, if the minimum brightness value of a pixel in the image to be processed is 0 and the maximum brightness value is 255, each integer in the range [0, 255] is used as a candidate tone mapping coefficient.

[0099] Substituting the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and a candidate tone mapping coefficient into the tone mapping loss function yields the loss value between the first and second regions. This loss value is the loss value corresponding to the candidate tone mapping coefficient. Following the same process, the loss value corresponding to each candidate tone mapping coefficient can be obtained.

[0100] For example, in the tone mapping loss function, the difference between the logarithm of the second brightness value and the sum of the candidate tone mapping coefficients, and the logarithm of the minimum brightness value and the sum of the candidate tone mapping coefficients, is used as the numerator. The difference between the logarithm of the maximum brightness value and the sum of the candidate tone mapping coefficients, and the logarithm of the minimum brightness value and the sum of the candidate tone mapping coefficients, is used as the denominator. The absolute value of the fraction formed by the numerator and denominator is used as the loss value corresponding to the candidate tone mapping coefficient.

[0101] The electronic device acquires the loss conditions, selects the loss values ​​that meet the loss conditions from the various loss values, and uses the candidate tone mapping coefficients corresponding to the loss values ​​that meet the loss conditions as the target tone mapping coefficients.

[0102] When there are multiple loss values ​​that meet the loss conditions, the candidate tone mapping coefficient corresponding to any one of the loss values ​​can be selected as the target tone mapping coefficient.

[0103] The loss condition can be either the minimum loss value or the loss value being less than or equal to a loss threshold. For example, the candidate tone mapping coefficients corresponding to the minimum loss value are selected from all loss values ​​and used as the target tone mapping coefficients.

[0104] In this embodiment, multiple candidate tone mapping coefficients are determined based on the minimum and maximum brightness values ​​of pixels in the image to be processed. For each candidate tone mapping coefficient, the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and the candidate tone mapping coefficient are substituted into the tone mapping loss function to calculate the loss value between the first and second regions. This yields the loss value corresponding to the candidate tone mapping coefficient, allowing the loss value to characterize the difference in brightness between the first and second regions. Furthermore, the loss value corresponding to each candidate tone mapping coefficient represents the different brightness differences between the first and second regions when using each candidate tone mapping coefficient. The candidate tone mapping coefficients corresponding to the loss values ​​that satisfy the loss conditions are used as target tone mapping coefficients. This allows for the selection of target tone mapping coefficients that reduce the brightness difference between the first and second regions, even to the point of minimum brightness difference, based on multiple loss values. Consequently, the tone mapping relationship formed based on the target tone mapping coefficients can more accurately perform tone mapping on the image.

[0105] In one embodiment, determining a first region in a first image and determining a second region in the image to be processed corresponding to the first region includes:

[0106] A first grayscale image corresponding to the first image is determined, and a first region in the first grayscale image and a mask image corresponding to the first region are determined; a second grayscale image corresponding to the image to be processed is determined, and a second region in the second grayscale image corresponding to the first region is determined based on the mask image.

[0107] The mask image corresponding to the first region represents the position of the first region in the image to be processed or the first grayscale image. The mask image can be used to filter out the second region corresponding to the first region in the image to be processed or the second grayscale image.

[0108] Specifically, the electronic device generates a first grayscale image of the first image based on each pixel of the first image. Further, the electronic device determines the color value of each pixel in each color channel of the first image, and generates the first grayscale image based on the color values ​​of each pixel in each color channel. For example, for each pixel in the first image, the average color value of the individual pixel in each color channel is calculated to obtain the pixel average value, which forms the first grayscale image. Each color channel includes a red channel, a green channel, and a blue channel.

[0109] The electronic device can determine a region of interest in a first grayscale image as a first region, or determine any region in the first image as the first region, or determine the first region according to the image type of the first image. The electronic device determines a mask image of the first region in the first grayscale image. For example, by setting the pixel values ​​of all pixels in the first region of the first grayscale image to the maximum value and setting the pixel values ​​of all pixels outside the first region to the minimum value, the corresponding mask image can be obtained.

[0110] The electronic device generates a second grayscale image of the image to be processed based on each pixel of the image to be processed. Further, the electronic device determines the color value of each pixel in each color channel of the image to be processed, and generates the second grayscale image based on the color values ​​of each pixel in each color channel. For example, for each pixel in the image to be processed, the average color value of the individual pixel in each color channel is calculated to obtain the pixel mean, and the pixel mean forms the second grayscale image.

[0111] The electronic device can filter out the second region in the second grayscale image by occluding the remaining areas in the second grayscale image other than the second region based on the mask image.

[0112] In this embodiment, determining the first grayscale image corresponding to the first image and the second grayscale image corresponding to the image to be processed can reduce the computational load of subsequent processing and improve processing speed. Determining the first region in the first grayscale image and the mask image corresponding to the first region allows for accurate determination of the position of the second region corresponding to the first region in the second grayscale image based on the mask image, thereby accurately obtaining the second region.

[0113] In one embodiment, the image to be processed is generated based on a first image and a second image, wherein the exposure time of the first image is greater than the exposure time of the second image.

[0114] Specifically, the electronic device can capture images of any scene using a camera to obtain a first image. After obtaining the first image, the electronic device can shorten the camera's exposure time and capture images of the same scene again to obtain a second image. The electronic device then fuses the first and second images to obtain the image to be processed.

[0115] In this embodiment, both the first image and the second image are preview images, and the exposure time of the first image is longer than that of the second image. Specifically, the electronic device can preview any scene using a camera to obtain a preview first image. After obtaining the preview first image, the camera's exposure time is shortened, and the same scene is previewed again to obtain a preview second image.

[0116] In this embodiment, the image to be processed is generated based on the first image and the second image. The exposure time of the first image is greater than that of the second image. That is, the first image is a normally exposed image and the second image is an underexposed image. The shorter exposure time of the underexposed image means that the bright areas in the image are less likely to be overexposed, so that the bright areas in the image to be processed generated based on the first image and the second image can present more detail information, thereby obtaining a clearer image.

[0117] In one embodiment, obtaining the image to be processed includes:

[0118] The image is previewed using a camera to obtain a first preview image; if the first image is overexposed, dynamic range compression processing is performed on the first image based on the exposure parameters to obtain the image to be processed.

[0119] Specifically, the electronic device previews an image using a camera, obtaining a first preview image. During the image preview, the electronic device determines if the preview image is overexposed. If the first image is overexposed, it performs dynamic range compression processing on the first image based on exposure parameters to obtain the image to be processed.

[0120] In this embodiment, when the first image is overexposed, dynamic range compression processing is performed on the first image based on the exposure parameters to obtain the image to be processed. This includes: when the first image is overexposed, determining the ratio of the brightness value of each pixel in the first image to the exposure parameters to obtain the image to be processed.

[0121] For example, if the first preview image is overexposed, to prevent the generated image from being overexposed, the exposure of the first preview image is linearly reduced by a factor of N. If the exposure parameter DRC (Dynamic Range Compression) = 3, the image exposure is reduced by a factor of 3 to prevent overexposure, thus obtaining an underexposed image to be processed. For example, if the brightness value of each pixel in the image to be processed is value_DRC_EV0, then value_DRC_EV0 = 1 / DRC * value_EV0, where value_EV0 is the brightness value of each pixel in the first image.

[0122] In this embodiment, an image preview is performed using a camera to obtain a first preview image; if the first image is overexposed, the exposure parameters of the camera are adjusted, and image acquisition is performed based on the adjusted camera to obtain the image to be processed.

[0123] In this embodiment, an image preview is performed using a camera to obtain a first preview image. If the first image is overexposed, dynamic range compression processing is performed on the first image based on the exposure parameters, so that the overexposed areas in the first image can be adjusted based on the exposure parameters, thereby obtaining an image to be processed that is not overexposed.

[0124] In one embodiment, an image processing method is provided, applied to an electronic device, comprising:

[0125] An image to be processed is generated based on a first image and a second image from the preview. The exposure time of the first image is longer than that of the second image. The brightness of the image to be processed is different from that of the first image.

[0126] Optionally, an image preview is performed using a camera to obtain a first preview image; if the first image is overexposed, dynamic range compression processing is performed on the first image based on exposure parameters to obtain an image to be processed; the brightness of the image to be processed is different from the brightness of the first image.

[0127] Next, the first grayscale image corresponding to the first image is determined, and the second grayscale image corresponding to the image to be processed is determined.

[0128] Furthermore, when the first image includes a facial image, a first facial region in the first grayscale image and a mask image corresponding to the first facial region are determined; the first facial region represents a first region in the first grayscale image; a second facial region in the second grayscale image corresponding to the first facial region is determined based on the mask image, and the second facial region represents a second region in the second grayscale image.

[0129] Optionally, if the first image does not include a facial image, a preset brightness range is obtained for filtering out intermediate brightness regions in the image; a first region is determined by each pixel in the first grayscale image that falls within the preset brightness range, and a mask image corresponding to the first region is determined; and a second region in the first grayscale image corresponding to the first region is determined based on the mask image.

[0130] Next, based on the brightness values ​​corresponding to each pixel in the first region, the first average brightness value corresponding to the first region is determined; based on the brightness values ​​corresponding to each pixel in the second region, the second average brightness value corresponding to the second region is determined.

[0131] Further, based on the minimum and maximum brightness values ​​of pixels in the image to be processed, multiple candidate tone mapping coefficients are determined. For each candidate tone mapping coefficient, the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and candidate tone mapping coefficient are substituted into the tone mapping loss function to calculate the loss value between the first region and the second region, thus obtaining the loss value corresponding to the candidate tone mapping coefficient. The candidate tone mapping coefficient corresponding to the minimum loss value among all loss values ​​is taken as the target tone mapping coefficient.

[0132] Furthermore, a preset tone mapping function is obtained, and the tone mapping relationship between the first region and the second region is determined based on the target tone mapping coefficient and the preset tone mapping function.

[0133] Next, based on the tone mapping relationship, global tone mapping is performed on the image to be processed to obtain the target image.

[0134] In this embodiment, the image to be processed is generated based on a normally exposed first image and an underexposed second image. The exposure time of the first image is longer than that of the second image. The shorter exposure time of the underexposed image makes it less likely for the bright areas in the image to be overexposed. This allows the bright areas in the image to be processed generated based on the first and second images to present more detail information, thereby obtaining a clearer image.

[0135] In cases where the first image includes a facial image, a first facial region in the first grayscale image is determined, and a mask image that can characterize the position of the first facial region in the first grayscale image is determined. Based on the mask image, a second facial region corresponding to the first facial region in the second grayscale image is accurately determined. The first facial region and the second facial region serve as the main body in the image, enabling the calculation of tone mapping relationships based on the average brightness of each pixel in the first facial region and the average brightness of each pixel in the second facial region. The resulting tone mapping relationship is more representative, thereby using the tone mapping relationship calculated from the main body region to perform more accurate global tone mapping on the image to be processed, and a target image with uniform tone can be obtained.

[0136] In the case that the first image does not include a face image, a preset brightness range is obtained to filter out the intermediate brightness areas in the image, so as to filter out the intermediate brightness areas (i.e., the first area and the second area) in the first grayscale image and the second grayscale image, thereby eliminating the interference of overly dark and overexposed areas in the image. The tone mapping relationship calculated using the average brightness of each pixel in the intermediate brightness area is more representative, and global tone mapping can be performed more accurately through this tone mapping relationship.

[0137] Furthermore, when calculating the tone mapping relationship, multiple candidate tone mapping coefficients are determined based on the minimum and maximum brightness values ​​of pixels in the image to be processed. For each candidate tone mapping coefficient, the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and the candidate tone mapping coefficient are substituted into the tone mapping loss function. This allows the calculation of the loss value between the first and second regions, obtaining the loss value corresponding to the candidate tone mapping coefficient. This loss value characterizes the difference in brightness between the first and second regions. The loss value corresponding to each candidate tone mapping coefficient represents the different differences in brightness between the first and second regions when using each candidate tone mapping coefficient. The candidate tone mapping coefficient corresponding to the minimum loss value is taken as the target tone mapping coefficient. The minimum loss value indicates the minimum brightness difference. Therefore, the target tone mapping coefficient that minimizes the brightness difference between the first and second regions can be selected based on multiple loss values, enabling the tone mapping relationship formed based on the target tone mapping coefficient to more accurately map the image.

[0138] Global tone mapping based on tone mapping relationships can compensate for the tone of the image to be processed, making the brightness of the image to be processed consistent with the brightness of the preview image. This can reduce the brightness differences between local areas in the image and avoid the problem of some areas being darker and some areas being brighter in the image to be processed, thereby obtaining a target image with uniform tone in all areas of the image.

[0139] In one embodiment, an image processing method is provided to perform global tone mapping on an HDR image, aligning the brightness of the HDR image with the brightness of a previewed EV0 image. The HDR image is the image to be processed, and the EV0 image is the first image. The processing procedure is as follows: Figure 3 As shown, it includes:

[0140] An HDR image is acquired by combining a normally exposed EV0 image with an underexposed EV- image. The underexposed EV- image is the second image.

[0141] If the EV0 image is not a portrait scene, i.e., it does not include facial images, then a first region is determined, formed by pixels whose brightness values ​​are within a preset brightness range [m,n] in the EV0 image, and the first average brightness value_EV0 corresponding to the first region is calculated. The purpose of determining the first region with the preset brightness range [m,n] is to extract the middle brightness area of ​​the EV0 image to eliminate interference from overly dark and overexposed areas.

[0142] A mask image is obtained for all pixels in the interval [m,n] in the EV0 image. The average brightness value_HDR of the pixels in the HDR image within the Mask is determined. That is, the second region in the HDR image is determined based on the mask image, and the second average brightness value_HDR corresponding to the second region is determined.

[0143] If the EV0 image is a portrait scene image, including a facial image, determine the first facial region in the EV0 image and the face bounding box representing the position of the first facial region in the EV0 image, and determine the first average brightness value_EV0 corresponding to the first facial region. Based on the face bounding box, determine the second facial region in the HDR image, and determine the second average brightness value_HDR corresponding to the second facial region.

[0144] Next, the luminance value of value_HDR needs to be mapped to value_EV0. Therefore, a globaltonemapping tone mapping function f is designed, and the tone mapping parameters of function f are calculated to make value_EV0 = f(value_HDR). The tone mapping parameters are the tone mapping coefficients.

[0145] The target HDR image is obtained by mapping the brightness values ​​of all pixels in the HDR image using the global tonemapping function f after calculating the tone mapping parameters.

[0146] like Figure 4 As shown, the image processing method in this embodiment performs global tone mapping on the HDR image, so that the brightness of the HDR image is aligned with the brightness of the previewed EVO image.

[0147] like Figure 5 The diagram shown is a flowchart illustrating the process of calculating the first average brightness value_EV0 corresponding to the first region in the EV0 image in this embodiment.

[0148] Step 502: Calculate the first grayscale image of the EV0 image. The specific calculation method for the grayscale value of each pixel in the first grayscale image is as follows:

[0149]

[0150] Where i is the pixel coordinate.

[0151] If it is a portrait scene, then proceed to step 504, which involves obtaining the following: Figure 6The first grayscale image shown contains a rectangular bounding box representing a face. This bounding box is automatically detected and acquired by the device, and step 506 is executed, which calculates the average brightness value of the pixels within the bounding box. Specifically, the sum of brightness values ​​(SUM) of the pixels within the bounding box is calculated, along with the number of pixels (Count). The average brightness value (value_EV0) within the bounding box is then the ratio of the sum (SUM) to the number of pixels (Count). The calculation formula is shown below:

[0152]

[0153] If it is not a portrait scene, then step 508 is executed, which calculates the first region in the first grayscale image whose brightness value is within the preset brightness range [m,n] and the mask image Mask. Figure 7 As shown, white pixels are those within the mask, and black pixels are those outside the mask. m and n are experimental values. The maximum brightness of the first grayscale image is set to 255, so m = 0.1 × 255 and n = 0.5 × 255. The values ​​of m and n are set to eliminate interference from excessively dark or bright pixels.

[0154] Execute step 510 to calculate the sum of the brightness values ​​of the pixels in the first region (SUM) and the number of pixels (Count) in the first region. The first average brightness value (value_EV0) of the first region is the ratio of the sum (SUM) to the number of pixels (Count). The calculation formula is as follows:

[0155]

[0156] like Figure 8 The diagram shown is a flowchart illustrating the process of calculating the average brightness of the second region in the HDR image in this embodiment.

[0157] Step 502: Calculate the second grayscale image of the HDR image. The specific calculation method for the grayscale value of each pixel in the second grayscale image is as follows:

[0158]

[0159] Where i is the pixel coordinate.

[0160] If it is a portrait scene, then step 804 is executed, which involves obtaining the face bounding box in the second grayscale image. The face bounding box is automatically detected and obtained by the device. Then, step 806 is executed, which involves calculating the second average brightness value of the pixels within the face bounding box in the second grayscale image. Specifically, the sum of the brightness values ​​SUM of the pixels within the face bounding box can be calculated, along with the number of pixels Count within the face bounding box. The second average brightness value value_HDR within the face bounding box is then the ratio of the sum SUM to the number of pixels Count. The calculation formula is as follows:

[0161]

[0162] If it is not a portrait scene, then step 808 is executed, that is, based on the mask image of the first region, the second region in the second grayscale image corresponding to the first region is determined.

[0163] Execute step 510 to calculate the sum of the luminance values ​​of the pixels in the second region (SUM) and the number of pixels (Count) in the second region. The second average luminance value (value_HDR) of the second region is the ratio of the sum (SUM) to the number of pixels (Count). The calculation formula is as follows:

[0164]

[0165] like Figure 9 The diagram shown is a flowchart illustrating the calculation of the parameters of the global tonemapping function f in this embodiment.

[0166] 1) First, initialize the preset tone mapping function of global tonemapping, where i is the coordinate of each pixel, min is the minimum brightness value of the HDR image, max is the maximum brightness value of the HDR image, and param is the parameter to be solved in the next step, i.e., the tone mapping parameter to be solved:

[0167]

[0168] 2) Calculate the parameter param from step one based on the first average brightness value_EV0 and the first average brightness value_HDR. First, substitute value_EV0 and value_HDR into the preset tone mapping function from step one to obtain the following function:

[0169]

[0170] 3) Solve for the parameter `param` using an iterative method. The specific calculation method is as follows: If `value_EV0` = 50, `value_HDR` = 20, `min` = 0, and `max` = 255, take integer values ​​from [0, 255] as candidate tone mapping coefficients for `param`. Substitute these integers into the tone mapping loss function below to obtain the loss value `diff` corresponding to each candidate tone mapping coefficient from 0 to 255. Select the candidate tone mapping coefficient with the smallest loss value `diff` as the value of the `param` parameter. For example, when `param` = 23, `diff` is the smallest. Therefore, through multiple iterations, `param` is finally determined to be 23. The tone mapping loss function is as follows:

[0171]

[0172] 4) Once the known parameters are obtained, the formula f for global tonemapping is derived. A schematic diagram of the curve of the global tonemapping function f is shown below. Figure 10 As shown, the horizontal axis represents the input brightness value, and the vertical axis represents the output brightness value. The known `param` parameter is the target tone mapping coefficient. Substituting this target tone mapping coefficient into the preset tone mapping function in the first step yields the target tone mapping function, as shown below:

[0173]

[0174] 5) Apply the target tone mapping function to the HDR image to obtain the HDR image after global tone mapping, i.e., the target image. Input[i] represents the brightness value of each pixel in the input HDR image, and Output[i] represents the brightness value of each pixel in the HDR image after global tone mapping.

[0175] In this embodiment, a global tone mapping method is designed for high dynamic range (HDR) images to make the brightness of the HDR image and the preview image more uniform. Specifically, the average brightness value value_HDR of the first region in the HDR image and the average brightness value value_EV0 of the second region in the preview image are calculated. Based on the designed tone mapping function, the brightness of the HDR image is accurately mapped to the brightness of the preview image. This can avoid the situation where some areas of the HDR image are brighter and some areas are darker, thereby making the tone of the HDR image more uniform and harmonious.

[0176] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0177] Based on the same inventive concept, this application also provides an image processing apparatus for implementing the image processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image processing apparatus embodiments provided below can be found in the limitations of the image processing method described above, and will not be repeated here.

[0178] In one embodiment, such as Figure 11 As shown, an image processing apparatus 1100 is provided, including: an acquisition module 1102, a region determination module 1104, a relationship determination module 1106, and a mapping module 1108, wherein:

[0179] The acquisition module 1102 is used to acquire the image to be processed, which is generated based on the first image, and the image brightness of the image to be processed is different from that of the first image.

[0180] The region determination module 1104 is used to determine a first region in the first image and to determine a second region in the image to be processed that corresponds to the first region.

[0181] The relationship determination module 1106 is used to determine the tone mapping relationship between the first region and the second region based on each pixel in the first region and each pixel in the second region.

[0182] The mapping module 1108 is used to perform global tone mapping on the image to be processed based on the tone mapping relationship to obtain the target image.

[0183] In this embodiment, a to-be-processed image generated based on a first image is obtained. The brightness of the to-be-processed image differs from that of the first image. A first region in the first image is determined, and a second region in the to-be-processed image corresponding to the first region is also determined. Based on the pixels in the first region and the pixels in the second region, a tone mapping relationship between the first and second regions is determined, thereby reflecting the tonal difference between the two corresponding regions. Based on the tone mapping relationship, global tone mapping is performed on the to-be-processed image to compensate for its tone, ensuring that the brightness of the to-be-processed image matches that of the first image. This reduces brightness differences between local regions in the image, avoiding the problem of some areas being too dark and others too bright, thus obtaining a target image with uniform tone across all regions.

[0184] In one embodiment, the region determination module 1104 is further configured to, when the first image includes a facial image, determine a first facial region in the first image; the first facial region represents a first region in the first image; and determine a second facial region in the image to be processed that corresponds to the first facial region, the second facial region representing a second region in the image to be processed.

[0185] In this embodiment, when the first image includes a facial image, a first facial region in the first image is determined, and a second facial region in the image to be processed corresponding to the first facial region is determined. The first facial region and the second facial region are the main body in the image, so that the tone mapping relationship can be calculated based on the pixels in the first facial region and the pixels in the second facial region. The obtained tone mapping relationship is more representative, so the tone mapping relationship calculated by the main body region is used to perform more accurate global tone mapping on the image to be processed, and a target image with uniform tone can be obtained.

[0186] In one embodiment, the region determination module 1104 is further configured to determine a first region formed by each pixel in the first image that is within a preset brightness range, in the case that the first image does not include a facial image;

[0187] Identify the second region in the image to be processed that corresponds to the first region.

[0188] In this embodiment, when the first image does not include a facial image, a first region formed by each pixel in the first image within a preset brightness range is determined, and a second region in the image to be processed corresponding to the first region is determined. This allows for the screening of regions with specific brightness, and the accurate calculation of tone mapping relationships based on these regions. Consequently, a more accurate global tone mapping is performed on the image to be processed according to the tone mapping relationships to obtain a target image with uniform tone.

[0189] When the preset brightness range is used to filter out the intermediate brightness area in the first image, the interference of overly dark and overexposed areas in the first image is excluded. The tone mapping relationship calculated using the intermediate brightness area and the corresponding second area is more representative, and global tone mapping can be performed more accurately through this tone mapping relationship.

[0190] In one embodiment, the relationship determination module 1106 is further configured to determine a first brightness value corresponding to the first region based on the brightness values ​​corresponding to each pixel in the first region; determine a second brightness value corresponding to the second region based on the brightness values ​​corresponding to each pixel in the second region; and determine a tone mapping relationship between the first region and the second region based on the first brightness value and the second brightness value.

[0191] In this embodiment, a first brightness value corresponding to the first region is determined based on the brightness value corresponding to each pixel in the first region, and a second brightness value corresponding to the second region is determined based on the brightness value corresponding to each pixel in the second region. Based on the first brightness value and the second brightness value, the tone mapping relationship between the first region and the second region can be accurately determined, thereby accurately representing the relationship between the first region and the second region in terms of brightness through the tone mapping relationship.

[0192] In one embodiment, the relationship determination module 1106 is further configured to obtain a tone mapping loss function, iterate the tone mapping loss function based on the first luminance value and the second luminance value to obtain a target tone mapping coefficient; obtain a preset tone mapping function, and determine the tone mapping relationship between the first region and the second region according to the target tone mapping coefficient and the preset tone mapping function.

[0193] In this embodiment, a tone mapping loss function is obtained. Based on a first brightness value and a second brightness value, the tone mapping loss function is iteratively processed to accurately calculate the target tone mapping coefficient. The target tone mapping coefficient is determined based on the brightness difference between the first region and the second region. A preset tone mapping function is obtained. Based on the target tone mapping coefficient and the preset tone mapping function, a tone mapping relationship that can be used to map the brightness between the first region and the second region can be accurately determined. Thus, the brightness of the image to be processed can be accurately compensated through the tone mapping relationship.

[0194] In one embodiment, the relationship determination module 1106 is further configured to determine multiple candidate tone mapping coefficients based on the minimum and maximum brightness values ​​of pixels in the image to be processed; for each candidate tone mapping coefficient, the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and candidate tone mapping coefficient are substituted into the tone mapping loss function to calculate the loss value between the first region and the second region, thereby obtaining the loss value corresponding to the candidate tone mapping coefficient; and the candidate tone mapping coefficients corresponding to the loss values ​​that satisfy the loss conditions among the loss values ​​are used as the target tone mapping coefficients.

[0195] In this embodiment, multiple candidate tone mapping coefficients are determined based on the minimum and maximum brightness values ​​of pixels in the image to be processed. For each candidate tone mapping coefficient, the minimum brightness value, maximum brightness value, first brightness value, second brightness value, and the candidate tone mapping coefficient are substituted into the tone mapping loss function to calculate the loss value between the first and second regions. This yields the loss value corresponding to the candidate tone mapping coefficient, allowing the loss value to characterize the difference in brightness between the first and second regions. Furthermore, the loss value corresponding to each candidate tone mapping coefficient represents the different brightness differences between the first and second regions when using each candidate tone mapping coefficient. The candidate tone mapping coefficients corresponding to the loss values ​​that satisfy the loss conditions are used as target tone mapping coefficients. This allows for the selection of target tone mapping coefficients that reduce the brightness difference between the first and second regions, even to the point of minimum brightness difference, based on multiple loss values. Consequently, the tone mapping relationship formed based on the target tone mapping coefficients can more accurately perform tone mapping on the image.

[0196] In one embodiment, the region determination module 1104 is further configured to determine a first grayscale image corresponding to the first image, and determine a first region in the first grayscale image and a mask image corresponding to the first region; determine a second grayscale image corresponding to the image to be processed, and determine a second region in the second grayscale image corresponding to the first region based on the mask image.

[0197] In this embodiment, determining the first grayscale image corresponding to the first image and the second grayscale image corresponding to the image to be processed can reduce the computational load of subsequent processing and improve processing speed. Determining the first region in the first grayscale image and the mask image corresponding to the first region allows for accurate determination of the position of the second region corresponding to the first region in the second grayscale image based on the mask image, thereby accurately obtaining the second region.

[0198] In one embodiment, the image to be processed is generated based on a first image and a second image, wherein the exposure time of the first image is greater than the exposure time of the second image.

[0199] In this embodiment, the image to be processed is generated based on the first image and the second image. The exposure time of the first image is greater than that of the second image. That is, the first image is a normally exposed image and the second image is an underexposed image. The shorter exposure time of the underexposed image means that the bright areas in the image are less likely to be overexposed, so that the bright areas in the image to be processed generated based on the first image and the second image can present more detail information, thereby obtaining a clearer image.

[0200] In one embodiment, the acquisition module 1102 is further configured to perform image preview through a camera to obtain a first preview image; and if the first image is overexposed, to perform dynamic range compression processing on the first image based on exposure parameters to obtain an image to be processed.

[0201] The image is previewed using a camera to obtain a first preview image. If the first image is overexposed, dynamic range compression processing is performed on the first image based on the exposure parameters. This allows the overexposed areas in the first image to be adjusted based on the exposure parameters, thereby obtaining a non-overexposed image to be processed.

[0202] Each module in the aforementioned image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0203] In one embodiment, a computer device is provided, which may be an electronic device or a server. Taking an electronic device as an example, its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0204] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0205] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of an image processing method.

[0206] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform an image processing method.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0208] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0209] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtain an image to be processed, wherein the image to be processed is generated based on a first image, and the brightness of the image to be processed is different from the brightness of the first image; A first region in the first image is determined, and a second region in the image to be processed corresponding to the first region is determined; Based on the brightness values ​​of each pixel in the first region and the brightness values ​​of each pixel in the second region, a tone mapping relationship between the first region and the second region is determined; the tone mapping relationship represents the brightness conversion relationship between the first region and the second region; through the tone mapping relationship, the brightness of the first region and the brightness of the second region are mapped to each other, so that the brightness of the first region and the brightness of the second region are consistent. Based on the tone mapping relationship, global tone mapping is performed on the image to be processed to obtain the target image.

2. The method according to claim 1, characterized in that, Determining a first region in the first image and determining a second region in the image to be processed corresponding to the first region includes: In the case where the first image includes a facial image, a first facial region is determined in the first image; the first facial region represents a first region in the first image; A second facial region corresponding to the first facial region in the image to be processed is determined, wherein the second facial region represents a second region in the image to be processed.

3. The method according to claim 1, characterized in that, Determining a first region in the first image and determining a second region in the image to be processed corresponding to the first region includes: In the case where the first image does not include a facial image, a first region is determined by each pixel in the first image that is within a preset brightness range; Determine the second region in the image to be processed that corresponds to the first region.

4. The method according to claim 1, characterized in that, Determining the tone mapping relationship between the first region and the second region based on each pixel in the first region and each pixel in the second region includes: Based on the brightness values ​​corresponding to each pixel in the first region, determine the first brightness value corresponding to the first region; Based on the brightness values ​​corresponding to each pixel in the second region, determine the second brightness value corresponding to the second region; Based on the first brightness value and the second brightness value, the tone mapping relationship between the first region and the second region is determined.

5. The method according to claim 4, characterized in that, Determining the tone mapping relationship between the first region and the second region based on the first brightness value and the second brightness value includes: Obtain the tone mapping loss function, and iterate the tone mapping loss function based on the first brightness value and the second brightness value to obtain the target tone mapping coefficient; Obtain a preset tone mapping function, and determine the tone mapping relationship between the first region and the second region based on the target tone mapping coefficient and the preset tone mapping function.

6. The method according to claim 5, characterized in that, The step of iteratively processing the tone mapping loss function based on the first brightness value and the second brightness value to obtain the target tone mapping coefficients includes: Based on the minimum and maximum brightness values ​​of pixels in the image to be processed, multiple candidate tone mapping coefficients are determined; For each candidate tone mapping coefficient, the minimum brightness value, the maximum brightness value, the first brightness value, the second brightness value, and the candidate tone mapping coefficient are substituted into the tone mapping loss function to calculate the loss value between the first region and the second region, thereby obtaining the loss value corresponding to the candidate tone mapping coefficient. The candidate tone mapping coefficients corresponding to the loss values ​​that satisfy the loss conditions are used as the target tone mapping coefficients.

7. The method according to claim 1, characterized in that, Determining a first region in the first image and determining a second region in the image to be processed corresponding to the first region includes: Determine the first grayscale image corresponding to the first image, and determine the first region in the first grayscale image and the mask image corresponding to the first region; A second grayscale image corresponding to the image to be processed is determined, and a second region in the second grayscale image corresponding to the first region is determined based on the mask image.

8. The method according to any one of claims 1 to 7, characterized in that, The image to be processed is generated based on a first image and a second image, wherein the exposure time of the first image is greater than the exposure time of the second image.

9. The method according to any one of claims 1 to 7, characterized in that, The process of acquiring the image to be processed includes: The image is previewed using a camera to obtain the first preview image; If the first image is overexposed, dynamic range compression is performed on the first image based on the exposure parameters to obtain the image to be processed.

10. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire an image to be processed, which is generated based on a first image, and the image brightness of the image to be processed is different from that of the first image; A region determination module is used to determine a first region in the first image and to determine a second region in the image to be processed that corresponds to the first region. The relationship determination module is used to determine the tone mapping relationship between the first region and the second region based on the brightness values ​​of each pixel in the first region and the brightness values ​​of each pixel in the second region; the tone mapping relationship represents the brightness conversion relationship between the first region and the second region; through the tone mapping relationship, the brightness of the first region and the brightness of the second region are mapped to each other, so that the brightness of the first region and the brightness of the second region are consistent. The mapping module is used to perform global tone mapping on the image to be processed based on the tone mapping relationship to obtain the target image.

11. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images