Image processing method and device, computer equipment and storage medium

Abstract

The invention relates to an image processing method and device, computer equipment and a storage medium. The method comprises the steps of obtaining a plurality of reference images corresponding to a target image; based on the gradient value of each pixel point of the target image, determining each target sub-image matched with the target image screening strategy in a plurality of reference images corresponding to the target image, and obtaining a fused image; and determining a target brightness value adjustment coefficient based on the fusion brightness value of each pixel point of the fusion image, and performing pixel value mapping on the brightness values of the pixel points contained in the target image to obtain a target adjustment image. On the basis of the combination of the brightness information of the multi-magnification sampling scale of the target image and the gradient of the image, the corresponding brightness value adjustment coefficient is determined on the basis of the brightness value of each pixel point of the target image, and the brightness of the brightness value mapping curve is adjusted. According to the invention, the brightness of the high-brightness area can be adaptively adjusted, the image information of the low-brightness area can be adaptively adjusted, and a better image processing effect is achieved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to an image processing method, apparatus, computer device, and storage medium. Background Art

[0002] With the continuous development of the field of image processing, a local tone mapping method has emerged. It involves dividing an image into blocks to obtain multiple block regions, and by statistically analyzing the histogram of each region, performing histogram equalization or other similar histogram regulation operations to obtain a cdf (cumulative distribution function) curve or a mapping curve. Based on this curve, pixels within the block region can be processed to obtain a processing result. Since each pixel point during the processing can be regarded as the combined result of the mapping curves of adjacent block regions, the mapping results of adjacent block regions have a high weight ratio, while those of distant block regions have a low weight ratio. As a result, the processing effect of pixel points usually exhibits obvious brightness mapping differences between blocks, leading to poor image processing effects. Summary of the Invention

[0003] Based on this, to address the above technical problems, it is necessary to provide an image processing method, apparatus, computer device, and storage medium that can ensure improved image processing effects and image quality.

[0004] In a first aspect, this application provides an image processing method, which includes:

[0005] Obtain multiple reference images corresponding to a target image, where each reference image is obtained by downsampling and upsampling based on multiple different sampling scales;

[0006] Based on the gradient values of each pixel point in the target image, determine, among the multiple reference images corresponding to the target image, each target sub - image that matches the target image screening strategy, and perform fusion processing on the target sub - images corresponding to each pixel point to obtain a fused image;

[0007] Determine a target brightness value adjustment coefficient based on the fused brightness values of each pixel point in the fused image, and perform pixel value mapping on the brightness values of the pixel points included in the target image based on the fused brightness values of each pixel point, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjusted image.

[0008] In one embodiment, the target brightness value adjustment coefficient includes the brightness adjustment coefficient corresponding to each pixel point; the determining of the target brightness value adjustment coefficient based on the fused brightness values of each pixel point in the fused image includes:

[0009] Based on the fusion brightness values of the pixels of the fusion image and the target brightness value adjustment coefficient curve, determine the brightness adjustment coefficients corresponding to the fusion brightness values of the respective pixels.

[0010] In one embodiment, the pixel value mapping of the brightness values of the pixels included in the target image based on the fusion brightness values of the respective pixels, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjustment image includes:

[0011] Perform mapping based on the fusion brightness values of the pixels of the fusion image and the gain algorithm to obtain an initial brightness value mapping curve;

[0012] Adjust the initial brightness value mapping curve based on the brightness adjustment coefficient to obtain a target brightness value mapping curve;

[0013] Perform pixel value mapping on the brightness values of the pixels included in the target image through the target brightness value mapping curve to obtain a target adjustment image.

[0014] In one embodiment, the performing pixel value mapping on the brightness values of the pixels included in the target image through the target brightness value mapping curve to obtain a target adjustment image includes:

[0015] Perform pixel value mapping on the brightness values of the pixels included in the target image through the target brightness value mapping curve to obtain an initial adjustment image;

[0016] Based on the brightness values of the respective pixels of the initial adjustment image and the brightness of the respective pixels included in the target image, calculate the gain values corresponding to the respective pixels;

[0017] Adjust the gain values corresponding to the respective pixels based on a target adjustment strategy to obtain the adjusted gain values of the respective pixels;

[0018] Process the brightness values of the pixels of the target image through the adjusted gain values of the respective pixels to obtain a target adjustment image.

[0019] In one embodiment, before the step of determining, in a plurality of reference images corresponding to the target image, the respective target sub - images that match the target image screening strategy based on the gradient values of the respective pixels of the target image, and performing fusion processing on the respective target sub - images corresponding to the respective pixels to obtain a fusion image, the method further includes:

[0020] Perform uniform brightness adjustment on the brightness values of the pixels of the initial image through a brightness adjustment algorithm to obtain a target image;

[0021] Calculate the gradient value of each pixel point based on the high-frequency operator and the brightness value of each pixel point of the target image;

[0022] Determine the gradient weight corresponding to each gradient value based on the target gradient weight strategy, and the target gradient weight strategy includes that the gradient value and the gradient weight are positively correlated.

[0023] In one embodiment, based on the gradient value of each pixel point of the target image, among the multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening strategy, and perform a fusion process on each target sub-image corresponding to each pixel point to obtain a fused image, including:

[0024] For each pixel point included in the target image, based on the gradient weight corresponding to the gradient value of the pixel point, among the multiple reference images corresponding to the target image, determine a target reference image, extract the first pixel point whose position coordinates match the position coordinates of the pixel point in the target reference image, and determine the image where the first pixel point is located as the target sub-image;

[0025] Perform a splicing and fusion process on each target sub-image corresponding to each pixel point included in the target image to obtain a fused image.

[0026] In one embodiment, perform mapping based on the fused brightness value of each pixel point of the fused image and the gain algorithm to obtain an initial brightness value mapping curve, including:

[0027] Divide the fused brightness value of each pixel point of the fused image based on multiple brightness value ranges to obtain multiple brightness regions, and calculate the brightness average value of each brightness region;

[0028] Process the brightness average value of each brightness region through the gain algorithm to obtain the gain value of each brightness region;

[0029] Perform point plotting calculation based on the gain value of each brightness region and the brightness average value of each brightness region to obtain an initial brightness value mapping curve, and the initial brightness value mapping curve is the mapping relationship between the fused brightness value and the target brightness value.

[0030] In one embodiment, determine the brightness adjustment coefficient corresponding to the fused brightness value of each pixel point based on the fused brightness value of each pixel point of the fused image and the target brightness value adjustment coefficient, including:

[0031] On the target brightness value adjustment coefficient curve, determine the adjustment coefficient that matches the fused brightness value of the pixel point of the fused image, and the brightness value and the brightness value adjustment coefficient in the target brightness value adjustment coefficient curve are negatively correlated.

[0032] In one embodiment, calculating gain values corresponding to each of the pixel points based on the brightness values of the pixel points of the initial adjustment image and the brightness of the pixel points included in the target image includes:

[0033] For each pixel point included in the target image, calculate a ratio of the brightness value of the pixel point in the initial adjustment image to the brightness value of the pixel point in the target image, and determine the ratio as the gain value of the pixel point.

[0034] In one embodiment, calculating gradient values of the pixel points based on a high-frequency operator and the brightness values of the pixel points of the target image includes:

[0035] For each pixel point, calculate the gradient value of the pixel point through the high-frequency operator, the brightness value of the pixel point, and the brightness values of adjacent pixel points of the pixel point.

[0036] In a second aspect, the present application further provides an image processing apparatus, where the apparatus includes:

[0037] A first acquisition module, configured to acquire a plurality of reference images corresponding to a target image, where each of the reference images is obtained by downsampling and upsampling based on a plurality of different sampling scales;

[0038] A first determination module, configured to determine, based on the gradient values of the pixel points of the target image, target sub-images that match a target image screening strategy from the plurality of reference images corresponding to the target image, and perform fusion processing on the target sub-images corresponding to the pixel points to obtain a fused image;

[0039] A first mapping module, configured to determine a target brightness value adjustment coefficient based on the fused brightness values of the pixel points of the fused image, and perform pixel value mapping on the brightness values of the pixel points included in the target image based on the fused brightness values of the pixel points, a gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjustment image.

[0040] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0041] Acquire a plurality of reference images corresponding to a target image, where each of the reference images is obtained by downsampling and upsampling based on a plurality of different sampling scales;

[0042] Based on the gradient values of the pixels of the target image, in multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening strategy, and perform fusion processing on the target sub-images corresponding to each of the pixels to obtain a fused image;

[0043] Determine a target brightness value adjustment coefficient based on the fused brightness values of the pixels of the fused image, and perform pixel value mapping on the brightness values of the pixels included in the target image based on the fused brightness values of the pixels, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjusted image.

[0044] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Obtain multiple reference images corresponding to the target image, and each of the reference images is obtained by downsampling and upsampling based on multiple different sampling scales;

[0046] Based on the gradient values of the pixels of the target image, in multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening strategy, and perform fusion processing on the target sub-images corresponding to each of the pixels to obtain a fused image;

[0047] Determine a target brightness value adjustment coefficient based on the fused brightness values of the pixels of the fused image, and perform pixel value mapping on the brightness values of the pixels included in the target image based on the fused brightness values of the pixels, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjusted image.

[0048] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Obtain multiple reference images corresponding to the target image, and each of the reference images is obtained by downsampling and upsampling based on multiple different sampling scales;

[0050] Based on the gradient values of the pixels of the target image, in multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening strategy, and perform fusion processing on the target sub-images corresponding to each of the pixels to obtain a fused image;

[0051] Determine a target brightness value adjustment coefficient based on the fusion brightness values of the pixel points of the fusion image, and perform pixel value mapping on the brightness values of the pixel points included in the target image based on the fusion brightness values of the pixel points, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjustment image.

[0052] In the above image processing method, device, computer device, storage medium, and computer program product, by combining the brightness information of multiple magnification sampling scales of the target image with the gradient of the image itself, and determining the corresponding brightness value adjustment coefficient based on the high and low brightness values of the pixel points of the target image, and performing brightness adjustment on the brightness values based on the brightness value adjustment coefficient, it can be ensured that the brightness value adjustment coefficient is determined based on the brightness value, realizing the adaptive adjustment of the brightness of the high-brightness area and the adjustment of the image information of the low-brightness area, achieving a better image processing effect and improving the image quality of brightness processing. Description of the Drawings

[0053] Figure 1 It is a schematic flowchart of the image processing method in an embodiment;

[0054] Figure 2 It is a schematic flowchart of the steps for obtaining the target adjustment image in an embodiment;

[0055] Figure 3 It is a schematic flowchart of the steps for determining the gradient weight in an embodiment;

[0056] Figure 4 It is a schematic flowchart of the steps for determining the fusion image in an embodiment;

[0057] Figure 5 It is a schematic flowchart of the steps for obtaining the initial brightness value mapping curve in an embodiment;

[0058] Figure 6 It is a schematic flowchart in another embodiment;

[0059] Figure 7 It is a schematic flowchart of the image preprocessing in an embodiment;

[0060] Figure 8a It is a schematic diagram of the brightness distribution in an embodiment;

[0061] Figure 8b It is a schematic diagram of the brightness distribution in an embodiment;

[0062] Figure 9 It is a schematic diagram of the mapping relationship between the gradient intensity and the weight in an embodiment;

[0063] Figure 10 It is a schematic diagram of the image for sampling processing in an embodiment;

[0064] Figure 11 Schematic diagram of the image after luminance weight fusion in one embodiment;

[0065] Figure 12 Schematic diagram of the mapping relationship between the luminance input value and the luminance output value in one embodiment;

[0066] Figure 13 Schematic diagram of the mapping relationship between the luminance value and the adjustment coefficient in one embodiment;

[0067] Figure 14 Block diagram of the structure of the image processing apparatus in one embodiment;

[0068] Figure 15 Internal structure diagram of a computer device in one embodiment. Detailed implementation manners

[0069] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] In one embodiment, as Figure 1 shown, a method for image processing is provided. In this embodiment, the application of this method to a terminal is taken as an example for illustration. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The above-mentioned terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented by an independent server or a server cluster composed of multiple servers. The terminal includes an image acquisition device. In this embodiment, the image processing method includes the following steps:

[0071] Step 102, obtaining a plurality of reference images corresponding to the target image.

[0072] Among them, each reference image is obtained by downsampling and upsampling the target image a target number of times based on the target sampling scale. The multiple reference images corresponding to the target image are reference images obtained by performing downsampling processing and upsampling processing on the target image a target number of times based on the target sampling scale, and the magnification factors of each reference image are different.

[0073] Specifically, the terminal can perform downsampling and upsampling on the target image a target number of times at a target sampling scale to obtain multiple reference images corresponding to the target image; in one example, the terminal can perform sampling processing a target number of times at a sampling scale of a target magnification to obtain reference images corresponding to multiple different magnifications; for example, the terminal can perform downsampling on the target image at a sampling scale of a target magnification to obtain a first downsampled image, and based on the sampling scale of the target magnification, perform upsampling on the first downsampled image to obtain a first reference image and output the first reference image; based on this, the terminal can perform downsampling on the first downsampled image at a sampling scale of a target magnification to obtain a second downsampled image, and determine an upsampling scale based on the number of sampling times, and perform upsampling on the second downsampled image to obtain a second reference image; the terminal can perform multiple iterative processes to obtain multiple reference images.

[0074] In a specific example, the sampling scale of the target magnification can be 2x. The terminal can perform downsampling on the target image (denoted as W, H) at a 2x scale to obtain a first downsampled image (denoted as W / 2, H / 2), and perform upsampling scale restoration processing on the first downsampled image to obtain a first reference image. The image size of the first reference image is the same as that of the target image, where the image size includes the length information and width information of the image; based on this, the terminal can perform downsampling on the first downsampled image again at a 2x scale to obtain a second downsampled image (denoted as W / 4, H / 4). Correspondingly, perform upsampling scale restoration processing on the second downsampled image to obtain a second reference image. The image size of the second reference image is the same as that of the target image; the terminal can perform downsampling on the second downsampled image again at a 2x scale to obtain a third downsampled image (denoted as W / 8, H / 8). Correspondingly, perform upsampling scale restoration processing on the third downsampled image to obtain a third reference image. The image size of the third reference image is the same as that of the target image; the terminal can perform downsampling on the third downsampled image again at a 2x scale to obtain a fourth downsampled image (denoted as W / 16, H / 16). Correspondingly, perform upsampling scale restoration processing on the third downsampled image to obtain a fourth reference image. The image size of the fourth reference image is the same as that of the target image. Based on this, the terminal can obtain multiple reference images, where each reference image is an image with the same size information as the target image obtained after downsampling processing and upsampling scale restoration processing.

[0075] Step 104: Based on the gradient values of the pixel points of the target image, in the multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening strategy, and perform fusion processing on the target sub-images corresponding to each pixel point to obtain a fused image.

[0076] Among them, the gradient value of a pixel represents the luminance difference between the luminance information of this pixel and that of adjacent pixels; the target image screening strategy can be to determine a reference image matching the gradient value based on the gradient values of each pixel. For example, each reference image is obtained through downsampling and upsampling processes with different magnifications. The target image screening strategy can be that the gradient value is negatively correlated with the magnification, that is, the target sub-image of a pixel with a smaller gradient value is determined in a reference image with a higher magnification, and the image information retained in this target sub-image is the low-frequency information in the reference image with a high magnification sampling.

[0077] Specifically, for each pixel among the multiple pixels included in the target image, the terminal can obtain the gradient value of this pixel, and based on the target image screening strategy, determine a target reference image matching this gradient value among multiple reference images, and in this target reference image, determine the image at the position coordinate consistent with the position coordinate of this pixel as the target sub-image of this pixel. The terminal can perform image fusion processing and image stitching processing based on the target sub-images of each pixel to obtain a fused image.

[0078] In one example, for each pixel among the multiple pixels included in the target image, the terminal can determine a gradient weight matching this gradient value based on the gradient value of this pixel and a preset gradient weight determination strategy, and based on this pixel's gradient weight and the target image screening strategy, determine a target reference image matching this gradient weight among multiple reference images. For example, the pixel area with a smaller gradient value is a smooth area, and the terminal determines that the gradient weight of this pixel is smaller based on the preset gradient weight determination strategy. Correspondingly, the target reference image determined based on the target image screening is the low-frequency information in the reference image with a high magnification sampling. The pixel area with a larger gradient value is a strong edge area, and the terminal determines that the gradient weight of this pixel is higher based on the preset gradient weight determination strategy. Correspondingly, the target reference image determined by the terminal based on the target image screening is the image information in the reference image with a low magnification sampling.

[0079] Step 106, determine a target brightness value adjustment coefficient based on the fused brightness values of the pixels of the fused image, and perform pixel value mapping on the brightness values of the pixels included in the target image based on the fused brightness values of the pixels, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjustment image.

[0080] Specifically, for each pixel point included in the target image, the terminal can determine the target brightness value adjustment coefficient corresponding to the pixel point through the fusion brightness value of the pixel point in the fusion image, and perform a mapping process on the brightness value of the pixel point based on the determined target brightness value adjustment coefficient, the fusion brightness values of the pixel points, and the gain algorithm, to obtain the brightness value after pixel value mapping of the pixel point; the terminal can adjust the brightness values of the respective pixel points included in the target image to the brightness values after pixel value mapping corresponding to the respective pixel points, respectively, to obtain a target adjustment image.

[0081] In this embodiment, by combining the brightness information of the multi-magnification sampling scale of the target image with the gradient of the image itself, and determining the corresponding brightness value adjustment coefficient based on the high or low brightness values of the respective pixel points of the target image, and performing brightness adjustment on the brightness value based on the brightness value adjustment coefficient, it can be ensured that the brightness value adjustment coefficient is determined based on the brightness value, realizing the adaptive adjustment of the brightness of the high-brightness area and realizing the adjustment of the image information of the low-brightness area, achieving a better image processing effect and improving the image quality of brightness processing.

[0082] In one embodiment, the target brightness value adjustment coefficient includes the brightness adjustment coefficients corresponding to the respective pixel points; the specific implementation manner of step 106, determining the target brightness value adjustment coefficient based on the fusion brightness values of the respective pixel points of the fusion image, includes:

[0083] Based on the fusion brightness values of the respective pixel points of the fusion image and the target brightness value adjustment coefficient curve, determine the brightness adjustment coefficients corresponding to the fusion brightness values of the respective pixel points.

[0084] Among them, the target brightness value adjustment coefficient curve can be used to adjust the mapping relationship between the fusion brightness value of each pixel point and the adjusted brightness value. This target brightness value adjustment coefficient curve is the corresponding relationship between the fusion brightness value of each pixel point of the fusion image and the brightness adjustment coefficient. The first coordinate axis represents the fusion brightness value of the pixel point, and the second coordinate axis represents the brightness adjustment coefficient.

[0085] Specifically, the terminal can also determine the brightness adjustment coefficient matching the fusion brightness value of each pixel point in the target brightness value adjustment coefficient curve based on the fusion brightness values of the respective pixel points of the fusion image.

[0086] In this embodiment, the corresponding brightness value adjustment coefficient can be determined based on the high or low brightness values of the respective pixel points of the target image, and the brightness value mapping curve can be adjusted in brightness based on the brightness value adjustment coefficient, which can ensure that the brightness value adjustment coefficient is determined based on the fusion brightness value and ensure the accuracy of coefficient determination.

[0087] In one embodiment, step 106, based on the fusion luminance values of the respective pixel points, the gain algorithm, and the target luminance value adjustment coefficient, performing pixel value mapping on the luminance values of the pixel points included in the target image to obtain a target adjustment image, the specific implementation manner includes:

[0088] Based on the fusion luminance values of the respective pixel points of the fusion image and the gain algorithm, perform mapping to obtain an initial luminance value mapping curve.

[0089] Among them, the initial luminance value mapping curve represents the mapping relationship between the fusion luminance values of the respective pixel points of the fusion image and the adjusted luminance values. The gain algorithm is used to determine the gain values in different luminance regions; the target luminance value adjustment coefficient curve can be used to adjust the mapping relationship between the fusion luminance values of the respective pixel points and the adjusted luminance values. This target luminance value adjustment coefficient curve is the corresponding relationship between the fusion luminance values of the respective pixel points of the fusion image and the luminance adjustment coefficient. The first coordinate axis represents the fusion luminance values of the pixel points, and the second coordinate axis represents the luminance adjustment coefficient.

[0090] Specifically, the terminal can perform luminance range division based on the fusion luminance values of the respective pixel points of the fusion image to obtain multiple luminance regions with different luminance ranges, and calculate the luminance gain of each luminance region based on a preset gain algorithm. Based on the fusion luminance values of the respective pixel points of the fusion image and the luminance gain of each luminance region calculated by the gain algorithm, obtain the initial luminance value mapping curve.

[0091] Based on the luminance adjustment coefficient, adjust the initial luminance value mapping curve to obtain a target luminance value mapping curve.

[0092] Specifically, for each pixel point among the multiple pixel points included in the luminance fusion image, the terminal can obtain the fusion luminance value of this pixel point, and determine the luminance adjustment coefficient matching the fusion luminance value of this pixel point in the target luminance value adjustment coefficient curve. Based on this, the terminal can adjust the initial luminance value mapping curve based on the determined luminance adjustment coefficient to obtain a target luminance mapping curve. Among them, the target luminance mapping curve represents the luminance value of the pixel point to be adjusted and the target luminance value to be adjusted to.

[0093] Through the target luminance value mapping curve, perform pixel value mapping on the luminance values of the pixel points included in the target image to obtain a target adjustment image.

[0094] Specifically, for each pixel point included in the target image, the terminal can perform pixel value mapping processing on the brightness value of the pixel point through the target brightness value mapping curve. The terminal can determine the fused brightness value of the pixel point in the fused image, determine the mapping relationship corresponding to the fused brightness value in the target brightness value mapping curve, and perform pixel value mapping processing on the brightness value of the pixel point in the target image based on the mapping relationship to obtain the brightness value after pixel value mapping of the pixel point. The terminal can adjust the brightness values of the respective pixel points included in the target image to the brightness values after pixel value mapping corresponding to the respective pixel points respectively to obtain the target adjusted image.

[0095] The above image processing method can combine the brightness information of the multi-magnification sampling scale of the target image with the gradient of the image itself to obtain the brightness value mapping curve, determine the corresponding brightness value adjustment coefficient based on the high and low of the brightness values of the respective pixel points of the target image, and perform brightness adjustment on the brightness value mapping curve based on the brightness value adjustment coefficient, which can ensure that the brightness value adjustment coefficient is determined based on the brightness value, realize the adaptive adjustment of the brightness of the high-brightness area and realize the adjustment of the image information of the low-brightness area, and achieve a better image processing effect and improve the image quality of brightness processing.

[0096] In one embodiment, as Figure 2 shown, the specific processing process of the step "performing pixel value mapping on the brightness value of the pixel point included in the target image through the target brightness value mapping curve to obtain the target adjusted image" includes:

[0097] Step 202, performing pixel value mapping on the brightness value of the pixel point included in the target image through the target brightness value mapping curve to obtain the initial adjusted image.

[0098] Specifically, for each pixel point included in the target image, the terminal can perform pixel value mapping processing on the brightness value of the pixel point through the target brightness value mapping curve. The terminal can determine the fused brightness value of the pixel point in the fused image, determine the mapping relationship corresponding to the fused brightness value in the target brightness value mapping curve, and perform pixel value mapping processing on the brightness value of the pixel point in the target image based on the mapping relationship to obtain the brightness value after pixel value mapping of the pixel point. The terminal can adjust the brightness values of the respective pixel points included in the target image to the brightness values after pixel value mapping corresponding to the respective pixel points respectively to obtain the initial adjusted image.

[0099] Step 204, calculating the gain value corresponding to each pixel point based on the brightness value of each pixel point of the initial adjusted image and the brightness of each pixel point included in the target image.

[0100] Among them, the gain value characterizes the difference degree of the brightness value of the pixel point before and after adjustment.

[0101] Specifically, for each pixel point included in the initial adjustment image, the terminal can determine the pixel point that matches the position coordinates of the pixel point in the target image, calculate the ratio of the brightness value of the pixel point in the initial adjustment image to the brightness value of the pixel point in the target image, and determine the ratio as the gain value of the pixel point.

[0102] Step 206: Based on the target adjustment strategy, adjust the gain values corresponding to each pixel point to obtain the adjusted gain values of each pixel point.

[0103] Among them, the target adjustment strategy can be a smoothing processing strategy, a mean processing strategy, etc.

[0104] Specifically, the terminal can adjust the gain values of each pixel point based on the target adjustment strategy to obtain the adjusted gain values of each pixel point. In one example, the target adjustment strategy can be a smoothing processing strategy. Based on this, the terminal can perform smoothing processing on the gain values corresponding to each pixel point to obtain the smoothed gain values of each pixel point.

[0105] Step 208: Process the brightness values of the pixel points of the target image through the adjusted gain values of each pixel point to obtain the target adjustment image.

[0106] Specifically, for each pixel point included in the target image, the terminal can calculate the target brightness value based on the brightness value of the pixel point in the target image and the adjusted gain values of each pixel point, and adjust the brightness value of the pixel point to the target brightness value to obtain the target adjustment image.

[0107] In this embodiment, by performing smoothing processing or mean processing on the gain values of each pixel point, the brightness value of the image can be adjusted again through the adjusted gain, avoiding the appearance of abrupt points and distorted points, and ensuring the image processing quality.

[0108] In one embodiment, before the step of determining each target sub - graph that matches the target image screening strategy from multiple reference images corresponding to the target image based on the gradient values of the pixel points of the target image and performing fusion processing on the target sub - graphs corresponding to each pixel point to obtain a fusion image, as Figure 3 shown, this image processing method further includes:

[0109] Step 302: Through a brightness adjustment algorithm, uniformly adjust the brightness values of the pixel points of the initial image to obtain a target image.

[0110] Among them, the brightness adjustment algorithm can be a Gamma correction algorithm, etc. The initial image can be the original image output by the photosensitive device in the terminal, and the target image can be an image with a brightness distribution meeting the uniform condition.

[0111] Specifically, the terminal can perform brightness adjustment processing on the acquired initial image. The initial image can be an image with uneven linear distribution of brightness data. After adjustment, the obtained target image can be an image with uniform distribution of brightness data in different brightness ranges.

[0112] In one example, the initial image can be raw format image data, or rgb format image data, or yuv format image data. When the initial image is raw data, the terminal can obtain the brightness information of each pixel point through low-pass filter convolution processing. It can also perform data domain conversion processing on the image to obtain frgb format image data, and perform color space conversion on the frgb data to obtain the target image. For example, the data of each pixel point in the rgb format image data can include channel values corresponding to multiple channels. For each pixel point, the terminal can perform weighted calculation based on the preset weights of each channel and the channel values corresponding to each channel to obtain the brightness value of the pixel point. Based on this, the terminal can perform brightness adjustment processing on the brightness values of each pixel point on the acquired image, so that the unevenly linearly distributed brightness data can be relatively evenly distributed in different brightness ranges.

[0113] Step 304: Calculate the gradient value of each pixel point based on the high-frequency operator and the brightness value of each pixel point of the target image.

[0114] Among them, the high operator can be a high-frequency information extraction operator for calculating the gradient value of each pixel point.

[0115] Specifically, the terminal can calculate through the high-frequency information extraction operator and the brightness value of each pixel point included in the target image to obtain the gradient value corresponding to each pixel point. Among them, the gradient value of the pixel point represents the smoothness of the pixel point. The larger the gradient value of the pixel point, the greater the difference degree between the brightness value of the pixel point and the brightness value of the adjacent pixel point, and this pixel point is a strong edge area. The smaller the gradient value of the pixel point, the smaller the difference degree between the brightness value of the pixel point and the brightness value of the adjacent pixel point, and this pixel point is a smooth display area.

[0116] Step 306: Determine the gradient weight corresponding to each gradient value based on the target gradient weight strategy.

[0117] Among them, the target gradient weight strategy includes that the gradient value and the gradient weight are positively correlated. The target gradient weight strategy also includes various forms of mapping relationships. In one possible implementation manner, the target gradient weight strategy can also include a negatively correlated association relationship between the gradient value and the gradient weight.

[0118] Specifically, the terminal can determine the gradient weights corresponding to the gradient values of each pixel point respectively based on the target gradient weight policy. In one example, the target gradient weight policy can be that the gradient value and the gradient weight are positively correlated, and the number of gradient weights included can be the same as the number of pixel points. In this way, the terminal can arrange the gradient values of each pixel point in descending order to obtain a pixel point sequence, and arrange the gradient weights in descending order to assign weights to each pixel point in the pixel point sequence. For example, the terminal can configure the gradient weight with the largest gradient weight value as the gradient weight of the pixel point with the largest gradient value until the gradient weight with the smallest gradient weight value is configured as the gradient weight of the pixel point with the smallest gradient value. Or, the target gradient weight policy can be the mapping relationship between the gradient value and the gradient weight, and the terminal can determine the gradient weights corresponding to the gradient values of each pixel point respectively based on the mapping relationship included in the target gradient weight policy.

[0119] In this embodiment, by determining the gradient weights corresponding to each pixel point respectively, the efficiency of determining the target reference images corresponding to each pixel point respectively can be improved, and the image processing efficiency can be improved.

[0120] In one embodiment, as Figure 4 shown, the specific processing process of the step "Based on the gradient values of the pixel points of the target image, in the multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening policy, and perform fusion processing on the target sub-images corresponding to each pixel point to obtain a fused image" includes:

[0121] Step 402, for each pixel point included in the target image, based on the gradient weight corresponding to the gradient value of the pixel point, in the multiple reference images corresponding to the target image, determine the target reference image, extract the first pixel point whose position coordinates match the position coordinates of the pixel point in the target reference image, and determine the image where the first pixel point is located as the target sub-image.

[0122] Specifically, for each pixel point among the multiple pixel points included in the target image, the terminal can obtain the gradient value of the pixel point and a preset gradient weight determination strategy, determine the gradient weight matching the gradient value, and based on the gradient weight of the pixel point and the target image screening strategy, determine the target reference image matching the gradient weight from multiple reference images. For example, the pixel point area with a smaller gradient value is a smooth area. The terminal determines that the gradient weight of this pixel point is smaller based on the preset gradient weight determination strategy. Correspondingly, the target reference image determined based on the target image screening is the low-frequency information in the high-magnification sampled reference image. The pixel point area with a larger gradient value is a strong edge area. The terminal determines that the gradient weight of this pixel point is higher based on the preset gradient weight determination strategy. Correspondingly, the target reference image determined by the terminal based on the target image screening is the image information in the low-magnification sampled reference image.

[0123] Based on this, for the pixel points in the target image, the terminal can determine the first pixel point in the target reference image based on the target position coordinates of the pixel point, and determine the image where the first pixel point is located as the target sub-image.

[0124] Step 404: Perform stitching and fusion processing on the target sub-images corresponding to the respective pixel points included in the target image to obtain a fused image.

[0125] Specifically, the terminal can obtain the target sub-images corresponding to the respective pixel points included in the target image, and perform stitching processing and fusion processing on the target sub-images corresponding to the respective pixel points according to the position coordinates of the respective pixel points in the target image to obtain a fused image.

[0126] In this embodiment, by the gradient values of the respective pixel points of the target image itself, screening is performed among the multiple reference images obtained after multiple sampling processes, that is, image stitching processing is performed based on images with different sampling magnifications. Information on better highlighted areas can be obtained in a relatively simple manner, and flat areas, edge areas, and weak texture areas can be better distinguished.

[0127] In one embodiment, as Figure 5 shown, the specific processing procedure of the step "perform mapping based on the fused brightness values of the respective pixel points of the fused image and the gain algorithm to obtain the initial brightness value mapping curve" includes:

[0128] Step 502: Divide the fused brightness values of the respective pixel points of the fused image based on multiple brightness value ranges to obtain multiple brightness areas, and calculate the brightness mean values of the respective brightness areas.

[0129] Among them, multiple brightness value ranges can be the first brightness range, the second brightness range, and the third brightness range. For example, the first brightness range can be a low brightness range, the second brightness range can be a medium brightness range, and the third brightness range can be a high brightness range. In one example, the brightness value interval of the low brightness area can be idx00 to idx01, the brightness value interval of the medium brightness area can be idx10 to idx11; the brightness value interval of the high brightness area can be idx20 to idx21.

[0130] Specifically, the terminal can perform brightness value statistics and brightness value differentiation on the brightness values of each pixel point included in the fused image based on the brightness value ranges of each brightness area, and obtain the first brightness area, the second brightness area, and the third brightness area corresponding to the fused image respectively.

[0131] Step 504, through a gain algorithm, process the brightness means of each brightness area to obtain the gain values of each brightness area.

[0132] Step 506, based on the gain values of each brightness area and the brightness means of each brightness area, perform dot plotting calculations to obtain an initial brightness value mapping curve.

[0133] Among them, the gain algorithm can be an exposure ratio split-frame gain calculation algorithm; the initial brightness value mapping curve is the mapping relationship between the fused brightness value and the target brightness value.

[0134] Specifically, the terminal can calculate the gain values of each brightness area through the exposure ratio split-frame gain calculation algorithm. The gain value of the first brightness area can be ev0, the gain value of the second brightness area can be ev1, and the gain value of the third brightness area can be ev2. The gain values corresponding to different brightness areas can be the slope values of the line graph input and the mapping graph output curve. The gains calculated by multiple exposure split-frames can be k0, k1, and k2 respectively. k0 is the brightness gain value of the first brightness area, k1 is the brightness gain value calculated for the second brightness area (medium brightness area), and k2 can be the brightness gain value (gain value) of the third brightness area (high brightness area).

[0135] In one example, the terminal can calculate the gain reference values corresponding to each brightness area respectively through a preset exposure ratio split-frame gain calculation algorithm, and adjust the gain values corresponding to the brightness area through the calculated gain reference values to obtain the adjusted gain values. Based on this, the terminal can obtain an initial brightness value mapping curve based on the adjusted gain values of each brightness area and the brightness means corresponding to each brightness area respectively. Among them, the initial brightness value mapping curve is the corresponding relationship between the brightness value to be adjusted and the corresponding target adjusted brightness value.

[0136] Specifically, the gain value of the third brightness region that the terminal can calculate can be ev2, and the terminal can also calculate the gain reference value corresponding to the gain value of the third brightness region. The gain reference value is used to adjust the gain value of the third brightness region. For example, the image frame (ev2 frame) of the third brightness region can be a linearly input frame, and the information retained on this image frame is the information of the highlighted region in the scene. In one implementation, the gain value corresponding to the ev2 frame can be 1. Through the gain reference value corresponding to the ev2 frame, the anchor point calculation of the highlighted region mapping curve can be adjusted.

[0137] The terminal can perform point plotting calculations based on the gain values of ev2 to ev0 and the brightness means of each brightness region to obtain anchor point values, perform interpolation calculations on the anchor point values, and expand the sampling rate to obtain a smooth high-sampling rate curve, that is, the initial brightness value mapping curve. For example, x00 can be the brightness mean x00 of the low-brightness region. After being multiplied by the ev0 gain k0, x00 is mapped to y00. The pixel gain in the abscissa range from x = 0 to x = x00 is k0. The low-brightness region uses the brightness gain value of ev0, that is, the pixel brightness of the low-brightness region uses ev0. Among them, the principle of multi-exposure frame splitting is exactly to use the exposure ratio as the gain of the low-brightness region. This gain is the maximum among all ev gains. Correspondingly, the terminal can multiply the brightness mean x01 of the medium-brightness region by the ev1 gain to obtain the gain brightness value y01. The purpose of this gain is to use the gain k1 used in the frame with medium exposure intensity of ev1 as the gain value of the brightness mean of the medium-brightness region. Correspondingly, the terminal can adjust based on the gain value 1 and the ev2 gain reference value to obtain the adjusted gain value.

[0138] In this embodiment, by obtaining the brightness value mapping curve through the gain values, gain reference values, and brightness means respectively corresponding to each brightness region, it can be ensured that the originally bright regions do not have excessive brightness gain, avoiding the overflow of the highlighted region range, that is, avoiding the phenomenon that the bright transition region gradually becomes brighter and the highlighted range becomes larger.

[0139] In one embodiment, the specific processing process of the step "determine the brightness adjustment coefficient corresponding to the fused brightness value of each pixel point of the fused image based on the fused brightness value of each pixel point of the fused image and the target brightness value adjustment coefficient" includes:

[0140] On the target brightness value adjustment coefficient curve, determine the adjustment coefficient that matches the fused brightness value of the pixel points of the fused image.

[0141] Among them, the target brightness value adjustment coefficient curve can be a mapping curve between the brightness value and the adjustment coefficient, and the brightness value and the brightness adjustment coefficient in the target brightness value adjustment coefficient curve are negatively correlated.

[0142] Specifically, for each pixel in the target image, the terminal can determine the fusion brightness value corresponding to the pixel in the fused image, and query in the target brightness value adjustment coefficient curve based on the fusion brightness value of the pixel to determine the adjustment coefficient that matches the fusion brightness value. Based on this, the terminal can adjust the initial brightness value mapping curve based on the adjustment coefficient to obtain the target brightness value mapping curve.

[0143] In this embodiment, by adaptively adjusting the initial brightness value mapping curve based on the fusion brightness value corresponding to the pixel and the regulation curve, the brightness values of pixels with different brightness values are improved, the brightness suppression of areas with relatively high local brightness is achieved, the brightness enhancement of areas with relatively low local brightness is achieved, and the image processing quality is ensured.

[0144] In one embodiment, the specific processing procedure of the step "calculate the gain value corresponding to each pixel based on the brightness value of each pixel in the initial adjustment image and the brightness of each pixel included in the target image" includes:

[0145] For each pixel included in the target image, calculate the ratio of the brightness value of the pixel in the initial adjustment image to the brightness value of the pixel in the target image, and determine the ratio as the gain value of the pixel.

[0146] Specifically, for each pixel included in the target image, the terminal can obtain the brightness value of the pixel in the target image, and based on the position coordinates of the pixel, determine the pixel with matching position coordinates in the initial adjustment image, and obtain the brightness value of the pixel. Based on this, the terminal can perform mapping gain calculation on the brightness value of the pixel in the initial adjustment image and the brightness value of the pixel in the target image, that is, calculate the ratio of the brightness value of the pixel in the initial adjustment image to the brightness value of the pixel in the target image, and determine the ratio as the gain value of the pixel.

[0147] In this embodiment, by calculating the gain value through the ratio of the brightness value of the pixel with the same position coordinates in the initial adjustment image to the brightness value in the target image, the accuracy of the gain value calculation can be ensured.

[0148] In one embodiment, the specific processing procedure of the step "calculate the gradient value of each pixel based on the high-frequency operator and the brightness value of each pixel in the target image" includes:

[0149] For each pixel, calculate the gradient value of the pixel through the high-frequency operator, the brightness value of the pixel, and the brightness values of the adjacent pixels of the pixel.

[0150] Among them, the high-frequency operator can be a high-frequency information extraction operator or a gradient calculation operator; in one example, the adjacent pixel points of the pixel point can be eight pixel points around the pixel point.

[0151] Specifically, for each pixel point in the target image, the terminal can calculate the brightness difference between the brightness value of the pixel point and the brightness values of the adjacent pixel points of the pixel point through a high-frequency information extraction operator, and determine the brightness difference as the gradient value of the pixel point. Based on this, the terminal can obtain the gradient values of each pixel point included in the target image, and obtain the gradient intensity distribution information of the entire image corresponding to the target image, that is, the gradient values of each pixel point.

[0152] In this embodiment, obtaining the gradient values of each pixel point through the gradient calculation operator can ensure the accuracy of the gradient value calculation.

[0153] This application also provides an application scenario that applies the above image processing method. Specifically, the application of the image processing method in this application scenario is as follows:

[0154] As Figure 6 shown, it is a schematic flowchart of the image processing method provided by the embodiment of this application. Among them, the output processing result image of the front-end processing flow can be image data 00 (initial image), and the image is preprocessed through an image preprocessing module to obtain filtered image data 01. Specifically, the terminal can perform image preprocessing based on the specific front-end processing method. The tonemapping data type can be raw type, rgb type, or yuv type. In the image data 00, it is usually raw type image data or rgb type image data. For the raw type image data, as Figure 7 shown, the specific process of the terminal preprocessing the image data 00 can be: first, perform an image data domain conversion on the image data 00, convert the original raw data into frgb data, and perform spatial conversion processing and brightness adjustment processing on the frgb type image data to obtain a target image. For example, the terminal can perform brightness adjustment processing based on Gamma correction to obtain the target image, that is, the filtered image data 01.

[0155] The terminal can input the result of the image preprocessing, "filtered image data 01", into the "filter module" to extract gradient information or high-frequency information. Specifically, the terminal can obtain the gradient intensity distribution of the "filtered image data 01" through a gradient calculation operator, that is, the gradient values corresponding to each pixel point globally in the "filtered image data 01"; the terminal can perform the calculation of "gradient weight mapping" based on the obtained "gradient intensity distribution", and finally obtain the weight sizes corresponding to the brightness information at different gradient positions of the entire image, that is, the gradient weights corresponding to the gradient values of each pixel point respectively.

[0156] The terminal can also input the result of image preprocessing, "filtered image data 01", into the multi-scale sampling process. Specifically, a method of sampling in a loop multiple times can be used to perform downsampling on the input image at different downsampling ratios, and then upsample them to the original image size respectively, so as to present brightness information at different frequencies on the original image size. Finally, in the "multi-scale sampling scale brightness information", multiple brightness distribution maps of the downsampled images at different ratios and then upsampled back to the original size are obtained, that is, multiple target reference images at different ratios. In one example, the terminal can perform downsampling on the filtered image data 01 to obtain a downsampling output, and based on the sampling layer count, send the downsampling output to the upsampling 00 module for upsampling. Correspondingly, the downsampling output is also re-input to the downsampling module to obtain a second downsampling output until the sampling layer count reaches the target sampling times, obtaining multiple brightness distribution maps of the downsampled images at different ratios and then upsampled back to the original size, that is, multiple target reference images at different ratios.

[0157] The terminal can combine the output result of "gradient weight mapping" with the "multi-scale sampling scale brightness information" and finally obtain the "brightness weight fusion" result. This "brightness weight fusion" result is a single image, containing the fused brightness values of each pixel. The terminal can perform "brightness statistics" processing on this "brightness weight fusion" result to generate "mapping relationship 01", that is, the target brightness value mapping curve. The terminal can send the input data "image data 00" into the "mapping relationship 01" for pixel value mapping to obtain the final mapped output pixel "pixel value mapping". The specific distribution of the full-image "mapping gain calculation" value required finally is obtained through the ratio calculation of "pixel value mapping" and the input data "image data 00", that is, the gain value of each pixel is obtained.

[0158] In one example, the terminal can also send the "filtered image data 01" into the "mapping relationship 01" for pixel value mapping to obtain the final mapped output pixel "pixel value mapping". The specific distribution of the full-image "mapping gain calculation" value required finally is obtained through the ratio calculation of "pixel value mapping" and the input data "filtered image data 01", that is, the gain value of each pixel is obtained.

[0159] The following is a specific embodiment, including: Step 1, obtaining of image data 00. The terminal can obtain high-dynamic linear input data, that is, image data 00, through the front-end processing flow module.

[0160] Step 2, perform "image preprocessing" on the high-dynamic "image data 00". The preprocessing process mainly includes: image data domain conversion, performing low-pass filtering convolution on the raw data to obtain the corresponding luminance information. Spatial conversion, converting the data in the original RGB space into data information with separated luminance and color. And performing luminance adjustment processing on the obtained luminance information so that the originally linearly distributed non-uniform luminance data (such as Figure 8a shown) can be more evenly distributed in different luminance ranges (such as Figure 8b shown), where the abscissa of the curve represents the position of each pixel point, and the ordinate represents the luminance value corresponding to each pixel point.

[0161] Step 3, the terminal can input the image preprocessing result "filtered image data 01" into the "filter" module. This module mainly provides a variety of high-frequency information extraction operators, namely gradient calculation operators. The terminal can calculate the gradient values of each pixel point of the "filtered image data 01" through the gradient calculation operator to obtain the "gradient intensity distribution" information of each pixel point in the whole image of the "filtered image data 01". Step 4, the terminal can perform "gradient weight mapping" on the "gradient intensity distribution" of each pixel point in the whole image obtained based on the mapping function (the mapping relationship between the gradient values and the gradient weights included in the target gradient weight strategy) to obtain the gradient weights of each pixel point included in the filtered image data 01. The mapping function in an example can be as Figure 9 shown. The function represented by the horizontal axis can be the gradient intensity, that is, the gradient value, and the function represented by the vertical axis value can be the weight, that is, the gradient weight values corresponding to each gradient value, specifically including the gradient value p01 and the gradient weight y01 corresponding to this gradient value p01, as well as the gradient value p00 and the gradient weight y00 corresponding to this gradient value p00, etc. The mapping function of this weight mapping can be various forms of functions or curves. This is only one example here.

[0162] Step 5, the terminal can input the "filtered image data 01" into the multi-scale sampling process. Determine how many scales of downsampled data there are through "sampling layer counting". For example, in the embodiments of the present application, 2x, 4x, 8x, 16x can be used as examples for illustration.

[0163] Step 5a, when the "filtered image data 01" is input into the multi-scale sampling process, it can first perform 2x (2 times) downsampling, and send the image after obtaining the 1 / 2 size (W / 2, H / 2) to the "upsampling 00" for upsampling processing, and finally obtain the image data with the original image width and height size (W, H), and record this processing process as one sampling layer.

[0164] Step 5b: Feed the 1 / 2-sized image obtained in the previous step into "Downsampling 00" again for the second downsampling process. An image data with a size of 1 / 4 (W / 4, H / 4) is obtained. And this data is fed into "Upsampling 00" for upsampling to the (W, H) size restoration process. The other path gives the data to the "Downsampling 00" data.

[0165] Step 5c: Based on Step 5a and Step 5b, put them into the execution process and execute it in a loop for multiple times until the number of loops reaches the preset number. Finally, the image brightness information of the (W, H) size after upsampling again after downsampling at 2x, 4x, 8x, and 16x magnification ratios is obtained in the "Multi-magnification Sampling Scale Brightness Information", that is, the target reference images after being processed at multiple different sampling magnification ratios are obtained.

[0166] Specifically, the image data of different magnification samplings in the "Multi-magnification Sampling Scale Brightness Information" is the image content information corresponding to different magnification ratios. There are more details in the low-magnification image, and there is more low-frequency information in the high-magnification sampled image. The high-frequency information is filtered out during the process of sampling and restoring back and forth. As Figure 10 shown, the image in the upper left corner is the target image, and the image in the upper right corner can be the reference image that is downsampled at 2x (2 times) and then upsampled back to the original image size; the image in the lower left corner can be the reference image that is downsampled at 4x (4 times) and then upsampled back to the original image size, that is, after the target image is downsampled twice at 2x (2 times), it is upsampled back to the original image size; the image in the lower right corner can be the reference image that is downsampled at 8x (8 times) and then upsampled back to the original image size, that is, after the target image is downsampled three times at 2x (2 times), it is upsampled back to the original image size. The clarity between the reference images is different, and there is a negative correlation relationship between the number of downsamplings and the clarity.

[0167] Step 6: The terminal can combine and process the "Gradient Weight Mapping" with the "Multi-magnification Sampling Scale Brightness Information"; the image data content obtained by the "Gradient Weight Mapping" is that the positions with larger gradients in the input image are set with lower weights, and the regions with smaller gradients have higher weights. Regarding this weight distribution, the positions with large gradients such as strong edges still retain the original brightness gradient change information, while the regions with small gradients have low weights, which means that these regions are flat areas and should retain the low-frequency brightness information left by high-magnification sampling. As Figure 11 shown, the left image can be the target image, and the right image can be the fused image obtained after the brightness weight fusion.

[0168] Specifically, the terminal can fuse the brightness image information of multiple different scales in the "Multi-magnification Sampling Scale Brightness Information" according to the distribution of the "Gradient Weight Mapping", and the finally obtained image brightness distribution data is as Figure 11As shown in the right image. In the relatively smooth area of the input image, low-frequency information obtained by high-magnification sampling is adopted, and at the strong edge positions, the image brightness information corresponding to the low-magnification or the original image data at the corresponding positions is retained.

[0169] Step 7, the terminal can perform local area statistics on the full-image brightness information obtained by "brightness weight fusion". For example, a single-pixel area statistical method can be adopted. That is, each pixel value in the full-image brightness information obtained by "brightness weight fusion" represents the brightness mean value at that position, and there is no need to divide the block area for brightness statistics.

[0170] Step 8, the terminal can calculate "mapping relationship 01" based on the fusion brightness value of each pixel point of "brightness weight fusion" to generate an initial brightness mapping curve. The specific process is as Figure 12 shown:

[0171] Step 8a, the terminal can perform "histogram statistics" processing on the input "image data 00" to obtain the histogram distribution information of the current image.

[0172] Step 8b, the terminal can divide the histogram range thresholds of different brightness intervals of high, medium, and low in the image through the "high brightness threshold" module, "medium brightness threshold" module, and "low brightness threshold" module to obtain the high brightness area, medium brightness area, and low brightness area. The terminal can adjust the high, medium, and low brightness histogram ranges by adjusting the idx numerical range.

[0173] Step 8c, the terminal can send the information of the "high brightness threshold" module, "medium brightness threshold" module, and "low brightness threshold" module and the information of the "histogram statistics" module into the "high brightness mean calculation module, medium brightness mean calculation module, low brightness mean calculation module" to obtain the brightness means of these three areas.

[0174] Step 8d, the terminal can obtain the gain values of each brightness area through a preset exposure ratio split-frame gain calculation algorithm, the gain value ev2 corresponding to the high brightness area, the gain value ev1 corresponding to the medium brightness area, and the gain value ev0 corresponding to the low brightness area. The terminal can also output the ev2 gain reference value through the "ev2 gain reference value" module, and this value can adjust the ev2 gain value output in the "exposure ratio split-frame gain calculation". Since the ev2 frame is a linear input frame, this frame is mainly used to collect the information of the high brightness area in the scene. The calculation result of this gain value is usually 1. The anchor point calculation of the high brightness area mapping curve can be adjusted through the "ev2 gain reference value".

[0175] Step 8e, the terminal can combine the ev2~ev0 gain values with the brightness values obtained by "high, medium, and low brightness mean calculation" and send them into the "anchor point 0 calculation module, anchor point 1 calculation module, and anchor point 2 calculation module. The anchor point calculation method is asFigure 12 As shown in the figure, x00 in the figure is the average brightness x00 of the low-brightness area output by "low-brightness average calculation". After being gain-adjusted by ev0 with k0, x00 is mapped to y00. The pixel gain in the interval from the abscissa x = 0 to x = x00 is all k0. That is, the low-brightness area adopts the brightness gain value of ev0, that is, the pixel brightness of ev0 in the low-brightness area. The principle of multi-exposure frame splitting is exactly to adopt the gain ratio of the low-brightness area. This gain is the maximum value among all ev gains.

[0176] Correspondingly, the terminal obtains the gain-adjusted brightness value y01 by gain-adjusting the average brightness x01 of the medium-brightness area through ev1. The purpose of this gain is to adopt the gain k1 used in the frame with medium exposure intensity of ev1 as the gain value for the average brightness of the medium-brightness area; the terminal processes x02 in the same way, and the ev2 gain is the gain value after adjusting the gain coefficient 1 with the "ev2 gain reference value". In the high-brightness area, we hope that the high-brightness average x02 can ensure the original bright area as much as possible without excessive brightness gain, otherwise it will cause the overflow of the high-brightness area range. That is, the brightness transition area gradually becomes too bright, resulting in an enlarged high-brightness range. That is to obtain the anchor point values of (x00, y00), (x01, y01), and (x02, y02).

[0177] Step 8f, the terminal can obtain the anchor point values of the high, medium, and low brightness areas through anchor point calculation, and send these anchor point values into the "interpolation calculation" module for sampling rate expansion to obtain a smooth high-sampling rate curve, that is, the initial brightness value mapping curve. The meaning represented by the horizontal axis is the input brightness value, and the meaning represented by the vertical axis value is the output brightness value.

[0178] Step 9, specifically, the terminal can send the input "image data 00" and the single-pixel brightness value of the "brightness statistics" module into the "mapping relationship 01" to calculate the target brightness value mapping curve at the position of this pixel finally.

[0179] The terminal can use the brightness information of the local area (taking the local area as a single pixel in this embodiment) obtained from the "brightness statistics" module to calculate the mapping curve adopted in the "mapping relationship 01". The target brightness value adjustment coefficient curve adopted in this scheme is as Figure 13As shown, Gaussian decay can be used as the weight setting for curve adjustment. When the local brightness of the input is low, it indicates that this area needs to be enhanced and brightened. The adjustment value scala val is close to 1 (keeping the original curve brightening amplitude unchanged) or greater than 1, so that the original curve brightening amplitude becomes larger. When the local brightness of the input is high, it indicates that the brightness of this area needs to be suppressed. The amplitude of the original curve should be reduced or even the scale val of the high-brightness area is 0. At this time, the value of the original curve amplitude (y - x) is multiplied by 0 to obtain a mapping curve of y = x, indicating that no brightening operation is performed on the brightness of this area. Among them, the abscissa represents the input brightness value (luma in), and the ordinate represents the adjustment coefficient (scale val).

[0180] The terminal can perform gain adjustment (the value of the curve function y - x) on the "mapping relationship 01 initial curve" by calculating the local area brightness obtained from the "brightness statistics" module, and output the finally calculated mapping curve, that is, the target brightness value mapping curve.

[0181] Step 10, the terminal can map the input "image data 01" using the mapping curve obtained from the "mapping relationship 01" to obtain "pixel value mapping", and perform "mapping gain calculation" based on the image data 00 or the filtered image data 01 and the pixel values after brightness mapping obtained from the "pixel value mapping" to obtain the gain values corresponding to all pixels in the whole image. It should be noted that the pixel points described in the embodiments of the present application can be single pixel points or multi-pixel regions including multiple single pixel points, such as pixel point regions of 2x2, 4x4, etc.; the embodiments of the present application can also extend from the linear interpolation mapping curve to multiple mapping relationships.

[0182] The calculation of the target brightness value mapping curve in the embodiments of the present application utilizes the brightness gain calculation of exposure frame splitting, differentiates the brightness improvement of high, medium, and low brightness regions, and makes the obtained mapping curve more reasonable and robust. In particular, the selection of the anchor points in the high-brightness region can better suppress the high-light diffusion information. Multi-magnification sampling can obtain better high-brightness region information with a relatively simple method. Local mapping of single-pixel regions is prone to such problems. It can better suppress the brightness of high-brightness light signs while brightening the dark areas. The advantage of local mapping of single-pixel regions is that it avoids the brightness block effect in scenes with high intensity in local block mapping.

[0183] The method provided in the embodiments of the present application can calculate a gain value for each pixel brightness as the mapping gain of the pixel, which can be compatible with the processing of the block scheme; secondly, through the process of multiple different magnification sampling and restoration, it can better obtain information of different frequencies in the image. This method has a good processing effect on many light sign regions, especially when there is some low-brightness text group information in the high-brightness light sign regions. The brightening operation on the low-brightness text patterns can be avoided through this method.

[0184] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0185] Based on the same inventive concept, an embodiment of the present application also provides an image processing apparatus for implementing the above-mentioned image processing method. The solution provided by this apparatus to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following image processing apparatus can refer to the limitations on the image processing method in the above text, and will not be repeated here.

[0186] In one embodiment, as Figure 14 shown, an image processing apparatus 1400 is provided. The image processing apparatus 1400 includes:

[0187] A first acquisition module 1402, configured to acquire a plurality of reference images corresponding to a target image, and each of the reference images is obtained by downsampling and upsampling based on a plurality of different sampling scales.

[0188] A first determination module 1404, configured to determine, based on the gradient values of the pixel points of the target image, each target sub-image that matches the target image screening strategy among the plurality of reference images corresponding to the target image, and perform fusion processing on the target sub-images corresponding to the pixel points to obtain a fused image.

[0189] A first mapping module 1406, configured to determine a target brightness value adjustment coefficient based on the fused brightness values of the pixel points of the fused image, and perform pixel value mapping on the brightness values of the pixel points included in the target image based on the fused brightness values of the pixel points, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjustment image.

[0190] Each module in the above image processing apparatus can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0191] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in Figure 15 . The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data of dimming control parameters. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an image processing method.

[0192] Those skilled in the art can understand that Figure 15 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0193] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0194] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0195] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0196] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties.

[0197] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0198] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0199] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtaining a plurality of reference images corresponding to the target image, where each of the reference images is obtained by downsampling and upsampling the target image a target number of times based on a target sampling scale; Based on the gradient values of the pixels of the target image, in the plurality of reference images corresponding to the target image, determining each target sub-image that matches the target image screening strategy, and performing fusion processing on the target sub-images corresponding to each of the pixels to obtain a fused image; Determining a target brightness value adjustment coefficient based on the fused brightness values of the pixels of the fused image, and based on the fused brightness values of the pixels, the gain algorithm, and the target brightness value adjustment coefficient, performing pixel value mapping on the brightness values of the pixels included in the target image to obtain a target adjusted image.

2. The method according to claim 1, wherein The target brightness value adjustment coefficient includes the brightness adjustment coefficients corresponding to each of the pixels; the determining the target brightness value adjustment coefficient based on the fused brightness values of the pixels of the fused image includes: Based on the fused brightness values of the pixels of the fused image and the target brightness value adjustment coefficient curve, determining the brightness adjustment coefficient corresponding to the fused brightness value of each of the pixels.

3. The method according to claim 2, characterized in that The performing pixel value mapping on the brightness values of the pixels included in the target image based on the fused brightness values of the pixels, the gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjusted image includes: Performing mapping based on the fused brightness values of the pixels of the fused image and the gain algorithm to obtain an initial brightness value mapping curve; Adjusting the initial brightness value mapping curve based on the brightness adjustment coefficient to obtain a target brightness value mapping curve; Through the target brightness value mapping curve, performing pixel value mapping on the brightness values of the pixels included in the target image to obtain a target adjusted image.

4. The method according to claim 3, wherein The performing pixel value mapping on the brightness values of the pixels included in the target image through the target brightness value mapping curve to obtain a target adjusted image includes: Through the target brightness value mapping curve, performing pixel value mapping on the brightness values of the pixels included in the target image to obtain an initial adjusted image; Based on the brightness values of the pixels of the initial adjusted image and the brightness of the pixels included in the target image, calculating the gain values corresponding to each of the pixels; Adjusting the gain values corresponding to each of the pixels based on a target adjustment strategy to obtain the adjusted gain values of each of the pixels; Through the adjusted gain values of each of the pixels, processing the brightness values of the pixels of the target image to obtain a target adjusted image.

5. The method according to claim 1, wherein Before the step of, based on the gradient values of the pixels of the target image, in the plurality of reference images corresponding to the target image, determining each target sub-image that matches the target image screening strategy, and performing fusion processing on the target sub-images corresponding to each of the pixels to obtain a fused image, the method further includes: Through a brightness adjustment algorithm, uniformly adjusting the brightness values of the pixels of the initial image to obtain a target image; Based on a high-frequency operator and the brightness values of the pixels of the target image, calculating the gradient values of each of the pixels; Determine the gradient weights corresponding to each gradient value based on the target gradient weight strategy, where the target gradient weight strategy includes that the gradient value and the gradient weight are positively correlated.

6. The method according to claim 5, characterized in that Based on the gradient values of each pixel point of the target image, in multiple reference images corresponding to the target image, determine each target sub-image that matches the target image screening strategy, and perform a fusion process on each target sub-image corresponding to each pixel point to obtain a fused image, including: For each pixel point included in the target image, based on the gradient weight corresponding to the gradient value of the pixel point, in multiple reference images corresponding to the target image, determine a target reference image, extract the first pixel point whose position coordinates in the target reference image match the position coordinates of the pixel point, and determine the image where the first pixel point is located as the target sub-image; Perform a splicing and fusion process on each target sub-image corresponding to each pixel point included in the target image to obtain a fused image.

7. The method according to claim 3, characterized in that, Based on the fused brightness values of each pixel point of the fused image and the gain algorithm, perform mapping to obtain an initial brightness value mapping curve, including: Based on multiple brightness value ranges, divide the fused brightness values of each pixel point of the fused image to obtain multiple brightness regions, and calculate the brightness mean value of each brightness region; Through the gain algorithm, process the brightness mean value of each brightness region to obtain the gain value of each brightness region; Based on the gain values of each brightness region and the brightness mean values of each brightness region, perform point plotting calculations to obtain an initial brightness value mapping curve, where the initial brightness value mapping curve is the mapping relationship between the fused brightness value and the target brightness value.

8. The method according to claim 2, wherein Based on the fused brightness values of each pixel point of the fused image and the target brightness value adjustment coefficient, determine the brightness adjustment coefficient corresponding to the fused brightness value of each pixel point, including: On the target brightness value adjustment coefficient curve, determine the adjustment coefficient that matches the fused brightness value of the pixel point of the fused image, and determine the adjustment coefficient as the brightness adjustment coefficient of the pixel point, where the brightness value and the brightness value adjustment coefficient in the target brightness value adjustment coefficient curve are negatively correlated.

9. The method according to claim 4, wherein Based on the brightness values of each pixel point of the initial adjustment image and the brightness of each pixel point included in the target image, calculate the gain value corresponding to each pixel point, including: For each pixel point included in the target image, calculate the ratio of the brightness value of the pixel point in the initial adjustment image to the brightness value of the pixel point in the target image, and determine the ratio as the gain value of the pixel point.

10. The method according to claim 5, characterized in that, Based on the high-frequency operator and the brightness values of each pixel point of the target image, calculate the gradient value of each pixel point, including: For each pixel point, calculate the gradient value of the pixel point through the high-frequency operator, the brightness value of the pixel point, and the brightness values of the adjacent pixel points of the pixel point.

11. An image processing apparatus, characterized in that, The device includes: A first acquisition module, configured to acquire multiple reference images corresponding to the target image, where each reference image is obtained by downsampling and upsampling based on multiple different sampling scales; A first determination module, configured to determine, based on gradient values of each pixel point of a target image, each target sub-image that matches a target image screening strategy from a plurality of reference images corresponding to the target image, and perform a fusion process on the target sub-images corresponding to each of the pixel points to obtain a fused image; A first mapping module, configured to determine a target brightness value adjustment coefficient based on the fused brightness values of each pixel point of the fused image, and perform pixel value mapping on the brightness values of the pixel points included in the target image based on the fused brightness values of each pixel point, a gain algorithm, and the target brightness value adjustment coefficient to obtain a target adjustment image.

12. An electronic device, including a memory and a processor, wherein a computer program is stored in the memory, characterized in that, When the computer program is executed by the processor, the processor is caused to execute the steps of the image processing method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 10 are implemented.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 10 are implemented.

Images