Tone mapping method, apparatus, electronic device and storage medium

Abstract

Provided in the present application are a tone mapping method, an apparatus, an electronic device and a storage medium. The method comprises: dividing an image into a plurality of sub-blocks, and acquiring a luminance histogram of each sub-block, the luminance histogram representing the frequency of pixels within the sub-block at different grayscale levels; acquiring a clipping threshold value corresponding to each sub-block, and correcting the luminance histogram of each sub-block according to the clipping threshold value corresponding to the sub-block, so as to generate a corrected histogram of each sub-block, wherein the frequency distribution across the different grayscale levels are equalized in the corrected histogram; solving a luminance mapping equation according to the luminance histogram and the corrected histogram of each sub-block, and establishing a luminance mapping relationship of each sub-block, the luminance mapping equation comprising a linear equation; and adjusting the image according to the luminance mapping relationship of each sub-block, so as to generate an adjusted image. Acquiring a clipping threshold value of each sub-block to perform equalization allows for more flexible adjustment of the local contrast, and using linear luminance mapping relationships ensures smoother transition between sub-blocks.

Description

Tone mapping method, device, electronic device and storage medium Technical Field

[0001] The present application relates to the field of image processing, and in particular to a tone mapping method, device, electronic device and storage medium. Background Art

[0002] Tone mapping is an image adjustment method widely used in the prior art, which can make the image displayed on the display more realistically.

[0003] In the prior art, image tone mapping methods involve segmenting each image block into categories, determining mapping parameters based on the categories of all pixels in each block, and finally performing tone mapping on the image block using the tone mapping curve. This method, which determines the final tone mapping curve based on category segmentation, has high computational complexity and significant resource overhead. Furthermore, the tone mapping relationship between pixels in adjacent domains is generated using bilinear interpolation, resulting in poor smoothness in the transition regions between neighboring domains.

[0004] Summary of the Invention

[0005] In view of this, the purpose of the present application is to provide a tone mapping method, device, electronic device and storage medium to solve the shortcomings of the above-mentioned existing tone mapping methods with high computational complexity.

[0006] In a first aspect, the present application provides a tone mapping method, which includes: dividing an image into multiple sub-blocks, obtaining a brightness histogram of each sub-block, and the brightness histogram is the frequency of pixels in the sub-block at different grayscale levels; obtaining a clipping threshold corresponding to each sub-block, and correcting the brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block to generate a corrected histogram for each sub-block, wherein the frequencies of different grayscale levels in the corrected histogram are evenly distributed; solving a brightness mapping equation based on the brightness histogram of the sub-block and the corrected histogram to establish a brightness mapping relationship for each sub-block, wherein the brightness mapping equation includes a linear equation; adjusting the image according to the brightness mapping relationship of each sub-block to generate an adjusted image.

[0007] The above-mentioned tone mapping method obtains the clipping threshold corresponding to each sub-block and performs equalization correction on the brightness histogram of the sub-block, thereby making the adjustment of local contrast more flexible, and using a linear brightness mapping relationship can make the transition between sub-blocks smoother. At the same time, the tone mapping method of the present application is simple and easy to implement, has low computational complexity, and reduces resource overhead.

[0008] In an optional embodiment of the present application, the method further includes: performing content detection on the image to obtain a content weight of each pixel; and establishing a brightness histogram of the sub-block based on the brightness value and content weight of each pixel in the sub-block.

[0009] In the above embodiment, content detection is performed, and the effect of the local high-frequency detail area on contrast adjustment is taken into consideration, so that the generated adjusted image has better contrast performance in the high-frequency detail area.

[0010] In an optional embodiment of the present application, content detection is performed on the image to obtain the content weight of each pixel, including: using a preset convolution kernel to convolve the pixel area corresponding to each pixel to obtain the convolution value of each pixel, and the convolution kernel is used for content detection; querying a preset weight table based on the convolution value corresponding to each pixel to obtain the content weight of each pixel.

[0011] In the above embodiment, when performing histogram statistics, the content weight of the current pixel should be added to the count of each pixel. By introducing the content weight generated by content detection, the contrast performance of the generated adjusted image in the high-frequency detail area will be better.

[0012] In an optional embodiment of the present application, the brightness histogram of the sub-block is established based on the brightness value and content weight of each pixel in the sub-block, including: judging the grayscale level of each pixel based on the brightness value of the pixel, and each grayscale level corresponds to at least one brightness value; establishing the brightness histogram based on all grayscale levels in the sub-block and the pixels corresponding to each grayscale level, and the frequency of each grayscale level in the brightness histogram is the sum of the number of pixels corresponding to the grayscale level and the content weights of all pixels corresponding to the grayscale level.

[0013] In an optional embodiment of the present application, obtaining the clipping threshold corresponding to each sub-block includes: obtaining the brightness relationship of each sub-block based on the comparison result of the average brightness of each sub-block and the average brightness of the image; and adjusting the preset clipping value according to the brightness relationship of each sub-block to obtain the clipping threshold corresponding to each sub-block.

[0014] In the above embodiment, different clipping thresholds are set for sub-blocks of different brightness levels. Different clipping thresholds are set for brighter or darker images, which allows for more flexible adjustment of local contrast.

[0015] In an optional embodiment of the present application, solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish the brightness mapping relationship of each sub-block includes: solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; obtaining a second brightness mapping relationship for each sub-block in the previous frame image of the image; and fusing the first brightness mapping relationship and the second brightness mapping relationship of each sub-block to establish a brightness mapping relationship for each sub-block.

[0016] In the above embodiment, the parameters of the linear equation of the brightness mapping relationship are obtained by fusing the parameters of the current frame and the parameters of the previous frame, which can avoid a large brightness difference between the previous frame image and the current frame image, making the transition between the two frames smoother.

[0017] In an optional embodiment of the present application, solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish the brightness mapping relationship of each sub-block includes: solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; obtaining a second brightness mapping relationship for each sub-block in the previous frame image of the image; obtaining a deviation relationship between the first brightness mapping relationship and the second brightness mapping relationship of each sub-block; if there is a deviation relationship of a sub-block greater than a deviation threshold, then using the second brightness mapping relationship of the sub-block as the brightness mapping relationship of the sub-block.

[0018] In the above embodiment, the two brightness mapping relationships of the sub-blocks in the previous and next frame images are compared. When the two brightness mapping relationships deviate greatly, the brightness mapping relationship of the previous frame image is taken as the brightness mapping relationship of the current frame image to avoid a large brightness difference between the previous and next frame images, making the transition between the previous and next frame images smoother.

[0019] In an optional embodiment of the present application, solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish the brightness mapping relationship of each sub-block includes: calculating the change in the probability density function of the corrected histogram image of each sub-block relative to the brightness histogram to obtain a first mapping relationship; converting the brightness value corresponding to each grayscale level of the brightness histogram in the sub-block based on the first mapping relationship to obtain multiple conversion values; solving the brightness mapping equation based on the brightness value corresponding to each grayscale level in the sub-block and the multiple conversion values ​​to obtain a first brightness mapping relationship corresponding to each grayscale level; based on the first brightness mapping relationship corresponding to each grayscale level, generating a brightness mapping relationship corresponding to each grayscale level, the brightness mapping relationship of a sub-block is the brightness mapping relationship of all grayscale levels in the sub-block.

[0020] In the above embodiment, the brightness value of the grayscale level and the conversion value are fitted to establish a brightness mapping relationship for each grayscale level. Each grayscale level has its own brightness mapping relationship, which makes the adjustment of the local contrast more flexible.

[0021] In an optional embodiment of the present application, the brightness mapping relationship corresponding to each grayscale level is generated based on the first brightness mapping relationship corresponding to each grayscale level, including: obtaining the second brightness mapping relationship corresponding to each grayscale level of the corresponding sub-block in the previous frame image of the image; fusing the first brightness mapping relationship corresponding to each grayscale level with the second brightness mapping relationship to generate a brightness mapping relationship corresponding to each grayscale level.

[0022] In an optional embodiment of the present application, when the brightness mapping equation is a linear equation, it includes a slope and an intercept. The image is adjusted according to the brightness mapping relationship of each sub-block to generate an adjusted image, including: obtaining multiple reference sub-blocks of the sub-block, the reference sub-blocks being adjacent sub-blocks of the sub-block; for any grayscale level in the sub-block, the slope of the brightness mapping relationship in the grayscale level of the sub-block and the multiple reference sub-blocks corresponding to the sub-block is used as the third coordinate axis, and the length and width are used as the first coordinate axis and the second coordinate axis respectively to establish a first coordinate system, and trilinear interpolation is performed based on the first coordinate system to generate an adjusted image. The method comprises the steps of: generating a first interpolation parameter, for each grayscale level in the sub-block, taking an intercept in a brightness mapping relationship of the sub-block and a plurality of reference sub-blocks corresponding to the sub-block at the grayscale level as a third coordinate axis, and establishing a second coordinate system with a length and a width as a first coordinate axis and a second coordinate axis respectively, performing trilinear interpolation based on the second coordinate system to generate a second interpolation parameter; establishing an interpolation brightness mapping relationship of the grayscale levels based on the first interpolation parameter and the second interpolation parameter; and adjusting the sub-block and the plurality of reference sub-blocks corresponding to the sub-block based on the interpolation brightness mapping relationship of each grayscale level in the sub-block to generate the adjusted image.

[0023] In the above embodiment, the mapping relationship of each gray level is generated by using the trilinear interpolation method, which can make the transition between image sub-blocks smoother.

[0024] In an optional embodiment of the present application, the brightness mapping equation is: Y=gain_a*X+gain_b; wherein Y is the brightness value of the pixel point in the corrected histogram, X is the brightness value of the pixel point in the brightness histogram, gain_a is the slope parameter of the brightness mapping relationship, and gain_b is the intercept parameter of the brightness mapping relationship.

[0025] In an optional embodiment of the present application, the image is adjusted according to the brightness mapping relationship of each sub-block to generate an adjusted image, including: obtaining a mapping sub-block and an interpolation sub-block in the image, each pixel in the mapping sub-block uses the brightness mapping relationship of the sub-block to adjust the brightness value, and each pixel in the interpolation sub-block uses the interpolation brightness mapping relationship to adjust the brightness value; obtaining multiple reference sub-blocks of the interpolation sub-block, the reference sub-block is an adjacent sub-block of the interpolation sub-block; using gain_a in the brightness mapping relationship of the interpolation sub-block and the multiple reference sub-blocks corresponding to the interpolation sub-block as the third coordinate axis, and the length and width as the first coordinate axis and the second coordinate axis respectively to establish a first coordinate system. A coordinate system is provided, and trilinear interpolation is performed based on the first coordinate system to generate first interpolation parameters; gain_b in the brightness mapping relationship of the interpolation sub-block and multiple reference sub-blocks corresponding to the interpolation sub-block is used as the third axis, and length and width are used as the first coordinate axis and the second coordinate axis respectively to establish a second coordinate system, and trilinear interpolation is performed based on the second coordinate system to generate second interpolation parameters; an interpolation brightness mapping relationship of the interpolation sub-block is established based on the first interpolation parameters and the second interpolation parameters; the brightness value of the mapping sub-block in the image is adjusted based on the brightness mapping relationship, and the brightness value of the pixel point of the interpolation sub-block in the image is adjusted based on the interpolation brightness mapping relationship to generate the adjusted image.

[0026] In the above embodiment, the mapping relationship of the interpolation sub-blocks is generated by using the trilinear interpolation method, which can make the transition between the image sub-blocks smoother.

[0027] In an optional embodiment of the present application, the brightness histogram of each sub-block is corrected according to the clipping threshold corresponding to the sub-block to generate a corrected histogram of each sub-block, including: performing multiple rounds of iteration until a stop condition is met, the iteration including: finding an adjustment level in the histogram of this round, the adjustment level being a grayscale level whose frequency is greater than the clipping threshold; obtaining a distribution value of this round according to the adjustment level, and setting the frequency of each adjustment level as the clipping threshold; calculating an increase value of evenly distributing the distribution value of this round to each grayscale level in the intermediate histogram of this round; if the frequency of the grayscale level is less than the clipping threshold, the frequency of the grayscale level in the intermediate histogram of the new round is the sum of the increase value and the frequency of this round; if the grayscale level is less than the clipping threshold, the frequency of the grayscale level in the intermediate histogram of the new round is the sum of the increase value and the frequency of this round; The frequency of the grayscale level is the clipping threshold, and the frequency of the grayscale level in the new round of intermediate histogram is the clipping threshold; a new round of intermediate histogram is generated according to the frequency of each grayscale level in the new round of intermediate histogram; wherein, the intermediate histogram of the first round of iteration is a brightness histogram, and the stopping condition includes that the distribution value cannot be evenly distributed to each grayscale level in the adjustment histogram, the distribution value of the first round of iteration is the sum of the frequency differences of all adjustment levels, the frequency difference is the frequency difference between the frequency of each adjustment level of the intermediate histogram of the current round and the clipping threshold, and the distribution value of other rounds is the sum of the remaining total value and the frequency difference, the remaining total value is the sum of all remaining values, and the remaining value is the added value that is not added.

[0028] In the above embodiment, the histogram can be balanced quickly and simply, and the occurrence of color stratification or dark areas in the adjusted image can be reduced.

[0029] In a second aspect, the present application also provides a tone mapping device, characterized in that the device includes: a division module, used to divide the image into multiple sub-blocks, and obtain a brightness histogram of each sub-block, wherein the brightness histogram is the frequency of pixels in the sub-block at different grayscale levels; a correction module, used to obtain a clipping threshold corresponding to each sub-block, and correct the brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block to generate a corrected histogram of each sub-block, wherein the frequencies of different grayscale levels in the corrected histogram are evenly distributed; an establishment module, used to solve a brightness mapping equation based on the brightness histogram of the sub-block and the corrected histogram, and establish a brightness mapping relationship for each sub-block, wherein the brightness mapping equation includes a linear equation; an adjustment module, used to adjust the image according to the brightness mapping relationship of each sub-block to generate an adjusted image.

[0030] In a third aspect, an embodiment of the present application provides an electronic device, characterized in that it includes: a processor and a memory; the processor and the memory communicate with each other through a bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method as described in any one of the first aspects.

[0031] In a fourth aspect, an embodiment of the present application provides an electronic device comprising: a memory and a processor, the processor being connected to the memory; the memory being used to store programs; and the processor being used to call the programs stored in the memory to execute a method as described in any one of the first aspects.

[0032] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is run by a processor, the method as described in any one of the first aspects is executed.

[0033] Other features and advantages of the present application will be described in the following description. The purpose and other advantages of the present application can be realized and obtained through the structures specifically pointed out in the written description and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a flowchart of a tone mapping method provided in an embodiment of the present application;

[0035] FIG2 is a schematic diagram of a sub-block division provided in an embodiment of the present application;

[0036] FIG3 is a schematic diagram of a brightness histogram provided in an embodiment of the present application;

[0037] FIG4 is a schematic diagram of a weight table provided in an embodiment of the present application;

[0038] FIG5 is a schematic diagram of a trilinear interpolation coordinate axis provided in an embodiment of the present application;

[0039] FIG6 is a schematic block diagram of a tone mapping device provided in an embodiment of the present application;

[0040] FIG7 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present application are described in detail with reference to the accompanying drawings. It should be noted that although the same elements are shown in different drawings, they will be represented by the same reference numerals. In the following description, specific details such as detailed configuration and components are provided to only help a comprehensive understanding of the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications to the embodiments described herein may be made without departing from the scope of the present application. In addition, for the sake of clarity and conciseness, descriptions of well-known functions and configurations have been omitted. The terms described below are defined in consideration of the functions in the present application and may vary according to the user, the user's intention or custom. Therefore, the definition of the terms should be determined based on the content throughout this specification.

[0042] The present application may have various modifications and various embodiments, and in the present application, the embodiments are described in detail below with reference to the accompanying drawings. However, it should be understood that the present application is not limited to the embodiments, but includes all modifications, equivalents and substitutes within the scope of the present application.

[0043] The terms used herein are only used to describe various embodiments of the present application and are not intended to limit the present application. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In the present application, it should be understood that the terms "including" or "having" indicate the presence of features, quantities, steps, operations, structural elements, parts or a combination thereof, and do not exclude the presence of one or more other features, quantities, steps, operations, structural elements, parts or a combination thereof, or the possibility of adding one or more other features, quantities, steps, operations, structural elements, parts or a combination thereof.

[0044] Unless defined differently, all terms used herein have the same meaning as understood by those skilled in the art to which this application belongs. Unless explicitly defined in this application, terms (such as those defined in general dictionaries) should be interpreted as having the same meaning as in the context of the relevant art and should not be interpreted as having an idealized or overly formal meaning.

[0045] The electronic device according to one embodiment may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to one embodiment of the disclosure, the electronic device is not limited to the above examples.

[0046] The term used in this application is not intended to limit the application, but is intended to include the various changes, equivalents or substitutes of corresponding embodiments. About the description of the accompanying drawings, similar reference numerals can be used to represent similar elements or related elements. Unless the relevant context clearly indicates otherwise, the singular form of the noun corresponding to the item can include one or more things. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" can include all possible combinations of the item enumerated together in the corresponding one in the phrase. As used herein, terms such as "the 1st", "the 2nd", "first" and "second" can be used to distinguish corresponding components from another component, and are not intended to limit components in other aspects (for example, importance or order). It is intended that if an element (e.g., a first element) is referred to as being “coupled with,” “coupled to,” “connected to,” or “connected to” another element (e.g., a second element), with or without the term “operably” or “communicatively,” it indicates that the element may be coupled with the other element directly (e.g., wired), wirelessly, or via a third element.

[0047] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "component," and "circuit." A module may be a single integrated component or its smallest unit or component adapted to perform one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0048] To facilitate understanding, the following first introduces the relevant terms and concepts involved in the embodiments of this application.

[0049] Tone mapping is a term originally used in photography. Because the brightness range that a printed photo can express is insufficient to represent the brightness range in the real world, simply linearly compressing the entire brightness range of the real world into the brightness range that a photograph can express would result in the loss of many details on both the light and dark ends, which is obviously undesirable. To overcome this problem, tone mapping was developed.

[0050] The concept of tone mapping has been further expanded in computer graphics to make images appear more realistic on monitors. The overall process can be described as follows: first, the average brightness of the scene is calculated based on the current scene, and then a suitable brightness domain is selected based on this average brightness. The image is then mapped to this brightness domain to obtain a result that matches the scene. For example, in some scenes, a brightly colored image will be generated. In other cases, in some scenes, an image with as much detail as possible or the maximum image contrast will be generated.

[0051] The tone mapping method in the existing technology currently has disadvantages such as high computational complexity and large additional resource overhead.

[0052] Based on the technical problems existing in the prior art, the present application proposes a tone mapping method, which obtains the clipping threshold corresponding to each sub-block and performs equalization correction on the brightness histogram of the sub-block, thereby making the adjustment of local contrast more flexible, and using a linear brightness mapping relationship to make the transition between sub-blocks smoother. At the same time, the tone mapping method of the present application is simple and easy to implement, has low computational complexity, and reduces resource overhead.

[0053] As shown in FIG1 , FIG1 is a flowchart of a tone mapping method shown in an embodiment of the present application. As shown in FIG1 , the method includes steps S110 to S140:

[0054] S110 , dividing the image into a plurality of sub-blocks, and obtaining a brightness histogram of each sub-block, wherein the brightness histogram is the frequency of pixels in the sub-block at different gray levels.

[0055] Acquire an image to be processed by tone mapping, and divide the image into multiple sub-blocks. For example, as shown in Figure 2, Figure 2 is a schematic diagram of a sub-block division shown in an embodiment of the present application. The image A shown in Figure 2 is an image to be processed by tone mapping. The image is divided into 4×5 sub-blocks, and the sub-blocks after division are shown as the image B in Figure 2. The sub-block division method in this application can be set according to requirements. The sub-block division method shown in Figure 2 is only an example and does not constitute a limitation to this application. For example, the A image can be divided into 3×3 sub-blocks.

[0056] For each sub-block, obtain the brightness histogram of the sub-block. Among them, the histogram is a type of chart widely used in statistics. It presents statistical data in a visual form and can show how many data points fall within a given numerical range. The given numerical range is called a class or bin (or bucket). The data frequency (frequency) corresponding to each class is indicated by a rectangle of a certain height. Luma represents the brightness of the pixel in the image. Luma refers to the brightness of the pixel in the image without considering the color information. In some scenes, since brightness refers to the difference in brightness and darkness of the displayed pixel, it is also called grayscale. The brightness value range is generally from 0 to 255, with white being 255 and black being 0.

[0057] In the brightness histogram of the embodiment of the present application, a category value interval is a brightness value value interval. For ease of description, a category in the brightness histogram is referred to as a grayscale level in this application. The frequency of each category is the number of pixels in the sub-block that are located within the category interval. For example, as shown in Figure 3, Figure 3 is a schematic diagram of a brightness histogram shown in an embodiment of the present application. The brightness histogram shown in Figure 3 is the brightness histogram of sub-block 1 in the B image shown in Figure 2. The horizontal axis in Figure 3 is the brightness value and the vertical axis is the frequency. In Figure 3, 8 grayscale levels are set, and the brightness values ​​in Figure 3 are divided into 8 intervals, namely 0-31, 32-63, 64-95, 96-127, 128-159, 160-191, 192-223, and 224-255. Then, the value interval of grayscale level 1 is 0-31, and the frequency of this grayscale level can be the number of pixels in sub-block 1 whose brightness values ​​are in the range of 0-31. In this application, the numerical range corresponding to the grayscale level can be freely set according to needs, for example, the numerical range can be set to 0-120, 121-200, 201-255, or each grayscale level can be set to correspond to a unique brightness value.

[0058] It should be noted that in this application, an image includes a luminance channel or similar features. For example, the image may be an image in a YUV encoding format, where the YUV image includes a luminance Y channel. Alternatively, when the image is in an RGB encoding format, the image may be converted to a YUV image, or a grayscale image of the RGB image may be obtained and used to establish the sub-blocks and histogram.

[0059] In one embodiment of the present application, the method further includes: performing content detection on the image to obtain a content weight of each pixel; and establishing a brightness histogram of the sub-block based on the brightness value and content weight of each pixel in the sub-block.

[0060] Different pixels in an image have different importance. If every pixel is counted as 1 during frequency statistics, the differences between different pixels are ignored. For example, in a sub-block containing a face, suppose there are two pixels, A and B. Pixel A belongs to the face, and pixel B belongs to the background. If the brightness value of pixel A is determined to be at the first grayscale level, the frequency of the first grayscale level is increased by 1. At the same time, if the brightness value of pixel B is determined to be at the first grayscale level, the frequency of the first grayscale level is also increased by 1. Obviously, both pixels are counted as 1, ignoring the importance of the face pixels, resulting in poor image performance after brightness mapping.

[0061] In this embodiment, different pixels are counted as different values ​​during frequency statistics to highlight the importance of different pixels.

[0062] Before histogram statistics, the image is first subjected to content detection. Content detection refers to the detection of features related to the image content. Image content detection can involve multiple features, and the selection of specific features depends on the application scenario. Common features for image content detection include: color features, texture features, shape features, edge features, structural features, context features, etc. These features can represent the content in the image. For example, color features are obtained by analyzing the color distribution in the image. The color features of different objects can be identified through color features, such as a red car, a blue sky, etc. For another example, texture features describe the texture patterns of different areas in the image. Different objects in the image can be identified through texture features, such as grass, brick walls, etc. A weight parameter for each pixel is obtained based on the detection results. This weight is referred to as the content weight in this application.

[0063] For example, an artificial intelligence model can be used to detect the degree of high-frequency detail expression of a sub-block and output a content weight of the pixel based on the degree of expression. In this embodiment, the higher the high-frequency detail, the greater the content weight should be assigned to the pixels in that area.

[0064] In an embodiment in which content detection is performed, when establishing a luminance histogram for each sub-block, the luminance value and content weight of each pixel can be combined for frequency statistics, assigning each pixel a different statistical value. For example, when establishing a luminance histogram, when a pixel is determined to belong to a grayscale level, the actual statistical value counted for the pixel is the content weight multiplied by the base number, where the base number can be the number of pixels. For example, assume a pixel has a luminance value of 30, a content weight of 2, and a pixel number of 1. If the luminance value of the pixel is determined to be within the numerical range of the first grayscale level in FIG3 (the numerical range of the first grayscale level is 0-31), then the pixel belongs to the first grayscale level. Assume that the frequency of the first grayscale level at this time is Cnt. The statistical value that should be counted for the pixel is 2×1=2. That is, the frequency of the first grayscale level of the pixel is Cnt+2.

[0065] In the above embodiment, content detection is performed, and the effect of the local high-frequency detail area on contrast adjustment is taken into consideration, so that the generated adjusted image has better contrast performance in the high-frequency detail area.

[0066] In one embodiment of the present application, content detection is performed on the image to obtain the content weight of each pixel, including: using a preset convolution kernel to convolve the pixel area corresponding to each pixel to obtain the convolution value of each pixel, and the convolution kernel is used for content detection; querying a preset weight table based on the convolution value corresponding to each pixel to obtain the content weight of each pixel.

[0067] In some embodiments of the present application, content detection and histogram creation may be:

[0068] A preset convolution kernel is used to convolve the pixel region corresponding to each pixel in the image. The pixel region is a pre-specified region representing a pixel. For example, a 3×3 region centered on a pixel is set as the pixel region of the pixel. For a pixel, the convolution value of the pixel is obtained by convolving the region with the preset convolution kernel. The convolution kernel is a pre-set convolution kernel for content detection, for example, a convolution kernel capable of detecting edges can be set.

[0069] After convolution of the pixel area corresponding to each pixel, the convolution value of each pixel can be obtained.

[0070] For each pixel, the preset weight table is queried based on the convolution value of the pixel to obtain the content weight of the pixel. The weight table is a pre-established correspondence. For example, the relationship between the convolution value and the content weight is a one-dimensional table. The corresponding curve of the convolution value and the content weight in the one-dimensional table is shown in Figure 4. Figure 4 is a schematic diagram of a weight table shown in an embodiment of the present application. The curve shown in Figure 4 establishes the relationship between high-frequency details and content weights, and is applied in subsequent statistical histograms. Through the content weight, when the histogram is counted, it is necessary not only to look at the grayscale brightness of the current point, but also to pay attention to the weight of its details.

[0071] In Figure 4, the horizontal axis Response represents the convolution value of the current point after convolution, and the vertical axis Weight is the content weight. Assuming that a pixel point has a Re (convolution value) after convolution, the curve shown in Figure 4 can be interpreted as:

[0072] if (Re<=Th1), weight=0; if the convolution value of the pixel is less than the preset first threshold Th1, the content weight of the pixel is 0.

[0073] if(Re>=Th2), weight=wgt_max; wherein, wgt_max is a customizable parameter, generally set in (5-10); if the convolution value of the pixel point is greater than or equal to the preset second threshold Th2, the content weight of the pixel point is wgt_max.

[0074] if(Th1 <Re<Th2), If the convolution value of the pixel is greater than the preset first threshold Th1 and less than the preset second threshold Th2, the content weight of the pixel is calculated by the formula: Calculated.

[0075] The content weight of each pixel can be obtained by looking up the table.

[0076] In one embodiment of the present application, the brightness histogram of the sub-block is established based on the brightness value and content weight of each pixel in the sub-block, including: judging the grayscale level of each pixel based on the brightness value of the pixel, and each grayscale level corresponds to at least one brightness value; establishing the brightness histogram based on all grayscale levels in the sub-block and the pixels corresponding to each grayscale level, and the frequency of each grayscale level in the brightness histogram is the sum of the number of pixels corresponding to the grayscale level and the content weights of all pixels corresponding to the grayscale level.

[0077] After obtaining the content weight, a brightness histogram is constructed based on the content weight and brightness value of each pixel. The grayscale frequency statistics for each pixel are as follows: For each pixel, first, the numerical interval corresponding to the brightness value of the pixel is determined. Based on the numerical interval corresponding to the brightness value, the corresponding grayscale level of the pixel can be determined. The existing frequency of the grayscale level is then added by 1 and the content weight of the pixel. For example, the current sub-block is sub-block 1 of the image in Figure B shown in Figure 2 , which has 8 grayscale levels. A pixel has a brightness value of 56 and a content weight of 7. For this pixel, first, it is determined that the pixel's brightness value is within the numerical interval of the second grayscale level (the numerical interval of the second grayscale level is 32-63). Assuming that the frequency counted for the second grayscale level is Cnt, the value to be counted for the second grayscale level is the number of pixels, 1, plus the content weight of 7. After counting the frequency of the pixel, the frequency of the second grayscale level is Cnt+1+7. All pixels within the sub-block are counted to create a luminance histogram for the sub-block. Using this statistical method, the frequency of each grayscale level in the graph is the sum of the number of pixels within the sub-block corresponding to the grayscale level's numerical range and the content weights of these pixels. It should be understood that the method of generating frequencies using content weights in the above embodiments of this application is merely illustrative, and this application does not limit the specific method of generating frequencies using content weights.

[0078] In the above embodiment, when performing histogram statistics, the content weight of the current pixel should be added to the count of each pixel. By introducing the content weight generated by content detection, the contrast performance of the generated adjusted image in the high-frequency detail area will be better.

[0079] S120 , obtaining a clipping threshold corresponding to each sub-block, and correcting the brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block to generate a corrected histogram of each sub-block, wherein the frequencies of different gray levels in the corrected histogram are evenly distributed.

[0080] Obtain the clipping threshold corresponding to each sub-block. Each sub-block has different brightness levels, so a different clipping threshold is set for each sub-block, allowing different sub-blocks to use different contrast adjustment methods, thereby making the generated adjusted image have higher contrast performance. For example, the grayscale ratio of each sub-block can be directly counted. The grayscale ratio represents the brightness level of the sub-block, and the clipping threshold of the sub-block is set based on the ratio.

[0081] The luminance histogram of each sub-block is modified based on the clipping threshold to generate a modified histogram for each sub-block. The resulting modified histogram has a balanced distribution of the frequencies of different grayscale levels. For example, based on the histogram shown in FIG3 , a luminance distribution curve 310 of the pixels in sub-block 1 can be calculated. This curve reveals that there are many high-luminance pixels and fewer low-luminance pixels in this sub-block, resulting in an uneven distribution and poor display quality. Therefore, the histogram needs to be modified to achieve a balanced distribution of pixels of different luminance levels. For example, the clipping threshold for sub-block 1 is shown as 320 in FIG3 . The peak portion of the distribution curve above the clipping threshold can be clipped. After clipping, the frequencies of the various grayscale levels on the distribution curve are relatively close. Alternatively, the peak portion of the distribution curve above the clipping threshold can be obtained and evenly distributed to the other grayscale levels. After this distribution, the frequencies of the various grayscale levels are relatively close.

[0082] In one embodiment of the present application, obtaining the clipping threshold corresponding to each sub-block includes: obtaining the brightness relationship of each sub-block based on the comparison result of the average brightness of each sub-block and the average brightness of the image; and adjusting the preset clipping value according to the brightness relationship of each sub-block to obtain the clipping threshold corresponding to each sub-block.

[0083] The brightness of the sub-block is determined by comparing it with the average brightness of the entire image, and the average brightness of each sub-block and the image is obtained. The average brightness can be calculated as follows:

[0084] Among them, statLumaSum is the sum of the brightness values ​​of the pixels included in the statistics, statPixelTotal is the total number of pixels included in the statistics, and gridMean is the average brightness.

[0085] For each sub-block, the average brightness of the sub-block is compared with the average brightness of the image to obtain the brightness relationship of the sub-block, which indicates the brightness level of the sub-block. For example, if the average brightness of sub-block 1 in Figure 2 is 100 and the average brightness of image A in Figure 2 is 200, based on the comparison result of 100 < 200, the brightness relationship of sub-block 1 is determined to be dark.

[0086] Adjust the preset clipping value based on the brightness relationship obtained to obtain the clipping threshold corresponding to each sub-block. For example, the clipping value is set to 120, and the brightness relationship of sub-block 1 indicates that the average brightness of the sub-block is less than the average brightness of the image. The current sub-block is dark, and the preset clipping value can be increased, such as increased to 130. 130 is the clipping threshold of sub-block 1, so that the clipping threshold of the darker sub-block is higher than the clipping threshold of the brighter sub-block. Correspondingly, when the average brightness of a sub-block is greater than the average brightness of the entire image, it is considered that the current sub-block is a little brighter, then the clipping threshold of the threshold should be lowered so that the clipping threshold of the sub-block is one level lower than the preset clipping threshold. The size of the increase or decrease in the clipping threshold can be set according to needs.

[0087] In the above embodiment, different clipping thresholds are set for sub-blocks of different brightness levels. Different clipping thresholds are set for brighter or darker images, which allows for more flexible adjustment of local contrast.

[0088] In one embodiment of the present application, the brightness histogram of each sub-block is corrected according to the clipping threshold corresponding to the sub-block to generate a corrected histogram of each sub-block, including: performing multiple rounds of iteration until a stop condition is met, the iteration including: finding an adjustment level in the histogram of this round, the adjustment level being a grayscale level whose frequency is greater than the clipping threshold; obtaining the allocation value of this round according to the adjustment level, and setting the frequency of each adjustment level as the clipping threshold; calculating the increase value of evenly distributing the allocation value of this round to each grayscale level in the intermediate histogram of this round; if the frequency of the grayscale level is less than the clipping threshold, the frequency of the grayscale level in the intermediate histogram of the new round is the sum of the increase value and the frequency of this round; if the grayscale level is less than the clipping threshold, the frequency of the grayscale level in the intermediate histogram of the new round is the sum of the increase value and the frequency of this round; The frequency of the level is the clipping threshold, and the frequency of the grayscale level in the new round of intermediate histogram is the clipping threshold; a new round of intermediate histogram is generated according to the frequency of each grayscale level in the new round of intermediate histogram; wherein, the intermediate histogram of the first round of iteration is a brightness histogram, and the stopping condition includes that the distribution value cannot be evenly distributed to each grayscale level in the adjustment histogram, the distribution value of the first round of iteration is the sum of the frequency differences of all adjustment levels, the frequency difference is the frequency difference between the frequency of each adjustment level of the intermediate histogram of the current round and the clipping threshold, and the distribution value of other rounds is the sum of the remaining total value and the frequency difference, the remaining total value is the sum of all remaining values, and the remaining value is the added value that is not added.

[0089] Perform multiple rounds of iterations until the preset iteration stop condition is met, then stop the iteration;

[0090] Each round of iteration includes:

[0091] First, sum the number of points exceeding the clipping threshold in this round:

[0092] Among them, Sumex is the number of pixels of the clipping threshold, clipTh is the clipping threshold, P represents the grayscale, and hist[p] represents the number of pixels within a grayscale.

[0093] Specifically, the adjustment level of the intermediate histogram of this round is obtained. The adjustment level is the grayscale level whose frequency is greater than the clipping threshold. For example, in the brightness histogram shown in Figure 3, for the clipping threshold 320, the adjustment levels are 4, 5, and 6. The frequencies of these three grayscale levels are all greater than the clipping threshold 320.

[0094] The distribution value for this round is obtained based on the adjustment level being searched. In the first iteration, the distribution value is the portion of the distribution curve of the intermediate histogram of this round that is greater than the clipping threshold. The intermediate histogram of the first round is the statistically obtained brightness histogram. For example, in the brightness histogram shown in Figure 3, for the first iteration, the differences between adjustment levels 4, 5, and 6 and the clipping threshold of 320 are calculated. For example, the differences between adjustment levels 4, 5, and 6 and the clipping threshold are calculated, resulting in differences of 50, 60, and 50. These differences are the frequency differences for each adjustment level. The sum of the frequency differences for each adjustment level is calculated, i.e., 50 + 60 + 50 = 160. After obtaining the distribution value, the frequency of each adjustment level is set to the clipping threshold. That is, the frequencies of the three grayscale levels of adjustment levels 4, 5, and 6 are all set to the clipping threshold.

[0095] Subsequently, the excess part is evenly distributed to each gray level. After the first distribution, hist[p] can be expressed as:

[0096] Among them, aveVal is the number of gray levels evenly distributed to each level. The remaining number of unallocated items is recorded as Res, and this allocation step is repeated to allow Res to continue to be allocated downwards until all items are allocated.

[0097] Specifically, the number of gray levels to which the assigned value is assigned (i.e., the gray level increase) is calculated, which is 160 / 8 = 20. For each gray level, first determine whether the frequency of the gray level is greater than the clipping threshold. If it is less than, obtain the current frequency of the gray level, calculate the sum of the frequency and the increase, and use the sum of the two as the frequency in the intermediate histogram of the next round of the gray level. For example, the frequency of gray level 3, 80, is less than the clipping threshold of 110, then 100 + 20 = 130, and 130 is the frequency of the gray level in the intermediate histogram of the next round.

[0098] If it is equal to the clipping threshold, the frequency is kept, that is, the clipping threshold is used as the frequency in the middle histogram of the next round of grayscale. For example, the frequency of grayscale 4 has been set to the clipping threshold of 110, then the frequency of the grayscale round remains unchanged.

[0099] A new round of intermediate histograms is generated based on the determined frequencies of each grayscale level in the new round of intermediate histograms. For example, the frequency of grayscale level 3 is set to 130, and the frequency of grayscale level 4 remains unchanged. A new round of intermediate histograms is generated based on the brightness histogram.

[0100] In subsequent iterations, the steps are the same as the first round, the only difference is the acquisition of the allocation value. Except for the first round, the allocation value of other rounds is the sum of the remaining total value and the frequency difference.

[0101] For example, for the second round of iteration: obtain the intermediate histogram of this round of iteration, find the adjustment level in the histogram of this round. For example, if the frequency of grayscale level 3 is 130, which is greater than the clipping threshold of 110, it is the adjustment level. According to the adjustment level, obtain the allocation value of this round, calculate the difference between the frequency of grayscale level 3 and the clipping threshold, and obtain the frequency difference of 20. Since grayscale levels 4, 5, and 6 did not receive additional values ​​in the first round of allocation, there are three additional values ​​that were not added. For the sake of distinction, these unadded values ​​are called residual values. Adding these three residual values ​​results in a total residual value of 60. That is, the allocation value in the second round of iteration is 60+20=80. Set the frequency of grayscale level 3 to 110.

[0102] Calculate the added value of assigning the assigned value to each grayscale level, i.e. 80 / 8=10. For each grayscale level, first determine whether the frequency of the grayscale level is greater than the clipping threshold. If it is less than, obtain the current frequency of the grayscale level, calculate the sum of the frequency and the added value, and use the sum of the two as the frequency in the intermediate histogram of the next round of the grayscale level. If it is equal to the clipping threshold, maintain the frequency, i.e. use the clipping threshold as the frequency in the intermediate histogram of the next round of the grayscale level. Based on the determined frequency of each grayscale level in the new round of intermediate histogram, generate a new round of intermediate histogram based on the intermediate histogram of the current round.

[0103] The iteration is repeated until a stopping condition is met. The stopping condition includes that the assigned value cannot be evenly distributed to each grayscale level in the adjusted histogram, for example, the frequency of all grayscale levels is equal to the clipping threshold, or that no grayscale level can still be less than the clipping threshold after receiving a new round of increments, or that the assigned value cannot be evenly distributed, for example, when the assigned value is 4, 4 / 8 cannot be divided evenly, and thus cannot be evenly distributed.

[0104] When the iteration stops, the intermediate histogram of that iteration is the corrected histogram.

[0105] In the above embodiment, the histogram can be balanced quickly and simply, and the occurrence of color stratification or dark areas in the adjusted image can be reduced.

[0106] S130: Solve the brightness mapping equation according to the brightness histogram of the sub-block and the modified histogram to establish the brightness mapping relationship of each sub-block.

[0107] For each sub-block, a brightness mapping relationship of the sub-block is established based on the change relationship between the brightness histogram and the corrected histogram of the sub-block. For example, the brightness histogram is relative to the corrected histogram, and the brightness distribution of the pixels in the sub-block is changed. Therefore, the distribution change relationship between the two can be used to solve the brightness mapping equation, thereby finding the change relationship of the sub-block brightness. This brightness change relationship is the brightness mapping relationship. In one embodiment of the present application, the brightness mapping equation solved is a linear equation. In another embodiment of the present application, the brightness mapping equation is nonlinear, such as a logarithmic function, etc. The present application does not limit the form of the brightness mapping equation.

[0108] In one embodiment of the present application, the brightness mapping equation is: Y = gain_a*X + gain_b; wherein Y is the brightness value of the pixel point in the corrected histogram, X is the brightness value of the pixel point in the brightness histogram, gain_a is the slope parameter of the brightness mapping relationship, and gain_b is the intercept parameter of the brightness mapping relationship.

[0109] After performing histogram equalization and obtaining a corrected histogram, the linear brightness mapping equation Y = gain_a * X + gain_b is solved based on the grayscale levels before and after correction in the two images to determine the specific gain_a and gain_b. Based on the solved gain_a and gain_b, the brightness mapping relationship of the sub-blocks can be established.

[0110] In one embodiment of the present application, solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish the brightness mapping relationship of each sub-block includes: solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; obtaining a second brightness mapping relationship for each sub-block in the previous frame image of the image; and fusing the first brightness mapping relationship and the second brightness mapping relationship of each sub-block to establish a brightness mapping relationship for each sub-block.

[0111] For each sub-block of the image, a brightness mapping equation is solved according to the transformation of the brightness histogram of the sub-block and the modified histogram to establish the brightness mapping relationship of the sub-block.

[0112] After establishing the brightness mapping relationship of each sub-block in the image, the brightness mapping relationship of each sub-block in the previous frame image in the image is obtained, and the sub-blocks in the previous frame image correspond one-to-one to the sub-blocks in the current image.

[0113] For the sake of distinction, for each sub-block, its brightness mapping relationship in the current image is called the first brightness mapping relationship, and its brightness mapping relationship in the previous frame image is called the second brightness mapping relationship;

[0114] For each sub-block, the first brightness mapping relationship and the second brightness mapping relationship are fused. For example, the first brightness mapping relationship and the second brightness mapping relationship are fused in a weighted manner.

[0115] For example: gain_a=(w1*gain_a_frm1)+((1-w1)*gain_a_frm2).

[0116] Where gain_a_frm1 is the gain_a of the previous frame image; gain_a_frm2 is the gain_a of the current frame image, and w1 is the preset weight parameter.

[0117] The same method can be used to integrate gain_b, gain_b = (w2*gain_b_frm1) + ((1-w2)*gain_b_frm2).

[0118] Where gain_b_frm1 is the gain_b of the previous frame image; gain_b_frm2 is the gain_b of the current frame image, and w2 is the preset weight parameter.

[0119] In the above embodiment, the parameters of the linear equation of the brightness mapping relationship are obtained by fusing the parameters of the current frame and the parameters of the previous frame, which can avoid a large brightness difference between the previous frame image and the current frame image, making the transition between the two frames smoother.

[0120] In one embodiment of the present application, solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish the brightness mapping relationship of each sub-block includes: solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; obtaining a second brightness mapping relationship for each sub-block in the previous frame image of the image; obtaining a deviation relationship between the first brightness mapping relationship and the second brightness mapping relationship of each sub-block; if there is a deviation relationship of a sub-block greater than a deviation threshold, using the second brightness mapping relationship of the sub-block as the brightness mapping relationship of the sub-block.

[0121] For each sub-block of the image, a brightness mapping relationship of the sub-block is established based on the transformation of the brightness histogram of the sub-block and the modified histogram. For the sake of distinction, this application refers to this mapping relationship as the first brightness mapping relationship.

[0122] After obtaining the first brightness mapping relationship, the brightness mapping relationship of each sub-block in the frame image before the current frame image is obtained, and the sub-blocks in the previous frame image correspond one-to-one with the sub-blocks in the current frame image. For ease of distinction, in the embodiment of the present application, the brightness mapping relationship of the sub-blocks in the previous frame image is referred to as the second brightness mapping relationship.

[0123] For each sub-block, obtain the deviation relationship between the first luminance mapping relationship and the second luminance mapping relationship of the sub-block. For example, for each sub-block, calculate the difference between gain_a and gain_b in the first luminance mapping relationship and the second luminance mapping relationship. Assume that gain_a_frm1 in the first luminance mapping relationship is 5, gain_a_frm2 in the second luminance mapping relationship is 8, and the difference between the two is 3. Suppose that gain_b_frm1 in the first luminance mapping relationship is 2, gain_b_frm2 in the second luminance mapping relationship is 6, and the difference between the two is 4. The sum of the two differences is 7, which is the deviation relationship between the first luminance mapping relationship and the second luminance mapping relationship.

[0124] If this value is greater than the preset threshold of 5, it indicates that the deviation relationship of the sub-block is greater than the deviation threshold, and the second brightness mapping relationship of the sub-block is used as the brightness mapping relationship of the sub-block. That is, gain_a is set to 8 and gain_b is set to 6 to establish the brightness mapping relationship.

[0125] The above deviation relationship is only an example, and it can also be in other forms, such as the deviation relationship is the difference between gain_a and gain_b. This application does not limit the form of the deviation relationship.

[0126] In the above embodiment, the two brightness mapping relationships of the sub-blocks in the previous and next frame images are compared. When the two brightness mapping relationships deviate greatly, the brightness mapping relationship of the previous frame image is taken as the brightness mapping relationship of the current frame image to avoid a large brightness difference between the previous and next frame images, making the transition between the previous and next frame images smoother.

[0127] S140 , adjusting the image according to the brightness mapping relationship of each sub-block to generate an adjusted image.

[0128] After obtaining the brightness mapping relationship for each sub-block, the brightness values ​​of the pixels within each sub-block are adjusted based on the brightness mapping relationship for that sub-block. For example, for sub-block 1, the brightness value calculated for each pixel within that sub-block using the brightness mapping relationship is calculated, and the brightness value of each pixel is set to the calculated brightness value.

[0129] After all sub-blocks are modified, the adjusted image can be obtained by splicing all sub-blocks.

[0130] The method adjusts the image according to the brightness mapping relationship of each sub-block to generate an adjusted image, including: obtaining a mapping sub-block and an interpolation sub-block in the image, adjusting the brightness value of each pixel in the mapping sub-block using the brightness mapping relationship of the sub-block, and adjusting the brightness value of each pixel in the interpolation sub-block using the interpolation brightness mapping relationship; obtaining multiple reference sub-blocks of the interpolation sub-block, wherein the reference sub-block is an adjacent sub-block of the interpolation sub-block; using gain_a in the brightness mapping relationship of the interpolation sub-block and the multiple reference sub-blocks corresponding to the interpolation sub-block as the third coordinate axis, and length and width as the first coordinate axis and second coordinate axis respectively to establish a first coordinate system based on the brightness mapping relationship of the interpolation sub-block and the interpolation sub-block. Performing trilinear interpolation on a first coordinate system to generate first interpolation parameters; establishing a second coordinate system using gain_b in a brightness mapping relationship between an interpolation sub-block and multiple reference sub-blocks corresponding to the interpolation sub-block as a third coordinate axis, and length and width as first and second coordinate axes, respectively; performing trilinear interpolation based on the second coordinate system to generate second interpolation parameters; establishing an interpolation brightness mapping relationship for the interpolation sub-block based on the first interpolation parameters and the second interpolation parameters; adjusting the brightness value of the mapping sub-block in the image based on the brightness mapping relationship, and adjusting the brightness value of the pixel point of the interpolation sub-block in the image based on the interpolation brightness mapping relationship to generate the adjusted image.

[0131] If the brightness mapping relationship of each sub-block is directly used to adjust the brightness of each sub-block, there may be a large difference in brightness between different sub-blocks, resulting in an unsmooth transition between sub-blocks. Therefore, an interpolation method can be used to calculate the brightness mapping relationship of the sub-blocks.

[0132] Obtain the mapped sub-blocks and interpolated sub-blocks in the image. The mapped sub-blocks and interpolated sub-blocks are pre-set. Each pixel in the mapped sub-block can directly use the brightness mapping relationship of the sub-block for brightness adjustment. Each pixel in the interpolated sub-block can also use the interpolated brightness mapping relationship for brightness adjustment. For example, for the B image shown in Figure 2, sub-blocks 1, 4, 17, and 20 can directly use the brightness mapping relationship of the sub-block for brightness adjustment. The other sub-blocks are interpolated sub-blocks, and each pixel in the interpolated sub-block has its brightness adjusted using the interpolated brightness mapping relationship.

[0133] For each interpolation sub-block, multiple reference sub-blocks of the interpolation sub-block are obtained. The reference sub-blocks are adjacent sub-blocks of the interpolation sub-block. The reference sub-blocks shown are found based on a preset adjacent relationship. For example, for sub-block 6, sub-blocks 7, 10, and 11 can be set as reference sub-blocks, or sub-blocks 5, 9, and 10 can be set as reference sub-blocks. This application does not limit the setting method of the reference sub-blocks.

[0134] After the interpolation sub-block is obtained, parameters such as gain_a and gain_b in the brightness mapping relationship between the interpolation sub-block and the multiple reference sub-blocks corresponding to the interpolation sub-block are obtained.

[0135] As shown in Figure 5, Figure 5 is a schematic diagram of the coordinate axes of trilinear interpolation shown in an embodiment of the present application. In Figure 5, gain_a is used as the third coordinate axis, such as the z-axis, length is used as the first coordinate axis, such as the X-axis, and width is used as the second coordinate axis, such as the Y-axis, to establish a first coordinate system. Trilinear interpolation is performed in this coordinate system to generate a first interpolation parameter, gain_a_inter. Gain_b is used as the third coordinate axis, such as the z-axis, length is used as the first coordinate axis, such as the X-axis, and width is used as the second coordinate axis, such as the Y-axis, to establish a second coordinate system. Trilinear interpolation is performed in this coordinate system to generate a second interpolation parameter, gain_b_inter. In other embodiments of the present application, gain_a can also be used as the X-axis of the coordinate system, length as the Y-axis, and width as the Z-axis. This application does not limit the parameters used for the coordinate axes.

[0136] The first interpolation parameter and the second interpolation parameter are reintroduced into the linear equation to obtain the interpolation brightness mapping relationship of the sub-block.

[0137] For each mapped sub-block, a new brightness value is calculated for each pixel in the sub-block according to the brightness mapping relationship of the sub-block, and the brightness value of the pixel is adjusted to the new value.

[0138] For each interpolation sub-block, a new brightness value is calculated for each pixel in the sub-block according to the interpolation brightness mapping relationship of the sub-block, and the brightness value of the pixel is adjusted to the new value.

[0139] After adjusting the brightness values ​​of the pixels in each sub-block, an adjusted image is generated.

[0140] In the above embodiment, the mapping relationship of the interpolation sub-blocks is generated by using the trilinear interpolation method, which can make the transition between the image sub-blocks smoother.

[0141] The above-mentioned tone mapping method obtains the clipping threshold corresponding to each sub-block and performs equalization correction on the brightness histogram of the sub-block, thereby making the adjustment of local contrast more flexible, and using a linear brightness mapping relationship can make the transition between sub-blocks smoother. At the same time, the tone mapping method of the present application is simple and easy to implement, with low computational complexity, and reduces resource overhead.

[0142] In one embodiment of the present application, solving the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish the brightness mapping relationship of each sub-block includes: calculating the change in the probability density function of the corrected histogram image of each sub-block relative to the brightness histogram to obtain a first mapping relationship; converting the brightness value corresponding to each grayscale level of the brightness histogram in the sub-block based on the first mapping relationship to obtain multiple conversion values; solving the brightness mapping equation based on the brightness value corresponding to each grayscale level in the sub-block and the multiple conversion values ​​to obtain a first brightness mapping relationship corresponding to each grayscale level; based on the first brightness mapping relationship corresponding to each grayscale level, generating a brightness mapping relationship corresponding to each grayscale level, the brightness mapping relationship of a sub-block is the brightness mapping relationship of all grayscale levels in the sub-block.

[0143] Assume that the pixel density function of the luminance histogram before equalization is p(x), and the width of each grayscale level is dx. Then the number of pixels in each grayscale level is p(x)dx. Assume that the pixel density function after equalization is p(y), and p(y) is a constant. The histogram equalization method is dy = p(x)dx. The p(x)dx pixels included in the luminance histogram width dx are mapped to the width dy of the corrected histogram. The number of pixels before and after the mapping remains unchanged, but the density changes.

[0144] Therefore, the relationship between the changes in brightness can be obtained by calculating the transformation of the probability density function. For example, the change relationship can be calculated by direct normalization (uniform distribution). For example, a nonlinear mapping method can be selected to establish a brightness change relationship so that the pixel value and the corresponding CDF (cumulative distribution function) do not completely conform to the uniform distribution, such as exponential distribution, Rayleigh distribution, etc. Alternatively, the brightness change relationship can be manually set. In this case, the brightness change relationship is a nonlinear mapping curve to achieve the purpose of amplifying and / or compressing part of the grayscale interval. The brightness change relationship obtained at this time is called the first mapping relationship.

[0145] After obtaining the first mapping relationship, the numerical interval corresponding to each grayscale level is converted according to the first mapping relationship, thereby obtaining a plurality of conversion values. In this embodiment, the numerical interval corresponding to the grayscale level should include at least two brightness values. According to the first mapping relationship, the numerical values ​​in the numerical interval corresponding to the grayscale level are partially or completely converted to obtain a plurality of conversion values. For example, for grayscale level 1 in FIG3 , the corresponding numerical interval is 0-31. 0-31 can be brought into the first mapping relationship to obtain a plurality of conversion values. Alternatively, multiple values ​​of the left boundary, such as 0-5, and multiple values ​​of the right boundary, such as 27-31, can be taken from the first mapping relationship to obtain a plurality of conversion values.

[0146] After all grayscale levels are converted, the original brightness value of each grayscale level and the conversion value corresponding to each brightness value can be obtained. For these values ​​of each grayscale relationship, the corresponding gain_a and gain_b are obtained by substituting them into a linear equation such as Y=gain_a*X+gain_b, thereby obtaining the brightness mapping relationship corresponding to each grayscale level. For example, the original brightness values ​​of the pixels at grayscale level 1 in Figure 3 are 10, 13, ..., 30, and after being substituted into the first mapping relationship, the corresponding conversion values ​​of the pixels are 23, 29..., 63. The linear equation is solved by these values ​​to obtain the linear equation of grayscale level 1 in Figure 3. For the sake of distinction, this linear equation is called the first brightness mapping relationship.

[0147] After obtaining the first mapping relationship for each grayscale level, an adjustment is performed based on the first mapping relationship corresponding to each grayscale level to generate a brightness mapping relationship corresponding to each grayscale level. The brightness mapping relationship of a sub-block includes the brightness mapping relationships of all grayscale levels within the sub-block. For example, for the histogram shown in Figure 3, its brightness mapping relationship includes brightness mapping relationships for 8 grayscale levels, that is, the sub-block includes 8 linear equations.

[0148] Furthermore, the brightness mapping relationship corresponding to each grayscale level is generated based on the first brightness mapping relationship corresponding to each grayscale level, including: obtaining the second brightness mapping relationship corresponding to each grayscale level of the corresponding sub-block in the previous frame image of the image; fusing the first brightness mapping relationship corresponding to each grayscale level with the second brightness mapping relationship to generate a brightness mapping relationship corresponding to each grayscale level.

[0149] After obtaining the first brightness mapping relationship of each grayscale level, the brightness mapping relationship of the previous frame image is obtained, and the previous frame image corresponds to the sub-blocks in the current frame image in a one-to-one manner.

[0150] For each grayscale level, obtain its corresponding brightness mapping relationship in the previous frame image, that is, obtain its brightness mapping relationship of the same grayscale level in the same sub-block. For example, for grayscale level 1 in the histogram shown in Figure 3, the brightness mapping relationship of grayscale level 1 in sub-block 1 in the previous frame image is obtained. For ease of distinction, this brightness mapping relationship is referred to as the second brightness mapping relationship in this embodiment.

[0151] The first brightness mapping relationship corresponding to each grayscale level is merged with the second brightness mapping relationship to generate a brightness mapping relationship corresponding to each grayscale level.

[0152] In the above embodiment, the brightness value of the grayscale level and the conversion value are fitted to establish a brightness mapping relationship for each grayscale level. Each grayscale level has its own brightness mapping relationship, which makes the adjustment of the local contrast more flexible.

[0153] Furthermore, when the brightness mapping equation is a linear equation, it includes a slope and an intercept. The image is adjusted according to the brightness mapping relationship of each sub-block to generate an adjusted image, including: obtaining multiple reference sub-blocks of the sub-block, the reference sub-blocks being adjacent sub-blocks of the sub-block, obtaining the brightness mapping relationship corresponding to each grayscale level in the sub-block; for any grayscale level in the sub-block, taking the slope of the brightness mapping relationship in the grayscale level of the sub-block and the multiple reference sub-blocks corresponding to the sub-block as the third coordinate axis, and the length and width as the first coordinate axis and the second coordinate axis respectively to establish a first coordinate system, and performing three-dimensional mapping based on the first coordinate system. Linear interpolation is performed to generate first interpolation parameters; for any grayscale level in the sub-block, an intercept in a brightness mapping relationship of the sub-block and multiple reference sub-blocks corresponding to the sub-block at that grayscale level is used as a third coordinate axis, and a first coordinate system is established with length and width as first and second coordinate axes respectively; trilinear interpolation is performed based on the first coordinate system to generate second interpolation parameters; an interpolation brightness mapping relationship of the grayscale levels is established based on the first interpolation parameters and the second interpolation parameters; and the sub-block and multiple reference sub-blocks corresponding to the sub-block are adjusted based on the interpolation brightness mapping relationship of each grayscale level in the sub-block to generate the adjusted image.

[0154] The linear equation of a grayscale brightness mapping relationship includes a slope and an intercept. For example, the linear equation is in the form of: Y = gain_a*X+gain_b, where gain_a is the slope and gain_b is the intercept.

[0155] For a subblock, multiple reference subblocks are obtained for the subblock. Reference subblocks are neighboring subblocks of the subblock found based on a preset adjacency relationship. In this embodiment, the subblock selection method is limited. For example, in the B-image shown in Figure 2, the number of reference subblocks should be three. The reference subblocks for subblock 10 can be 11, 14, and 15, or subblocks 9, 13, and 14.

[0156] For any grayscale level in the sub-block, the first parameter of the slope in the brightness mapping relationship of the grayscale level of the sub-block and the multiple reference sub-blocks corresponding to the sub-block is used as the third coordinate axis, and the length and width are used as the first coordinate axis and the second coordinate axis respectively, and trilinear interpolation is performed to generate the first interpolation parameter. The intercept is used as the third coordinate axis, and the length and width are used as the first coordinate axis and the second coordinate axis respectively, and trilinear interpolation is performed to generate the second interpolation parameter.

[0157] For a grayscale level, obtain its brightness mapping relationship in each reference sub-block. For example, when the reference sub-blocks of sub-block 10 are 11, 14, and 15, for grayscale level 1 of this sub-block, obtain the brightness mapping relationship of grayscale level 1 in reference sub-blocks 11, 14, and 15. The grayscale value range of each sub-block is the same.

[0158] In this embodiment, trilinear interpolation is used to obtain the brightness mapping relationship of the grayscale levels of the sub-blocks and perform trilinear interpolation. Trilinear interpolation requires eight points in space for interpolation. In this embodiment, four sub-blocks are selected, each representing a point, so the four sub-blocks have four points. Assuming that the reference sub-blocks of B image sub-block 10 are 11, 14, and 15, and the length and width are used as the X-axis and Y-axis respectively, the coordinates of the four sub-blocks 10, 11, 14, and 15 are (1, 2), (3, 2), (1, 4), and (3, 4), respectively.

[0159] The slope gain_a in the grayscale level is used as the third coordinate axis, such as the Z axis, to establish a first coordinate system. Assuming that the gain_a of sub-blocks 10, 11, 14, and 15 are 5, 6, 7, and 8 respectively, the coordinate points of the four sub-blocks 10, 11, 14, and 15 in the coordinate system are (1, 2, 5), (1, 2, 0), (3, 2, 6), (3, 2, 0), (1, 4, 7), (1, 4, 0), (3, 4, 8), and (3, 4, 0). The coordinates of the third coordinate axis in the newly added points can be other values ​​besides 0, and this application does not limit this.

[0160] After obtaining the eight coordinate points and their coordinates, trilinear interpolation can be performed to generate a new point. The coordinate of the third axis of this new point is the gain_a of the interpolated brightness relationship of the four sub-blocks. In this embodiment, the slope obtained by interpolation is called the first interpolation parameter.

[0161] Similarly, the slope gain_b in the grayscale level is used as the third coordinate axis, such as the Z axis, to establish a second coordinate system. Assuming that the gain_a of sub-blocks 10, 11, 14, and 15 are 3, 4, 5, and 6 respectively, we can get (1, 2, 3), (1, 2, 0), (3, 2, 4), (3, 2, 0), (1, 4, 5), (1, 4, 0), (3, 4, 6), and (3, 4, 0). Similarly, the coordinates of the third coordinate axis in the newly added points can be other values ​​besides 0, and this application does not limit this.

[0162] After obtaining the eight coordinate points and their coordinates, trilinear interpolation can be performed to generate a new point. The coordinate of the third axis of this new point is the gain_b of the interpolated brightness relationship of the four sub-blocks. In this embodiment, the intercept obtained by interpolation is called the second interpolation parameter.

[0163] The first interpolation parameter and the second interpolation parameter are brought into the linear equation to generate the grayscale interpolation brightness mapping relationship.

[0164] After interpolating each gray level, an interpolated brightness mapping relationship of each gray level can be obtained.

[0165] The grayscale levels of the four sub-blocks 10, 11, 14, and 15 all use interpolated brightness mapping.

[0166] For each pixel in the four sub-blocks 10, 11, 14, and 15, the original brightness value of the pixel in the image is substituted into the interpolated brightness mapping relationship to obtain a new brightness value, and the brightness of the pixel is set to the new brightness value. For example, for a pixel in the image, assuming its brightness value is 100, it is determined to belong to the fourth grayscale level. The interpolated brightness mapping relationship of the fourth grayscale level is used to calculate the brightness after 100 is converted to 150, and the brightness of the pixel is set to 150.

[0167] In the above embodiment, the mapping relationship of each gray level is generated by using the trilinear interpolation method, which can make the transition between image sub-blocks smoother.

[0168] As shown in FIG6 , FIG6 shows a schematic block diagram of a tone mapping apparatus 600 provided in an embodiment of the present application. The tone mapping apparatus 600 includes: a division module 610 , a correction module 620 , a creation module 630 , and an adjustment module 640 .

[0169] A division module 610 is used to divide the image into multiple sub-blocks and obtain a brightness histogram of each sub-block, wherein the brightness histogram is the frequency of pixels in the sub-block at different gray levels;

[0170] a correction module 620 for obtaining a clipping threshold corresponding to each sub-block, and correcting the brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block to generate a corrected histogram for each sub-block, wherein the frequencies of different gray levels in the corrected histogram are evenly distributed;

[0171] An establishment module 630 is configured to solve a brightness mapping equation based on the brightness histogram of the sub-block and the modified histogram to establish a brightness mapping relationship for each sub-block, wherein the brightness mapping relationship is a linear equation;

[0172] The adjustment module 640 is configured to adjust the image according to the brightness mapping relationship of each sub-block to generate an adjusted image.

[0173] In one embodiment of the present application, the brightness mapping equation is: Y = gain_a*X + gain_b; wherein Y is the brightness value of the pixel point in the corrected histogram, X is the brightness value of the pixel point in the brightness histogram, gain_a is the slope parameter of the brightness mapping relationship, and gain_b is the intercept parameter of the brightness mapping relationship.

[0174] In one embodiment of the present application, the device also includes a content detection module, which is used to perform content detection on the image and generate a content weight for each pixel; and establish a brightness histogram of the sub-block based on the brightness value and content weight of each pixel in the sub-block.

[0175] In one embodiment of the present application, the content detection module further includes:

[0176] A convolution submodule, configured to perform convolution on the pixel area corresponding to each pixel using a preset convolution kernel to obtain a convolution value for each pixel, wherein the convolution kernel is used for content detection;

[0177] The weight acquisition submodule is used to query the preset weight table according to the convolution value corresponding to each pixel point to obtain the content weight of each pixel point.

[0178] In one embodiment of the present application, the content detection module also includes: a histogram establishment submodule, which is used to determine the grayscale level of each pixel based on the brightness value of the pixel, and each grayscale level corresponds to at least one brightness value; based on all grayscale levels in the sub-block and the pixels corresponding to each grayscale level, the brightness histogram is established, and the frequency of each grayscale level in the brightness histogram is the sum of the number of pixels corresponding to the grayscale level and the content weights of all pixels corresponding to the grayscale level.

[0179] In one embodiment of the present application, the device further includes a shear value generating module,

[0180] The shearing value generation module is used to obtain the brightness relationship of each sub-block based on the comparison result of the average brightness of each sub-block and the average brightness of the image; and adjust the preset shearing value according to the brightness relationship of each sub-block to obtain the shearing threshold corresponding to each sub-block.

[0181] In one embodiment of the present application, the establishment module is specifically used to solve the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; obtain the second brightness mapping relationship of each sub-block in the previous frame image of the image; and fuse the first brightness mapping relationship and the second brightness mapping relationship of each sub-block to establish a brightness mapping relationship for each sub-block.

[0182] In one embodiment of the present application, the establishment module is specifically used to solve the brightness mapping equation based on the brightness histogram and the corrected histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; obtain the second brightness mapping relationship of each sub-block in the previous frame image of the image; obtain the deviation relationship between the first brightness mapping relationship and the second brightness mapping relationship of each sub-block; if there is a deviation relationship of a sub-block greater than a deviation threshold, then use the second brightness mapping relationship of the sub-block as the brightness mapping relationship of the sub-block.

[0183] In one embodiment of the present application, the establishment module includes: a calculation submodule, which is used to calculate the change in the probability density function of the corrected histogram image of each sub-block relative to the brightness histogram to obtain a first mapping relationship; a conversion submodule, which is used to convert the brightness value corresponding to each grayscale level of the brightness histogram in the sub-block according to the first mapping relationship to obtain multiple conversion values; a fitting submodule, which is used to solve the brightness mapping equation according to the brightness value corresponding to each grayscale level in the sub-block and the multiple conversion values ​​to obtain the first brightness mapping relationship corresponding to each grayscale level; and a generation submodule, which is used to generate the brightness mapping relationship corresponding to each grayscale level based on the first brightness mapping relationship corresponding to each grayscale level. The brightness mapping relationship of a sub-block is the brightness mapping relationship of all grayscale levels in the sub-block.

[0184] In one embodiment of the present application, the generation submodule is specifically used to obtain the second brightness mapping relationship corresponding to each grayscale level of the corresponding sub-block in the previous frame image of the image; merge the first brightness mapping relationship corresponding to each grayscale level with the second brightness mapping relationship to generate a brightness mapping relationship corresponding to each grayscale level.

[0185] In one embodiment of the present application, when the brightness mapping equation is a linear equation, it includes a slope and an intercept. The adjustment module is specifically configured to obtain multiple reference sub-blocks of the sub-block, where the reference sub-blocks are adjacent sub-blocks of the sub-block. For any grayscale level in the sub-block, a first coordinate system is established using the slope of the brightness mapping relationship of the sub-block and the multiple reference sub-blocks corresponding to the sub-block at that grayscale level as a third coordinate axis, and the length and width are used as the first coordinate axis and the second coordinate axis, respectively. Trilinear interpolation is performed based on the first coordinate system to generate first interpolation parameters. For any grayscale level in the sub-block, an intercept of the brightness mapping relationship of the sub-block and the multiple reference sub-blocks corresponding to the sub-block at that grayscale level is used as the third coordinate axis, and the length and width are used as the first coordinate axis and the second coordinate axis, respectively. Trilinear interpolation is performed based on the second coordinate system to generate second interpolation parameters. An interpolated brightness mapping relationship for the grayscale level is established based on the first interpolation parameters and the second interpolation parameters. The sub-block and the multiple reference sub-blocks corresponding to the sub-block are adjusted based on the interpolated brightness mapping relationship for each grayscale level in the sub-block to generate the adjusted image.

[0186] In one embodiment of the present application, the adjustment module is specifically used to obtain the mapping sub-block and the interpolation sub-block in the image, and each pixel in the mapping sub-block uses the brightness mapping relationship of the sub-block to adjust the brightness value, and each pixel in the interpolation sub-block uses the interpolation brightness mapping relationship to adjust the brightness value; obtain multiple reference sub-blocks of the interpolation sub-block, and the reference sub-block is an adjacent sub-block of the interpolation sub-block; use gain_a in the brightness mapping relationship of the interpolation sub-block and the multiple reference sub-blocks corresponding to the interpolation sub-block as the third coordinate axis, and use the length and width as the first coordinate axis and the second coordinate axis respectively to establish a first coordinate system, and perform trilinear interpolation is performed to generate first interpolation parameters; gain_b in the brightness mapping relationship between the interpolation sub-block and multiple reference sub-blocks corresponding to the interpolation sub-block is used as the third coordinate axis, and the length and width are used as the first coordinate axis and the second coordinate axis respectively to establish a second coordinate system, and trilinear interpolation is performed based on the second coordinate system to generate second interpolation parameters; an interpolation brightness mapping relationship is established for the interpolation sub-block based on the first interpolation parameters and the second interpolation parameters; the brightness value of the mapping sub-block in the image is adjusted based on the brightness mapping relationship, and the brightness value of the pixel point of the interpolation sub-block in the image is adjusted based on the interpolation brightness mapping relationship to generate the adjusted image.

[0187] In one embodiment of the present application, the correction module is specifically used to perform multiple rounds of iteration until a stop condition is met, and the iteration includes: finding the adjustment level in the histogram of this round, the adjustment level is the grayscale level whose frequency is greater than the clipping threshold; obtaining the allocation value of this round according to the adjustment level, and setting the frequency of each adjustment level as the clipping threshold; calculating the increase value of evenly distributing the allocation value of this round to each grayscale level in the intermediate histogram of this round; if the frequency of the grayscale level is less than the clipping threshold, the frequency of the grayscale level in the new intermediate histogram is the sum of the increase value and the frequency of this round; if the frequency of the grayscale level is the clipping threshold, the grayscale level is clipped. The frequency of the level in the new round of intermediate histogram is the clipping threshold; the intermediate histogram of the new round is generated according to the frequency of each grayscale level in the new round of intermediate histogram; wherein, the intermediate histogram of the first round of iteration is the brightness histogram, and the stopping condition includes that the distribution value cannot be evenly distributed to each grayscale level in the adjustment histogram, the distribution value of the first round of iteration is the sum of the frequency differences of all adjustment levels, the frequency difference is the frequency difference between the frequency of each adjustment level of the intermediate histogram of the current round and the clipping threshold, and the distribution value of other rounds is the sum of the remaining total value and the frequency difference, the remaining total value is the sum of all remaining values, and the remaining value is the added value that is not added.

[0188] As shown in Figure 7, Figure 7 shows a schematic block diagram of an electronic device 700 provided in an embodiment of the present application. The electronic device 700 includes: a processor 710 and a memory 720;

[0189] The processor 710, the memory 720 and each component are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 720 is used to store computer programs, such as the software function module shown in Figure 6, that is, the tone mapping device 600. Among them, the tone mapping device 600 includes at least one software function module that can be stored in the memory 720 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 700. The processor 710 is used to execute the executable module stored in the memory 720, such as the software function module or computer program included in the tone mapping device 600.

[0190] Among them, the memory 720 can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0191] The processor 710 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a microprocessor, etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. Alternatively, the processor 710 may also be any conventional processor, etc.

[0192] An embodiment of the present application further provides a non-volatile computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored. When the computer program is run by a computer such as the above-mentioned electronic device 400, the tone mapping method shown above is executed.

[0193] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0195] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0196] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a computer-readable storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a laptop, a server, or an electronic device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0197] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A tone mapping method, characterized in that: The method comprises: Divide the image into multiple sub-blocks and obtain a brightness histogram of each sub-block, wherein the brightness histogram is the frequency of pixels in the sub-block at different gray levels; Obtaining a clipping threshold corresponding to each sub-block, and correcting a brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block to generate a corrected histogram of each sub-block, wherein the frequencies of different gray levels in the corrected histogram are evenly distributed; Solving a brightness mapping equation according to the brightness histogram of the sub-block and the modified histogram to establish a brightness mapping relationship for each sub-block, wherein the brightness mapping equation includes a linear equation; The image is adjusted according to the brightness mapping relationship of each sub-block to generate an adjusted image.

2. The method according to claim 1, characterized in that The method further comprises: Performing content detection on the image to obtain a content weight of each pixel; The brightness histogram of the sub-block is established based on the brightness value and content weight of each pixel in the sub-block.

3. The method according to claim 2, characterized in that The performing content detection on the image to obtain the content weight of each pixel includes: Convolve the pixel area corresponding to each pixel using a preset convolution kernel to obtain a convolution value for each pixel. The convolution kernel is used for content detection. The preset weight table is queried according to the convolution value corresponding to each pixel to obtain the content weight of each pixel.

4. The method according to claim 2, characterized in that The step of establishing a brightness histogram of the sub-block based on the brightness value and content weight of each pixel in the sub-block includes: Determine the grayscale level of each pixel based on its brightness value, where each grayscale level corresponds to at least one brightness value. The brightness histogram is established based on all gray levels in the sub-block and the pixels corresponding to each gray level. The frequency of each gray level in the brightness histogram is the sum of the number of pixels corresponding to the gray level and the content weights of all pixels corresponding to the gray level.

5. The method according to claim 1, wherein The obtaining of the clipping threshold corresponding to each sub-block includes: Obtaining a brightness relationship of each sub-block according to a comparison result of the average brightness of each sub-block and the average brightness of the image; The preset clipping value is adjusted according to the brightness relationship of each sub-block to obtain the clipping threshold corresponding to each sub-block.

6. The method according to claim 1, characterized in that Solving the brightness mapping equation based on the brightness histogram of the sub-block and the modified histogram to establish the brightness mapping relationship of each sub-block includes: Solving the brightness mapping equation according to the brightness histogram and the modified histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; Acquire a second brightness mapping relationship of each sub-block in a previous frame image of the image; The first brightness mapping relationship and the second brightness mapping relationship of each sub-block are merged to establish a brightness mapping relationship of each sub-block.

7. The method according to claim 1, characterized in that Solving the brightness mapping equation based on the brightness histogram of the sub-block and the modified histogram to establish the brightness mapping relationship of each sub-block includes: Solving the brightness mapping equation according to the brightness histogram and the modified histogram of the sub-block to establish a first brightness mapping relationship for each sub-block; Acquire a second brightness mapping relationship of each sub-block in a previous frame image of the image; Obtaining a deviation relationship between a first brightness mapping relationship and a second brightness mapping relationship for each sub-block; If the deviation relationship of a sub-block exists and is greater than the deviation threshold, the second brightness mapping relationship of the sub-block is used as the brightness mapping relationship of the sub-block.

8. The method according to claim 1, characterized in that Solving the brightness mapping equation based on the brightness histogram of the sub-block and the modified histogram to establish the brightness mapping relationship of each sub-block includes: Calculating a change in the probability density function of the corrected histogram image of each sub-block relative to the brightness histogram to obtain a first mapping relationship; According to the first mapping relationship, converting the brightness value corresponding to each grayscale level of the brightness histogram in the sub-block to obtain multiple conversion values; Solving the brightness mapping equation according to the brightness value corresponding to each grayscale level in the sub-block and the multiple conversion values ​​to obtain a first brightness mapping relationship corresponding to each grayscale level; Based on the first brightness mapping relationship corresponding to each gray level, a brightness mapping relationship corresponding to each gray level is generated. The brightness mapping relationship of a sub-block is the brightness mapping relationship of all gray levels in the sub-block.

9. The method according to claim 8, characterized in that The generating of the brightness mapping relationship corresponding to each grayscale level based on the first brightness mapping relationship corresponding to each grayscale level includes: Obtaining a second brightness mapping relationship corresponding to each grayscale level of a corresponding sub-block in a previous frame of the image; The first brightness mapping relationship corresponding to each grayscale level is merged with the second brightness mapping relationship to generate a brightness mapping relationship corresponding to each grayscale level.

10. The method according to claim 8, characterized in that When the brightness mapping equation is a linear equation including a slope and an intercept, the image is adjusted according to the brightness mapping relationship of each sub-block to generate an adjusted image, including: Acquire multiple reference sub-blocks of the sub-block, where the reference sub-blocks are adjacent sub-blocks of the sub-block; For any grayscale level in a sub-block, a first coordinate system is established using the slope of the brightness mapping relationship between the sub-block and the multiple reference sub-blocks corresponding to the sub-block at that grayscale level as the third coordinate axis, and the length and width as the first coordinate axis and the second coordinate axis, respectively. Trilinear interpolation is performed based on the first coordinate system to generate first interpolation parameters. For any grayscale level in the sub-block, a second coordinate system is established using the intercept in the brightness mapping relationship of the sub-block and the multiple reference sub-blocks corresponding to the sub-block at that grayscale level as the third coordinate axis, and the length and width as the first coordinate axis and the second coordinate axis respectively, and trilinear interpolation is performed based on the second coordinate system to generate second interpolation parameters; Establishing an interpolation brightness mapping relationship of grayscale levels according to the first interpolation parameter and the second interpolation parameter; The sub-block and a plurality of reference sub-blocks corresponding to the sub-block are adjusted according to the interpolation brightness mapping relationship of each grayscale level in the sub-block to generate the adjusted image.

11. The method according to claim 1, wherein The brightness mapping equation is: Y=gain_a*X+gain_b; Wherein, Y is the brightness value of the pixel in the corrected histogram, X is the brightness value of the pixel in the brightness histogram, gain_a is the slope parameter of the brightness mapping relationship, and gain_b is the intercept parameter of the brightness mapping relationship.

12. The method according to claim 11, characterized in that The step of adjusting the image according to the brightness mapping relationship of each sub-block to generate an adjusted image includes: Obtaining a mapping sub-block and an interpolation sub-block in the image, adjusting the brightness value of each pixel in the mapping sub-block using the brightness mapping relationship of the sub-block, and adjusting the brightness value of each pixel in the interpolation sub-block using the interpolation brightness mapping relationship; Acquire multiple reference sub-blocks of the interpolation sub-block, where the reference sub-blocks are adjacent sub-blocks of the interpolation sub-block; A first coordinate system is established by using gain_a in the brightness mapping relationship between the interpolation sub-block and the multiple reference sub-blocks corresponding to the interpolation sub-block as the third coordinate axis, and the length and width as the first coordinate axis and the second coordinate axis respectively, and performing trilinear interpolation based on the first coordinate system to generate first interpolation parameters; Using gain_b in the brightness mapping relationship between the interpolation sub-block and the multiple reference sub-blocks corresponding to the interpolation sub-block as the third axis, and length and width as the first coordinate axis and second coordinate axis respectively to establish a second coordinate system, and performing trilinear interpolation based on the second coordinate system to generate second interpolation parameters; Establishing an interpolation brightness mapping relationship of the interpolation sub-block according to the first interpolation parameter and the second interpolation parameter; The brightness value of the mapped sub-block in the image is adjusted according to the brightness mapping relationship, and the brightness value of the pixel point of the interpolation sub-block in the image is adjusted according to the interpolation brightness mapping relationship to generate the adjusted image.

13. The method according to any one of claims 1 to 12, characterized in that The step of correcting the brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block to generate a corrected histogram of each sub-block includes: Perform multiple iterations until a stopping condition is met, wherein the iterations include: Find the adjustment level in the histogram of this round, where the adjustment level is the grayscale level whose frequency is greater than the clipping threshold; Obtain the allocation value for this round according to the adjustment level, and set the frequency of each adjustment level as the cutting threshold; Calculate the increase in value of evenly distributing the distribution value of this round to each gray level in the middle histogram of this round; If the frequency of a gray level is less than the clipping threshold, the frequency of the gray level in the new round of intermediate histogram is the sum of the increase value and the frequency of the current round; If the frequency of the gray level is the clipping threshold, the frequency of the gray level in the new round of intermediate histogram is the clipping threshold; Generate a new round of intermediate histograms based on the frequency of each gray level in the new round of intermediate histograms; The intermediate histogram of the first iteration is the brightness histogram, and the stopping condition includes that the distribution value cannot be evenly distributed to each gray level in the adjustment histogram. The distribution value of the first iteration is the sum of the frequency differences of all adjustment levels, and the frequency difference is the sum of the frequency differences of the current round. The frequency of each adjustment level of the intermediate histogram is the difference between the frequency of the clipping threshold. The distribution value of other rounds is the sum of the remaining total value and the frequency difference. The remaining total value is the sum of all remaining values, and the remaining value is the added value that is not added.

14. A tone mapping device, characterized in that: The device comprises: A division module is used to divide the image into multiple sub-blocks and obtain a brightness histogram of each sub-block, wherein the brightness histogram is the frequency of pixels in the sub-block at different gray levels; a correction module, configured to obtain a clipping threshold corresponding to each sub-block, and to correct a brightness histogram of each sub-block according to the clipping threshold corresponding to the sub-block, thereby generating a corrected histogram of each sub-block, wherein the frequencies of different gray levels in the corrected histogram are evenly distributed; An establishment module, configured to solve a brightness mapping equation based on the brightness histogram and the modified histogram of the sub-blocks, and establish a brightness mapping relationship for each sub-block, wherein the brightness mapping equation includes a linear equation; The adjustment module is used to adjust the image according to the brightness mapping relationship of each sub-block to generate an adjusted image.

15. An electronic device, characterized in that: include: processor and memory; The processor and the memory communicate with each other via a bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the tone mapping method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a computer, the tone mapping method according to any one of claims 1 to 13 is executed.

Images