Image processing method, device, image processing equipment and storage medium

Abstract

Image processing method, apparatus, image processing device and storage medium, wherein the image processing method comprises: obtaining an original high dynamic range image, and dividing the original high dynamic range image into multiple regions; calculating the histograms corresponding to the multiple regions respectively, and generating the tone mapping curves corresponding to the multiple regions according to the histograms; obtaining the low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively; combining the low dynamic range images corresponding to the multiple regions to obtain a complete low dynamic range image. The above image processing method can enhance the details of the image and improve the image quality.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing, and in particular to an image processing method, an image processing device, an image processing equipment and a storage medium. Background Art

[0002] In recent years, High Dynamic Range (HDR) image processing technology has become very important and increasingly common in the field of image processing because it can provide more dynamic range and image details. With the rapid development of high dynamic range image processing technology, people can more easily obtain high dynamic range images with real scenes.

[0003] However, the existing traditional display devices such as low dynamic range (LDR) LCDs and projectors cannot truly reproduce high dynamic range images. In order to solve this problem, a tone mapping algorithm is proposed. For example, the research methods of commonly used tone mapping algorithms can be divided into global tone mapping algorithms and local tone mapping algorithms.

[0004] However, when the existing tone mapping algorithms perform dynamic range compression on the complete image, whether it is a global tone mapping algorithm or a local tone mapping algorithm, only a fixed tone mapping algorithm is used for all areas of the entire image, without considering the content distribution of the image, which will lead to the loss of local details and fail to achieve ideal image quality. Summary of the invention

[0005] In view of this, embodiments of the present invention provide an image processing method, an apparatus, an image processing device, and a storage medium, which can enhance the details of an image and improve the image quality.

[0006] First, an embodiment of the present invention provides an image processing method, including:

[0007] Acquire an original high dynamic range image, and divide the original high dynamic range image into a plurality of regions;

[0008] Calculating histograms corresponding to the multiple regions respectively, and generating tone mapping curves corresponding to the multiple regions according to the histograms;

[0009] Obtaining low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively;

[0010] The low dynamic range images corresponding to the multiple regions are combined to obtain a complete low dynamic range image.

[0011] Optionally, dividing the original high dynamic range image into a plurality of regions comprises:

[0012] The original high dynamic range image is evenly divided into blocks to obtain a plurality of regular regions.

[0013] Optionally, dividing the original high dynamic range image into a plurality of regions comprises:

[0014] Obtaining brightness levels corresponding to pixel points in each area of ​​the original high dynamic range image;

[0015] According to a preset brightness level range and based on the brightness levels corresponding to the pixels in each area, the original high dynamic range image is divided into a plurality of irregular areas.

[0016] Optionally, the respectively calculating histograms corresponding to the multiple regions, and generating tone mapping curves corresponding to the multiple regions according to the histograms, includes:

[0017] For the multiple regions, the brightness of each region is divided into m brightness levels according to the brightness value of the pixel points, and m+1 anchor points are formed at the endpoints of the m brightness levels, where m is a natural number greater than or equal to 1;

[0018] For the multiple regions, respectively count the number of pixels distributed at each brightness level to obtain a brightness histogram corresponding to each region with continuously changing brightness, and construct an initial linear tone mapping curve of the brightness level of the original high dynamic range image in each region according to m+1 anchor points corresponding to the brightness of each region;

[0019] According to the number of pixels at each brightness level in the brightness histogram corresponding to each region, the maximum brightness level, average brightness level and highlight brightness level of the pixels of the original high dynamic range image corresponding to each region are obtained, wherein the maximum brightness level is the maximum brightness value corresponding to the pixels corresponding to each region, the average brightness level is the average brightness value of all the pixels corresponding to each region, and the highlight brightness level is between the maximum brightness level and the average brightness level;

[0020] According to the highlight brightness level of the original high dynamic range image corresponding to each of the multiple regions and the target display brightness level, the initial linear tone mapping curve corresponding to each of the multiple regions is adjusted so that the brightness level of the original high dynamic range image corresponding to each of the multiple regions is linearly mapped to the target display brightness level, thereby obtaining a compressed tone mapping curve corresponding to each of the multiple regions;

[0021] According to the number of pixels of m brightness levels in the brightness histogram corresponding to each of the multiple regions, the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each region are adjusted to obtain the tone mapping curve corresponding to each region.

[0022] Optionally, adjusting the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each region according to the number of pixels of m brightness levels in the brightness histogram corresponding to each region of the multiple regions to obtain the tone mapping curve corresponding to each region includes:

[0023] For any one of the m+1 anchor points in the compressed tone mapping curve corresponding to each area, determine the size of the number of pixels corresponding to the brightness level on both sides thereof, and when the number of pixels with a smaller brightness level is greater than the number of pixels with a larger brightness level, increase the value of the anchor point, and when the number of pixels with a smaller brightness level is less than or equal to the number of pixels with a larger brightness level, decrease the value of the anchor point.

[0024] Optionally, the highlight brightness level is a brightness level corresponding to when the number of accumulated pixel points from the maximum brightness level downward in all pixel points corresponding to each area of ​​the multiple areas meets a preset condition.

[0025] Optionally, the image processing method further includes: adjusting the preset condition according to a target display brightness level.

[0026] Optionally, the preset condition satisfied by the accumulated number of pixels includes:

[0027] The accumulated number of pixels satisfies a preset number of pixels;

[0028] The ratio of the accumulated number of pixels to the total number of pixels satisfies a preset value.

[0029] Accordingly, an embodiment of the present invention further provides an image processing device, the image processing device comprising:

[0030] An image division unit, adapted to acquire an original high dynamic range image and divide the original high dynamic range image into a plurality of regions;

[0031] An image tone mapping curve calculation unit, adapted to respectively calculate histograms corresponding to the plurality of regions, and generate tone mapping curves corresponding to the plurality of regions according to the histograms;

[0032] An image tone mapping unit, adapted to obtain low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively;

[0033] The image combining unit is adapted to combine the low dynamic range images corresponding to the multiple regions to obtain a complete low dynamic range image.

[0034] Correspondingly, an embodiment of the present invention also provides an image processing device, which includes a memory and a processor, and the memory stores computer instructions that can be executed on the processor. It is characterized in that when the processor runs the computer instructions, it executes the steps of the image processing method described in any of the aforementioned embodiments.

[0035] Correspondingly, an embodiment of the present invention further provides a computer-readable storage medium on which computer instructions are stored, wherein the computer instructions, when executed, execute the steps of the image processing method described in any of the aforementioned embodiments.

[0036] The image processing method in the embodiment of the present invention is adopted, by dividing the acquired original high dynamic range image to obtain multiple regions, and calculating the histograms corresponding to the multiple regions, generating the tone mapping curves corresponding to the multiple regions according to the histograms, and obtaining the low dynamic range images corresponding to the multiple regions according to the tone mapping curves of the multiple regions, and then combining the low dynamic range images corresponding to the multiple regions to obtain a complete low dynamic range image. Since the brightness distribution of different regions of the original high dynamic range image is not uniform, by dividing the original high dynamic range image into regions, the brightness distribution in each region obtained is relatively more consistent. Therefore, applying different tone mapping curves to the original high dynamic range image corresponding to each region can more realistically reflect the details of each region of the image. Therefore, the complete low dynamic range image obtained by combining the low dynamic range images corresponding to each region has more image details, so the image quality can be improved.

[0037] Furthermore, by evenly dividing the original high dynamic range image into blocks, a plurality of regular regions are obtained, and the segmentation method is simple and the amount of data processing is relatively small, thus saving processing resources.

[0038] Furthermore, by obtaining the brightness levels corresponding to the pixels in each area of ​​the original high dynamic range image, and dividing the original high dynamic range image into multiple irregular areas according to a preset brightness level range and based on the brightness levels corresponding to the pixels in each area, pixels with similar brightness can be divided into the same area, and then corresponding tone mapping curves are used for different areas to obtain corresponding low dynamic range images, wherein, since pixels with similar brightness are in the same area, their corresponding tone mapping curves can more fully reflect the image details and improve the image quality.

[0039] Furthermore, according to the number of m brightness level pixels in the brightness histogram corresponding to each of the multiple regions, the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each of the multiple regions are adjusted, so that the distribution of pixels of brightness levels on the left and right sides of any anchor point can be more uniform. In the tone mapping process, the merging of brightness with too many pixels can be reduced, so that the mapped image can better present local details.

[0040] Furthermore, by adjusting the preset conditions, different highlight brightness levels are obtained, thereby being able to adapt to different target display brightness levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the embodiments of this specification or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flow chart of an image processing method in an embodiment of the present invention is shown.

[0043] Figure 2 A schematic diagram of dividing an original high dynamic range image in an embodiment of the present invention is shown.

[0044] Figure 3 A schematic diagram showing another method of dividing an original high dynamic range image in an embodiment of the present invention is shown.

[0045] Figure 4 A flow chart of a method for generating a corresponding region tone mapping curve of an original high dynamic range image in an embodiment of the present invention is shown.

[0046] Figure 5 A brightness histogram of a corresponding area of ​​an original high dynamic range image according to an embodiment of the present invention is shown.

[0047] Figure 6 A tone mapping curve of a corresponding area of ​​an original high dynamic range image according to an embodiment of the present invention is shown.

[0048] Figure 7 A tone mapping curve of a corresponding area of ​​another original high dynamic range image according to an embodiment of the present invention is shown.

[0049] Figure 8 A tone mapping curve of a corresponding area of ​​another original high dynamic range image according to an embodiment of the present invention is shown.

[0050] Fig. 9A schematic structural diagram of an image processing device in an embodiment of the present invention is shown.

[0051] Fig.10 A schematic structural diagram of an image processing device in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0052] As can be seen from the background technology, when processing high dynamic range images, commonly used tone mapping algorithms include global tone mapping algorithms and local tone mapping algorithms. Among them, the global tone mapping algorithm uses the same tone mapping curve for the entire image. The algorithm is simple, but it is easy to cause the loss of local details; the local tone mapping algorithm decomposes the image into a basic layer and a detail layer through a filter, and each layer uses a different global tone mapping curve to compress the dynamic range, and then combines the two to obtain the target image, but the obtained image is prone to halo and the mapping process is difficult to control.

[0053] From the above, it can be seen that the existing high dynamic range image processing methods will cause the loss of local details, and the obtained low dynamic range images will have poor quality, affecting the user experience.

[0054] In order to enhance the details of an image and improve the image quality, an embodiment of the present invention provides an image processing method, which divides an original high dynamic range image into regions and applies different tone mapping curves to the original high dynamic range image corresponding to each region to map a low dynamic range image.

[0055] In order to enable those skilled in the art to better understand, implement and realize the embodiments of this specification, the following is a detailed description through specific embodiments with reference to the accompanying drawings.

[0056] Reference Figure 1 The flowchart of the image processing method in the embodiment of the present invention is shown. In the embodiment of the present invention, a low dynamic range image can be obtained specifically through the following steps.

[0057] S11, acquiring an original high dynamic range image, and dividing the original high dynamic range image into a plurality of regions.

[0058] In specific implementations, for most images, the brightness distribution of different regions is uneven. If the brightness of the original dynamic range image is directly compressed, the resulting low dynamic range image is prone to missing local details. In the tone mapping process, in order to more faithfully reflect the brightness distribution of different regions in the image, the original high dynamic range image can be divided into multiple different regions, and different tone mapping curves can be applied to images in different regions.

[0059] In the embodiment of the present invention, the original high dynamic range image may be divided into regular areas or irregular areas.

[0060] S12, calculating histograms corresponding to the multiple regions respectively, and generating tone mapping curves corresponding to the multiple regions according to the histograms.

[0061] Specifically, the brightness histogram of the original high dynamic range image of each area divided by step S11 can be calculated respectively, and according to the brightness histogram corresponding to each area among the multiple areas, the initial linear tone mapping curve of the brightness level of the original high dynamic range image in each area is constructed, and the brightness corresponding to the initial linear tone mapping curve corresponding to each area is compressed and dynamically adjusted to obtain the tone mapping curve corresponding to each area.

[0062] S13, obtaining low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively.

[0063] Specifically, the tone mapping curve corresponding to each of the multiple regions is obtained in step S12, and the tone mapping curve corresponding to each region is applied to the original high dynamic range image corresponding to each region to obtain the low dynamic range image corresponding to the original high dynamic range image of each region.

[0064] S14, combining the low dynamic range images corresponding to the multiple regions to obtain a complete low dynamic range image.

[0065] After obtaining the low dynamic range images corresponding to each region in the original high dynamic range image through steps S11 to S13, the low dynamic range images corresponding to the different regions can be combined to obtain a complete low dynamic range image corresponding to the original high dynamic range image.

[0066] As can be seen from the above, the above embodiments can enhance the details of the image and improve the image quality. Specifically, since the brightness distribution of different regions of the original high dynamic range image is not uniform, by dividing the original high dynamic range image into regions, the brightness distribution in each region is relatively more consistent. Therefore, applying different tone mapping curves to the original high dynamic range image corresponding to each region can more realistically reflect the details of each region of the image.

[0067] In order to enable those skilled in the art to better understand and implement, the following is a detailed description through specific application scenarios in conjunction with the accompanying drawings.

[0068] In the embodiment of the present invention, different division methods may be used according to the type of the acquired original high dynamic range image to divide the original high dynamic range image into regular areas or irregular areas.

[0069] Generally, since the brightness difference corresponding to the pixels in adjacent areas of the image is relatively small, in a specific implementation, the original high dynamic range image can be directly divided into uniform blocks to obtain a plurality of regular areas.

[0070] As a specific example, refer to Figure 2 FIG. 1 is a schematic diagram of dividing an original high dynamic range image in an embodiment of the present invention, as shown in FIG. Figure 2 As shown, by dividing the original high dynamic range image P1, a high dynamic range image with 3*3 regular regions can be obtained, that is, Figure 2 In other embodiments of the present invention, the original high dynamic range image may be evenly divided into a plurality of regular regions such as 2*2, 4*4, etc.

[0071] By evenly dividing the original high dynamic range image into blocks, a plurality of regular regions are obtained. The segmentation method is simple and the amount of data processing is relatively small, thus saving processing resources. In a specific implementation, the size of the high dynamic range image block region can be determined according to specific hardware conditions and image quality requirements.

[0072] In a specific implementation, for most high dynamic range images, the brightness distribution of pixels in each area of ​​the image is uneven, and the sensitivity of the human eye to images with different brightness levels is different, so the image can be divided based on the brightness level of the pixels in the image, and the original high dynamic range image can be divided into multiple irregular areas. Specifically, the brightness level corresponding to the pixels in each area of ​​the original high dynamic range image can be obtained, and the image can be divided into multiple irregular areas according to the brightness level range and based on the brightness level corresponding to the pixels in each area, so as to achieve the application of different tone mapping curves to images in different brightness areas.

[0073] As a specific example, refer to Figure 3 FIG. 1 is a schematic diagram of another embodiment of the present invention showing the division of an original high dynamic range image. Figure 3As shown, the original high dynamic range image P2 has 10000 brightness levels. According to the preset brightness level range, its brightness level can be divided into three intervals, that is, the first interval is 0-3000nits, the second interval is 3001-6000nits, and the third interval is 6001-10000nits. According to the intervals, a high dynamic range image P3 with three brightness changes and multiple irregular areas can be obtained, wherein the dark area A in the figure represents an image whose image pixel brightness level is in the first interval of 0-3000nits, the medium bright area B represents an image whose image pixel brightness level is in the second interval of 3001-6000nits, and the bright area C represents an image whose image pixel brightness level is in the third interval of 6001-10000nits.

[0074] By dividing the original high dynamic range image into multiple irregular areas according to a preset brightness level range and based on the brightness levels corresponding to the pixels in each area, pixels with similar brightness can be divided into the same area, and then corresponding tone mapping curves are used for different areas to obtain corresponding low dynamic range images, wherein, since pixels with similar brightness are in the same area, their corresponding tone mapping curves can more fully reflect image details and improve image quality.

[0075] It should be noted that the above-mentioned high dynamic range image region division method is only an example and does not limit the embodiments of this specification to divide the original high dynamic range image into regions by other methods.

[0076] In a specific implementation, after the acquired original high dynamic range image is divided into a plurality of different regions, the image of each region can be processed separately to generate a tone mapping curve corresponding to each region. Figure 4 The flowchart of the tone mapping curve generation method shown in the figure can obtain the tone mapping curve corresponding to each area through the following steps.

[0077] S41 , for the multiple regions, dividing the brightness of each region into m brightness levels according to the brightness values ​​of the pixels, and forming m+1 anchor points at the endpoints of the m brightness levels.

[0078] Here, m is a natural number greater than or equal to 1.

[0079] In a specific implementation, the brightness of different areas can be divided into different brightness levels according to the brightness value of the pixel. For a high dynamic range image with a large difference between the maximum brightness and the minimum brightness, the brightness levels obtained by dividing according to the brightness value of the pixel can be relatively more. In addition, the division of brightness levels is also restricted by the display hardware. Different types of display screens can display different brightness levels. Therefore, the brightness can be divided according to the difference between the maximum brightness and the minimum brightness of the image and the actual application scenario.

[0080] As a specific example, the brightness in the corresponding area of ​​the original high dynamic range image can be divided into 12 brightness levels according to the brightness values ​​of the pixels, and 13 anchor points are formed.

[0081] S42, for the multiple regions, respectively count the number of pixels distributed at each brightness level to obtain a brightness histogram corresponding to each region with continuously changing brightness, and construct an initial linear tone mapping curve of the brightness level of the original high dynamic range image in each region according to the m+1 anchor points corresponding to the brightness of each region.

[0082] As a specific example, the number of pixels corresponding to any brightness level in the 12 brightness levels in each area of ​​the original high dynamic range image can be counted to obtain a brightness histogram of the corresponding area. Figure 5 The figure shows a brightness histogram of a corresponding area of ​​an original high dynamic range image, wherein the horizontal axis represents the brightness level in the corresponding area of ​​the original high dynamic range image, the black columns represent the number of pixels corresponding to each brightness level (the specific numbers are not shown), and the endpoints between the lower side of the horizontal axis and the black column graph represent anchor points.

[0083] Depend on Figure 5 It can be seen that the brightness of the corresponding area of ​​the original high dynamic range image has 12 brightness levels and 13 anchor points, and the distribution of pixels corresponding to each brightness level is not uniform. It can be seen that the pixels of the original high dynamic range image in the corresponding area are mostly distributed at the 3rd, 7th and 8th brightness levels.

[0084] The brightness of the corresponding area of ​​the original high dynamic range image is divided into 12 brightness levels according to the brightness values ​​of the pixels, and 13 anchor points can be formed at the endpoints of the 12 brightness levels. The anchor points can be used to construct an initial linear tone mapping curve of the brightness levels of the corresponding area of ​​the original high dynamic range image.

[0085] Specifically, the brightness of the original high dynamic range image can be divided into 12 brightness levels according to the brightness values ​​of the pixels, and 13 anchor points are formed at the endpoints of the 12 brightness levels. The tone mapping curve of the initial original high dynamic range is constructed according to the 13 anchor points of the brightness levels of the corresponding areas of the original high dynamic range image, and the following can be obtained: Figure 6 An initial linear tone mapping curve of a corresponding area of ​​an original high dynamic range image is shown, wherein the abscissa and the ordinate both represent the brightness level of the original high dynamic range image.

[0086] It should be noted that, since the original high dynamic range image is not compressed, mapped, or otherwise processed when constructing the initial linear tone mapping curve of the brightness levels of pixels in the corresponding area of ​​the original high dynamic range image, the slope of the obtained initial linear tone mapping curve is equal to 1.

[0087] S43, obtaining the maximum brightness level, average brightness level and highlight brightness level of the pixel points of the original high dynamic range image corresponding to each area according to the number of pixel points of each brightness level in the histogram corresponding to each area.

[0088] Among them, the maximum brightness level is the maximum brightness value corresponding to the pixel points corresponding to each area, the average brightness level is the average brightness value of all pixel points corresponding to each area, and the highlight brightness level is between the maximum brightness level and the average brightness level.

[0089] As a specific example, refer to Figure 5 The brightness histogram shown is composed of Figure 5 It can be seen that the maximum brightness level of the corresponding area of ​​the original high dynamic range image is the maximum brightness value corresponding to the 12th brightness level. In other embodiments of the present invention, if the number of pixels in the 12th brightness level is 0 or does not exist, and the number of pixels corresponding to the 11th brightness level is not 0, then the maximum brightness level is the maximum brightness value corresponding to the 11th brightness level.

[0090] In a specific implementation, the average brightness level can be used to characterize the overall brightness of the entire complete image, and the highlight brightness level changes with the average brightness level. Specifically, the higher the average brightness level of the image, the higher the highlight brightness level; the lower the average brightness level of the image, the lower the highlight brightness. Therefore, the highlight brightness level can be adjusted according to the average brightness level.

[0091] In the embodiment of the present invention, the highlight brightness level is the brightness level corresponding to when the number of accumulated pixels from the maximum brightness level downward in all pixels corresponding to each of the multiple regions meets a preset condition. The preset condition includes: the accumulated number of pixels meets a preset number of pixels, and the ratio of the accumulated number of pixels to all the number of pixels meets a preset value.

[0092] It should be noted that the accumulated number of pixels satisfies the preset condition that the accumulated number of pixels is greater than or equal to the number of pixels in the preset condition or the ratio of the accumulated number of pixels to the total number of pixels is greater than or equal to a preset value. The embodiment of the present invention does not impose any restrictions on this.

[0093] For ease of understanding, the process of obtaining the highlight brightness level is described below using a specific example.

[0094] As a specific example, the original high dynamic range image of a certain area after division has 300 pixels, and the maximum brightness level is 10000nits. The preset condition is that the number of pixels preset from the maximum brightness level down among all pixels is 30. If the image with the maximum brightness level of 10000nits has 5 pixels, which is much less than the preset 30 pixels, then continue to obtain pixels with lower brightness levels. Assuming that when obtaining pixels of an image with a brightness level of 9900nits, 30 pixels are accumulated, the preset number of pixels is met, so the brightness level of 9900nits can be used as the highlight brightness level.

[0095] As another specific example, the original high dynamic range image of a certain area after division has 300 pixels, and the maximum brightness level is 10000 nits. The preset condition is that the pixels preset from the maximum brightness level downward account for 6% of all pixels. If the image with the highest brightness level of 10000 nits has 5 pixels, and its ratio to 300 is less than 6%, then continue to acquire pixels with lower brightness levels. Assuming that when acquiring pixels of an image with a brightness level of 9000 nits, the ratio of the accumulated pixels to the total number of pixels 300 is greater than 6%, the brightness level of 9000 nits is used as the highlight brightness level. Otherwise, continue to acquire pixels of the next brightness level, for example, the number of pixels of images with brightness levels of 8500 nits and 8000 nits, until the ratio of the total number of pixels acquired for the first time to the total number of pixels is greater than or equal to 6%, and the brightness level corresponding to the last pixel acquired is used as the highlight brightness level.

[0096] In the embodiment of the present invention, the preset condition can be adjusted according to the target display brightness level to obtain different highlight brightness levels. By adjusting the preset condition, different highlight brightness levels are obtained, so that different target display brightness levels can be adapted.

[0097] It should be noted that the preset conditions may be adjusted automatically according to the target display brightness level, or may be adjusted manually to adapt to different target display brightness levels, so as to obtain images of different qualities.

[0098] In a specific implementation, in order to ensure that the low dynamic range image obtained by mapping the original high dynamic range image through the tone mapping curve can better display local details, the highlight brightness level may be between the maximum brightness level and the average brightness level.

[0099] S44, adjusting the initial linear tone mapping curve corresponding to each area according to the highlight brightness level of the original high dynamic range image corresponding to each area of ​​the multiple areas and the target display brightness level, so that the highlight brightness level of the original high dynamic range image corresponding to each area is linearly mapped to the target display brightness level, and obtaining the compressed tone mapping curve corresponding to each area.

[0100] In a specific implementation, since the brightness level of the original high dynamic range image is much greater than the target display brightness level, if the brightness level of the original high dynamic range image is directly mapped according to the corresponding initial linear tone mapping curve, the local details of the obtained low dynamic range image are easily lost, so it is necessary to compress the brightness level of the original high dynamic range image to adapt to the target display brightness level. Specifically, the initial linear tone mapping curve can be adjusted according to the proportional relationship between the highlight brightness level of the original high dynamic range image corresponding to each area of ​​the multiple areas and the target display brightness level, so that the brightness level of the original high dynamic range image is linearly mapped to the target display brightness level.

[0101] As a specific example, the target display brightness level is set to be k*highlight brightness level, where k is a proportional coefficient, whose value is less than 1 and greater than 0, and the brightness values ​​corresponding to all pixels in the original high dynamic range image are multiplied by k at the same time to linearly compress the brightness values ​​of the pixels in the original high dynamic range image to the target display brightness value, thereby obtaining Figure 7 The compressed tone mapping curve is shown in curve 7A.

[0102] like Figure 7The tone mapping curve of the corresponding area of ​​another original high dynamic range image in the embodiment of the present invention is shown, wherein the abscissa represents the brightness level of the original high dynamic range image, the ordinate represents the target display brightness level, curve 7A represents the compressed tone mapping curve, and curve 7B represents the initial linear tone mapping curve. Since the overall brightness of the original high dynamic range image is reduced after compression, the slope of the compressed tone mapping curve 7A is smaller than the slope of the initial linear tone mapping curve 7B. However, in the process of compressing the original high dynamic range image, the brightness values ​​corresponding to all pixels are compressed according to the same proportional coefficient. Therefore, the compressed tone mapping curve 7A obtained by compressing the image still maintains a linear relationship.

[0103] S45, adjusting the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each region according to the number of pixels of m brightness levels in the brightness histogram corresponding to each region of the multiple regions, to obtain the tone mapping curve corresponding to each region.

[0104] In a specific implementation, since the brightness level of the original high dynamic range image is high, even if the brightness in the original high dynamic range image is divided into several brightness levels according to the brightness value of the pixel points, the distribution of the number of pixels corresponding to each brightness level is not uniform, resulting in too many pixels at some brightness levels. By adjusting the values ​​of the m+1 anchor points in the compression tone mapping curve corresponding to each of the multiple regions, the distribution of the pixels at the brightness levels on the left and right sides of any anchor point can be made more uniform. When compressing the image, the merging of the brightness of the pixels can be reduced, and the mapped image can better present local details. Therefore, any anchor point value can be dynamically adjusted according to the number of pixels corresponding to the brightness levels on the left and right sides of the anchor point value.

[0105] Specifically, for any one of the m+1 anchor points, the size of the number of pixels corresponding to the brightness level on both sides thereof is determined, and when the number of pixels with a smaller brightness level is greater than the number of pixels with a larger brightness level, the value of the anchor point is increased, and when the number of pixels with a smaller brightness level is less than the number of pixels with a larger brightness level, the value of the anchor point is decreased, so that the following can be obtained: Figure 8 Mid tone mapping curve 8A.

[0106] like Figure 8A tone mapping curve of a corresponding area of ​​another original high dynamic range image in an embodiment of the present invention is shown, wherein the horizontal axis represents the brightness level of the original high dynamic range image, the vertical axis represents the target display brightness level, curve 8A represents the tone mapping curve formed by dynamic adjustment of the anchor point, curve 8B represents the compressed tone mapping curve, and curve 8C represents the initial linear tone mapping curve. The anchor point value is adjusted by the size of the number of pixels corresponding to the brightness levels on the left and right sides of the anchor point, resulting in that the relationship between the brightness level of the original high dynamic range image and the target display brightness level no longer satisfies a linear relationship. Therefore, the anchor points can be connected by using cubic polynomials, Bessel polynomials, etc. to obtain the tone mapping curve 8A.

[0107] In order to enable those skilled in the art to better understand and apply the adjustment process of the anchor point value, a specific example is provided below for explanation.

[0108] As a specific example, Anc[4] represents the value of the fourth anchor point, Anc[5] represents the value of the fifth anchor point, Anc[3] represents the value of the third anchor point, Anc[4]_L_num represents the number of pixels in the brightness level to the left of the fourth anchor point, and Anc[4]_R_num represents the number of pixels in the brightness level to the right of the fourth anchor point.

[0109] When Anc[4]_L_num>Anc[4]_R_num, the value of the fourth anchor point can be adjusted according to the formula Anc[4]=Anc[4]+α*(Anc[5]-Anc[4]); when Anc[4]_L_num≤Anc[4]_R_num, the value of the fourth anchor point can be adjusted according to the formula Anc[4]=Anc[4]-β*(Anc[4]-Anc[3]), where α and β are adjustment coefficients, and both values ​​are greater than or equal to 0 and less than or equal to 1.

[0110] It should be noted that the above calculation of the anchor point is only an example. Other anchor points can be adjusted according to the above formula and will not be described in detail here.

[0111] In an embodiment of the present invention, the anchor point is dynamically adjusted by the number of pixel points corresponding to the grayscale on both sides of any anchor point in the brightness histogram so that the grayscale pixels on both sides of the anchor point are distributed more evenly. In the tone mapping process, the merging of brightness with too many pixels can be reduced. Therefore, the mapped image can better present local details.

[0112] It should be noted that the above process is only an explanation of the method for generating the tone mapping curve of one of the multiple different regions. For other regions, the corresponding tone mapping curves can still be generated according to the above process, which will not be repeated here.

[0113] In a specific implementation, a tone mapping curve of a corresponding area of ​​the original high dynamic range image is obtained, and the tone mapping curve can be applied to the original high dynamic range image of the corresponding area to obtain a low dynamic range image of the corresponding area, and then the low dynamic range images of the corresponding areas are combined to obtain a complete low dynamic range image.

[0114] The embodiment of the present invention also provides an image processing device corresponding to the above-mentioned image processing method, which is described in detail below with reference to the accompanying drawings through specific embodiments. It should be noted that the image processing device described below can be considered as a functional module required to implement the processing method provided by the embodiment of the present invention; the content of the image processing device described below can be referred to in correspondence with the content of the image processing method described above.

[0115] Reference Fig. 9 The structure block diagram of an image processing device in an embodiment of the present invention is shown in FIG. 9 . In the embodiment of the present invention, the image processing device 90 may include:

[0116] An image division unit 91 is adapted to obtain an original high dynamic range image and divide the original high dynamic range image into a plurality of regions;

[0117] An image tone mapping curve calculation unit 92 calculates histograms corresponding to the multiple regions respectively, and generates tone mapping curves corresponding to the multiple regions according to the histograms;

[0118] An image tone mapping unit 93, adapted to obtain low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively;

[0119] The image combining unit 94 is adapted to combine the low dynamic range images corresponding to the multiple regions to obtain a complete low dynamic range image.

[0120] In a specific implementation, the image division unit 91 may include an image acquisition module 911 and an image segmentation module 912, wherein the image acquisition module 911 is suitable for acquiring an original high dynamic range image, and the image segmentation module 912 is suitable for dividing the original high dynamic range image into multiple regions.

[0121] In a specific implementation, the image segmentation module 912 may divide the original high dynamic range image into regular regions or irregular regions. Specifically, the image segmentation module 912 may evenly divide the acquired original high dynamic range image into blocks to obtain a plurality of regular regions, or may divide the original high dynamic range image into a plurality of irregular regions according to a preset brightness level range.

[0122] After the image division unit 91 divides the acquired original high dynamic range image into multiple regions, it is necessary to calculate the tone mapping curves corresponding to the images of different regions. Specifically, for the multiple regions, the image tone mapping curve calculation unit 92 can divide the brightness of each region into m brightness levels according to the brightness value of the pixel points, and form m+1 anchor points at the endpoints of the m brightness levels, and then count the number of pixels falling on each brightness level, respectively, to obtain a brightness histogram of the continuously changing brightness corresponding to each region in the multiple regions, and construct an initial linear tone mapping curve of the brightness level of the corresponding region of the original high dynamic range image according to the m+1 anchor points corresponding to the brightness of each region in the multiple regions.

[0123] Since the brightness distribution of pixels in the original high dynamic range image is uneven, if the original high dynamic range image is directly mapped according to the initial linear tone mapping curve to obtain a low dynamic range image, local details of the obtained low dynamic range image are easily lost, so the initial linear tone mapping curve needs to be adjusted.

[0124] In an embodiment of the present invention, the maximum brightness level, highlight brightness level, and average brightness level corresponding to the pixel points corresponding to each of the multiple regions may be first obtained. Specifically, the maximum brightness level, average brightness level, and highlight brightness level of the pixel points of the original high dynamic range image corresponding to each of the multiple regions may be obtained according to the number of pixel points at each brightness level in the histogram corresponding to each of the multiple regions, wherein the maximum brightness level is the maximum brightness value corresponding to the pixel points corresponding to each of the multiple regions, the average brightness level is the average brightness value of all pixel points corresponding to each of the multiple regions, and the highlight brightness level is between the maximum brightness level and the average brightness level.

[0125] Furthermore, the image tone mapping curve calculation unit 92 can adjust the initial linear tone mapping curve corresponding to each of the multiple regions according to the relationship between the highlight brightness level of the original high dynamic range image corresponding to each of the multiple regions and the target display brightness level, so that the brightness level of the original high dynamic range image corresponding to each of the multiple regions is linearly mapped to the target display brightness level, thereby obtaining the compressed tone mapping curve corresponding to each of the regions.

[0126] In a specific implementation, since the number of pixels corresponding to each brightness level is not evenly distributed, in order to make the pixel distribution of the brightness levels on the left and right sides of any anchor point more even, when compressing the image, the anchor point value can be adjusted according to the number of pixels corresponding to the brightness levels on the left and right sides of the anchor point value. Specifically, the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each of the multiple regions can be adjusted according to the number of pixels of m brightness levels in the brightness histogram corresponding to each of the multiple regions, so as to obtain the tone mapping curve corresponding to each of the multiple regions.

[0127] As a specific example, when the number of pixels with brightness levels on the left side of the anchor point is greater than the number of pixels with brightness levels on its right side, the value of the anchor point is increased, and when the number of pixels with brightness levels on the left side of the anchor point is less than or equal to the number of pixels with brightness levels on its right side, the value of the anchor point is decreased.

[0128] After the original high dynamic range images of each region are calculated by the image tone mapping curve calculation unit 92, the tone mapping curve corresponding to each region can be obtained. The image tone mapping unit 93 can obtain the low dynamic range image corresponding to the original high dynamic range image of each region according to the tone mapping curve corresponding to each region, and the image combination unit 94 combines the low dynamic range images corresponding to the original high dynamic range images of each region to obtain a complete low dynamic range image.

[0129] The embodiments of this specification also provide corresponding image processing devices for implementing image processing, such as Fig.10 As shown, the image processing device 100 may include a memory 101 and a processor 102, wherein the memory 101 stores computer instructions that can be run on the processor 102, wherein the processor 102 executes the steps of the image processing method described in any of the above embodiments when running the computer instructions. For details, please refer to the above embodiments and corresponding drawings, which will not be repeated here.

[0130] In a specific implementation, for the high dynamic range images of each area obtained after the original high dynamic range image is divided, the processor 102 can process the high dynamic range images of each area in turn, or randomly select the original high dynamic range image of any area for processing; or use parallel or batch processing to process the original high dynamic range images of multiple areas at the same time until the low dynamic range images of each area are obtained.

[0131] In the specific implementation, Fig.10 As shown, the image processing device 100 may further include a display 103 suitable for displaying low dynamic range images.

[0132] In a specific implementation, the image processing device 100 may further include an expansion interface 104, which is suitable for connecting with other devices to achieve data interaction. For example, the image processing device 100 may be connected to other electronic display devices through the expansion interface 104 to play enhanced images or pictures containing enhanced images.

[0133] In a specific implementation, the memory 101 , the processor 102 , the display 103 and the expansion interface 104 may communicate with each other via a bus.

[0134] In the present embodiment, the image processing device 100 may be a set-top box, a smart TV, a personal computer, a tablet, or other electronic devices including a display device.

[0135] In a specific implementation, the processor can be implemented by a processing chip such as a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), etc., or can be implemented by an application specific integrated circuit (ASIC) or one or more integrated circuits configured to implement the embodiments of this specification.

[0136] The memory may include a random access memory (RAM) memory, and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0137] The embodiment of this specification also provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed, the steps of the image processing method described in any of the above embodiments of this specification can be executed. The computer-readable storage medium can be various suitable readable storage media such as an optical disk, a mechanical hard disk, and a solid-state hard disk. The instructions stored on the computer-readable storage medium execute the steps of the image processing method described in any of the above embodiments. For details, please refer to the above embodiments and will not be repeated here.

[0138] The computer-readable storage medium may include, for example, any suitable type of memory unit, memory device, memory item, memory medium, storage device, storage item, storage medium and / or storage unit, such as memory, removable or non-removable medium, erasable or non-erasable medium, writable or rewritable medium, digital or analog medium, hard disk, floppy disk, compact disk read-only memory (CD-ROM), compact disk recordable (CD-R), compact disk rewritable (CD-RW), optical disk, magnetic medium, magneto-optical medium, removable memory cards or disks, various types of digital versatile discs (DVDs), magnetic tapes, cassettes, etc.

[0139] Although the embodiments of the present disclosure are disclosed as above, the embodiments of the present disclosure are not limited thereto. Any person skilled in the art may make various changes and modifications without departing from the spirit and scope of the embodiments of the present disclosure. Therefore, the protection scope of the embodiments of the present disclosure shall be subject to the scope defined by the claims.

Claims

1. An image processing method, characterized in that: include: Acquire an original high dynamic range image, and divide the original high dynamic range image into a plurality of regions; Calculating histograms corresponding to the multiple regions respectively, and generating tone mapping curves corresponding to the multiple regions according to the histograms; The method of respectively calculating the histograms corresponding to the multiple regions and generating the tone mapping curves corresponding to the multiple regions according to the histograms includes: for the multiple regions, dividing the brightness of each region into m brightness levels according to the brightness values ​​of the pixels, and forming m+1 anchor points at the endpoints of the m brightness levels, wherein m is a natural number greater than or equal to 1; for the multiple regions, respectively counting the number of pixels distributed at each brightness level to obtain a brightness histogram corresponding to each region with continuously changing brightness, and respectively constructing an initial linear tone mapping curve of the brightness level of the original high dynamic range image in each region according to the m+1 anchor points corresponding to the brightness of each region; respectively obtaining the maximum brightness level, average brightness level, and highlight brightness of the pixels of the original high dynamic range image corresponding to each region according to the number of pixels at each brightness level in the brightness histogram corresponding to each region. level, wherein the maximum brightness level is the maximum brightness value corresponding to the pixel points corresponding to each region, the average brightness level is the average brightness value of all the pixel points corresponding to each region, and the highlight brightness level is between the maximum brightness level and the average brightness level; according to the highlight brightness level and the target display brightness level of the original high dynamic range image corresponding to each region of the multiple regions, the initial linear tone mapping curve corresponding to each region is adjusted so that the brightness level of the original high dynamic range image corresponding to each region is linearly mapped to the target display brightness level, and the compressed tone mapping curve corresponding to each region is obtained; according to the number of pixels of m brightness levels in the brightness histogram corresponding to each region of the multiple regions, the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each region are adjusted to obtain the tone mapping curve corresponding to each region; Obtaining low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively; The low dynamic range images corresponding to the multiple regions are combined to obtain a complete low dynamic range image.

2. The image processing method according to claim 1, characterized in that: The step of dividing the original high dynamic range image into a plurality of regions comprises: The original high dynamic range image is evenly divided into blocks to obtain a plurality of regular regions.

3. The image processing method according to claim 1, characterized in that: The step of dividing the original high dynamic range image into a plurality of regions comprises: Obtaining brightness levels corresponding to pixel points in each area of ​​the original high dynamic range image; According to a preset brightness level range and based on the brightness levels corresponding to the pixels in each area, the original high dynamic range image is divided into a plurality of irregular areas.

4. The image processing method according to claim 1, characterized in that: The adjusting the values ​​of the m+1 anchor points in the compressed tone mapping curve corresponding to each region according to the number of pixels of m brightness levels in the brightness histogram corresponding to each region of the multiple regions to obtain the tone mapping curve corresponding to each region includes: For any one of the m+1 anchor points in the compressed tone mapping curve corresponding to each area, determine the size of the number of pixels corresponding to the brightness level on both sides thereof, and when the number of pixels with a smaller brightness level is greater than the number of pixels with a larger brightness level, increase the value of the anchor point, and when the number of pixels with a smaller brightness level is less than or equal to the number of pixels with a larger brightness level, decrease the value of the anchor point.

5. The image processing method according to claim 1, characterized in that: The highlight brightness level is a brightness level corresponding to when the number of accumulated pixel points from the maximum brightness level downward in all pixel points corresponding to each area of ​​the multiple areas meets a preset condition.

6. The image processing method according to claim 5, characterized in that: Also includes: The preset condition is adjusted according to the target display brightness level.

7. The image processing method according to claim 5, characterized in that: The preset conditions satisfied by the accumulated number of pixels include: The accumulated number of pixels satisfies a preset number of pixels; The ratio of the accumulated number of pixels to the total number of pixels satisfies a preset value.

8. An image processing device, characterized in that: include: An image division unit, adapted to acquire an original high dynamic range image and divide the original high dynamic range image into a plurality of regions; An image tone mapping curve calculation unit, adapted to respectively calculate histograms corresponding to the plurality of regions, and generate tone mapping curves corresponding to the plurality of regions according to the histograms; The method of respectively calculating the histograms corresponding to the multiple regions and generating the tone mapping curves corresponding to the multiple regions according to the histograms includes: for the multiple regions, dividing the brightness of each region into m brightness levels according to the brightness values ​​of the pixels, and forming m+1 anchor points at the endpoints of the m brightness levels, wherein m is a natural number greater than or equal to 1; for the multiple regions, respectively counting the number of pixels distributed at each brightness level to obtain a brightness histogram corresponding to each region with continuously changing brightness, and respectively constructing an initial linear tone mapping curve of the brightness level of the original high dynamic range image in each region according to the m+1 anchor points corresponding to the brightness of each region; respectively obtaining the maximum brightness level, average brightness level, and highlight brightness of the pixels of the original high dynamic range image corresponding to each region according to the number of pixels at each brightness level in the brightness histogram corresponding to each region. level, wherein the maximum brightness level is the maximum brightness value corresponding to the pixel points corresponding to each region, the average brightness level is the average brightness value of all the pixel points corresponding to each region, and the highlight brightness level is between the maximum brightness level and the average brightness level; according to the highlight brightness level and the target display brightness level of the original high dynamic range image corresponding to each region of the multiple regions, the initial linear tone mapping curve corresponding to each region is adjusted so that the brightness level of the original high dynamic range image corresponding to each region is linearly mapped to the target display brightness level, and the compressed tone mapping curve corresponding to each region is obtained; according to the number of pixels of m brightness levels in the brightness histogram corresponding to each region of the multiple regions, the values ​​of m+1 anchor points in the compressed tone mapping curve corresponding to each region are adjusted to obtain the tone mapping curve corresponding to each region; An image tone mapping unit, adapted to obtain low dynamic range images corresponding to the multiple regions according to the tone mapping curves corresponding to the multiple regions respectively; The image combining unit is adapted to combine the low dynamic range images corresponding to the multiple regions to obtain a complete low dynamic range image.

9. An image processing device, comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, characterized in that: When the processor runs the computer instructions, the steps of the image processing method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed, the steps of the image processing method according to any one of claims 1 to 7 are executed.

Images