Image tone mapping method, device, equipment, medium and product

Abstract

The invention provides an image tone mapping method and device, equipment, a medium and a product, and the method comprises the steps: obtaining a to-be-processed image, and dividing the to-be-processed image into a plurality of color blocks; and based on the plurality of color blocks, performing tone mapping on the to-be-processed image to obtain a processed image. Therefore, compared with direct processing of chromaticity and brightness channels of the image, the method has the following effects: independent hue adjustment can be performed on different areas of the image through color block division, and local details and color features of the image can be better reserved; the division of the color blocks can be dynamically adjusted according to the complexity of the image content, so that the tone mapping can better adapt to different types of images and target display equipment; through color block processing, smoother tone adjustment can be performed in each color block, so that halo artifacts and false colors are reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to an image tone mapping method, device, equipment, medium and product. Background Art

[0002] In modern image processing technology, HDR (High Dynamic Range) images have received widespread attention because of their ability to capture rich details in natural scenes. The dynamic range of natural scenes is generally very high, that is, the ratio of maximum illumination to minimum illumination is large, which can reach 14 orders of magnitude. Due to its high brightness value, traditional displays cannot achieve such high brightness, resulting in overexposure or underexposure of captured images. Therefore, high dynamic range images need to be processed by tone mapping before they can be displayed on ordinary displays. The image compressed by tone mapping of high dynamic range images needs to keep as many detailed textures as possible, keep the corresponding color information, and avoid halo phenomenon to obtain high-quality images.

[0003] In recent years, researchers have been working on designing various edge-preserving filters to achieve tone mapping, dividing HDR images into detail layers and preservation layers, and processing the two layers separately. However, since filtering methods are limited in processing complex image structures, they are prone to produce halo artifacts and false colors.

[0004] Although the tone mapping algorithm based on brightness edge preservation can improve the quality of the tone mapping image to a certain extent, since the nonlinear filter used each time is fixed and not flexible enough, it cannot adapt to all structures and edges in the original image, which will cause halo artifacts and false colors. Summary of the invention

[0005] The embodiments of the present application provide an image tone mapping method, device, equipment, medium and product to solve the technical problem of the tone mapping method in the related art that is prone to cause halo artifacts and false colors.

[0006] In order to solve the above technical problems, this application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides an image tone mapping method, the method comprising:

[0008] Acquire an image to be processed, and divide the image to be processed into a plurality of color blocks;

[0009] Based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image.

[0010] Optionally, each of the color blocks includes the following three components: a color structure component, a color block mean component, and a color change component;

[0011] Based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image, which includes:

[0012] Perform K-means clustering on the color structure components of all color blocks to divide the color structure components of all color blocks into K clusters;

[0013] Processing each of the K clusters based on a principal component analysis method to obtain a subspace corresponding to each cluster;

[0014] Map the color structure component of each color block to the subspace of the corresponding cluster to obtain the mapped color structure component of each color block;

[0015] Performing enhancement processing on the color block mean component of each color block to obtain an enhanced color block mean component of each color block; and performing enhancement processing on the color change component of each color block to obtain an enhanced color change component of each color block;

[0016] The processed color blocks are determined based on the mapped color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and the processed color blocks are merged to obtain the processed image.

[0017] Optionally, the color block mean component of each color block is enhanced to obtain the enhanced color block mean component of each color block; and the color change component of each color block is enhanced to obtain the enhanced color change component of each color block, which includes:

[0018] The color change component of each color block is enhanced based on the arctan function to obtain the enhanced color change component of each color block;

[0019] And based on the preset weight value corresponding to each color block, the color block mean component of each color block is enhanced to obtain the enhanced color block mean component of each color block.

[0020] Optionally, determining a processed color block based on the mapped color structure component, the enhanced color block mean component, and the enhanced color change component of each color block, and merging the processed color blocks to obtain the processed image includes:

[0021] The mapped color structure component of each color block is enhanced based on the arctan function to obtain the enhanced color structure component of each color block;

[0022] The processed color blocks are determined based on the enhanced color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and the processed color blocks are merged to obtain the processed image.

[0023] Optionally, the color block mean component of each color block is enhanced to obtain the enhanced color block mean component of each color block, including:

[0024] The color block mean component of each color block is decomposed to obtain the sub-color structure component and the sub-color block mean component corresponding to each color block mean component;

[0025] Perform K-means clustering on the sub-color structure components corresponding to each color block mean component to divide all sub-color structure components into K clusters;

[0026] Based on the principal component analysis method, each of the K clusters divided based on all sub-color structure components is processed to obtain the subspace corresponding to each cluster;

[0027] Map each sub-color structure component to the subspace of the corresponding cluster to which it belongs, and obtain the mapped sub-color structure component of each color block mean component;

[0028] Performing enhancement processing on each sub-color block mean component respectively to obtain an enhanced sub-color block mean component;

[0029] An enhanced color block mean component of each color block is obtained based on the mapped sub-color structure component and the enhanced sub-color block mean component of each color block mean component.

[0030] Optionally, obtaining an image to be processed and dividing the image to be processed into a plurality of color blocks includes:

[0031] Dividing the image to be processed into a plurality of overlapping color blocks with a step size of a preset value;

[0032] Wherein, based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image, which includes:

[0033] Based on the multiple overlapping color blocks, tone mapping is performed on the image to be processed to obtain the image to be processed after tone mapping;

[0034] Performing image post-processing on the image to be processed after tone mapping to obtain the processed image, wherein the image post-processing includes at least one of the following: averaging the overlapping parts between adjacent color blocks in the overlapping color blocks, clamping a preset proportion of pixels in the overlapping color blocks, and pixel stretching the overlapping color blocks.

[0035] Optionally, obtaining an image to be processed and dividing the image to be processed into a plurality of color blocks includes:

[0036] Acquire an image to be processed; perform logarithmic transformation on the image to be processed to obtain a logarithmic transformed image to be processed;

[0037] The image to be processed after logarithmic transformation is divided into multiple color blocks.

[0038] In a second aspect, an embodiment of the present application provides an image tone mapping device, the device comprising:

[0039] An acquisition module, used for acquiring an image to be processed and dividing the image to be processed into a plurality of color blocks;

[0040] An execution module is used to perform tone mapping on the image to be processed based on the multiple color blocks to obtain a processed image.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of an image tone mapping method as described in the first aspect are implemented.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of an image tone mapping method as described in the first aspect are implemented.

[0043] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of an image tone mapping method as described in the first aspect.

[0044] The method shown in the embodiment of the present application realizes tone mapping by processing image color blocks, which has the following effects compared to directly processing the chroma and brightness channels of the image: by dividing the color blocks, the tone of different areas of the image can be adjusted independently, which can better preserve the local details and color characteristics of the image. In contrast, processing the overall chroma and brightness channels will lead to the loss of some details or unnatural colors. The division of color blocks can be dynamically adjusted according to the complexity of the image content, so that the tone mapping can better adapt to different types of images and target display devices. In the tone mapping process, directly processing the chroma and brightness channels may introduce artifacts or unnatural color transitions, while through color block processing, smoother tone adjustments can be made within each color block, thereby reducing these problems. In summary, the tone mapping method based on color blocks can provide higher flexibility and accuracy, making the processed images more natural and fidelity. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0046] Figure 1 A flowchart of an image tone mapping method provided in an embodiment of the present application;

[0047] Figure 2 A flowchart of an image tone mapping method provided in an embodiment of the present application;

[0048] Figure 3 A flowchart of an image tone mapping method provided in an embodiment of the present application;

[0049] Figure 4 A flowchart of an image tone mapping method (single scale) provided in an embodiment of the present application;

[0050] Figure 5 A flowchart of an image tone mapping method (multi-scale) provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of a processed image provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of a processed image provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of a processed image provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of a processed image provided in an embodiment of the present application;

[0055] Fig.10 A flowchart of an image tone mapping method provided in an embodiment of the present application;

[0056] Fig.11 A structural block diagram of a network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] Figure 1 An image tone mapping method provided according to an embodiment of the present application is shown, such as Figure 1 As shown, the method includes:

[0059] Step S101, obtaining an image to be processed, and dividing the image to be processed into a plurality of color blocks;

[0060] Step S102: Based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image.

[0061] In a possible implementation, step S101, obtaining an image to be processed, and dividing the image to be processed into multiple color blocks includes: dividing the image to be processed into multiple overlapping color blocks with a preset stride; correspondingly, step S102, based on the multiple color blocks, performing tone mapping on the image to be processed to obtain a processed image includes: based on the multiple overlapping color blocks, performing tone mapping on the image to be processed to obtain the image to be processed after tone mapping; performing image post-processing on the image to be processed after tone mapping to obtain the processed image, wherein the image post-processing includes at least one of the following: averaging the overlapping parts between adjacent color blocks in the overlapping color blocks, clamping a preset proportion of pixels in the overlapping color blocks, and pixel stretching the overlapping color blocks.

[0062] In a possible implementation, step S101, obtaining an image to be processed, and dividing the image to be processed into a plurality of color blocks includes: obtaining an image to be processed; performing a logarithmic transformation on the image to be processed to obtain a logarithmically transformed image to be processed; and dividing the logarithmically transformed image to be processed into a plurality of color blocks.

[0063] In a possible implementation, each color block includes the following three components: a color structure component, a color block mean component, and a color variation component; Figure 2 As shown, step S102, based on multiple color blocks, tone mapping is performed on the image to be processed, and the processed image obtained includes:

[0064] Step S201, performing K-means clustering processing on the color structure components of all color blocks to divide the color structure components of all color blocks into K clusters;

[0065] Step S202: Process each of the K clusters based on the principal component analysis method to obtain a subspace corresponding to each cluster;

[0066] Step S203, mapping the color structure component of each color block to the subspace of the corresponding cluster to which it belongs, to obtain the mapped color structure component of each color block;

[0067] Step S204, performing enhancement processing on the color block mean component of each color block to obtain an enhanced color block mean component of each color block; and performing enhancement processing on the color change component of each color block to obtain an enhanced color change component of each color block;

[0068] Step S205 , determining a processed color block based on the mapped color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and merging the processed color blocks to obtain a processed image.

[0069] Among them, step S204, enhancing the color block mean component of each color block to obtain the enhanced color block mean component of each color block; and enhancing the color change component of each color block to obtain the enhanced color change component of each color block, including: enhancing the color change component of each color block based on the arctan function to obtain the enhanced color change component of each color block; and based on the preset weight value corresponding to each color block, enhancing the color block mean component of each color block to obtain the enhanced color block mean component of each color block.

[0070] Among them, step S205, determining the processed color blocks based on the mapped color structure components, the enhanced color block mean components and the enhanced color change components of each color block, and merging the processed color blocks to obtain the processed image includes: enhancing the mapped color structure components of each color block based on the arctan function to obtain the enhanced color structure components of each color block; determining the processed color blocks based on the enhanced color structure components, the enhanced color block mean components and the enhanced color change components of each color block, and merging the processed color blocks to obtain the processed image.

[0071] In a possible implementation, the color block mean component of each color block is enhanced to obtain the enhanced color block mean component of each color block, including: decomposing the color block mean component of each color block to obtain the sub-color structure component and sub-color block mean component corresponding to each color block mean component; performing K-means clustering on the sub-color structure component corresponding to each color block mean component to divide all sub-color structure components into K clusters; processing each cluster of the K clusters divided based on all sub-color structure components based on the principal component analysis method to obtain the subspace corresponding to each cluster; mapping each sub-color structure component to the subspace of the corresponding cluster to obtain the mapped sub-color structure component of each color block mean component; enhancing each sub-color block mean component to obtain the enhanced sub-color block mean component; obtaining the enhanced color block mean component of each color block based on the mapped sub-color structure component and the enhanced sub-color block mean component of each color block mean component.

[0072] It should be noted that the embodiment of the present application provides an image adaptive tone mapping method based on the native color gamut and clustering of the display, which is used to solve the problem that ordinary displays used by cloud computers that can only display 16-bit brightness cannot directly display HDR images well, and the problem that traditional tone mapping is prone to produce halo artifacts and false colors.

[0073] like Figure 3 As shown, it is a flow chart of the image adaptive tone mapping method based on the native color gamut and clustering of the display provided in the embodiment of the present application. First, the high dynamic RGB image is preprocessed, then the color block is decomposed, and then clustered, that is, clustered into multiple clusters according to the color structure components, and the PCA (Principal Component Analysis) method is used for tone mapping to convert the HDR image into an LDR (Low Dynamic Range) image. The algorithm can retain the details in the image as much as possible and improve the contrast of the image as much as possible, thereby obtaining a higher quality image. Below, the image adaptive tone mapping method based on the native color gamut and clustering of the display provided in the embodiment of the present application is described in detail.

[0074] like Figure 4 FIG. 1 is a flow chart of an image adaptive tone mapping (single scale) method based on the native color gamut and clustering of a display provided in an embodiment of the present application. The steps are as follows:

[0075] Collect real-time HDR images, preprocess the images, and initialize them through logarithmic transformation; then, decompose the initialized HDR images into multiple overlapping color blocks with a stride of 2 (or specific settings according to actual needs). Decompose the color blocks into three parts: color structure, color block mean, and color change; then, use the k-means clustering method to divide them into multiple clusters according to the color structure components, and use PCA to learn adaptive subspaces for each cluster, transform the subspaces into a more compact domain to achieve tone mapping, reconstruct the color blocks through dynamic range adjustment, and finally summarize and post-process.

[0076] The preprocessing process includes: initializing it through logarithmic transformation, as shown in the following formula:

[0077] L(i,j,c)=log(I(i,j,c)·10 6 +1), where I is the input HDR image, (i, j) represents the position of the pixel, c represents the R, G, B channels, and c∈{r,g,b}.

[0078] The color block decomposition process includes: dividing into a plurality of overlapping color blocks with a stride of 2, and each color block can be divided into three components, as shown in the following formula:

[0079]

[0080] Among them, the first component retains detailed local structure information and is the color structure component; the second component reflects the color difference between the three channels and is the color change component; the third component is the average value of the three channels of all pixels, which is called the color block mean.

[0081]

[0082] Among them, the clustering method adopted is the K-means clustering method, including: the K-means clustering method is to divide the data into K groups, first randomly select K objects as the initial cluster centers, then calculate the distance between each object and the initial cluster center, and then assign it to the cluster center closest to it. The cluster centers and the objects assigned to them represent a cluster cluster. Every time an object is assigned, the cluster center of the cluster will be recalculated according to the existing objects in the cluster. This process will be repeated until a certain termination condition is met. The termination condition can be that no object is reassigned to a different cluster, no cluster center changes again, and the sum of squared errors is locally minimized. The method shown in the embodiment of the present application is to cluster the vectorized color structure components, and the use of K-means clustering for it not only has low computational cost, but also has very good results.

[0083] The PCA transformation matrix process includes: for each of the K clusters, Φ is used to represent the vector The covariance matrix of , where Ф is semi-positive definite, and the orthogonal matrix Q is the set of eigenvectors of Ф. According to the formula:

[0084] Φ=QΛQ -1 ; P = Q T ;

[0085] The PCA transformation matrix P can be obtained. Because the structures in a color block are similar, the vectors associated with the first few largest eigenvalues ​​can be used to represent the most important common structure in the cluster. Using the transformation matrix P, each color block can be Transform to the PCA domain.

[0086]

[0087] From the above, we can see that the coefficients in y are sparser than those in x, where large coefficients represent the principle structure of the image, medium coefficients represent the details of the image, and small coefficients represent noise and trivial structures. In addition, the first few coefficients in y are meaningful, while the others are very small and close to zero. Therefore, the dynamic range of y is more conducive to compression than that of x.

[0088] The process of color block reconstruction includes: adjusting m, and The value of is used to realize the tone mapping of the color block x. Transform to PCA domain and process y. Due to the trivial structure, fluctuations and noise are caused by Therefore, in order to obtain a stable tone mapping result, the noise can be effectively suppressed by filtering out the smallest PCA coefficient through an empirical threshold. While removing noise, the original structure and details of the image should be preserved as much as possible, so a suitable threshold should be selected, which is generally set to 0.1max, where max represents The absolute value of the maximum value of the coefficient. All coefficients less than 0.1max are zero.

[0089] At this time, the coefficients still need to be adjusted through curves to complete the tone mapping task, that is, it is necessary to lock the larger PCA coefficients in y and appropriately enhance the smaller coefficients in y, so that the details of the image can be better preserved. The suitable curves are arctan and sigmod functions. Since the arctan function has a strong transition ability in both shadows and highlights, and the function is oddly symmetric about the 0 point and can handle negative numbers in PCA transformation, the arctan function is used to adjust the coefficients of y as follows:

[0090] The parameter a is used to control the shape of the curve. The smaller a is, the stronger the compression effect on y is; the larger a is, the weaker the compression effect on y is.

[0091] You can also use the arctan function to adjust the color change component The adjustment formula is as follows:

[0092]

[0093] The parameter b is also used to control the shape of the curve. The slow change of the mean value m of the color block can be compressed by multiplying the weight w. After adjusting the parameters, the tone mapping patch of x can be reconstructed. The specific operations are as follows:

[0094]

[0095] Where w is a scalar, 0≤w≤1.

[0096] The aggregation and post-processing process includes: aggregating the processed color blocks to reconstruct the tone-mapped low-dynamic LDR image, first putting the tone-mapped color blocks back to their original positions, and averaging the overlapping parts of adjacent color blocks. In the post-processing stage, in order to enhance the contrast, the maximum and minimum values ​​of 1% pixels are clamped. In order to make full use of the dynamic range of the scene, each color block pixel is linearly stretched between 0 and 1.

[0097] Correspondingly, the process of extending the adaptive tone mapping method from a single scale to multiple scales is as follows Figure 5 shown. Figure 5 A schematic diagram of an image adaptive tone mapping (multi-scale) method based on display native color gamut and clustering provided in an embodiment of the present application includes the following steps:

[0098] Real-time HDR images are collected, preprocessed, and initialized by logarithmic transformation, wherein the images can be obtained from the Kodak database; then the color blocks are decomposed into multiple overlapping color blocks with a stride of 2, and the color blocks are decomposed into three parts: color structure, color block mean, and color change; the color block mean is further decomposed into color structure and color block mean, and the single scale is transformed into multiple scales; the k-means clustering method is used to divide it into multiple clusters according to the color structure component; for each cluster, PCA is used to learn an adaptive subspace, and the subspace is converted into a more compact domain to achieve tone mapping; the color blocks are reconstructed by dynamic range adjustment; and finally, summary and post-processing are performed.

[0099] It should be noted that the method shown in the embodiment of the present application can also construct a tone mapping model, and based on Figure 4 and / or Figure 5 The tone mapping model is trained in the described manner; a new image to be processed is input into the trained tone mapping model, and a processed image is output, thereby improving the efficiency of image tone mapping.

[0100] Compared with the prior art, the effect of the embodiment of the present application is: although the tone mapping algorithm based on brightness edge preservation can improve the quality of the tone mapping image to a certain extent, since the nonlinear filter used each time is fixed and inflexible, it cannot adapt to all structures and edges in the original image, which will cause halo artifacts and false colors. In contrast, the method shown in the embodiment of the present application can make full use of the redundancy of the image through clustering. The method shown in the embodiment of the present application realizes tone mapping by processing image color blocks instead of processing the chrominance and brightness channels of the image. The color blocks are decomposed into three parts: color block mean, color change and color structure. There are repeated structures in natural images. Therefore, according to the color structure component, similar color blocks can be divided into a group, that is, a cluster, and the PCA matrix is ​​used to define the subspace of the color blocks in the cluster. Each subspace can be projected into a more compact domain, so that tone mapping can be performed more effectively in it. Compared with other tone mapping methods, the method based on statistical clustering makes good use of the characteristics of global redundancy of the image to decompose the local structure of the image, so that the processed LDR image has more details and color adaptability. It is specifically reflected in the following four points:

[0101] (1) Instead of using a deterministic edge-preserving filter, a statistical clustering method is used to better represent the local color structure of the HDR image. Each color block will be adaptively processed according to the PCA transformation matrix of its cluster.

[0102] (2) Tone mapping is performed in the PCA transform domain, where the coefficients have clear physical meanings and are therefore easy to adjust, so that tone mapping can be performed effectively and the HDR image can be compressed into an LDR image.

[0103] (3) The brightness channel is not extracted and layer separation is not performed. Instead, the brightness and chromaticity information of the image are processed simultaneously by extracting color blocks.

[0104] (4) Using offline PCA transformation learning, compared with online clustering, the offline clustering method will greatly shorten the running time.

[0105] Now, in combination with specific application scenarios, the image adaptive tone mapping method based on the native color gamut and clustering of the display provided in the embodiment of the present application is described in detail.

[0106] In the clustering-based tone mapping method, the mean component of each color block will form a smooth grayscale image of the original image. Since the color blocks of size 7*7*3 are extracted in the horizontal and vertical directions, the resolution of the average image is 1 / 4 of the original image. It can be seen that there is still a certain amount of texture information in the average image. If it is compressed with weight w, it will definitely lead to the loss of some texture information. Therefore, it is proposed to expand to multiple scales to make up for this deficiency. Expanding to dual scales can better retain details, and at the same time, offline subspace learning is used to reflect adaptation. In theory, it can be expanded to more scales, but the computational cost and reporting situation should be considered. Generally, it will not be infinitely divided, and it can be set according to actual needs. Among them, the process of expanding the adaptive tone mapping method from a single scale to multiple scales includes: decomposing the mean image color block into color structure and color block mean. There is no color change component here because the mean image is a grayscale image. The color block mean here is still the average value of all pixels in the color block, and the color structure is obtained by subtracting the average value from the color block. Then clustering and PCA transformation can be used to process the color structure component in the color block.

[0107] The offline subspace learning method includes: using the K-means clustering method, in the MATLAB R2016a programming environment on a PC equipped with an i7-4790 CPU, 4G HZ and 64GB memory, it takes about 123 seconds to process an image of size 1000*750*3 (color block size: 7*7*3).

[0108] In order to save time and reduce computational costs, a PCA transformation matrix that is pre-clustered and uses an external data set can be used. The method shown in the embodiment of the present application uses the Kodak database as a training set, extracts approximately 300,000 color blocks, and the size of the color blocks is 7*7*3, and clusters are performed by calculating their color structure components. Each cluster has a cluster average and a PCA transformation matrix. In the test phase, for each patch of the input image, the corresponding cluster is determined based on the minimum Euclidean distance between its color structure component and the cluster centroid. Finally, the PCA transformation matrix of the cluster is used to process the color block. Compared with online clustering, the offline clustering method greatly shortens the running time, and it only takes about 6 seconds to process an image of a size of 1000*750*3, which is about 20 times faster than the online method. And in terms of objective evaluation, this method obtains tone mapping results similar to those of the online method.

[0109] In the embodiment of the present application, the Kodak database is used as a training set. The effectiveness of the clustering-based adaptive tone mapping algorithm proposed in the present application is verified by experiments. Figure 6 to Figure 9This is a visual result interface of offline PCA conversion learning. When the program is running, click the training button first. After the training is completed, click the test button to display the processed LDR image. In order to ensure the generality of the experimental results, four groups of HDR images were selected as research objects. The quality of the processed images was objectively compared from three perspectives: the image quality evaluation index TMQI score Q value, fidelity and naturalness.

[0110] Among the four images, the left one is the effect of the HDR image opened directly with matlab, and the right one is the effect of the LDR image obtained after the HDR image is processed by tone mapping. Objectively, the structural fidelity score of the LDR test image is represented by S; the statistical naturalness score is represented by N; and the total score of TMQI (Image quality evaluation index) is represented by Q. And the larger the values ​​of S, N and Q, the better. The scores of the three indicators are shown in Table 1.

[0111] Table 1 Scoring results of three indicators

[0112] Figure 6 Figure 7 Figure 8 Fig. 9 S 0.012252 0.024146 0.011506 0.0123 N 2.5972e-12 9.0531e-13 2.981e-12 6.5843e-12 Q 0.20961 0.25772 0.20564 0.20986

[0113] From the above data structure fidelity score S, since S5>S7>S4>S6, Figure 5 The structure fidelity is the best and the detail texture is the clearest. Figure 8 The worst, followed by the others. From the above data statistics, we can see that since N7>N6>N4>N5, Fig. 9 The naturalness is the greatest, that is, the processed image is the most natural. Figure 7 The most unnatural, followed by the others. From the above data, the total score Q value of TMQI is as follows: Q5>Q7>Q4>Q6, so we can get: Figure 7 The overall treatment effect is the best. Fig. 9 Secondly, Figure 6 again, Figure 8 Worst.

[0114] Subjectively, from the comparison of the left and right images, the image on the left has low contrast and many details and textures are unclear, while the image on the right has relatively enhanced contrast and clearer details and textures; it can basically be used as a normal image. From the perspective of fidelity and naturalness, Figure 6 The fidelity is slightly worse, and the detailed texture is lost; the naturalness is also slightly worse, and the global contrast is not high. Figure 7 The fidelity and details are good, but the naturalness is poor and the global contrast is low; Figure 8 The fidelity is poor, with many details blurred, but the naturalness is good and the global contrast is high. Fig. 9The fidelity of the image is slightly poor, and many details are not clear, but the naturalness is very high and the global contrast is very high. Compared with naturalness, the fidelity of the image is more important, so fidelity is given priority, and naturalness is considered second. Therefore, the final result of the subjective analysis is Figure 7 The overall treatment effect is the best. Fig. 9 Secondly, Figure 6 again, Figure 8 Worst. In summary, the quality of these four images after processing is not bad, they can be used as normal images, and the subjective and objective analysis results coincide with each other. It can be seen that the results of the experiment are relatively successful.

[0115] The brightness value required by HDR images cannot be achieved by general hardware, and traditional displays can only display 16-bit brightness. In order to solve this problem, tone mapping is required. The display effect of the mapped image on the display is much better than direct display, but traditional tone mapping is prone to produce halo artifacts and false colors. On this basis, the embodiment of the present application proposes a tone mapping algorithm, namely, an adaptive tone mapping algorithm based on clustering, which can retain the details in the image as much as possible and improve the contrast of the image as much as possible, thereby obtaining a higher quality image (simulation experimental results show that this method has good results). And because offline learning obtains adaptability, that is, there is no need to manually adjust parameters through training, which greatly reduces the complexity.

[0116] In addition, the method provided in the embodiment of the present application has a broad market application prospect in the field of image processing, and provides a very effective solution to the problems currently to be solved in the industry. The image processed by this method has more details and color adaptability, and can solve the problems that may arise when cloud computer products display HDR images on different displays. Specifically, at present, when using a cloud computer to watch HDR images on different display devices, due to the different hardware and software configurations of various displays, directly displaying HDR images on these devices may result in poor display effects. In the method shown in the embodiment of the present application, the HDR image can be converted into an LDR image for display, and the HDR image can be converted into an LDR image through tone mapping to adapt to different display devices, which can improve the display effect of viewing HDR images on a cloud computer, and can ensure that a more unified and optimized visual effect is presented on different cloud computer display devices, so that users can get a good viewing experience.

[0117] Fig.10 An image tone mapping device according to an embodiment of the present application is shown. Fig.10 As shown, the device 10 comprises:

[0118] An acquisition module 1001 is used to acquire an image to be processed and divide the image to be processed into a plurality of color blocks;

[0119] The execution module 1002 is used to perform tone mapping on the image to be processed based on multiple color blocks to obtain a processed image.

[0120] In a possible implementation, each color block includes the following three components: a color structure component, a color block mean component, and a color variation component;

[0121] The execution module 1002 is further used to perform K-means clustering processing on the color structure components of all color blocks to divide the color structure components of all color blocks into K clusters;

[0122] Based on the principal component analysis method, each cluster in the K clusters is processed to obtain the subspace corresponding to each cluster;

[0123] Map the color structure component of each color block to the subspace of the corresponding cluster to obtain the mapped color structure component of each color block;

[0124] Performing enhancement processing on the color block mean component of each color block to obtain an enhanced color block mean component of each color block; and performing enhancement processing on the color change component of each color block to obtain an enhanced color change component of each color block;

[0125] A processed color block is determined based on the mapped color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and the processed color blocks are merged to obtain a processed image.

[0126] In a possible implementation, the execution module 1002 is further configured to perform enhancement processing on the color change component of each color block based on an arctan function to obtain an enhanced color change component of each color block;

[0127] And based on the preset weight value corresponding to each color block, the color block mean component of each color block is enhanced to obtain the enhanced color block mean component of each color block.

[0128] In a possible implementation, the execution module 1002 is further configured to perform enhancement processing on the mapped color structure component of each color block based on an arctan function to obtain an enhanced color structure component of each color block;

[0129] The processed color blocks are determined based on the enhanced color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and the processed color blocks are merged to obtain a processed image.

[0130] In a possible implementation, the execution module 1002 is further configured to decompose the color block mean component of each color block to obtain the sub-color structure component and the sub-color block mean component corresponding to each color block mean component;

[0131] Perform K-means clustering on the sub-color structure components corresponding to each color block mean component to divide all sub-color structure components into K clusters;

[0132] Based on the principal component analysis method, each of the K clusters divided based on all sub-color structure components is processed to obtain the subspace corresponding to each cluster;

[0133] Map each sub-color structure component to the subspace of the corresponding cluster to which it belongs, and obtain the mapped sub-color structure component of each color block mean component;

[0134] Performing enhancement processing on each sub-color block mean component respectively to obtain an enhanced sub-color block mean component;

[0135] An enhanced color block mean component of each color block is obtained based on the mapped sub-color structure component and the enhanced sub-color block mean component of each color block mean component.

[0136] In a possible implementation, the acquisition module 1001 is further configured to divide the image to be processed into a plurality of overlapping color blocks with a step size of a preset value;

[0137] Wherein, based on multiple color blocks, tone mapping is performed on the image to be processed, and the processed image includes:

[0138] The execution module 1002 is also used to perform image post-processing on the image to be processed after tone mapping to obtain a processed image, wherein the image post-processing includes at least one of the following: averaging the overlapping parts between adjacent color blocks in the overlapping color blocks, clamping a preset proportion of pixels in the overlapping color blocks, and pixel stretching the overlapping color blocks.

[0139] In a possible implementation, the acquisition module 1001 is further used to acquire the image to be processed; perform logarithmic transformation on the image to be processed to obtain the image to be processed after logarithmic transformation; and divide the image to be processed after logarithmic transformation into multiple color blocks.

[0140] In summary, the device shown in the embodiment of the present application realizes tone mapping by processing image color blocks, which has the following effects compared to directly processing the chrominance and brightness channels of the image: through color block division, different areas of the image can be adjusted independently to better preserve the local details and color characteristics of the image. In contrast, processing the overall chrominance and brightness channels will result in the loss of some details or unnatural colors. The division of color blocks can be dynamically adjusted according to the complexity of the image content, so that the tone mapping can better adapt to different types of images and target display devices. During the tone mapping process, directly processing the chrominance and brightness channels may introduce artifacts or unnatural color transitions, while through color block processing, smoother tone adjustments can be made within each color block, thereby reducing these problems. In summary, the color block-based tone mapping method can provide higher flexibility and accuracy, making the processed image more natural and fidelity.

[0141] The present application embodiment provides an electronic device 11, such as Fig.11 As shown, the electronic device 11 includes: a processor 1101, a memory 1102, and a program stored in the memory 1102 and executable on the processor 1101. When the program is executed by the processor 1101, the steps of the image tone mapping method shown in the above embodiment are implemented.

[0142] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image tone mapping method shown in the above embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0143] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above-mentioned image tone mapping method and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0144] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0145] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0146] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image tone mapping method, characterized in that: The method comprises: Acquire an image to be processed, and divide the image to be processed into a plurality of color blocks; Based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image.

2. The method according to claim 1, characterized in that Each of the color blocks includes the following three components: a color structure component, a color block mean component, and a color change component; Based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image, which includes: Perform K-means clustering on the color structure components of all color blocks to divide the color structure components of all color blocks into K clusters; Processing each of the K clusters based on a principal component analysis method to obtain a subspace corresponding to each cluster; Map the color structure component of each color block to the subspace of the corresponding cluster to obtain the mapped color structure component of each color block; Performing enhancement processing on the color block mean component of each color block to obtain an enhanced color block mean component of each color block; and performing enhancement processing on the color change component of each color block to obtain an enhanced color change component of each color block; The processed color blocks are determined based on the mapped color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and the processed color blocks are merged to obtain the processed image.

3. The method according to claim 2, characterized in that Performing enhancement processing on the color block mean component of each color block to obtain an enhanced color block mean component of each color block; The color change component of each color block is enhanced, and the enhanced color change component of each color block includes: The color change component of each color block is enhanced based on the arctan function to obtain the enhanced color change component of each color block; And based on the preset weight value corresponding to each color block, the color block mean component of each color block is enhanced to obtain the enhanced color block mean component of each color block.

4. The method according to claim 2, characterized in that: Determining the processed color blocks based on the mapped color structure component, the enhanced color block mean component, and the enhanced color change component of each color block, and merging the processed color blocks to obtain the processed image includes: The mapped color structure component of each color block is enhanced based on the arctan function to obtain the enhanced color structure component of each color block; The processed color blocks are determined based on the enhanced color structure component, the enhanced color block mean component and the enhanced color change component of each color block, and the processed color blocks are merged to obtain the processed image.

5. The method according to claim 2, characterized in that: The color block mean component of each color block is enhanced, and the enhanced color block mean component of each color block includes: The color block mean component of each color block is decomposed to obtain the sub-color structure component and the sub-color block mean component corresponding to each color block mean component; Perform K-means clustering on the sub-color structure components corresponding to each color block mean component to divide all sub-color structure components into K clusters; Based on the principal component analysis method, each of the K clusters divided based on all sub-color structure components is processed to obtain the subspace corresponding to each cluster; Map each sub-color structure component to the subspace of the corresponding cluster to which it belongs, and obtain the mapped sub-color structure component of each color block mean component; Performing enhancement processing on each sub-color block mean component respectively to obtain an enhanced sub-color block mean component; An enhanced color block mean component of each color block is obtained based on the mapped sub-color structure component and the enhanced sub-color block mean component of each color block mean component.

6. The method according to claim 1, characterized in that Acquiring an image to be processed, and dividing the image to be processed into a plurality of color blocks comprises: Dividing the image to be processed into a plurality of overlapping color blocks with a step size of a preset value; Wherein, based on the multiple color blocks, tone mapping is performed on the image to be processed to obtain a processed image, which includes: Based on the multiple overlapping color blocks, tone mapping is performed on the image to be processed to obtain the image to be processed after tone mapping; Performing image post-processing on the image to be processed after tone mapping to obtain the processed image, wherein the image post-processing includes at least one of the following: averaging the overlapping parts between adjacent color blocks in the overlapping color blocks, clamping a preset proportion of pixels in the overlapping color blocks, and pixel stretching the overlapping color blocks.

7. The method according to claim 1, characterized in that Acquiring an image to be processed, and dividing the image to be processed into a plurality of color blocks comprises: Acquire an image to be processed; perform logarithmic transformation on the image to be processed to obtain a logarithmic transformed image to be processed; The image to be processed after logarithmic transformation is divided into multiple color blocks.

8. An image tone mapping device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed and dividing the image to be processed into a plurality of color blocks; An execution module is used to perform tone mapping on the image to be processed based on the multiple color blocks to obtain a processed image.

9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of an image tone mapping method as claimed in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of an image tone mapping method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of an image tone mapping method as claimed in any one of claims 1 to 7.

Images