Method of tone mapping, computer program product, electronic device and storage medium

Abstract

The application provides a tone mapping method, a computer program product, an electronic device and a storage medium, and is applied to the field of image processing. Wherein, the global tone mapping model is used for tone mapping of the image to be adjusted, which can ensure the uniformity of the overall brightness distribution of the target image, and the local tone mapping model is used for tone mapping of the image to be processed, which can ensure the high contrast of the target image. Since the hybrid tone mapping model is obtained by fusing the global tone mapping model and the local tone mapping model, the hybrid tone mapping model is used for tone mapping of the image to be adjusted, which can ensure the uniformity of the overall brightness distribution of the target image and the high contrast of the target image, thereby solving the technical problem that the tone mapping method in the prior art cannot consider the uniformity of the overall brightness distribution of the image and the high contrast.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a tone mapping method, computer program product, electronic device, and storage medium. Background Technology

[0002] Dynamic range refers to the relative ratio between the brightest and darkest parts of a scene. In the real world, a scene contains a high degree of visual information and brightness variation; or, high dynamic range (HDR) image data captured by advanced devices also has a high dynamic range. However, current mainstream display devices, such as liquid crystal displays (LCDs) and projectors, can generally only display 256 levels of brightness values. Therefore, HDR image data needs to be processed into low dynamic range (LDR) data before it can be displayed on current mainstream display devices.

[0003] Tone mapping is a technique used to convert HDR image data into LDR image data. Its basic principle is to establish a mapping from high bit pixel values ​​(e.g., 0 to 65535) to low bit pixel values ​​(e.g., 0 to 255), thereby converting HDR image data into LDR image data, which can then be displayed on a monitor.

[0004] However, existing tone mapping methods cannot simultaneously ensure both the uniformity of the overall brightness distribution of an image and a high contrast. If the overall brightness distribution of the tone-mapped image is uniform, a significant loss of contrast will occur. If the contrast of the tone-mapped image is high, the overall brightness distribution will be uneven. Summary of the Invention

[0005] The purpose of this application is to provide a tone mapping method, computer program product, electronic device, and storage medium to solve the technical problem that the tone mapping method in the prior art cannot simultaneously achieve uniformity of the overall brightness distribution of the image and high contrast.

[0006] In a first aspect, embodiments of this application provide a tone mapping method, comprising: acquiring an image to be adjusted; wherein the pixel values ​​in the image to be adjusted are within a first value range, and the pixel values ​​in a target image corresponding to the image to be adjusted are within a second value range, wherein the first value range is greater than the second value range; establishing a global tone mapping model corresponding to the image to be adjusted; performing image segmentation on the image to be adjusted to obtain multiple local images, and establishing multiple local tone mapping models corresponding to the multiple local images; fusing the global tone mapping model with each of the local tone mapping models to obtain multiple mixed tone mapping models; mapping each local image to a corresponding local target image using the corresponding mixed tone mapping model; and fusing the multiple local target images to obtain the target image. In the above scheme, using a global tone mapping model to perform tone mapping on the image to be adjusted can ensure the uniformity of the overall brightness distribution of the target image, while using a local tone mapping model to perform tone mapping on the image to be processed can ensure a high contrast of the target image. Since the hybrid tone mapping model is obtained by fusing the global tone mapping model and the local tone mapping model, using the hybrid tone mapping model to perform tone mapping on the image to be adjusted can ensure both the uniformity of the overall brightness distribution of the target image and the high contrast of the target image. This solves the technical problem that the existing tone mapping methods cannot simultaneously ensure the uniformity of the overall brightness distribution and the high contrast of the image.

[0007] In an optional implementation, the global tone mapping model and the local tone mapping model have the same structure. In the above scheme, the global tone mapping model and the local tone mapping model can both have the same structure, which facilitates subsequent fusion of the global tone mapping model and the local tone mapping model, thereby ensuring both the uniformity of the overall brightness distribution of the image and high contrast.

[0008] In an optional implementation, both the global tone mapping model and the local tone mapping model are piecewise linear functions, and both models use the same segmentation method for the first value range. In this scheme, both the global and local tone mapping models can be represented as piecewise linear functions, and they use the same segmentation method for the first value range of the image to be adjusted. This facilitates subsequent fusion of the global and local tone mapping models, thereby achieving a balance between the uniformity of the overall brightness distribution and high contrast of the image.

[0009] In an optional implementation, fusing the global tone mapping model with each local tone mapping model to obtain multiple hybrid tone mapping models includes: for each linear function segment in the global tone mapping model, fusing that segment with the corresponding linear function segment in the local tone mapping model to obtain the corresponding linear function segment in the hybrid tone mapping model. In the above scheme, fusing each linear function segment in the global tone mapping model with the corresponding linear function segment in the local tone mapping model results in a hybrid tone mapping model that is still a piecewise linear function, thus balancing the uniformity of the overall brightness distribution of the image with high contrast.

[0010] In an optional implementation, establishing a global tone mapping model corresponding to the image to be adjusted includes: constructing a global histogram based on the image to be adjusted and the segmentation method; wherein the global histogram includes multiple pixel value ranges and the luminance distribution probability corresponding to each pixel value range, the luminance distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the image to be adjusted falls within that pixel range; determining the slope of the linear function of the global tone mapping model on each pixel value range based on the global histogram, resulting in multiple slopes; wherein each slope is proportional to the luminance distribution probability corresponding to that pixel value range; determining the global tone mapping model based on the multiple slopes; and / or, establishing multiple local tone mapping models corresponding to the multiple local images. The global tone mapping model includes: for each local image, constructing a local histogram based on the local image and the segmentation method; wherein the local histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range, the brightness distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the local image falls within that pixel value range; determining the slope of the linear function of the global tone mapping model on each pixel value range based on the global histogram, resulting in multiple slopes; wherein the multiple slopes minimize the mean square error between the contrast of the local image and the contrast of the locally mapped image, the locally mapped image being the image obtained after mapping the local image based on the local tone mapping model; and determining the local tone mapping model based on the multiple slopes. In the above scheme, during the establishment of the global tone mapping model, the uniformity of the overall brightness distribution of the image can be considered, so that using the global tone mapping model to perform tone mapping on the image to be adjusted can ensure the uniformity of the overall brightness distribution of the target image; during the establishment of the local tone mapping model, the contrast of the local image can be preserved, so that using the local tone mapping model to perform tone mapping on the image to be processed can ensure that the contrast of the target image is high.

[0011] In an optional implementation, the step of segmenting the image to be adjusted into multiple local images includes: segmenting the image to be adjusted using a sliding window to obtain the multiple local images. In the above scheme, the image to be adjusted can be segmented using an active window, thereby enabling the establishment of local tone mapping models and global tone mapping models based on the segmented images to map the image tone.

[0012] In an optional implementation, there is overlap between two adjacent local images in the image to be adjusted. In the above scheme, the image to be adjusted can be segmented using an active window approach, and the resulting adjacent local images may overlap, thereby improving the uniformity of brightness distribution among the local images.

[0013] In an optional implementation, the area of ​​the overlapping portion between two adjacent local images in the image to be adjusted is half the area of ​​the local image.

[0014] Secondly, embodiments of this application provide a tone mapping apparatus, comprising: an acquisition module for acquiring an image to be adjusted; wherein the range of pixel values ​​in the image to be adjusted is a first range, and the range of pixel values ​​in a target image corresponding to the image to be adjusted is a second range, the first range being greater than the second range; an establishment module for establishing a global tone mapping model corresponding to the image to be adjusted; a segmentation module for performing image segmentation on the image to be adjusted to obtain multiple local images, and establishing multiple local tone mapping models corresponding to the multiple local images; wherein both the global tone mapping model and the local tone mapping model are used to map pixel values ​​within the first range to pixel values ​​within the second range; a first fusion module for fusing the global tone mapping model with each local tone mapping model to obtain multiple mixed tone mapping models; a mapping module for mapping each local image to a corresponding local target image using the corresponding mixed tone mapping model; and a second fusion module for fusing the multiple local target images to obtain the target image. In the above scheme, using a global tone mapping model to tone map the image to be adjusted can ensure the uniformity of the overall brightness distribution of the target image, while using a local tone mapping model to tone map the image to be processed can ensure the high contrast of the target image. Since the hybrid tone mapping model is obtained by fusing the global tone mapping model and the local mapping model, using the hybrid tone mapping model to tone map the image to be adjusted can ensure both the uniformity of the overall brightness distribution of the target image and the high contrast of the target image, thus solving the technical problem that existing tone mapping methods cannot simultaneously achieve both the uniformity of the overall brightness distribution and high contrast of the image.

[0015] In an optional implementation, the global tone mapping model and the local tone mapping model have the same structure. In the above scheme, the global tone mapping model and the local tone mapping model can both have the same structure, which facilitates subsequent fusion of the global tone mapping model and the local tone mapping model, thereby ensuring both the uniformity of the overall brightness distribution of the image and high contrast.

[0016] In an optional implementation, both the global tone mapping model and the local tone mapping model are piecewise linear functions, and both models use the same segmentation method for the first value range. In this scheme, both the global and local tone mapping models can be represented as piecewise linear functions, and they use the same segmentation method for the first value range of the image to be adjusted. This facilitates subsequent fusion of the global and local tone mapping models, thereby achieving a balance between the uniformity of the overall brightness distribution and high contrast of the image.

[0017] In an optional implementation, the first fusion module is specifically used to: fuse each linear function segment in the global tone mapping model with a corresponding linear function segment in the local tone mapping model to obtain a corresponding linear function segment in the mixed tone mapping model. In the above scheme, each linear function segment in the global tone mapping model can be fused with a corresponding linear function segment in the local tone mapping model to obtain a mixed tone mapping model that is still a piecewise linear function, thereby balancing the uniformity of the overall brightness distribution of the image with high contrast.

[0018] In an optional implementation, the establishing module is specifically used for: constructing a global histogram based on the image to be adjusted and the segmentation method; wherein, the global histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range, the brightness distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the image to be adjusted falls within that pixel range; determining the slope of the linear function of the global tone mapping model on each pixel value range based on the global histogram, resulting in multiple slopes; wherein, each slope is proportional to the brightness distribution probability corresponding to that pixel value range; determining the global tone mapping model based on the multiple slopes; and / or, the segmentation module is specifically used for: for each local image, based on... A local histogram is constructed based on the local image and the segmentation method. The local histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range. The brightness distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the local image falls within that range. The slope of the linear function of the global tone mapping model on each pixel value range is determined based on the global histogram, resulting in multiple slopes. These multiple slopes minimize the mean square error between the contrast of the local image and the contrast of the locally mapped image. The locally mapped image is the image obtained after mapping the local image based on the local mapping model. The local tone mapping model is determined based on the multiple slopes. In the above scheme, during the establishment of the global tone mapping model, the uniformity of the overall brightness distribution of the image can be considered, ensuring the uniformity of the overall brightness distribution of the target image when using the global tone mapping model to tone map the image to be adjusted. During the establishment of the local tone mapping model, the contrast of the local image can be preserved, ensuring high contrast of the target image when using the local tone mapping model to tone map the image to be processed.

[0019] In an optional implementation, the segmentation module is specifically used to: segment the image to be adjusted using a sliding window to obtain the multiple local images. In the above scheme, the image to be adjusted can be segmented using an active window, thereby enabling the establishment of a local tone mapping model and a global tone mapping model based on the segmented images to map the image tone.

[0020] In an optional implementation, there is overlap between two adjacent local images in the image to be adjusted. In the above scheme, the image to be adjusted can be segmented using an active window approach, and the resulting adjacent local images may overlap, thereby improving the uniformity of brightness distribution among the local images.

[0021] In an optional implementation, the area of ​​the overlapping portion between two adjacent local images in the image to be adjusted is half the area of ​​the local image.

[0022] Thirdly, embodiments of this application provide a computer program product, including computer program instructions, which are read and executed by a processor to perform the method described in the first aspect.

[0023] Fourthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus; the processor and the memory communicate with each other via the bus; the memory stores computer program instructions executable by the processor, and the processor can execute the method described in the first aspect by calling the computer program instructions.

[0024] Fifthly, embodiments of this application provide a computer-readable storage medium that stores computer program instructions, which, when executed by a computer, cause the computer to perform the method described in the first aspect.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, embodiments of this application are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart of a tone mapping method provided in an embodiment of this application;

[0028] Figure 2 A structural block diagram of a tone mapping device provided in an embodiment of this application;

[0029] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0031] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beautification, makeup, medical aesthetics, and intelligent temperature measurement.

[0032] Please refer to Figure 1 , Figure 1 A flowchart of a tone mapping method provided in this application embodiment, which can be applied to electronic devices, and specifically may include the following:

[0033] Step S101: Obtain the image to be adjusted.

[0034] Step S102: Establish a global tone mapping model corresponding to the image to be adjusted.

[0035] Step S103: Perform image segmentation on the image to be adjusted to obtain multiple local images, and establish multiple local tone mapping models corresponding to the multiple local images.

[0036] Step S104: Fuse the global tone mapping model with each local tone mapping model to obtain multiple mixed tone mapping models.

[0037] Step S105: For each local image, map it to the corresponding local target image using the corresponding blending tone mapping model.

[0038] Step S106: Fuse multiple local target images to obtain the target image.

[0039] Specifically, the image to be adjusted in step S101 refers to the image that needs tone mapping, while the target image in step S106 refers to the image obtained after tone mapping of the image to be adjusted. The tone mapping method converts a high dynamic range image into a low dynamic range image. Dynamic range can be understood as the range of pixel values ​​in an image, or as the number of bits occupied by each pixel in the image. Therefore, the first range of pixel values ​​in the image to be adjusted should be greater than the second range of pixel values ​​in the target image.

[0040] For example, the first value range of the image to be adjusted can be 0 to 65535, and the second value range of the target image can be 0 to 255; that is, the image to be adjusted with pixel values ​​in the range of 0 to 65535 can be mapped to the target image with pixel values ​​in the range of 0 to 255 through the tone mapping method provided in the embodiments of this application.

[0041] It is understood that the specific values ​​mentioned above are merely an example provided by the embodiments of this application, and those skilled in the art can adjust the values ​​of the first and second value ranges according to the actual situation.

[0042] Furthermore, this application does not specifically limit the implementation method of the electronic device acquiring the image to be adjusted, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the image to be adjusted can be an image acquired in real time by an image acquisition device, which can send the image to be adjusted to the electronic device; or, the image to be adjusted can be an image pre-stored locally, which the electronic device can directly read; or, the image to be adjusted can be an image stored in the cloud, which the electronic device can read, etc.

[0043] Then, based on the image to be adjusted obtained in step S101, a global tone mapping model and a local tone mapping model can be established respectively. Both the global tone mapping model and the local tone mapping model are used to map pixel values ​​within a first value range to pixel values ​​within a second value range.

[0044] As the name suggests, a global tone mapping model applies the same mapping rules to all pixels in the image to be adjusted. Therefore, a global tone mapping model can be directly built based on the complete image to be adjusted. Based on this global tone mapping model, all pixels in the image to be adjusted can be mapped using the same mapping rules.

[0045] Local tone mapping (LTM) models apply the same mapping rules to local pixels in an image to be adjusted, while different local pixels may be mapped using different rules. Therefore, the image to be adjusted can be segmented into multiple local images. Then, a corresponding LTM model can be generated based on each local image, resulting in multiple LTM models. Each LTM model maps all pixels in the corresponding local image using the same rules; different LTM models can map pixels in different local images using different rules.

[0046] It should be noted that the specific implementation methods for establishing the global tone mapping model, segmenting the image to be adjusted, and establishing the local tone mapping model will be described in detail in subsequent embodiments, and will not be introduced here.

[0047] It is understood that the embodiments of this application do not impose specific limitations on the order of establishing the global tone mapping model and the local tone mapping model, and those skilled in the art can also make appropriate adjustments according to the actual situation. For example, both models can be established simultaneously; the global tone mapping model can be established first, followed by the local tone mapping model; or the local tone mapping model can be established first, followed by the global tone mapping model.

[0048] Next, after obtaining a global tone mapping model and multiple local tone mapping models, each local tone mapping model can be fused with the global tone mapping model to obtain a corresponding mixed tone mapping model; multiple local tone mapping models will result in multiple corresponding mixed tone mapping models. Therefore, a local image will correspond to one local tone mapping model and also to one mixed tone mapping model.

[0049] Next, each blending tone mapping model is used to map the corresponding local image, resulting in a local target image for each local image. It's understandable that the blending tone mapping model can also map pixel values ​​from a first value range to a second value range. That is, the range of pixel values ​​in the local image is greater than the range of pixel values ​​in the corresponding local target image. In this way, a local image with a high dynamic range is converted into a local target image with a low dynamic range.

[0050] Finally, the multiple local target images obtained from the transformation are fused to obtain the target image. It can be understood that the process of fusing multiple local target images can be similar to the process of segmenting the image to be adjusted into multiple local images; in other words, the process of fusing the local target images can be the reverse of segmenting the image to be adjusted, thereby ensuring that the fused target image is the transformed image of the image to be adjusted.

[0051] The following example illustrates the above implementation method:

[0052] First, obtain the image P1 to be adjusted; then, establish a global tone mapping model G based on P1, and segment P1 into n local images IN1, IN2, ..., IN n Based on IN1, a corresponding local tone mapping model L1 is established, ..., based on IN n Establish the corresponding local tone mapping model L n This yields n local tone mapping models L1, L2, ..., L n Next, G and L1 are fused to obtain the corresponding blended tone mapping model GL1, ..., G and L n The corresponding blending tone mapping model GL is obtained by fusion. n This yields n blending tone mapping models GL1, GL2, ..., GL n Next, GL1 is used to map IN1 to the corresponding local target image OUT1, ..., using GL n IN n Mapped to the corresponding local target image OUT n This yields n local target images OUT1, OUT2, ..., OUT n Finally, merge OUT1, OUT2, ..., OUT n The target image P2 is obtained; wherein the range of pixel values ​​in P1 (i.e., the first range of values) is greater than the range of pixel values ​​in P2 (i.e., the second range of values).

[0053] In the above scheme, using a global tone mapping model to tone map the image to be adjusted can ensure the uniformity of the overall brightness distribution of the target image, while using a local tone mapping model to tone map the image to be processed can ensure the high contrast of the target image. Since the hybrid tone mapping model is obtained by fusing the global tone mapping model and the local mapping model, using the hybrid tone mapping model to tone map the image to be adjusted can ensure both the uniformity of the overall brightness distribution of the target image and the high contrast of the target image, thus solving the technical problem that existing tone mapping methods cannot simultaneously achieve both the uniformity of the overall brightness distribution and high contrast of the image.

[0054] Furthermore, based on the above embodiments, the global tone mapping model and the local tone mapping model provided in this application embodiment may have the same structure.

[0055] Specifically, for example, both the global tone mapping model and the local tone mapping model can be piecewise linear functions; or, both the global tone mapping model and the local tone mapping model can be linear functions; or, both the global tone mapping model and the local tone mapping model can be exponential functions, etc.

[0056] It is understood that the embodiments of this application do not impose specific limitations on the specific implementation methods of the global tone mapping model and the local tone mapping model, and those skilled in the art can make appropriate choices according to the actual situation.

[0057] In the above scheme, the global tone mapping model and the local tone mapping model can have the same structure, which facilitates the subsequent fusion of the global tone mapping model and the local tone mapping model, thereby taking into account both the uniformity of the overall brightness distribution of the image and high contrast.

[0058] Furthermore, based on the above embodiments, the global tone mapping model and the local tone mapping model provided in this application embodiment can both be piecewise linear functions, and the global tone mapping model and the local tone mapping model have the same segmentation method for the first value range.

[0059] Specifically, assuming the pixel value in the image to be adjusted is l, its first value range is 0 to l. max The pixel value in the target image is v, and its second value range is 0 to v. max .

[0060] Since both global and local tone mapping models are piecewise linear functions, their x-axis represents the pixel value before mapping, and their y-axis represents the pixel value after mapping. Therefore, both global and local tone mapping models can be segmented based on a first value range of the image to be adjusted, and the segmentation method should be the same. After segmentation, within a certain range of pixel values, the pixel value before mapping and the pixel value after mapping have a linear relationship.

[0061] It is understood that the embodiments of this application provide various ways to segment the first value range, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the first value range can be divided equally. For instance, if the first value range is 0 to 65535, it can be divided into 16 segments, each with 4096 pixel values; or, the first value range of the image to be adjusted can be 0 to 1... maxDivided by the piecewise factor δ Segment; where, when k=1, the k-th segment represents the pixel value range [0,δ), when When k represents the pixel value range [k*δ, (k+1)*δ), the kth segment represents the pixel value range [k*δ, (k+1)*δ].

[0062] As one implementation method, the global tone mapping model and the local tone mapping model can be represented as piecewise linear functions as follows:

[0063] v(l)=v k +s k *(ll k );

[0064] Where v(l) is the pixel value v in the image to which the pixel value l in the image to be adjusted is mapped. Let l be the slope of the piecewise linear function in the k-th segment. k =k*δ is the pixel value in the image to be adjusted corresponding to the starting position of the kth segment, v k That is, v(l) k ) represents the pixel value in the image to which the starting position of the k-th segment is mapped.

[0065] It is understood that the specific implementation of the global tone mapping model and the local tone mapping model provided in this application is only an example provided by this application. Depending on the actual situation, those skilled in the art can make appropriate adjustments to the specific implementation of the global tone mapping model and the local tone mapping model. For example, the global tone mapping model and the local tone mapping model can also be nonlinear functions, etc.

[0066] In the above scheme, both the global tone mapping model and the local tone mapping model can be represented by piecewise linear functions, and the segmentation method of the first value range of the image to be adjusted is the same for both. This facilitates the subsequent fusion of the global tone mapping model and the local tone mapping model, thereby taking into account both the uniformity of the overall brightness distribution of the image and high contrast.

[0067] Furthermore, since both the global tone mapping model and the local tone mapping model provided in this application embodiment are piecewise linear functions, the above step S104 may specifically include the following:

[0068] For each linear function segment in the global tone mapping model, the linear function segment is fused with the corresponding linear function segment in the local tone mapping model to obtain the corresponding linear function segment in the mixed tone mapping model.

[0069] Specifically, since both the global tone mapping model and the local tone mapping model are piecewise linear functions, and they divide the first value range in the same way, the k-th segment in the global tone mapping model has the same x-coordinate as the k-th segment in the local tone mapping model. In other words, fusing the global tone mapping model and the local tone mapping model can be understood as fusing a corresponding linear function segment from the global tone mapping model and the local tone mapping model; further, it can also be understood as fusing the slope of the global tone mapping model and the slope of the local tone mapping model.

[0070] As one implementation method, during the fusion process, the global tone mapping model and the local tone mapping model can be weighted and fused based on a fusion strategy. This fusion strategy can be predetermined by the user; it can be adjusted by the electronic device as needed; or it can be adjusted by the user as needed, etc. This application does not impose specific limitations on this, and those skilled in the art can make appropriate adjustments according to the actual situation.

[0071] It is understood that when the tone mapping method provided in this application embodiment is applicable to scenarios where the uniformity of the overall brightness distribution of the image is highly required, the weight of the global tone mapping model can be larger; when the tone mapping method provided in this application embodiment is applicable to scenarios where the contrast of the image is highly required, the weight of the local tone mapping model can be larger.

[0072] Taking the piecewise linear function provided in the above embodiment as an example, the fused mixed tone mapping model can be expressed as:

[0073] s GL,k =α*s G,k +(1-α)*s L,k ;

[0074] Among them, s GL,k For the mixed tone mapping model corresponding to the k-th segment, s G,k Let s be the slope of the global tone mapping model corresponding to the k-th segment. L,k s G,k Let α be the slope of the local tone mapping model corresponding to the k-th segment, α be the weight of the global tone mapping model corresponding to the k-th segment, and (1-α) be the weight of the local tone mapping model corresponding to the k-th segment.

[0075] In the above scheme, each linear function in the global tone mapping model can be fused with the corresponding linear function in the local tone mapping model to obtain a mixed tone mapping model that is still a piecewise linear function, thereby taking into account both the uniformity of the overall brightness distribution of the image and high contrast.

[0076] Furthermore, since both the global tone mapping model and the local tone mapping model provided in this application embodiment are piecewise linear functions, the step of establishing the global tone mapping model corresponding to the image to be adjusted in step S102 may specifically include the following:

[0077] Step 1) Construct a global histogram based on the image to be adjusted and the segmentation method.

[0078] Step 2) Determine the slope of the linear function of the global tone mapping model in each pixel value range based on the global histogram, and obtain multiple slopes.

[0079] Step 3) Determine the global tone mapping model based on multiple slopes.

[0080] Specifically, a global histogram can first be constructed based on the image to be adjusted and the segmentation method used in the above embodiments to segment the first value range. The global histogram includes multiple pixel value ranges (horizontal axis) and the luminance distribution probability corresponding to each pixel value range (vertical axis). The luminance distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the image to be adjusted falls within that pixel range.

[0081] For global tone mapping models, considering the uniformity of the overall brightness distribution of the image, we can refer to the concept of histogram equalization, assuming that for a piecewise brightness distribution probability p k A larger segment k should occupy a larger range of brightness distribution in the mapped image. Therefore, the following formula can be obtained:

[0082]

[0083] Where N is the total number of segments in the piecewise function, p i p j Let be the probabilities that the pixel values ​​of the image to be adjusted fall into the i-th and j-th segments, respectively. Then, substitute the above formula into the previously given slope calculation formula. The following formula can be obtained:

[0084]

[0085] Where, p k The probability that a pixel value of the image to be adjusted falls into the k-th segment can be used to determine the slope s of the k-th segment. k The luminance distribution probability p corresponding to this range of pixel values k Proportional.

[0086] Then, the above s k Substituting the values ​​into the piecewise linear function yields the corresponding global tone mapping model.

[0087] It should be noted that in piecewise linear functions, v k It can be based on v k-1 and s k The calculation yields the result. For example, if v0 = 0, then v1 = s1 * δ.

[0088] In the above scheme, the uniformity of the overall brightness distribution of the image can be considered during the process of establishing the global tone mapping model, so that the use of the global tone mapping model to perform tone mapping on the image to be adjusted can ensure the uniformity of the overall brightness distribution of the target image.

[0089] Furthermore, based on the fact that both the global tone mapping model and the local tone mapping model provided in this application embodiment are piecewise linear functions, the establishment of multiple local tone mapping models corresponding to multiple local images in step S103 above may specifically include the following:

[0090] Step 1): For each local image, construct a local histogram based on the local image and the segmentation method.

[0091] Step 2) Determine the slope of the linear function of the global tone mapping model in each pixel value range based on the global histogram, and obtain multiple slopes.

[0092] Step 3) Determine the local tone mapping model based on multiple slopes.

[0093] Specifically, a local histogram can first be constructed based on the local image and the segmentation method used in the above embodiments to segment the first value range. Similar to the global histogram, the local histogram includes multiple pixel value ranges (horizontal axis) and the brightness distribution probability corresponding to each pixel value range (vertical axis). The brightness distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the image to be adjusted falls within that pixel range.

[0094] For the local tone mapping model, image contrast is considered. Assuming we only focus on the k-th segment of the linear piecewise function, and that the contrast between pixel values ​​falling within this segment in the local image is C. k The contrast after tone mapping is There is clearly a slope s for the current segment. k The lower the value, the more likely it is to map a larger range of pixel values ​​to a smaller range of pixel values, meaning the contrast will become lower; similarly, s k The higher the value, the higher the contrast ratio will be, and when s k When the value is 1, the contrast is well preserved. Therefore, we can obtain:

[0095]

[0096] Extending the above analysis to all segments of a linear piecewise function, if we want to minimize the contrast loss of the tone-mapped image, we can minimize the following mean square error:

[0097]

[0098] Where Loss(s0,…,s N ) represents the mean square error between the contrast of the local image and the contrast of the locally mapped image, where the locally mapped image is the image obtained after mapping the local image based on the local mapping model; p(C) represents the contrast of the image to be adjusted falling within this range of brightness values, where C is the mean square error. k The distribution of .

[0099] Will Substituting into the above formula and simplifying, we get:

[0100]

[0101] Because, C k Both p(C) and s are only related to the image to be adjusted, and not to the image itself. k Since it is irrelevant, the above equation can be simplified to solving for:

[0102]

[0103] At the same time, it should be ensured that the maximum pixel value after piecewise linear function mapping is equal to the expressible upper limit v of pixel values ​​in the target image. max ,Right now:

[0104]

[0105] Therefore, based on the above constraints, using the Lagrange operator method combined with the KKT conditions, we have:

[0106]

[0107]

[0108]

[0109] We can solve for:

[0110]

[0111] Substitute λ into s k That is:

[0112]

[0113] It can be seen that the multiple slopes obtained minimize the mean square error between the contrast of the local image and the contrast of the local mapped image.

[0114] Then, the above s k Substituting the values ​​into the piecewise linear function yields the corresponding local tone mapping model.

[0115] It should be noted that, similar to the global tone mapping model, the piecewise linear function in the local tone mapping model, v k Similarly, it can be based on v k-1 and s k Calculated.

[0116] In the above scheme, during the process of establishing the local tone mapping model, the contrast of the local image can be preserved so that the tone mapping of the image to be processed can be performed using the local tone mapping model to ensure that the contrast of the target image is high.

[0117] Furthermore, based on the above embodiments, the step S103 of segmenting the image to be adjusted into multiple local images may specifically include the following:

[0118] The image to be adjusted is segmented by using a sliding window method to obtain multiple local images.

[0119] Specifically, this application provides various implementation methods for segmenting the image to be adjusted, such as: segmenting the image to be adjusted using a sliding window method; segmenting the image to be adjusted by determining the size of a local image, etc. Those skilled in the art can make appropriate adjustments according to the actual situation.

[0120] It should be noted that during the segmentation process of the image to be adjusted, there may be overlap between two adjacent local images in the image to be adjusted, or there may be no overlap.

[0121] For example, assuming the window size is window_size = L and the window movement step is step = L / 2, an image to be adjusted with an original size of (H, W) can be segmented into... A local image of size (window_size, window_size).

[0122] In the above scheme, the image to be adjusted can be segmented by a sliding window. The adjacent local images obtained by the segmentation can overlap, thereby improving the uniformity of brightness distribution among the local images.

[0123] Please refer to Figure 2 , Figure 2This application provides a structural block diagram of a tone mapping device 200, which may include: an acquisition module 201 for acquiring an image to be adjusted; wherein the pixel values ​​in the image to be adjusted are within a first range, and the pixel values ​​in the target image corresponding to the image to be adjusted are within a second range, wherein the first range is greater than the second range; an establishment module 202 for establishing a global tone mapping model corresponding to the image to be adjusted; and a segmentation module 203 for performing image segmentation on the image to be adjusted to obtain multiple local images, and establishing a model corresponding to the multiple local images. Multiple local tone mapping models; wherein, both the global tone mapping model and the local tone mapping model are used to map pixel values ​​within the first value range to pixel values ​​within the second value range; a first fusion module 204 is used to fuse the global tone mapping model with each of the local tone mapping models respectively to obtain multiple mixed tone mapping models; a mapping module 205 is used to map each local image to a corresponding local target image using the corresponding mixed tone mapping model; a second fusion module 206 is used to fuse multiple local target images to obtain the target image.

[0124] In an optional implementation, the global tone mapping model and the local tone mapping model have the same structure. In the above scheme, the global tone mapping model and the local tone mapping model can both have the same structure, which facilitates subsequent fusion of the global tone mapping model and the local tone mapping model, thereby ensuring both the uniformity of the overall brightness distribution of the image and high contrast.

[0125] In this embodiment, using a global tone mapping model to tone map the image to be adjusted ensures the uniformity of the overall brightness distribution of the target image, while using a local tone mapping model ensures high contrast. Since the hybrid tone mapping model is obtained by fusing the global and local tone mapping models, using it to tone map the image to be adjusted can guarantee both the uniformity of the overall brightness distribution and high contrast of the target image. This solves the technical problem that existing tone mapping methods cannot simultaneously achieve both uniformity of overall brightness distribution and high contrast.

[0126] Furthermore, both the global tone mapping model and the local tone mapping model are piecewise linear functions, and the global tone mapping model and the local tone mapping model use the same segmentation method for the first value range.

[0127] In the embodiments of this application, both the global tone mapping model and the local tone mapping model can be represented by piecewise linear functions, and the segmentation method of the first value range of the image to be adjusted is the same for both. This facilitates the subsequent fusion of the global tone mapping model and the local tone mapping model, thereby taking into account both the uniformity of the overall brightness distribution of the image and a high contrast.

[0128] Furthermore, the first fusion module 204 is specifically used to: for each linear function in the global tone mapping model, fuse the linear function with the corresponding linear function in the local tone mapping model to obtain the corresponding linear function in the mixed tone mapping model.

[0129] In this embodiment, each linear function in the global tone mapping model can be fused with the corresponding linear function in the local tone mapping model to obtain a mixed tone mapping model that is still a piecewise linear function, thereby taking into account both the uniformity of the overall brightness distribution of the image and a high contrast.

[0130] Further, the establishment module 202 is specifically used for: constructing a global histogram based on the image to be adjusted and the segmentation method; wherein, the global histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range, the brightness distribution probability corresponding to each pixel value range refers to the probability that the pixel value in the image to be adjusted falls into that pixel range; determining the slope of the linear function of the global tone mapping model on each pixel value range based on the global histogram, obtaining multiple slopes; wherein, each slope is proportional to the brightness distribution probability corresponding to that pixel value range; determining the global tone mapping model based on the multiple slopes; and / or, the segmentation module 203 is specifically used for: for each local image, based on... A local histogram is constructed based on the local image and the segmentation method; wherein, the local histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range, the brightness distribution probability corresponding to each pixel value range refers to the probability that a pixel value in the local image falls within that pixel value range; the slope of the linear function of the global tone mapping model on each pixel value range is determined based on the global histogram, resulting in multiple slopes; wherein, the multiple slopes minimize the mean square error between the contrast of the local image and the contrast of the locally mapped image, the locally mapped image being the image obtained after mapping the local image based on the local mapping model; the local tone mapping model is determined based on the multiple slopes.

[0131] In the embodiments of this application, during the process of establishing a global tone mapping model, the uniformity of the overall brightness distribution of the image can be considered, so that the tone mapping of the image to be adjusted using the global tone mapping model can ensure the uniformity of the overall brightness distribution of the target image; during the process of establishing a local tone mapping model, the contrast of the local image can be preserved, so that the tone mapping of the image to be processed using the local tone mapping model can ensure that the contrast of the target image is high.

[0132] Furthermore, the segmentation module 203 is specifically used to: segment the image to be adjusted by means of a sliding window to obtain the multiple local images.

[0133] In this embodiment of the application, the image to be adjusted can be segmented by an active window. The adjacent local images obtained by the segmentation may overlap, so that a local tone mapping model and a global tone mapping model can be established based on the segmented images to map the image tone.

[0134] Furthermore, there is overlap between two adjacent local images in the image to be adjusted.

[0135] In this embodiment of the application, the image to be adjusted can be segmented by using an active window. The adjacent local images obtained by segmentation may overlap, thereby improving the uniformity of brightness distribution among the local images.

[0136] Furthermore, the area of ​​the overlapping portion between two adjacent local images in the image to be adjusted is half the area of ​​the local image.

[0137] Please refer to Figure 3 , Figure 3 This application provides a structural block diagram of an electronic device 300, which includes at least one processor 301, at least one communication interface 302, at least one memory 303, and at least one communication bus 304. The communication bus 304 enables direct communication between these components, the communication interface 302 facilitates signaling or data communication with other node devices, and the memory 303 stores machine-readable instructions executable by the processor 301. When the electronic device 300 is running, the processor 301 communicates with the memory 303 via the communication bus 304, and the aforementioned tone mapping method is executed when the machine-readable instructions are invoked by the processor 301.

[0138] For example, the processor 301 in this embodiment of the application can read a computer program from the memory 303 via the communication bus 304 and execute the computer program to implement the following method: Step S101: Obtain the image to be adjusted. Step S102: Establish a global tone mapping model corresponding to the image to be adjusted. Step S103: Perform image segmentation on the image to be adjusted to obtain multiple local images, and establish multiple local tone mapping models corresponding to the multiple local images. Step S104: Fuse the global tone mapping model with each local tone mapping model respectively to obtain multiple mixed tone mapping models. Step S105: For each local image, map it to the corresponding local target image using the corresponding mixed tone mapping model. Step S106: Fuse the multiple local target images to obtain the target image.

[0139] The processor 301 includes one or more, which can be an integrated circuit chip with signal processing capabilities. The processor 301 can be a general-purpose processor, including a Central Processing Unit (CPU), a Microcontroller Unit (MCU), a Network Processor (NP), or other conventional processors; it can also be a special-purpose processor, including a Neural-network Processing Unit (NPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Furthermore, when there are multiple processors 610, some can be general-purpose processors, and others can be special-purpose processors.

[0140] The memory 303 includes one or more, which may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0141] Understandable. Figure 3 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown. Figure 3 The components shown can be implemented using hardware, software, or a combination thereof. In the embodiments of this application, electronic device 300 can be, but is not limited to, physical devices such as desktop computers, laptops, smartphones, smart wearable devices, and in-vehicle devices, or virtual devices such as virtual machines. Furthermore, electronic device 300 is not necessarily a single device; it can be a combination of multiple devices, such as a server cluster, etc.

[0142] This application also provides a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes computer program instructions. When the computer program instructions are executed by a computer, the computer can perform the steps of the tone mapping method described in the above embodiments, such as: acquiring an image to be adjusted; wherein the range of pixel values ​​in the image to be adjusted is a first range, and the range of pixel values ​​in the target image corresponding to the image to be adjusted is a second range, the first range being greater than the second range; establishing a global tone mapping model corresponding to the image to be adjusted; performing image segmentation on the image to be adjusted to obtain multiple local images, and establishing multiple local tone mapping models corresponding to the multiple local images; wherein both the global tone mapping model and the local tone mapping model are used to map pixel values ​​within the first range to pixel values ​​within the second range; fusing the global tone mapping model with each local tone mapping model to obtain multiple mixed tone mapping models; for each local image, mapping it to a corresponding local target image using the corresponding mixed tone mapping model; and fusing the multiple local target images to obtain the target image.

[0143] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0144] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0146] It should be noted that if the function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0147] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0148] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A tone mapping method, characterized in that, include: Obtain the image to be adjusted; wherein the range of pixel values ​​in the image to be adjusted is a first range, and the range of pixel values ​​in the target image corresponding to the image to be adjusted is a second range, and the first range is greater than the second range. Establish a global tone mapping model corresponding to the image to be adjusted; The image to be adjusted is segmented to obtain multiple local images, and multiple local tone mapping models corresponding to the multiple local images are established; The global tone mapping model is fused with each of the local tone mapping models to obtain multiple mixed tone mapping models; For each local image, the corresponding blending tone mapping model is used to map it into the corresponding local target image; The target image is obtained by fusing multiple local target images; Wherein, both the global tone mapping model and the local tone mapping model are piecewise linear functions, and the global tone mapping model and the local tone mapping model have the same segmentation method for the first value range; The establishment of a global tone mapping model corresponding to the image to be adjusted includes: A global histogram is constructed based on the image to be adjusted and the segmentation method; wherein, the global histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range, and the brightness distribution probability corresponding to each pixel value range refers to the probability that the pixel value in the image to be adjusted falls into that pixel range; The slope of the linear function of the global tone mapping model in each pixel value range is determined based on the global histogram, resulting in multiple slopes; each slope is proportional to the luminance distribution probability corresponding to that pixel value range. The global tone mapping model is determined based on the multiple slopes; The establishment of multiple local tone mapping models corresponding to the multiple local images includes: For each local image, a local histogram is constructed based on the local image and the segmentation method; wherein, the local histogram includes multiple pixel value ranges and the brightness distribution probability corresponding to each pixel value range, and the brightness distribution probability corresponding to each pixel value range refers to the probability that the pixel value in the local image falls into that pixel value range; The slope of the linear function of the local tone mapping model in each pixel value range is determined based on the local histogram, resulting in multiple slopes. Among these slopes, the mean square error between the contrast of the local image and the contrast of the locally mapped image is minimized. The locally mapped image is the image obtained by mapping the local image based on the local tone mapping model. The local tone mapping model is determined based on the multiple slopes.

2. The tone mapping method according to claim 1, characterized in that, The global tone mapping model and the local tone mapping model have the same structure.

3. The tone mapping method according to claim 1, characterized in that, The global tone mapping model is fused with each local tone mapping model to obtain multiple mixed tone mapping models, including: For each linear function segment in the global tone mapping model, the linear function segment is fused with the corresponding linear function segment in the local tone mapping model to obtain the corresponding linear function segment in the mixed tone mapping model.

4. The tone mapping method according to any one of claims 1-3, characterized in that, The step of segmenting the image to be adjusted into multiple local images includes: The image to be adjusted is segmented by using a sliding window method to obtain multiple local images.

5. The tone mapping method according to claim 4, characterized in that, There is an overlap between the two local images obtained by segmenting adjacent regions from the image to be adjusted.

6. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-5.

7. An electronic device, characterized in that, include: Processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, and the processor can invoke the computer program instructions to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-5.

Images