Video inverse tone mapping method and system, terminal equipment and medium

Abstract

The invention discloses a video inverse tone mapping method and system, terminal equipment and a medium, and relates to the technical field of video image processing. The method comprises the following steps: converting a video frame SDR image to be reconstructed from an RGB color space to an ICtCp color space, and decoupling the video frame SDR image into a brightness channel and a chrominance channel; performing brightness hierarchical mapping processing on the brightness channel to obtain an HDR image brightness channel after dynamic range expansion; performing chromaticity mapping processing on the chromaticity channel to obtain an HDR image chromaticity channel after color gamut expansion; and combining the two channels and converting the two channels to an RGB color space to obtain a reconstructed HDR image. Brightness and chromaticity decoupling is realized based on the ICtCp space, light weight and processing precision are considered, end side deployment is adapted, the dynamic range and color gamut of the video are effectively expanded, highlight and dark part details are recovered, and the visual effect is improved.

Description

A video inverse tone mapping method, system, terminal device and medium Technical Field

[0001] This invention relates to the field of video image processing technology, and in particular to a video inverse tone mapping method, system, terminal device, and medium. Background Technology

[0002] High Dynamic Range (HDR) video, with its wider dynamic range, broader color gamut, and richer details, has become a development trend in the video display field. However, mainstream video content is still mainly based on Standard Dynamic Range (SDR), and there is an urgent need for Inverse Tone Mapping (ITM) technology to convert SDR to HDR.

[0003] Existing ITM methods mainly fall into two categories. One category is mapping methods based on traditional look-up tables (LUTs), which are lightweight but have weak generalization ability and are prone to quantization artifacts. The other category is deep learning-driven methods, which can improve conversion accuracy, but the models are complex, computationally expensive, and difficult to adapt to edge device deployment. In addition, most methods do not achieve effective decoupling of luminance and chrominance processing, resulting in asynchronous dynamic range expansion and color gamut expansion, leading to problems such as loss of highlight or shadow details and color distortion.

[0004] Therefore, there is an urgent need for a video inverse tone mapping method that balances lightweight deployment with conversion accuracy to fill the gap in existing technologies. Summary of the Invention

[0005] The technical problem this invention aims to solve is that, in the field of video image processing, existing methods struggle to simultaneously achieve effective expansion of dynamic range and color gamut, accurate restoration of highlights and shadow details, and cannot meet the low-computing power deployment requirements of edge devices, resulting in poor HDR reconstruction effects or difficulty in practical application. Therefore, an effective solution is urgently needed to address these technical problems.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: Firstly, the present invention provides a video inverse tone mapping method, the method comprising: converting the SDR image of the video frame to be reconstructed from the RGB color space to the IC color space. t C pA color space is used to decouple luminance and chrominance into luminance and chrominance channels. The decoupled luminance channels are subjected to luminance layer mapping to obtain the luminance channel of the HDR image with extended dynamic range. The decoupled chrominance channels are subjected to chrominance mapping to obtain the chrominance channel of the HDR image with extended color gamut. The luminance and chrominance channels of the HDR image are merged and converted to the RGB color space to obtain the reconstructed HDR image.

[0007] In one implementation, the step of performing luminance layer mapping processing on the decoupled luminance channels to obtain the luminance channels of the dynamically extended HDR image includes: downsampling the decoupled luminance channels; segmenting the downsampled luminance channels and performing a masking operation to obtain a luminance information map; simultaneously inputting the luminance information map into a layered luminance processing network and a pixel encoder to obtain a luminance channel set of layered luminance and a weight map corresponding to each pixel; and multiplying the luminance channel set of layered luminance and the weight map corresponding to each pixel to obtain the luminance channels of the dynamically extended HDR image.

[0008] In one implementation, the step of segmenting and masking the downsampled luminance channel to obtain a luminance information map includes: inputting the downsampled luminance channel into a threshold learning network to obtain a bright area threshold and a dark area threshold; and based on the bright area threshold and the dark area threshold, segmenting and masking the downsampled luminance channel to obtain a luminance information map; wherein the luminance information map is a three-channel information map, which consists of a luminance channel in the bright area, a luminance channel of the original luminance, and a luminance channel in the dark area.

[0009] In one implementation, the step of simultaneously inputting the luminance information map into a hierarchical luminance processing network and a pixel encoder to obtain a luminance channel set and a weight map corresponding to each pixel of the hierarchical luminance includes: inputting the three-channel information map into the hierarchical luminance processing network to obtain a 1DLUT set; performing a lookup table operation on the 1DLUT set for the luminance channels at the original resolution to obtain a luminance channel set for the hierarchical luminance; and inputting the three-channel information map into the pixel encoder to obtain a weight map corresponding to each pixel of the hierarchical luminance channels.

[0010] In one implementation, the hierarchical luminance processing network includes three parallel feature extraction networks and three corresponding LUT generators. The feature extraction networks are used to extract context information of different luminance regions in the three-channel information map, and the LUT generators are used to generate LUTs of different luminances based on the context information and combine them to form the 1DLUT set.

[0011] In one implementation, the pixel encoder has a symmetrical structure and includes a feature extraction module, a downsampling feature extraction module, and an upsampling feature fusion module. The downsampling feature extraction module includes an average pooling layer and a feature extraction module. The upsampling feature fusion module includes a transposed convolutional block, a channel concatenation block, and a feature extraction module. The step of inputting the three-channel information map into the pixel encoder to obtain a weight map corresponding to each pixel of the layered brightness channel includes: the three-channel information map undergoes downsampling once by the feature extraction module and downsampling three times by the downsampling feature extraction module to obtain an encoded feature map; the encoded feature map undergoes upsampling processing four times by the upsampling feature fusion module, and during each upsampling process, the feature map from the downsampling process is concatenated with the feature map from the downsampling process through the channel concatenation block; and the upsampling output is restored to the original resolution through linear interpolation to obtain the weight map corresponding to each pixel.

[0012] In one implementation, the step of performing chroma mapping processing on the decoupled chroma channels to obtain the chroma channels of the HDR image with extended color gamut includes: downsampling the decoupled chroma channels; extracting chroma context information of the downsampled chroma channels using a chroma feature extraction network; generating a 2DLUT for the chroma channels based on the chroma context information using a 2DLUT generator; and performing a lookup operation on the 2DLUT for the chroma channels at the original resolution to obtain the chroma channels of the HDR image with extended color gamut.

[0013] Secondly, embodiments of the present invention also provide a video inverse tone mapping system, the system comprising: a luminance and chrominance decoupling module, used to convert the SDR image of the video frame to be reconstructed from the RGB color space to the IC color space. t C p The system uses a color space to decouple luminance and chrominance into luminance and chrominance channels; a luminance layer mapping processing module performs luminance layer mapping processing on the decoupled luminance channels to obtain the luminance channel of the HDR image with expanded dynamic range; a chrominance mapping processing module performs chrominance mapping processing on the decoupled chrominance channels to obtain the chrominance channel of the HDR image with expanded color gamut; and a merging and conversion module merges the luminance and chrominance channels of the HDR image and converts them to the RGB color space to obtain the reconstructed HDR image.

[0014] Thirdly, embodiments of the present invention also provide a terminal device, the terminal device including a memory, a processor, and a video inversion tone mapping program stored in the memory and executable on the processor, wherein when the processor executes the video inversion tone mapping program, it implements the steps of the video inversion tone mapping method described in any of the above schemes.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a video inversion tone mapping program, wherein when the video inversion tone mapping program is executed by a processor, it implements the steps of the video inversion tone mapping method described in any of the above schemes.

[0016] Beneficial Effects: This invention discloses a video inverse tone mapping method, system, terminal device, and medium, relating to the field of video image processing technology. The method first converts the SDR image of the video frame to be reconstructed from the RGB color space to IC color space. t C p A color space is used to decouple luminance and chrominance into luminance and chrominance channels. Then, luminance layer mapping is performed on the decoupled luminance channel to obtain the luminance channel of the HDR image with expanded dynamic range, and chrominance mapping is performed on the decoupled chrominance channel to obtain the chrominance channel of the HDR image with expanded color gamut. Finally, the luminance and chrominance channels of the HDR image are merged and converted to the RGB color space to obtain the reconstructed HDR image. This invention utilizes IC... t C p Spatial decoupling of luminance and chrominance, and channel-specific mapping enable simultaneous adaptation of dynamic range expansion and color gamut expansion, effectively restoring highlights and shadow details in video and avoiding color distortion. Simultaneously, the lightweight design of the layered luminance processing network and lookup table mapping significantly reduces computational power consumption, adapting to edge device deployments and enabling real-time HDR reconstruction of high-definition video, balancing processing accuracy and practicality, and achieving a balance between reconstruction quality and required computational power. Attached Figure Description

[0017] Figure 1 is a flowchart of a specific implementation of the video inverse tone mapping method provided in an embodiment of the present invention.

[0018] Figure 2 is a schematic diagram of the overall algorithm framework of the video inverse tone mapping method provided in the embodiment of the present invention.

[0019] Figure 3 is a schematic diagram of the HLPN module structure of the video inverse tone mapping method provided in an embodiment of the present invention.

[0020] Figure 4 is a schematic diagram of the PixelDecoder module structure of the video inverse tone mapping method provided in an embodiment of the present invention.

[0021] Figure 5 is a schematic diagram of the video inverse tone mapping device provided in an embodiment of the present invention.

[0022] Figure 6 is a block diagram illustrating the internal structure of the terminal device provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0024] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0025] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, "first control information" and "second control information" are only used to distinguish different control information and do not limit their order.

[0027] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0028] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] In the field of video image processing technology, visual signals are an important carrier for humans to obtain natural information. Video quality mainly depends on two dimensions: color gamut breadth and dynamic range. Color gamut breadth determines the color coverage of the imaging system, affecting the richness and accuracy of image colors, while dynamic range determines the light intensity distribution range of the imaging system, affecting the image's brightness and contrast. High Dynamic Range (HDR) video, possessing both high dynamic range and wide color gamut characteristics, typically boasts a maximum brightness of up to 1000 cd / m², and its color gamut usually conforms to the ITU-R BT.2020 standard. It can present visual effects that more closely resemble real-world scenes and has become an important development direction for video display technology.

[0030] However, due to limitations in equipment cost and distribution conditions, mainstream digital video content is currently stored and distributed in Standard Dynamic Range (SDR) format, with its brightness range typically limited to around 100 cd / m² and its color gamut usually following the Rec.709 standard. This limitation results in severe compression or even loss of detail information in bright or high-contrast scenes. To address this issue, Inverse Tone Mapping (ITM) technology has emerged. Its goal is to reconstruct SDR images or video content into signals that conform to HDR display characteristics while maintaining the original visual style and color consistency, thus restoring and expanding its brightness dynamic range and color gamut coverage.

[0031] Existing ITM technologies are mainly divided into two categories. One category is based on end-to-end deep learning methods. These methods often employ black-box regression strategies. Some solutions jointly model super-resolution (SR) and inverse tone mapping (ITM), using information from multiple adjacent frames to compensate for overexposed areas and restore HDR details. Other solutions improve image detail reproduction and color naturalness through a three-level cascaded network structure consisting of global color mapping, local detail enhancement, and specular consistency modeling. However, these methods generally suffer from high model complexity and high computational requirements, making them difficult to implement in edge devices such as televisions.

[0032] Another category is based on lightweight look-up tables (LUTs). Some schemes predict and dynamically fuse multiple LUT weights using lightweight convolutional neural networks (CNNs); others learn adaptive-interval 3D LUTs to improve color reproduction in real-time image enhancement through non-uniform sampling; still others design separable image adaptive LUT frameworks, combining 1DLUTs (One-Dimensional Look-Up Tables) and 3DLUTs (Three-Dimensional Look-Up Tables) for combined enhancement guided by content features. However, these low-computational-power lightweight methods have not been fully developed in the field of inverse tone mapping. Even if some schemes reallocate the quantization precision of 3DLUTs in the color gamut and luminance dimensions based on the adaptive LUT concept, the low-computational-power advantage is not preserved due to the deep neural network structure design, leading to difficulties in edge applications.

[0033] Furthermore, existing ITM methods generally do not effectively decouple the luminance and chrominance information of video images, making it difficult to accurately adapt dynamic range expansion and color gamut expansion to the information characteristics of different channels. This can easily lead to problems such as overexposure in bright areas, loss of detail in dark areas, and distortion of color and luminance matching, making it impossible to take into account the dynamic range, color performance, and detail reproduction of HDR videos.

[0034] Therefore, existing methods in the field of video inverse tone mapping (ITM) technology suffer from two drawbacks. One is that deep learning methods require high computational power and are difficult to deploy on edge devices, while lightweight LUT methods are underdeveloped and lack adaptability in the ITM field. The second drawback is the failure to effectively decouple luminance and chrominance, making it difficult to simultaneously optimize dynamic range expansion, color gamut expansion, and detail restoration during HDR reconstruction. Ultimately, existing technologies struggle to simultaneously meet the dual requirements of effective dynamic range expansion, accurate color gamut adaptation, complete restoration of brightness and darkness details, and low-computational-power deployment and real-time processing on edge devices. Consequently, the effectiveness of HDR video reconstruction and its practical applicability cannot be reconciled. Therefore, an effective solution is urgently needed to address these technical problems.

[0035] This embodiment provides a video inverse tone mapping method, as shown in Figure 1, which specifically includes the following steps: Step S100: Convert the SDR image of the video frame to be reconstructed from the RGB color space to the IC color space. t C p Color space is used to decouple luminance and chrominance into luminance channels and chrominance channels.

[0036] In this embodiment, the RGB color space is the mainstream color space for image display and storage. It presents various colors by combining the brightness of the three primary color channels: red, green, and blue. Specifically, it can be the sRGB standard commonly used in electronic devices. t C p The (Intensity, Chrominance-t, Chrominance-p) color space is designed for High Dynamic Range (HDR) displays. It effectively separates luminance and chrominance information, facilitating separate optimization. The luminance channel (I-channel) is the IC... t C p The channels in space that express the brightness and darkness of an image determine the brightness levels and contrast of the image. The chroma channel (TP channel) is composed of the T channel and the P channel, which together express the color information of the image. The T channel corresponds to hue-related information, and the P channel corresponds to saturation-related information.

[0037] As shown in Figure 2, the algorithm framework first converts the input SDR video frame from the RGB color space to the IC color space. t C p Color space. In this embodiment, 4K resolution (3840×2160) video frames are used. Due to IC t C p The color space itself is designed for HDR content and does not have a direct conversion path for Standard Dynamic Range (SDR) images. Therefore, a preprocessing step needs to be performed before color space conversion.

[0038] Specifically, firstly, the non-linear coded RGB values ​​of the SDR image are converted to linear luminance values ​​using Gamma 2.4 mapping. Gamma 2.4 mapping is a commonly used non-linear to linear luminance conversion method in the imaging field, used to restore the true light intensity distribution of the image. Next, the linear RGB values ​​of the Rec.709 color gamut are converted to the Rec.2020 color gamut. The Rec.2020 color gamut is a commonly used color gamut standard for HDR content, with a coverage much larger than the Rec.709 color gamut. Finally, the luminance values ​​are amplified according to the SDR luminance standard to ensure that the converted data range is compatible with the IC. t C p Space handling requirements.

[0039] The above preprocessing steps are represented by Equations 1, 2, and 3: Equation 1:

[0040] in, This represents the non-linear encoded RGB value under the Rec.709 color gamut. This indicates that a Gamma2.4 mapping transformation will be performed. This represents the linearly encoded RGB value under the Rec.709 color gamut.

[0041] Formula 2:

[0042] in, , , These represent the linear brightness values ​​of the red, green, and blue channels in the corresponding color gamut. The subscript 2020 indicates the Rec.2020 color gamut, and the subscript 709 indicates the Rec.709 color gamut.

[0043] Formula 3:

[0044] in, This represents the linearly encoded RGB value under the Rec.2020 color gamut after brightness amplification. Multiplying by 100 is to adjust the brightness range according to the SDR brightness standard for IC compatibility. t C p Spatial transformation.

[0045] After completing the preprocessing, the result is According to IC t C p A color space conversion standard that converts linear RGB values ​​of the Rec.2020 color gamut to IC. t C p Spatial data enables the decoupling of the luminance and chrominance channels.

[0046] The separated brightness and chromaticity can be optimized in a targeted manner to avoid mutual interference, providing a foundation for dynamic range expansion and color gamut expansion, ensuring that color consistency is not destroyed when brightness is increased, and brightness levels are not affected when color is optimized.

[0047] Step S200: Perform luminance layer mapping processing on the decoupled luminance channel to obtain the luminance channel of the HDR image after dynamic range extension.

[0048] In this embodiment, luminance layer mapping processing is a targeted optimization operation for the luminance channel. Based on the luminance characteristics of different regions of the image, it performs differentiated dynamic range expansion, specifically by processing dark areas, normal luminance, and bright areas separately. Dynamic range expansion refers to increasing the limited luminance range of an SDR image to the wide luminance range of an HDR image while preserving the original details of the image's brightness and darkness.

[0049] Specifically, the luminance channel is first pre-processed to divide it into different luminance levels. Then, an appropriate mapping strategy is selected based on the characteristics of each level to avoid the loss of details in dark areas or overexposure in bright areas caused by traditional global mapping. Through layered processing, the expansion needs of different luminance areas are precisely matched. The luminance of dark areas can be appropriately increased to restore details, the luminance increase of bright areas can be reasonably controlled to avoid overexposure, and the original visual style is maintained in normal luminance areas.

[0050] Compared to traditional single mapping strategies, this processing method can more accurately restore the details of the image in different brightness areas, making the expanded HDR image richer in brightness and darkness, more natural in contrast, while maintaining the visual consistency of the original image and avoiding distortion problems such as overall brightness or darkness.

[0051] In one implementation, the step of performing luminance layer mapping processing on the decoupled luminance channels to obtain the luminance channels of the dynamically extended HDR image specifically includes the following steps: Step S210, downsampling the decoupled luminance channels; Step S220, segmenting the downsampled luminance channels and performing a masking operation to obtain a luminance information map; Step S230, simultaneously inputting the luminance information map into the layered luminance processing network and the pixel encoder to obtain the luminance channel set of the layered luminance and the weight map corresponding to each pixel; Step S240, multiplying the luminance channel set of the layered luminance and the weight map corresponding to each pixel to obtain the luminance channels of the dynamically extended HDR image.

[0052] In this embodiment, as shown in Figure 2, the luminance channel is first downsampled. Downsampling is a process that reduces the image resolution to decrease the computational cost of subsequent network operations and adapt to the computational limitations of the edge device. In this embodiment, the specific downsampling method can be to reduce the original 4K resolution (3840×2160) luminance channel to a resolution of 236×256. The downsampling process can use a mean sampling algorithm to ensure that the overall distribution of luminance information is not distorted. The 236×256 resolution is chosen because it can meet the feature extraction requirements of the subsequent network and keep the computational cost within the capacity of the edge device. This embodiment is applied to the MediaTek MTK9655 platform, which only supports convolution operations on video frames with a resolution of 512 or lower.

[0053] Subsequently, the downsampled luminance channels are segmented and masked to obtain a luminance information map. The segmentation operation divides the downsampled luminance channels into different regions based on a luminance threshold, while the masking operation selectively preserves and enhances information in each region, ultimately integrating them to form the luminance information map. Threshold segmentation enables the identification of luminance regions, and the masking operation highlights the luminance characteristics of each region, providing a clear processing target for subsequent layered mapping. The luminance information map is integrated multi-channel data that can simultaneously carry feature information from different luminance regions, facilitating parallel processing by the subsequent network.

[0054] Subsequently, the luminance information map is simultaneously input into the Hierarchical Luminance Processing Network (HLPN) and the pixel encoder to obtain the luminance channel set and the corresponding weight map for each pixel of the hierarchical luminance. The HLPN is a network specifically designed to generate luminance mapping rules; its function is to generate a targeted look-up table (LUT) based on the regional features of the luminance information map. The pixel encoder is a network used to generate fusion weights, which can assign appropriate fusion weights to each pixel based on its local features.

[0055] By simultaneously inputting the luminance information map into two networks, parallel processing of mapping rule generation and fusion weight calculation can be achieved, improving overall computational efficiency. The luminance channel set of the layered luminance is a set of luminance channels processed by different mapping rules at the original resolution. Each channel corresponds to the mapping result of a luminance region, while the weight map corresponding to each pixel is weight data consistent with the original resolution, used to control the contribution ratio of each mapping result in the final output.

[0056] Finally, the luminance channel set of the layered luminance is multiplied by the corresponding weight map for each pixel to obtain the luminance channel of the HDR image after dynamic range extension. The multiplication operation can be a pixel-by-pixel element-wise multiplication, where the pixel value of each channel in the layered luminance channel set is multiplied by the corresponding weight value in the weight map, and all multiplication results are summed to obtain the final luminance value. The results of different mapping rules are adaptively fused according to pixel-level weights to ensure that each pixel receives the most suitable luminance mapping effect.

[0057] By employing a process of downsampling to reduce computational power, segmentation masking to extract features, parallel processing with dual networks, and pixel-level fusion, accurate hierarchical mapping of brightness is achieved while controlling computational power consumption. The downsampling step reduces the computational power requirements of intermediate processing, making it suitable for edge deployment; the segmentation masking step ensures accurate identification of brightness regions; parallel processing with dual networks improves computational efficiency; and pixel-level fusion ensures optimal brightness mapping for each pixel, ultimately achieving effective expansion of the dynamic range while preserving rich details of light and dark areas.

[0058] In one implementation, the step of segmenting and masking the downsampled luminance channel to obtain a luminance information map specifically includes the following steps: Step S221: Input the downsampled luminance channel into a threshold learning network to obtain a bright area threshold and a dark area threshold; Step S222: Based on the bright area threshold and the dark area threshold, segment and mask the downsampled luminance channel to obtain a luminance information map; wherein, the luminance information map is a three-channel information map, which consists of a luminance channel in the bright area, a luminance channel in the original luminance, and a luminance channel in the dark area.

[0059] In this embodiment, the downsampled luminance channel is first input into a threshold learning network to obtain the bright and dark thresholds. The threshold learning network is a lightweight network specifically designed for adaptively learning luminance thresholds. Its function is to automatically learn the critical values ​​that distinguish between dark areas and normal luminance, and between normal luminance and bright areas, based on the distribution of the input luminance channel data. This network randomly initializes learnable parameters and continuously optimizes them through a training process, ultimately outputting the dark threshold. and brightness threshold These are used to guide the segmentation of dark and bright areas, respectively.

[0060] The threshold generation process is represented as follows:

[0061]

[0062]

[0063]

[0064] in, and These are randomly generated initial learnable parameters from the network. Their initial values ​​are randomly set before training begins and dynamically adjusted according to the loss function during training. This represents the theoretical maximum value of the brightness channel of the input SDR image, i.e., the upper limit of the brightness of the SDR image; and It is a weighting factor used to control and The range of variation of the threshold is constrained by the sigmoid activation function to ensure that the threshold does not exceed a reasonable range, avoid oversegmentation or undersegmentation, and make threshold learning more stable.

[0065] Dark threshold This is the threshold value that distinguishes dark areas from normal brightness areas; areas with brightness below this value are considered dark areas; the brightness threshold is... The threshold value that distinguishes bright areas from normal brightness areas; areas with brightness higher than this value are considered bright areas.

[0066] Subsequently, based on the brightness and darkness thresholds, the downsampled brightness channel is segmented and masked to obtain a brightness information map. Specifically, the segmentation operation is based on... and The luminance channel is divided into three regions: dark region, normal luminance region, and bright region. The masking operation then enhances the information in these three regions through a smooth transition of weight allocation, avoiding abrupt changes in the segmentation boundaries. The specific mathematical expression is as follows:

[0067]

[0068]

[0069]

[0070] in, and These are the mask weights for dark and bright areas, respectively; This represents the raw value of the brightness channel of the input SDR image; This is a hyperparameter used to control the smooth transition range of mask weights, and can be adjusted according to the actual scenario; This represents the theoretical minimum value of the luminance channel space of the input SDR image, i.e., the lower limit of the luminance of the SDR image. and These are the brightness channel values ​​for the dark and bright areas after the masking operation.

[0071] The final luminance information map is a three-channel map, consisting of a highlight luminance channel, a raw luminance channel, and a dark area luminance channel. The highlight luminance channel carries the luminance information of the bright areas after masking enhancement, the dark area luminance channel carries the luminance information of the dark areas after masking enhancement, and the raw luminance channel carries the complete luminance information before region segmentation. These three channels are combined for subsequent layering processing.

[0072] The adaptive threshold learning method achieves accurate segmentation of brightness regions, avoiding the problem that fixed thresholds are difficult to adapt to different images. Furthermore, the smooth mask operation avoids boundary distortion caused by region segmentation, ensuring the naturalness of brightness transition. Meanwhile, the three-channel information map provides a clear and comprehensive processing object for subsequent layer mapping, laying the foundation for targeted optimization of different brightness regions.

[0073] In one implementation, the step of simultaneously inputting the luminance information map into a hierarchical luminance processing network and a pixel encoder to obtain the luminance channel set and the weight map corresponding to each pixel of the hierarchical luminance is specifically included in the following steps: Step S231: Input the three-channel information map into the hierarchical luminance processing network to obtain a 1DLUT set; Step S232: Perform a lookup operation on the 1DLUT set for the luminance channels at the original resolution to obtain the luminance channel set of the hierarchical luminance; Step S233: Input the three-channel information map into the pixel encoder to obtain the weight map corresponding to each pixel of the hierarchical luminance channels.

[0074] In this embodiment, the three-channel information map is first input into the hierarchical luminance processing network (HLPN) to obtain a 1DLUT set. Specifically, the three-channel information map contains feature information from three channels: highlight area, original luminance, and dark area. After inputting it into the hierarchical luminance processing network (HLPN), the network extracts and analyzes the features of each channel separately. The structure of the HLPN can include three parallel feature extraction networks and three corresponding LUT generators. The three feature extraction networks correspond to the three channels of the three-channel information map, respectively, and are specifically designed to extract the contextual information of each luminance region. This contextual information includes features such as the luminance distribution of pixels within the region and the luminance correlation of adjacent pixels, which can comprehensively reflect the luminance characteristics of the region.

[0075] Three LUT generators each receive contextual information from the corresponding feature extraction network output and generate a targeted One-Dimensional Look-Up Table (1DLUT) based on this information. A 1DLUT is a data structure that stores mapping relationships, quickly mapping input luminance values ​​to output luminance values ​​through the correspondence between indices and output values. The 1DLUT generated for the highlight channel optimizes the dynamic range of bright areas, the 1DLUT generated for the dark channel optimizes dark details, and the 1DLUT generated for the original luminance channel ensures overall luminance balance.

[0076] Three 1DLUTs are combined to form a 1DLUT set, which covers the adaptation mapping rules for different brightness regions, providing a mapping basis for lookup table operations. The parallel structure of HLPN ensures that feature extraction and LUT generation for the three channels are performed synchronously, improving computational efficiency. At the same time, targeted LUT generation ensures that the mapping rules for each brightness region can accurately match its features.

[0077] Subsequently, a lookup operation is performed on the 1DLUT set to obtain a luminance channel set with layered luminance. Specifically, the luminance channel at the original resolution is the complete luminance data without downsampling, preserving all the detailed information of the image. The lookup operation refers to using the luminance value of each pixel in the channel as an index to find the corresponding output value in the three 1DLUTs in the 1DLUT set, resulting in three different luminance channels: a highlight mapping channel based on the bright area 1DLUT, a dark area mapping channel based on the dark area 1DLUT, and a normal luminance mapping channel based on the original luminance 1DLUT.

[0078] These three channels together form a luminance channel set for layered luminance, with each channel corresponding to a luminance mapping strategy. The advantage of table lookup is its fast computation speed; it eliminates the need for complex real-time calculations, completing the mapping simply through index lookup, effectively reducing computational power consumption and adapting to the real-time processing requirements of edge devices. Furthermore, since each 1DLUT is optimized for a specific luminance region, the channels obtained after table lookup accurately enhance the luminance characteristics of the corresponding region: high-brightness mapping channels improve the dynamic range of bright areas, dark-brightness mapping channels restore details in dark areas, and normal-brightness mapping channels maintain the naturalness of midtones.

[0079] Finally, the three-channel information map is input into the pixel encoder to obtain a weight map corresponding to each pixel of the layered brightness channels. The pixel encoder is a symmetric network specifically designed to generate pixel-by-pixel fusion weights. It extracts deep features from the three-channel information map through the encoding process and then maps these features back to the original resolution through the decoding process, generating weight data corresponding to each layered brightness channel. Each pixel value in the weight map represents the contribution ratio of each layered brightness channel to the corresponding pixel during fusion. For example, if a pixel has a high weight value for its bright areas, the result of the high-brightness mapping channel will be adopted more during fusion.

[0080] The pixel encoder takes a three-channel information map as input, which can simultaneously acquire feature information from different brightness regions. Based on this information, it determines the brightness attribute of each pixel—whether it belongs to a dark area, normal brightness, or bright area—and its specific features in that area, and then assigns appropriate weights. The generated weight map is consistent with the original resolution and contains three weight channels, corresponding to the three channels in the brightness channel set of the layered brightness, ensuring that each layered channel has a corresponding fusion weight.

[0081] By employing targeted LUT generation, rapid lookup table mapping, and pixel-level weight allocation, accuracy and efficiency in luminance mapping are achieved. The targeted generation of the 1DLUT set ensures the adaptation of mapping rules for different luminance regions, the lookup table operation guarantees real-time processing capabilities, and the weight map generated by the pixel encoder provides precise pixel-level control for subsequent fusion. The synergistic effect of these three elements allows the luminance channel set of layered luminance to retain the optimization effects of each region while achieving overall brightness balance and coordination through subsequent fusion.

[0082] In one implementation, the hierarchical luminance processing network includes three parallel feature extraction networks and three corresponding LUT generators. The feature extraction networks are used to extract context information of different luminance regions in the three-channel information map, and the LUT generators are used to generate LUTs of different luminances based on the context information and combine them to form the 1DLUT set.

[0083] In this embodiment, the Hierarchical Brightness Processing Network (HLPN) generates targeted mapping rules for different brightness regions through a parallel structure. Figure 3 illustrates the structure of the HLPN module. Specifically, the HLPN module contains three parallel feature extraction networks and three corresponding LUT generators, forming a symmetrical parallel architecture. The three feature extraction networks have the same structure, all referencing the feature extraction structure of the Separable Image-Adaptive Look-Up Tables (SepLUT) model, and can be simple 5-layer Convolutional Neural Networks (CNNs). This structure is lightweight yet possesses strong feature extraction capabilities, accurately capturing contextual information of different brightness regions while controlling computational power consumption.

[0084] The structure of the three LUT generators can also refer to the LUT generator design of the SepLUT model, which is specifically used to convert the context information output by the feature extraction network into a specific 1DLUT mapping relationship. Each feature extraction network and its corresponding LUT generator form an independent processing branch. The three branches correspond to the brightness channel of the highlight area, the brightness channel of the original brightness, and the brightness channel of the dark area of ​​the three-channel information map, respectively, and process the feature information of each channel in parallel to generate a targeted 1DLUT.

[0085] In HLPN, during feature extraction and LUT generation, the three channels of the three-channel infographic are first input into three processing branches. Each feature extraction network extracts features from the input channels layer by layer. Specifically, the first convolutional layer extracts basic brightness features, the middle three convolutional layers refine the feature representation, and the fifth convolutional layer outputs the final contextual information. This contextual information includes key features such as the pixel distribution pattern of the brightness region, brightness gradient changes, and relationships between adjacent regions, reflecting the brightness characteristics of that region.

[0086] Subsequently, after receiving context information, the LUT generator converts the feature vectors into 1DLUT mapping data through a fully connected layer. The 1DLUT has a length of 256, covering the complete range of SDR image brightness values. Each index corresponds to an input brightness value, and the stored value is the mapped output brightness value. A brightness-optimized 1DLUT is generated for the branch of the highlight channel to expand the dynamic range of highlights and avoid overexposure; a shadow-optimized 1DLUT is generated for the branch of the shadow channel to enhance shadow brightness and restore details; and a globally balanced 1DLUT is generated for the branch of the original brightness channel to ensure overall brightness consistency.

[0087] Both dark and bright LUTs can be generated using non-uniform sampling. By optimizing the distribution of sampling points, the mapping accuracy can be improved. Specifically, non-uniform sampling allocates more sampling points in bright areas with dense pixels and fewer sampling points in sparse areas, based on the pixel distribution probability of the bright region, making the mapping relationship more closely match the actual brightness distribution.

[0088] The generation of non-uniform sampling points can be represented as:

[0089]

[0090]

[0091]

[0092]

[0093] in, The range of the divided brightness area is determined by the dark area threshold or the bright area threshold; This indicates the size of the blocks after uniformly dividing the brightness range; Indicates falling on the 1st The number of pixels in each block; Indicates the first The statistical probability of each block reflects the density of pixels in that block; This represents the cumulative distribution function, reflecting brightness values ​​less than or equal to... The total probability of a pixel; N represents the number of sampling points of the LUT; Indicates the first The uniform probability of a sampling point is determined by the number of points in the LUT; This indicates that the probability on the left is closest. brightness point , Indicates the probability closest to the right. brightness point ; Let represent a linear interpolation function, and let be the probability obtained through linear interpolation. The brightness values ​​of the sampling points adopt a similar idea to histogram equalization, which distributes more sampling points in areas with a high number of brightness points and fewer sampling points in areas with a low number of brightness points, thus achieving non-uniform sampling under different brightness levels.

[0094] In the specific process of non-uniform sampling points, the pixel distribution probability of each block within the brightness area is first calculated. Calculate the cumulative distribution function Then generate uniformly distributed sampling point probabilities. ; through the inverse function Find and The corresponding brightness value, if If a direct match is not possible, linear interpolation is used. Calculate the accurate sampling point brightness Finally, based on these non-uniformly distributed sampling points, the mapping relationship of 1DLUT is generated.

[0095] The parallel structure of the HLPN module enables simultaneous processing of different brightness regions, improving computational efficiency. Referencing the lightweight network structure of SepLUT, it ensures low computational consumption and adaptability for edge deployment. Targeted LUT generation ensures that the mapping rules for each brightness region accurately match its features; the bright LUT optimizes the dynamic range, while the dark LUT restores details. The non-uniform sampling strategy further improves mapping accuracy, making the sampling points more closely match the actual pixel distribution and avoiding the problem of insufficient mapping accuracy in densely pixel areas caused by uniform sampling. The synergistic effect of these three elements results in a 1DLUT set that comprehensively covers the optimization needs of different brightness regions, providing a high-quality mapping basis for subsequent lookup operations.

[0096] In one implementation, the pixel encoder has a symmetrical structure and includes a feature extraction module, a downsampling feature extraction module, and an upsampling feature fusion module. The downsampling feature extraction module includes an average pooling layer and a feature extraction module. The upsampling feature fusion module includes a transposed convolutional block, a channel concatenation block, and a feature extraction module. The step of inputting the three-channel information map into the pixel encoder to obtain a weight map corresponding to each pixel of the layered brightness channel specifically includes the following steps: Step S2331: The three-channel information map undergoes downsampling by the feature extraction module once and downsampling by the feature extraction module three times to obtain an encoded feature map; Step S2332: The encoded feature map undergoes upsampling processing by the upsampling feature fusion module four times, and during each upsampling process, the feature map from the downsampling process is concatenated with the feature map from the downsampling process through the channel concatenation block; Step S2333: The upsampled output is restored to the original resolution through linear interpolation to obtain the weight map corresponding to each pixel.

[0097] In this embodiment, the pixel encoder is a network with a symmetrical structure. Its core function is to generate pixel-wise fusion weights based on a three-channel information map. The module structure of the pixel encoder is shown in Figure 4. The structure of the pixel encoder is similar to that of the U-Net network, exhibiting a symmetrical encoder-decoder architecture, including a feature extraction module, a downsampling feature extraction module, and an upsampling feature fusion module. In this embodiment, the feature extraction module uses the Ghost Module, the downsampling feature extraction module uses the Ghost ModuleDNN, and the upsampling feature fusion module uses the Ghost ModuleUNN. Among them, Ghost Module is a lightweight feature extraction component, similar in function to traditional CNN, but it reduces computational consumption by using grouped convolutions and feature recombination, significantly reducing floating-point operations (FLOPs) while maintaining feature extraction capabilities; Ghost ModuleDNN is the core component of the encoding stage, consisting of an average pooling layer and Ghost Module. The average pooling layer is used to reduce the resolution of the feature map and retain global features, while Ghost Module is used to extract deep features. The combination of the two achieves the encoding functions of dimensionality reduction and feature extraction; Ghost ModuleUNN is the core component of the decoding stage, consisting of a transposed convolutional block, a channel concatenation block, and Ghost Module. The transposed convolutional block is used to increase the resolution of the feature map, the channel concatenation block is used to fuse the feature maps corresponding to the encoding stage, and Ghost Module is used to refine the features. The combination of the three achieves the decoding functions of dimensionality increase, feature fusion, and detail extraction.

[0098] In the pixel encoder's encoding and decoding process, the input is first acquired and initialized. The input to the pixel encoder is a three-channel infographic with a resolution of 236×256 and 3 channels. After input, a Ghost Module downsampling is performed, reducing the resolution from 236×256 to 128×128 and increasing the number of channels from 3 to 16. Through the lightweight feature extraction of the Ghost Module, the features of the three-channel infographic are initially condensed, and the data dimensions are adjusted to adapt to the subsequent encoding process.

[0099] The next stage is the encoding phase, also known as the downsampling process. This encoding phase consists of three Ghost Module DNN processes, each halving the resolution and doubling the number of channels. In the first Ghost Module DNN process, the input resolution is 128×128 with 16 channels. After an average pooling layer, the resolution is reduced to 64×64, and the Ghost Module increases the number of channels to 32, resulting in a 64×64 feature map with 32 channels. In the second Ghost Module DNN process, the input resolution is 64×64 with 32 channels. After an average pooling layer, the resolution is reduced to 32×32, and the Ghost Module increases the number of channels to 64, resulting in a 32×32 feature map with 64 channels. In the third Ghost Module DNN processing, the input resolution is 32×32 and the number of channels is 64. After the average pooling layer, the resolution is reduced to 16×16 and the number of channels is increased to 128 by the Ghost Module. The output is a feature map with a resolution of 16×16 and a number of channels of 128, which is the encoded feature map.

[0100] During the encoding phase, feature information is condensed through progressive downsampling, and deep semantic features of the three-channel information map are extracted. At the same time, sufficient feature expressive power is retained by doubling the number of channels. The average pooling layer ensures the global consistency of features during the downsampling process, while the Ghost Module achieves efficient feature extraction with low computational consumption.

[0101] Following the encoding stage is the decoding stage, which is the upsampling process. Specifically, the decoding stage includes four GhostModuleUNN processes. The first three processes double the resolution and halve the number of channels, while the fourth process restores the resolution to the pre-encoding size and adapts the number of channels to the weight map requirements. In the first GhostModuleUNN process, the encoded feature map (16×16, 128 channels) is input, and the transposed convolutional block increases the resolution to 32×32 and reduces the number of channels to 64. The channel concatenation block then concatenates the feature map (32×32, 64 channels) with the output feature map from the second encoding stage, resulting in 128 channels. The Ghost Module refines the concatenated feature map, outputting a 32×32, 64-channel feature map. In the second GhostModuleUNN processing, a 32×32, 64-channel feature map is input. The transposed convolutional block increases the resolution to 64×64 and reduces the number of channels to 32. Channel concatenation is then performed with the output feature map (64×64, 32 channels) from the first processing in the encoding stage, resulting in 64 channels. After Ghost Module refinement, a 64×64, 32-channel feature map is output. In the third GhostModuleUNN processing, a 64×64, 32-channel feature map is input. The transposed convolutional block increases the resolution to 128×128 and reduces the number of channels to 16. Channel concatenation is then performed with the feature map (128×128, 16 channels) downsampled by the initial Ghost Module in the encoding stage, resulting in 32 channels. After Ghost Module refinement, a 128×128, 16-channel feature map is output. During the fourth Ghost Module UNN processing, the input is a 128×128, 16-channel feature map. The transposed convolutional block increases the resolution to 236×256 and reduces the number of channels to 3. Without channel concatenation, the features are directly refined through the Ghost Module, and the output is a 236×256, 3-channel feature map.

[0102] The decoding stage restores resolution through progressive upsampling, while channel concatenation fuses shallow and deep features from the encoding stage, ensuring that the weight map reflects both global semantics and local details. Transposed convolutional blocks efficiently improve resolution, channel concatenation blocks guarantee feature integrity, and the Ghost Module refines feature representation.

[0103] Finally, the resolution is restored and a weight map is generated. Through linear interpolation, the 236×256 feature map output by the fourth GhostModuleUNN iteration is restored to the original 4K resolution (3840×2160), resulting in a pixel-by-pixel weight map. Linear interpolation ensures a smooth transition of weight values ​​during resolution enhancement, avoiding detail distortion. The final weight map contains three channels, corresponding to the three channels in the brightness channel set of the layered brightness, representing the contribution ratio of the corresponding layered channel during fusion.

[0104] Step S300: Perform chroma mapping processing on the decoupled chroma channels to obtain the chroma channels of the HDR image after gamut expansion.

[0105] In this embodiment, chroma mapping processing is an optimization operation targeting the chroma channel. Its purpose is to expand the Rec.709 color gamut of the SDR image to the Rec.2020 color gamut of the HDR image, while maintaining color accuracy and naturalness. Color gamut expansion refers to broadening the range of colors an image can display, making the colors richer and closer to the real scene. This can be achieved through color space conversion, adaptive feature mapping, and other methods.

[0106] Specifically, the contextual information of the chroma channels is first extracted to capture the patterns and characteristics of color distribution. Then, based on this information, an appropriate mapping relationship is constructed to ensure that colors do not shift or become distorted during the color gamut expansion process. The T and P channels of the chroma channels jointly determine the color performance of the image. Therefore, the mapping process needs to consider the coordinated optimization of both channels simultaneously to avoid color imbalance caused by adjusting a single channel.

[0107] The expanded color gamut can display more color details that the original SDR image cannot show, making the colors of HDR images more vivid and realistic, while maintaining the consistency of color and brightness matching, avoiding problems such as color distortion in bright areas and dull colors in dark areas, thus improving the overall visual experience.

[0108] In one implementation, the step of performing chroma mapping processing on the decoupled chroma channels to obtain the chroma channels of the HDR image with extended color gamut specifically includes the following steps: Step S310, downsampling the decoupled chroma channels; Step S320, using a chroma feature extraction network to extract the chroma context information of the downsampled chroma channels; Step S330, generating a 2DLUT for the chroma channels based on the chroma context information using a 2DLUT generator; Step S340, performing a lookup table operation on the 2DLUT for the chroma channels at the original resolution to obtain the chroma channels of the HDR image with extended color gamut.

[0109] In this embodiment, as shown in Figure 2, the steps of chroma mapping processing include feature extraction, mapping rule generation, and table lookup mapping. Color gamut expansion is specifically performed on the chroma channel to ensure the color richness and accuracy of HDR images.

[0110] First, the chroma channels are downsampled. The chroma channels, including the T and P channels, carry the complete color information of the image. The downsampling operation reduces the original 4K resolution chroma channels to 236×256 resolution. This can be achieved using bilinear interpolation, which maintains the continuity of color distribution and avoids color distortion while reducing resolution. The purpose of downsampling is to reduce the computational cost of subsequent feature extraction networks, maintaining the same downsampling resolution as the luma channel for parallel processing. Simultaneously, the 236×256 resolution preserves sufficient chroma feature information to meet the needs of subsequent processing.

[0111] Subsequently, a Chrominance Feature Extraction Network (CFEN) is used to extract chroma context information from the downsampled chroma channels. CFEN is a lightweight network for extracting chroma features. Its structure is based on the feature extraction part of a hierarchical luminance processing network, employing a 3-layer CNN architecture that is both lightweight and efficient. The function of this network is to extract chroma context information from the downsampled chroma channels, including color distribution patterns, hue variation trends, saturation gradients, and color relationships between adjacent pixels. This information comprehensively reflects the color characteristics of the image, providing a basis for subsequent 2DLUT generation.

[0112] The input to CFEN is a 236×256 resolution chroma channel, i.e., two channels. The first convolutional layer extracts basic chroma features, the second convolutional layer refines the feature representation, and the third convolutional layer outputs a 256-dimensional chroma context information vector. This vector condenses the two-dimensional chroma feature map into a one-dimensional feature vector, facilitating the subsequent LUT generator to learn mapping relationships while maintaining the integrity and relevance of the chroma features.

[0113] Subsequently, based on the chroma context information, a 2DLUT for the chroma channels is generated using a 2DLUT generator. The 2DLUT generator is a network specifically designed for generating Two-Dimensional Look-Up Tables (2DLUTs). It consists of fully connected layers and is used to convert the chroma context information vector into a two-dimensional mapping between the T-channel and P-channel. A 2DLUT is a two-dimensional data structure that uses the T-channel and P-channel values ​​as two-dimensional indices to store the corresponding output chroma values, enabling simultaneous co-mapping of the T-channel and P-channel.

[0114] Specifically, the 256-dimensional chroma context information vector output by CFEN is input into the 2DLUT generator. After transformation through two fully connected layers, a 64×64 2DLUT is generated. The 64×64 size ensures mapping accuracy while controlling storage and computational consumption, making it suitable for edge deployment. The generated 2DLUT is optimized for the expansion from the Rec.709 color gamut to the Rec.2020 color gamut, ensuring that the mapped chroma values ​​cover the range of the Rec.2020 color gamut while maintaining color naturalness and accuracy. For example, for compressed saturated colors in SDR images, the 2DLUT will appropriately expand their saturation range, allowing them to present richer color levels in HDR images.

[0115] Finally, a lookup operation is performed on the chroma channels of the original resolution image in 2DLUT to obtain the chroma channels of the HDR image after gamut expansion. The original resolution chroma channels retain the complete color details of the image. The lookup operation uses the T and P values ​​of each pixel in the channel as two-dimensional indices to find the corresponding output T and P values ​​in 2DLUT, achieving fast mapping of chroma values. Since 2DLUT stores the complete mapping relationship, the lookup operation does not require complex calculations and can be completed simply by index lookup, resulting in fast operation speed and effectively reducing the computational power consumption of real-time processing.

[0116] During the lookup process, if the T or P value of a pixel exceeds the index range of 2DLUT, boundary interpolation is used to determine the output value, ensuring that all pixels receive a valid mapping result. The mapped chroma channel expands its color gamut from Rec.709 to Rec.2020, offering a wider color range and richer saturation, capable of displaying color details that the original SDR image cannot show.

[0117] By employing a process of downsampling to reduce computational power, CFEN to extract features, 2DLUT to generate mapping rules, and lookup table for fast mapping, efficient color gamut expansion of the chroma channel is achieved while controlling computational power consumption. The downsampling step reduces the computational power requirements of intermediate processing; CFEN accurately extracts chroma features, ensuring the adaptability of the mapping rules; the 2DLUT generator specifically generates color gamut expansion mapping relationships, ensuring the accuracy of color expansion; and the lookup table operation enables efficient real-time mapping, adapting to the processing needs of edge devices. The final generated chroma channel covers the Rec.2020 standard, with natural and rich colors, working in conjunction with the expanded luminance channel to form a complete HDR image color system.

[0118] Step S400: Merge the luminance and chrominance channels of the HDR image and convert them to the RGB color space to obtain the reconstructed HDR image.

[0119] In this embodiment, as shown in Figure 2, channel merging integrates the luminance channel (after dynamic range expansion) and the chroma channel (after color gamut expansion) to form a complete IC. t C p Spatial HDR data integration ensures that the spatial resolution and pixel alignment of the two channels are completely consistent, avoiding image misalignment or information loss. Color space conversion is the process of IC... t C p The HDR data in the color space is converted back to the RGB color space to adapt to the output requirements of mainstream display devices. The conversion process is performed according to the color space conversion matrix that is the opposite of the previous one, ensuring the reversibility and accuracy of the data.

[0120] The reconstructed HDR image must simultaneously meet the characteristics of high dynamic range and wide color gamut, with brightness range conforming to the HDR standard, color gamut covering the Rec.2020 standard, and retaining the visual style and content integrity of the original SDR image.

[0121] By combining channels and converting color spaces, the brightness and chromaticity information optimized in the previous steps is integrated into a directly usable HDR image, realizing the coordinated implementation of dynamic range expansion and color gamut expansion. The final output HDR image is superior to the original SDR image in terms of brightness and darkness levels and color richness, and is compatible with mainstream display devices.

[0122] To verify the effectiveness of the model, experimental development and testing were conducted under the parameters of the MediaTek MTK9655 platform. In addition, to demonstrate the advancement of this embodiment, it was trained and tested on the HDR1K dataset along with several existing advanced models, and objective evaluation metrics were used.

[0123] Objective evaluation metrics include three aspects: HDR-VDP-3 (High-Dynamic-Range Visual-Difference-Predictor version 3). And floating-point operations (FLOPs).

[0124] Among them, the higher the HDR-VDP-3 value, the better the performance; The smaller the value, the better the performance; floating-point number is defined as the total number of floating-point operations during model inference, and is often used to measure the computational complexity of the model. This experimental design is based on the MTK6595 chip in a real-world application context. The chip's maximum load computing power is 1.5T (1T=1×10^3G) FLOPs. To meet the needs of real-world applications, the model design aims to achieve efficient image quality improvement within the allowable computing power range. Therefore, experimental tests are conducted on the above three indicators.

[0125] The experimental results of different ITM methods on the HDR1K dataset are shown in Table 1. Within the allowable range of computing power limit (1.5T), the objective indicators have reached the state-of-the-art results. That is, this embodiment achieves real-time enhancement on the edge and realizes excellent visual effect improvement.

[0126] Table 1:

[0127] In summary, the method proposed in this embodiment utilizes IC t C p Spatial decoupling of luminance and chrominance, and channel-specific mapping enable simultaneous adaptation of dynamic range expansion and color gamut expansion, effectively restoring highlights and shadow details in video and avoiding color distortion. Simultaneously, the lightweight design of the layered luminance processing network and lookup table mapping significantly reduces computational power consumption, adapting to edge device deployments and enabling real-time HDR reconstruction of high-definition video, balancing processing accuracy and practicality, and achieving a balance between reconstruction quality and required computational power.

[0128] As shown in Figure 5, this embodiment of the invention provides a video inverse tone mapping system, which includes: a luminance and chrominance decoupling module 10, a luminance layer mapping processing module 20, a chrominance mapping processing module 30, and a merging and conversion module 40.

[0129] Specifically, the luminance and chrominance decoupling module 10 is used to convert the SDR image of the video frame to be reconstructed from the RGB color space to the IC color space. t C p The color space is used to decouple luminance and chrominance into luminance channels and chrominance channels; the luminance layer mapping processing module 20 is used to perform luminance layer mapping processing on the decoupled luminance channels to obtain the luminance channels of the HDR image with expanded dynamic range; the chrominance mapping processing module 30 is used to perform chrominance mapping processing on the decoupled chrominance channels to obtain the chrominance channels of the HDR image with expanded color gamut; the merging and conversion module 40 is used to merge the luminance channels and chrominance channels of the HDR image and convert them to the RGB color space to obtain the reconstructed HDR image.

[0130] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which is shown in Figure 6. The terminal device includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected via a system bus. The processor of the terminal device provides computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the terminal device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a video inverse tone mapping method. The display screen of the terminal device can be a liquid crystal display screen or an e-ink display screen. The temperature sensor of the terminal device is pre-installed inside the terminal device for detecting the operating temperature of the internal components.

[0131] Those skilled in the art will understand that the principle block diagram shown in Figure 6 is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. A specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] In one embodiment, a terminal device is provided, including a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs including instructions for performing operations as described in the embodiments of the methods above.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video inverse tone mapping method, characterized in that, The method includes: converting the SDR image of the video frame to be reconstructed from the RGB color space to the IC color space. t C p A color space is used to decouple luminance and chrominance into luminance and chrominance channels. The decoupled luminance channels are subjected to luminance layer mapping to obtain the luminance channel of the HDR image with extended dynamic range. The decoupled chrominance channels are subjected to chrominance mapping to obtain the chrominance channel of the HDR image with extended color gamut. The luminance and chrominance channels of the HDR image are merged and converted to the RGB color space to obtain the reconstructed HDR image.

2. The video inverse tone mapping method according to claim 1, characterized in that, The step of performing luminance layer mapping processing on the decoupled luminance channels to obtain the luminance channels of the dynamically extended HDR image includes: downsampling the decoupled luminance channels; segmenting the downsampled luminance channels and performing a masking operation to obtain a luminance information map; simultaneously inputting the luminance information map into a layered luminance processing network and a pixel encoder to obtain a luminance channel set for layered luminance and a weight map corresponding to each pixel; and multiplying the luminance channel set for layered luminance and the weight map corresponding to each pixel to obtain the luminance channels of the dynamically extended HDR image.

3. The video inverse tone mapping method according to claim 2, characterized in that, The step of segmenting and masking the downsampled luminance channel to obtain a luminance information map includes: inputting the downsampled luminance channel into a threshold learning network to obtain a bright area threshold and a dark area threshold; and performing segmentation and masking operations on the downsampled luminance channel based on the bright area threshold and the dark area threshold to obtain a luminance information map; wherein the luminance information map is a three-channel information map, which consists of a luminance channel in the bright area, a luminance channel in the original luminance, and a luminance channel in the dark area.

4. The video inverse tone mapping method according to claim 3, characterized in that, The step of simultaneously inputting the luminance information map into the hierarchical luminance processing network and the pixel encoder to obtain the luminance channel set and the weight map corresponding to each pixel of the hierarchical luminance includes: inputting the three-channel information map into the hierarchical luminance processing network to obtain the 1DLUT set; performing a lookup operation on the 1DLUT set for the luminance channels at the original resolution to obtain the luminance channel set of the hierarchical luminance; and inputting the three-channel information map into the pixel encoder to obtain the weight map corresponding to each pixel of the hierarchical luminance channels.

5. The video inverse tone mapping method according to claim 4, characterized in that, The hierarchical brightness processing network includes three parallel feature extraction networks and three corresponding LUT generators. The feature extraction networks are used to extract context information of different brightness regions in the three-channel information map. The LUT generators are used to generate LUTs of different brightness based on the context information and combine them to form the 1DLUT set.

6. The video inverse tone mapping method according to claim 4, characterized in that, The pixel encoder has a symmetrical structure and includes a feature extraction module, a downsampling feature extraction module, and an upsampling feature fusion module. The downsampling feature extraction module includes an average pooling layer and a feature extraction module. The upsampling feature fusion module includes a transposed convolutional block, a channel concatenation block, and a feature extraction module. The process of inputting the three-channel information map into the pixel encoder to obtain a weight map corresponding to each pixel of the layered brightness channel includes: the three-channel information map undergoes downsampling once by the feature extraction module and downsampling three times by the downsampling feature extraction module to obtain an encoded feature map; the encoded feature map undergoes upsampling four times by the upsampling feature fusion module, and during each upsampling process, the feature map from the downsampling process is concatenated with the feature map from the downsampling process through the channel concatenation block; and the upsampling output is restored to the original resolution through linear interpolation to obtain the weight map corresponding to each pixel.

7. The video inverse tone mapping method according to claim 1, characterized in that, The step of performing chroma mapping processing on the decoupled chroma channels to obtain the chroma channels of the HDR image with extended color gamut includes: downsampling the decoupled chroma channels; extracting chroma context information of the downsampled chroma channels using a chroma feature extraction network; generating a 2DLUT for the chroma channels based on the chroma context information using a 2DLUT generator; and performing a lookup operation on the 2DLUT for the chroma channels at the original resolution to obtain the chroma channels of the HDR image with extended color gamut.

8. A video inverse tone mapping system, characterized in that, The system includes a luminance and chrominance decoupling module for converting the SDR image of the video frame to be reconstructed from the RGB color space to the IC color space. t C p The system uses a color space to decouple luminance and chrominance into luminance and chrominance channels; a luminance layer mapping processing module performs luminance layer mapping processing on the decoupled luminance channels to obtain the luminance channel of the HDR image with expanded dynamic range; a chrominance mapping processing module performs chrominance mapping processing on the decoupled chrominance channels to obtain the chrominance channel of the HDR image with expanded color gamut; and a merging and conversion module merges the luminance and chrominance channels of the HDR image and converts them to the RGB color space to obtain the reconstructed HDR image.

9. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a video inversion tone mapping program stored in the memory and executable on the processor. When the processor executes the video inversion tone mapping program, it implements the steps of the video inversion tone mapping method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a video inversion tone mapping program, which, when executed by a processor, implements the steps of the video inversion tone mapping method as described in any one of claims 1-7.

Images