Neural Network Luminance Re-grading for HDR Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in effectively creating secondary dynamic range images for High Dynamic Range (HDR) video coding, particularly in communicating and displaying HDR images across devices with varying display capabilities.
Innovation Solution
The use of neural networks for luminance re-grading, which involves a first neural network processing circuit to determine parameter values for a parametric re-grading equation and a second neural network processing circuit to adjust these parameters based on sensor measurements, allowing for the creation of secondary images with adjusted dynamic ranges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural networks are used for luminance re-grading to create secondary dynamic range images, then image quality and adaptability to different display capabilities are improved, but device complexity and computational requirements increase
Solution Approach 1:
The neural networks are trained in advance offline to learn optimal luminance re-grading mappings from HDR to various LDR display ranges. This preliminary training phase allows the system to store pre-computed transformation parameters, so that during actual operation, the system only needs to apply these pre-learned transformations rather than performing complex real-time neural network computations, thus reducing device complexity while maintaining high adaptability
Solution Approach 2:
The luminance re-grading process is segmented into distinct functional components: the first neural network processes pixel luminances to generate initial re-grading parameters, while the second neural network refines these parameters based on specific display characteristics. This segmentation allows each network to specialize in specific aspects of the transformation, improving overall efficiency and reducing the computational burden on individual components
2Manufacturing precision
If multiple neural network processing circuits are used to determine parametric re-grading equations, then manufacturing precision and image quality are improved, but device complexity increases
Solution Approach 1:
Two neural network processing circuits are combined in a multi-stage architecture where the first network generates initial re-grading parameters and the second network refines them. This merging of specialized functions achieves higher precision than a single network could provide, while the modular combination allows each component to be optimized independently, managing overall complexity through functional integration
3Adaptability or versatility
If parametric re-grading equations with multiple partial functions are used, then adaptability to different display dynamic ranges is improved, but computational complexity increases
Solution Approach 1:
The parametric re-grading equation uses dynamic parameters that are automatically adjusted by the neural networks based on the input HDR image characteristics and target display capabilities. This dynamic adaptation allows a single flexible equation structure to handle various display dynamic ranges, replacing the need for multiple fixed equations and reducing computational complexity through parameter optimization rather than structural complexity
Data Source
AI summary
To obtain a practical yet versatile and accurate apparatus for luminance re-grading (300) of an input high dynamic range image (IM_HDR) of first luminance dynamic range into a second image (IM_DR2) of second luminance dynamic range, the inventors propose the apparatus for luminance re-grading (300) of an input high dynamic range image (IM_HDR) of first luminance dynamic range into a second image (IM_DR2) of second luminance dynamic range, wherein a maximum luminance of the second image may be higher or lower than a maximum luminance of the input high dynamic range image, the apparatus comprising:—a first neural network processing circuit (301), having as input a set of pixel luminances of the input high dynamic range image (IM_HDR), wherein the first neural network processing circuit has at least two sets of outputs (S1, S2), wherein the sets of outputs have output nodes for supplying parameter values of a parametric re-grading equation, wherein the parametric re-grading equation is composed of at least two partial functions an amount of which in the re-grading equation being controlled by at least two corresponding output parameter values (P11, P21); wherein a first output set (S1) comprises at least two parameter values (P11, P21) which determine a first shape of the re-grading function, and a second output set (S2) comprises at least two corresponding parameter values (P12, P22) which determine an alternative second shape of the re-grading function, which being determined by the same parametric definition but using different values for its parameters gets a different shape;—a second neural network processing circuit (302), which has as input at least one measurement value (La) from at least one sensor (311), and as output nodes supplying a set of weights corresponding in amount to at least the number of sets of outputs of the first neural network processing circuit;—a combiner (303), arranged to add a first parameter value (P11) of the first output set of the first neural network processing circuit multiplied by a corresponding first weight (w11) from the second neural network processing circuit, to a corresponding parameter value (P12) for the same re-grading function defining parameter of the second output set of the first neural network processing circuit multiplied by a corresponding second weight (w12) from the second neural network processing circuit, yielding a final parameter value (P1_f); and—a luma mapping circuit (320) arranged to map input lumas of the input high dynamic range image (IM_HDR) with a luma mapping function (F_L) which is defined by at least two parameters comprising the final parameter value (P1_f) to obtain output lumas of the second image (IM_DR2).