A tone mapping method, device and equipment of a high dynamic range image and a storage medium

Abstract

The application discloses a tone mapping method and device of a high dynamic range image, equipment and a storage medium, relates to the technical field of video processing, and comprises the following steps: acquiring dynamic metadata of a high dynamic range image to be processed, determining a curve source mode according to curve source indication information in the dynamic metadata; generating curve generation data by using data form indication information, and then generating a tone mapping curve; determining display conditions corresponding to the curve according to target display condition information, and performing display adaptation processing on the curve by combining display capability parameters of a current display device and the curve source mode, so that an adapted mapping curve is obtained; and finally, the curve is used for brightness adjustment of the image, and an image adapted to the current display device is output, so that the efficiency of tone mapping of the high dynamic range image is improved.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a tone mapping method, apparatus, device, and storage medium for high dynamic range images. Background Technology

[0002] Currently, with the rapid development of multimedia display technology, High Dynamic Range (HDR) video technology is gradually becoming mainstream. Compared to traditional Standard Dynamic Range (SDR) video, HDR video can provide a wider brightness range and richer color details, thus bringing users a more realistic and immersive visual experience. However, the production and playback of HDR video faces a core challenge: there is a difference between the peak brightness of the display device used for mastering (usually reaching thousands of nits) and the actual peak brightness of the end-user's display device (ranging from a few hundred nits to thousands of nits).

[0003] To achieve good visual presentation across these differences, the industry typically employs Global Tone Mapping (GTM) technology. This technology maps the brightness range of the master content to the displayable range of the target display device, ensuring that the image retains detail and contrast on any display device, regardless of brightness. In current HDR video transmission standards (such as HDRVivid), parameters related to the tone mapping curve are transmitted as dynamic metadata along with the video stream. The receiving decoder then reconstructs the mapping curve based on this metadata and applies it to the actual pixel data.

[0004] Current dynamic metadata schemes in standards typically use names like statistical mode and curve mode to distinguish the content carried by the metadata. This classification method has limitations, as it lacks a source of mapping curves. The source of these curves is crucial for the adaptation strategy at the decoding end.

[0005] Current standards rely solely on statistics or curve identifiers, failing to indicate to the decoder whether the currently received curve is automatically generated or derived from a reference fit. This prevents the decoder from deterministically selecting the correct mapping and display adaptation based on the curve's origin. Directly cropping or scaling the brightness of curves from the reference fit model could easily disrupt the colorist's carefully designed inflection point positions, highlight roll-off shapes, and overall visual style, resulting in distorted output images. Conversely, imposing overly stringent shape-preserving constraints on automatically generated curves limits the system's ability to freely optimize display capabilities, reducing display quality. Furthermore, existing mechanisms struggle to achieve deterministic backoff processing based on frame-wide auxiliary information when curve parameters are unreliable or the device is unsupported.

[0006] As can be seen from the above, improving the efficiency of tone mapping for high dynamic range (HDR) images is a problem that urgently needs to be solved. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a tone mapping method, apparatus, device, and storage medium for high dynamic range (HDR) images, which can improve the efficiency of tone mapping for HDR images. The specific solution is as follows: In a first aspect, this application provides a tone mapping method for high dynamic range images, comprising: Determine the dynamic metadata corresponding to the high dynamic range image to be processed; the dynamic metadata includes curve source indication information, data shape indication information, and target display condition information. The curve source mode corresponding to the high dynamic range image to be processed is determined based on the curve source indication information; the curve source mode includes an automatic generation mode and a reference fitting mode. The corresponding curve generation data is determined using the data form indication information, and a tone mapping curve is generated based on the curve generation data; Based on the target display condition information, the target display conditions corresponding to the tone mapping curve are determined, and the display capability parameters of the current display device are obtained. The tone mapping curve is then subjected to display adaptation processing based on the display capability parameters and the curve source mode to obtain the display-adapted mapping curve. The brightness of the high dynamic range image to be processed is adjusted using the post-adaptation mapping curve to obtain an output image adapted to the current display device.

[0008] Optionally, determining the dynamic metadata corresponding to the high dynamic range image to be processed includes: Obtain metadata update flag information, and determine whether the current high dynamic range image to be processed carries new dynamic metadata compared with the previous high dynamic range image to be processed based on the metadata update flag information; When the metadata update flag indicates that the current high dynamic range image to be processed carries new dynamic metadata compared to the previous high dynamic range image to be processed, the cache parameters corresponding to the current high dynamic range image to be processed, including curve source indication information, data form indication information and target display condition information, are obtained, and the cache parameters are set as the dynamic metadata corresponding to the high dynamic range image to be processed. When the metadata update flag indicates that the current high dynamic range image to be processed does not carry new dynamic metadata compared to the previous high dynamic range image to be processed, the cache parameter corresponding to the previous high dynamic range image to be processed is set to the dynamic metadata corresponding to the high dynamic range image to be processed.

[0009] Optionally, determining the curve source mode corresponding to the high dynamic range image to be processed based on the curve source indication information includes: The curve source mode is determined based on the curve source indication information and the high dynamic range image to be processed; the curve source mode is used to characterize the mode for generating the tone mapping curve corresponding to the high dynamic range image to be processed; When the curve source mode characterizes the tone mapping curve as a curve automatically determined based on the content features of the high dynamic range image to be processed and the display capability parameters of the current display device, the curve source mode is set to automatic generation mode; The reference target and source content corresponding to the high dynamic range image to be processed are determined, and the mapping relationship between the reference target and the source content is determined. When the curve source mode represents the tone mapping curve as a curve obtained by fitting based on the mapping relationship, the curve source mode is a reference fitting mode.

[0010] Optionally, the step of determining the corresponding curve generation data using the data form indication information and generating a tone mapping curve based on the curve generation data includes: The data type used to indicate the curve generation data is determined based on the data form indication information; When the data type is characterized as a statistical mode, a corresponding first curve generation data is generated based on the data type and the data form indication information, and the brightness distribution statistics of the high dynamic range image to be processed are determined based on the first curve generation data, so as to generate a tone mapping curve based on the brightness distribution statistics. When the data type is characterized as a curve mode, a corresponding second curve generation data is generated based on the data type and the data form indication information, and the curve control parameters corresponding to the curve generation data are determined based on the second curve generation data, so as to generate a corresponding tone mapping curve based on the curve control parameters. When the data type is characterized as a mixed mode, a tone mapping curve is generated based on the brightness distribution statistics and the curve control parameters.

[0011] Optionally, generating a tone mapping curve based on the curve generation data includes: When the data type is characterized as a curve mode, the control point information corresponding to the curve control parameters, which describes the shape of the tone mapping curve, is determined; the control point information includes the number of control points, control point coordinates, control point slope, and piecewise function parameters; The shape of the tone mapping curve is determined based on the control point information, and a corresponding tone mapping curve is generated based on the curve shape. When the data type is characterized as a mixed mode, it is determined whether the curve control parameter meets a preset valid condition. If the curve control parameter meets the preset valid condition, a tone mapping curve is generated based on the curve control parameter. The preset valid condition is that the curve control parameter is valid, or the current display device supports displaying the curve control parameter. If the curve control parameters do not meet the preset valid conditions, a tone mapping curve is generated based on the brightness distribution statistics.

[0012] Optionally, the step of determining the target display conditions corresponding to the tone mapping curve based on the target display condition information, and obtaining the display capability parameters of the current display device, to perform display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode, to obtain a display-adapted tone mapping curve, includes: The display capability parameters corresponding to the current display device are determined, and the target display conditions corresponding to the tone mapping curve are determined based on the target display condition information; the target display condition information includes target peak brightness, black level information, transfer function information, and color gamut information. The difference between the target display conditions and the display capability parameters is determined, and the tone mapping curve is displayed and adapted based on the difference and the curve source mode to obtain the display-adapted mapping curve.

[0013] Optionally, the step of performing display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode to obtain the display-adapted mapping curve includes: When the curve source mode is the reference fitting mode, the tone mapping curve is subjected to constraints on high-conservation light compression characteristics, brightness change trends and visual style to obtain the first display adaptation mapping curve. When the curve source mode is automatic generation mode, the display effect corresponding to the current display device is determined based on the display capability parameters, and the display device capability is optimized based on the display effect to obtain the second display adaptation mapping curve.

[0014] Secondly, this application provides a tone mapping apparatus for high dynamic range images, comprising: The dynamic metadata generation module is used to determine the dynamic metadata corresponding to the high dynamic range image to be processed; the dynamic metadata includes curve source indication information, data form indication information, and target display condition information; The curve source mode determination module is used to determine the curve source mode corresponding to the high dynamic range image to be processed based on the curve source indication information; the curve source mode includes an automatic generation mode and a reference fitting mode. A tone mapping curve generation module is used to determine the corresponding curve generation data using the data form indication information, and to generate a tone mapping curve based on the curve generation data; The curve display adaptation module is used to determine the target display conditions corresponding to the tone mapping curve based on the target display condition information, and to obtain the display capability parameters of the current display device, so as to perform display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode, and obtain the display-adapted mapping curve. The image adjustment module is used to adjust the brightness of the high dynamic range image to be processed using the display adaptation mapping curve to obtain an output image adapted to the current display device.

[0015] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the aforementioned tone mapping method for high dynamic range images.

[0016] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned tone mapping method for high dynamic range images.

[0017] As can be seen from the above, before performing tone mapping on a high dynamic range image, this application first determines the dynamic metadata corresponding to the image to be processed, and specifically introduces curve source indication information independent of data format into the dynamic metadata. Based on this curve source indication information, it is possible to determine at the decoding end whether the tone mapping curve was generated via an automatic generation mode or a reference fitting mode.

[0018] Therefore, this application first needs to parse the curve source in the metadata architecture and perform differentiated display adaptation processing based on the source. In this way, for curves carrying the colorist's subjective intentions in the reference fitting mode, the decoding end can implement constraint processing to preserve high light compression characteristics, brightness change trends, and visual style, ensuring that the color grading style is faithfully reproduced on any target display device; for curves in the automatic generation mode, free optimization without conformal constraints can be performed based on the current display capability parameters, improving the efficiency of tone mapping for high dynamic range images and enhancing the user experience of high dynamic range video transmission and playback. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a flowchart of a tone mapping method for high dynamic range images disclosed in this application; Figure 2 This is a schematic diagram of the tone mapping device for high dynamic range images disclosed in this application; Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The production and playback of HDR video face a core challenge: the peak brightness of the display device used for mastering differs from the actual peak brightness of the end-user's display device. To address this, this application provides a tone mapping method for high dynamic range (HDR) images, which can improve the efficiency of tone mapping for HDR images.

[0023] See Figure 1 As shown, this embodiment of the invention discloses a tone mapping method for high dynamic range images, including: Step S11: Determine the dynamic metadata corresponding to the high dynamic range image to be processed; the dynamic metadata includes curve source indication information, data form indication information, and target display condition information.

[0024] In this embodiment, after receiving the encapsulated video stream at the video decoding end, the supplemental enhancement information (SEI) or independent metadata track in the stream is first parsed to obtain the metadata update flag information (metadata_update_flag) of the high dynamic range image to be processed (which can be a video frame or a scene-level image to be processed).

[0025] The aforementioned metadata update flag information is a key element designed in this application embodiment to reduce bitstream transmission overhead. In continuously transmitted video frames, if the tone mapping parameters do not change, it is unnecessary to carry the complete metadata payload in every subsequent frame. The specific processing logic includes: Obtain the metadata update flag information to determine whether the current high dynamic range image to be processed carries new dynamic metadata compared to the previous high dynamic range image to be processed. If the value of the metadata update flag (metadata_update_flag) is "1", it indicates that the current high dynamic range image to be processed carries entirely new dynamic metadata, and the decoding end must parse the new parameters.

[0026] When the metadata update flag indicates that the current high dynamic range (HMR) image to be processed carries new dynamic metadata compared to the previous HMR image, the cache parameters corresponding to the current HMR image to be processed, including curve source indication information, data shape indication information, and target display condition information, are obtained, and the cache parameters are set to the dynamic metadata corresponding to the HMR image to be processed. In other words, the decoding end saves the parsed new parameters to the hardware or software cache as the currently valid parameters for subsequent mapping calculations.

[0027] When the metadata update flag indicates that the current high dynamic range image to be processed does not carry new dynamic metadata compared to the previous high dynamic range image to be processed, the cache parameters corresponding to the previous high dynamic range image to be processed are set to the dynamic metadata corresponding to the current high dynamic range image to be processed. In this case, the parameter settings of the previous frame are directly reused as the parameters of the current high dynamic range image to be processed, thereby saving computational overhead and communication bandwidth.

[0028] It is worth mentioning that the dynamic metadata in this application embodiment specifically includes the following three newly designed core information types, which together drive the subsequent efficient processing: Curve source indication information (curve_source_mode): This indicates how the current tone mapping curve was defined. Unlike existing technologies that classify curves based on "data format," the curve source indication information defines the curve's "birth attribute." Its value range includes at least two modes: "automatic generation mode" and "reference fitting mode." For example, when curve_source_mode=0, the curve is in automatic generation mode; when curve_source_mode=1, the curve is in reference fitting mode; other values ​​can be reserved for future standard expansions.

[0029] Payload type: This field describes the physical payload type of the curve generation data. This field is independent of the curve source information, meaning that multiple data types can exist regardless of whether it's in "automatic generation mode" or "reference fitting mode". For example, the payload type can be set to: 0 for statistical mode, 1 for curve mode, and 2 for mixed mode. Statistical mode means that the metadata payload contains luminance distribution statistics of the source image content (e.g., pixel luminance histogram), which will be used by the decoding end to generate the final tone mapping curve. Curve mode means that the metadata payload contains curve control parameters already generated by the preprocessor, which the decoding end needs to directly reconstruct. Mixed mode means that the metadata payload contains both luminance distribution statistics and curve control parameters, forming a primary / backup structure.

[0030] Target display condition information (target_data): This field describes the physical parameters of the reference target display device or color anchor environment corresponding to the tone mapping curve generated by the current encoder. It typically includes target peak brightness information, black level information, transfer function (EOTF) information, and color gamut information. After obtaining this information at the decoder, the curve can be compared with the actual display capabilities of the current physical display device, thus providing a benchmark for subsequent display adaptation.

[0031] Step S12: Determine the curve source mode corresponding to the high dynamic range image to be processed based on the curve source indication information; the curve source mode includes automatic generation mode and reference fitting mode.

[0032] In this embodiment, the decoding end needs to determine the curve source mode of the high dynamic range image to be processed based on the parsed curve source indication information (curve_source_mode). The specific method is as follows: First, the decoding end determines the curve source mode corresponding to the high dynamic range image to be processed based on the value of the curve source indication information and the current high dynamic range image to be processed. The curve source mode characterizes the source method used to generate the tone mapping curve corresponding to the high dynamic range image to be processed.

[0033] When the curve source indication information (e.g., curve_source_mode=0) indicates that the tone mapping curve is automatically determined based on the content features of the high dynamic range image to be processed (e.g., pixel brightness histogram, variance, contrast, etc.) and the display capability parameters of the current display device (i.e., the peak brightness, black level limit, etc. of the physical display screen at the decoding end), the decoding end sets the curve source mode to automatic generation mode. In this mode, the system does not require additional manual intervention or a reference target image; it is completely controlled automatically by the algorithm.

[0034] When the curve source indication information (such as curve_source_mode=1) indicates that the tone mapping curve is a curve obtained by fitting based on the mapping relationship between the reference target and the source content, the decoder sets the curve source mode to the reference fitting mode.

[0035] Specifically, during the encoding stage, the preprocessing device first determines a corresponding reference target image (this reference image is usually a high-quality image obtained by a colorist with rich color grading experience through fine-tuning the source content under specific color grading anchor brightness conditions), and then determines the mapping relationship between the reference target and the source content. In order to enable the decoding end to reproduce the colorist's artistic intent on a series of target display devices, the encoding end records the above mapping relationship through curve fitting.

[0036] By distinguishing between these two curve source modes, the subsequent processing logic at the decoding end can execute a display adaptation strategy based on this semantic identifier.

[0037] Step S13: Use the data form indication information to determine the corresponding curve generation data, and generate a tone mapping curve based on the curve generation data.

[0038] In this embodiment, after determining the curve source mode, this application embodiment needs to read the specific curve generation data payload based on the data format indication information (payload_type) in the metadata, and complete the initial curve generation. This step is specifically as follows: The data type used to generate the indicator curve is determined based on the data format indication information. Depending on the field application, the data format typically includes, but is not limited to, the following three: When the data type is represented as a statistical mode, the corresponding first curve generation data is generated based on the data type and data form indication information. Based on this first curve generation data, the luminance distribution statistics of the high dynamic range image to be processed are determined, and a tone mapping curve is generated based on these statistics. In this mode, the encoder does not directly send the curve coordinates to the decoder; instead, it sends the histogram statistics of the luminance components of the source content, the characteristics of the master display device, and necessary generation constraints. The decoder needs to run an internally preset curve generation algorithm to dynamically calculate the optimal tone mapping curve suitable for the current display device based on the received statistical information and its own actual display capabilities.

[0039] When the data type is represented as a curve mode, corresponding second curve generation data is generated based on the data type and data form indication information. Based on this second curve generation data, the curve control parameters corresponding to the curve generation data are determined, and a corresponding tone mapping curve is generated based on these parameters. In this mode, the encoder directly transmits specific curve control parameters (such as control point coordinates and piecewise function parameters) to the decoder. The decoder only needs to parse these parameters and reconstruct a smooth, continuous tone mapping curve through interpolation. This mode significantly reduces the computational complexity of the decoder and is suitable for scenarios where the preprocessing stage has already completed a complex curve design, requiring the post-processing stage to strictly adhere to the predetermined curve for tone mapping.

[0040] When the data type is represented in hybrid mode, a tone mapping curve is generated based on luminance distribution statistics and curve control parameters. In this mode, the encoder simultaneously sends luminance distribution statistics and curve control parameters to the decoder. The hybrid mode was designed to address the potential system vulnerability caused by relying solely on one of the aforementioned two modes.

[0041] In the further processing of the above hybrid mode, the decoding end also needs to execute robustness judgment logic: When the data type is represented as a curve mode, the control point information corresponding to the curve control parameters is determined to describe the shape of the tone mapping curve; the control point information includes the number of control points, the coordinates of the control points, the slope of the control points, and the piecewise function parameters.

[0042] To reconstruct the curve designed in the preprocessing stage, the decoding stage needs to obtain the aforementioned control point information. This includes the number of control points (e.g., the number of nodes selected when using cubic spline interpolation or Catmull-Rom interpolation), the coordinates of the control points (the mapping coordinates between the brightness input and output), the slope of the control points (e.g., the tangent direction), and the piecewise function parameters (e.g., the coefficients of each polynomial). Based on this information, the decoding stage can determine the shape of the tone mapping curve corresponding to the high dynamic range image to be processed, and generate a complete mapping curve for subsequent brightness adjustment based on this curve shape.

[0043] When the data type is represented as a mixed mode, it is determined whether the curve control parameters meet the preset valid conditions. If the curve control parameters meet the preset valid conditions, a tone mapping curve is generated based on the curve control parameters. The preset valid conditions are that the curve control parameters are valid (e.g., the control point coordinates are not monotonically increasing, causing curve distortion, numerical overflow, etc.), or the current display device supports displaying the curve control parameters (e.g., the physical limit of the current display device can meet the peak brightness of the curve target display conditions).

[0044] If the curve control parameters do not meet the preset valid conditions, a tone mapping curve is generated based on the brightness distribution statistics. If the curve control parameters are unreliable, or the current device cannot support rendering the main curve generated by the preprocessor, the decoder will immediately abandon the main curve and instead use the brightness distribution statistics transmitted in the same frame to regenerate a backup tone mapping curve using an automatic generation algorithm. This ensures normal display of the image and avoids black screen, color distortion, or screen flickering. This approach of having built-in backtracking material in the same frame effectively avoids processing delays caused by backtracking data across frames or scenes.

[0045] Step S14: Determine the target display conditions corresponding to the tone mapping curve based on the target display condition information, and obtain the display capability parameters of the current display device. Perform display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode to obtain the display-adapted mapping curve.

[0046] In this embodiment, after generating the tone mapping curve, the original tone mapping curve needs to undergo display adaptation processing. Since the display capabilities of actual decoding terminal devices vary greatly (e.g., different brightness, color gamut, and contrast), the original curve must be adapted to the actual display environment.

[0047] First, determine the display capability parameters corresponding to the current display device, and then determine the target display conditions corresponding to the tone mapping curve based on the target display condition information. As mentioned above, the target display condition information specifically includes the target peak brightness (e.g., 2000 nits), black level information (e.g., 0.0001 nits), transfer function information (e.g., using HLG (Hybrid Log-Gamma) or PQ (Perceptual Quantizer) standards), and color gamut information (e.g., BT.2020 or DCI-P3 color gamut).

[0048] Next, the difference between the target display conditions and the display capability parameters is determined, and the tone mapping curve is adapted for display based on the difference and the curve source mode. For example, if the target peak brightness in the target display condition information is 2000 nits, while the peak brightness of the current display device is only 500 nits, the decoder needs to determine the down-mapping loss and shrink and crop the highlight portion of the original tone mapping curve.

[0049] It is worth mentioning that, based on the different curve source modes, the display adaptation processing logic described in the embodiments of this application also has fundamental differences: When the curve source mode is the reference fitting mode: because the curve carries the colorist's subjective intentions, including specific visual inflection points, highlight roll-off shapes, and overall color tension, it cannot be arbitrarily cropped. The decoding end will perform constraint processing on the tone mapping curve to preserve highlight compression characteristics, brightness change trends, and visual style, resulting in the first display-adapted mapping curve.

[0050] Specifically, the constraint processing includes: when adapting to local display devices with lower brightness, the decoder locks and fixes the original control point coordinates and slope in the low-to-medium brightness area, ensuring that the brightness mapping relationship of all pixels below a certain threshold remains strictly consistent with the original curve; for the high brightness range, the decoder performs non-linear highlight compression while preserving the highlight roll-off characteristics. This processing method is called "conformal constraint," and its core is to ensure that the aesthetic style established by the colorist in the brightness environment of the color grading anchor point can be reproduced to the greatest extent possible on all display devices.

[0051] When the curve source mode is automatic generation mode: Since this curve does not contain irreproducible subjective artistic style, it is purely based on the objective statistical information of the input image and the target display capability. Therefore, the decoding end determines the display effect corresponding to the current display device based on the display capability parameters, and optimizes the tone mapping curve based on the display device capability to obtain the second display-adapted mapping curve.

[0052] In this mode, the decoding end is not subject to conformal constraints at all. Instead, it can adopt a standard display adaptation strategy, move control points more freely, rewrite parameters, and even recalculate an optimal curve based entirely on the actual effect target of the current display device to maximize the use of the physical capabilities of the display device to present the dynamic contrast of the image and achieve the best native viewing experience.

[0053] By employing the differentiated adaptation strategy based on different curve sources, the playback quality of HDR images on irregularly shaped display devices has been improved.

[0054] Step S15: Adjust the brightness of the high dynamic range image to be processed using the display adaptation mapping curve to obtain an output image adapted to the current display device.

[0055] In this embodiment, after obtaining the post-adaptation mapping curve (i.e., the first post-adaptation mapping curve or the second post-adaptation mapping curve) at the decoding end, the embodiment of this application needs to perform the final mapping application process.

[0056] The decoding end takes the luminance component of the high dynamic range image to be processed (such as the Y component of the YUV color space, or the luminance index derived from RGB pixel values) as the independent variable and inputs it into the post-adaptation mapping curve to calculate the mapped output luminance value. At the same time, depending on the application requirements, the matching color restoration algorithm (such as color saturation compensation, color gamut mapping conversion, etc.) can also be applied to finally output an HDR image with excellent visual effects that is perfectly adapted to the current display device.

[0057] Furthermore, embodiments of this application also introduce temporal stabilization processing and abnormal rollback processing mechanisms at the decoding end or post-processing end to improve the stability of video playback.

[0058] In terms of temporal stabilization, when metadata is frequently updated frame by frame, the decoder can smooth curve changes based on scene change flags (such as Scene Change Flag), parameter change thresholds (such as curve parameter amplitude changes exceeding a preset percentage), or preset filtering rules (such as weighted smoothing of curve parameters in adjacent frames). Within a scene, when metadata inherits parameters from the preceding high dynamic range image to be processed, the decoder keeps the tone mapping curve parameters and filtering parameters unchanged until a new update flag (metadata_update_flag=1) is received, thereby avoiding image jitter or flickering caused by frequent metadata switching.

[0059] Regarding abnormal rollback handling, when the decoding end detects that the curve parameter is out of bounds, non-monotonic, or the local display device explicitly does not support the curve control parameter, the system will initiate an abnormal rollback. For different data formats, the rollback logic has a clear priority order: If the data format indication information of the high dynamic range image to be processed is a mixed mode (payload_type indicates that it carries both statistical parameters and curve parameters), the decoder will directly parse and utilize the brightness distribution statistics carried in the same frame to generate a brand new backtrack curve according to the calculation process of the automatic generation mode. This backtrack curve is "backtrack material built into the same frame", without waiting for data from other frames, thus ensuring the timeliness and determinism of abnormal backtracking and preventing large areas of black or screen distortion in the image.

[0060] If the data format indication information of the current high dynamic range image to be processed is non-mixed mode (such as only curve parameters), the decoder will backtrack across frames and use the parameters of the previous valid high dynamic range image to be processed as a backup backtrack curve to ensure a stable transition of the output image.

[0061] For new extended fields that are not yet recognized or defined in the bitstream, the decoder can skip them directly based on the length information of the syntax definition, thus maintaining compatibility with the bitstream of older decoders.

[0062] As can be seen from the above, the tone mapping method for high dynamic range images provided in this application embodiment is designed to make the curve source indication information independent of the data form indication information, enabling the decoding end to accurately perceive the "birth attribute" of the curve. During the display adaptation process, the decoding end will apply targeted adaptation (conformal constraint and free optimization are executed separately) that differs from traditional processing methods, so as to preserve the subjective color adjustment intention while releasing the physical capabilities of the display; when an anomaly occurs, the same-frame backoff strategy in the hybrid mode improves the robustness of HDR video playback.

[0063] Accordingly, see Figure 2 As shown, this application also provides a tone mapping apparatus for high dynamic range images, comprising: The dynamic metadata generation module 11 is used to determine the dynamic metadata corresponding to the high dynamic range image to be processed; the dynamic metadata includes curve source indication information, data form indication information, and target display condition information. The curve source mode determination module 12 is used to determine the curve source mode corresponding to the high dynamic range image to be processed based on the curve source indication information; the curve source mode includes an automatic generation mode and a reference fitting mode. The tone mapping curve generation module 13 is used to determine the corresponding curve generation data using the data form indication information, and to generate a tone mapping curve based on the curve generation data. The curve display adaptation module 14 is used to determine the target display conditions corresponding to the tone mapping curve based on the target display condition information, and to obtain the display capability parameters of the current display device, so as to perform display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode, and obtain the display adapted mapping curve. The image adjustment module 15 is used to adjust the brightness of the high dynamic range image to be processed using the display adaptation mapping curve to obtain an output image adapted to the current display device.

[0064] In some specific embodiments, the dynamic metadata generation module 11 may specifically include: The update flag determination unit is used to obtain metadata update flag information, so as to determine whether the current high dynamic range image to be processed carries new dynamic metadata compared with the previous high dynamic range image to be processed based on the metadata update flag information. The new parameter update unit is used to obtain cache parameters corresponding to the current high dynamic range image to be processed, including curve source indication information, data form indication information and target display condition information, when the metadata update flag information indicates that the current high dynamic range image to be processed carries new dynamic metadata compared with the previous high dynamic range image to be processed, and set the cache parameters as the dynamic metadata corresponding to the high dynamic range image to be processed. The old parameter reuse unit is used to set the cache parameters corresponding to the previous high dynamic range image to the dynamic metadata corresponding to the high dynamic range image to be processed when the metadata update flag information indicates that the current high dynamic range image to be processed does not carry new dynamic metadata compared to the previous high dynamic range image to be processed.

[0065] In some specific embodiments, the curve source pattern determination module 12 may specifically include: The pattern parsing unit is used to determine the curve source pattern based on the curve source indication information and the high dynamic range image to be processed; the curve source pattern is used to characterize the pattern of generating the tone mapping curve corresponding to the high dynamic range image to be processed; An automatic setting unit is used to set the curve source mode to an automatic generation mode when the curve source mode characterizes the tone mapping curve as a curve automatically determined based on the content features of the high dynamic range image to be processed and the display capability parameters of the current display device. The reference fitting setting unit is used to determine the reference target and source content corresponding to the high dynamic range image to be processed, and to determine the mapping relationship between the reference target and the source content. When the curve source mode represents the tone mapping curve as a curve obtained by fitting based on the mapping relationship, the curve source mode is the reference fitting mode.

[0066] In some specific embodiments, the tone mapping curve generation module 13 may specifically include: A data type acquisition unit is used to determine the data type used to indicate the data generated by the curve based on the data form indication information. The statistical pattern generation unit is used to generate corresponding first curve generation data based on the data type and the data form indication information when the data type is characterized as a statistical pattern, and to determine the brightness distribution statistics of the high dynamic range image to be processed based on the first curve generation data, so as to generate a tone mapping curve based on the brightness distribution statistics. A curve pattern generation unit is used to generate corresponding second curve generation data based on the data type and the data form indication information when the data type is characterized as a curve pattern, and to determine the curve control parameters corresponding to the curve generation data based on the second curve generation data, so as to generate a corresponding tone mapping curve based on the curve control parameters. A blending mode generation unit is used to generate a tone mapping curve based on the brightness distribution statistics and the curve control parameters when the data type is characterized as a blending mode.

[0067] In some specific embodiments, the tone mapping curve generation module 13 may specifically include: The control point parsing unit is used to determine the control point information corresponding to the curve control parameters to describe the shape of the tone mapping curve when the data type is represented as a curve mode; the control point information includes the number of control points, control point coordinates, control point slope, and piecewise function parameters; the control point unit determines the curve shape of the tone mapping curve based on the control point information, and generates a corresponding tone mapping curve based on the curve shape. The primary / backup judgment and fallback unit is used to determine whether the curve control parameters meet preset validity conditions when the data type is characterized as a mixed mode. If the curve control parameters meet the preset validity conditions, a tone mapping curve is generated based on the curve control parameters. The preset validity conditions are that the curve control parameters are valid or the current display device supports displaying the curve control parameters. If the curve control parameters do not meet the preset validity conditions, a tone mapping curve is generated based on the brightness distribution statistics.

[0068] In some specific embodiments, the curve display adaptation module 14 may specifically include: An environmental parameter comparison unit is used to determine the display capability parameters corresponding to the current display device, and to determine the target display conditions corresponding to the tone mapping curve based on the target display condition information; the target display condition information includes target peak brightness, black level information, transfer function information, and color gamut information; An adaptation execution unit is used to determine the difference between the target display conditions and the display capability parameters, and to perform display adaptation processing on the tone mapping curve based on the difference and the curve source mode to obtain a display-adapted mapping curve.

[0069] In some specific embodiments, the curve display adaptation module 14 may specifically include: The conformal adaptation unit is used to perform constraint processing on the tone mapping curve to preserve the high-light compression characteristics, brightness change trend and visual style when the curve source mode is the reference fitting mode, so as to obtain the first display-adapted mapping curve. The free adaptation unit is used to determine the display effect corresponding to the current display device based on the display capability parameters when the curve source mode is automatic generation mode, and to optimize the display device capability of the tone mapping curve based on the display effect to obtain a second display-adapted mapping curve.

[0070] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the tone mapping method for high dynamic range images disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0071] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0072] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0073] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Unix, Linux, etc. In addition to including a computer program capable of performing the tone mapping method for high dynamic range images executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0074] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned tone mapping method for high dynamic range images. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0076] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0077] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0078] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0079] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A tone mapping method for high dynamic range images, characterized in that, include: Determine the dynamic metadata corresponding to the high dynamic range image to be processed; The dynamic metadata includes curve source indication information, data form indication information, and target display condition information; The curve source mode corresponding to the high dynamic range image to be processed is determined based on the curve source indication information; the curve source mode includes an automatic generation mode and a reference fitting mode. The corresponding curve generation data is determined using the data form indication information, and a tone mapping curve is generated based on the curve generation data; Based on the target display condition information, the target display conditions corresponding to the tone mapping curve are determined, and the display capability parameters of the current display device are obtained. The tone mapping curve is then subjected to display adaptation processing based on the display capability parameters and the curve source mode to obtain the display-adapted mapping curve. The brightness of the high dynamic range image to be processed is adjusted using the post-adaptation mapping curve to obtain an output image adapted to the current display device.

2. The tone mapping method for high dynamic range images according to claim 1, characterized in that, The determination of the dynamic metadata corresponding to the high dynamic range image to be processed includes: Obtain metadata update flag information, and determine whether the current high dynamic range image to be processed carries new dynamic metadata compared with the previous high dynamic range image to be processed based on the metadata update flag information; When the metadata update flag indicates that the current high dynamic range image to be processed carries new dynamic metadata compared to the previous high dynamic range image to be processed, the cache parameters corresponding to the current high dynamic range image to be processed, including curve source indication information, data form indication information and target display condition information, are obtained, and the cache parameters are set as the dynamic metadata corresponding to the high dynamic range image to be processed. When the metadata update flag indicates that the current high dynamic range image to be processed does not carry new dynamic metadata compared to the previous high dynamic range image to be processed, the cache parameter corresponding to the previous high dynamic range image to be processed is set to the dynamic metadata corresponding to the high dynamic range image to be processed.

3. The tone mapping method for high dynamic range images according to claim 1, characterized in that, Determining the curve source mode corresponding to the high dynamic range image to be processed based on the curve source indication information includes: The curve source mode is determined based on the curve source indication information and the high dynamic range image to be processed; the curve source mode is used to characterize the mode for generating the tone mapping curve corresponding to the high dynamic range image to be processed; When the curve source mode characterizes the tone mapping curve as a curve automatically determined based on the content features of the high dynamic range image to be processed and the display capability parameters of the current display device, the curve source mode is set to automatic generation mode; The reference target and source content corresponding to the high dynamic range image to be processed are determined, and the mapping relationship between the reference target and the source content is determined. When the curve source mode represents the tone mapping curve as a curve obtained by fitting based on the mapping relationship, the curve source mode is a reference fitting mode.

4. The tone mapping method for high dynamic range images according to claim 1, characterized in that, The step of determining the corresponding curve generation data using the data form indication information, and generating a tone mapping curve based on the curve generation data, includes: The data type used to indicate the curve generation data is determined based on the data form indication information; When the data type is characterized as a statistical mode, a corresponding first curve generation data is generated based on the data type and the data form indication information, and the brightness distribution statistics of the high dynamic range image to be processed are determined based on the first curve generation data, so as to generate a tone mapping curve based on the brightness distribution statistics. When the data type is characterized as a curve mode, a corresponding second curve generation data is generated based on the data type and the data form indication information, and the curve control parameters corresponding to the curve generation data are determined based on the second curve generation data, so as to generate a corresponding tone mapping curve based on the curve control parameters. When the data type is characterized as a mixed mode, a tone mapping curve is generated based on the brightness distribution statistics and the curve control parameters.

5. The tone mapping method for high dynamic range images according to claim 4, characterized in that, The process of generating a tone mapping curve based on the curve generation data includes: When the data type is characterized as a curve mode, the control point information corresponding to the curve control parameters, which describes the shape of the tone mapping curve, is determined; the control point information includes the number of control points, control point coordinates, control point slope, and piecewise function parameters; The shape of the tone mapping curve is determined based on the control point information, and a corresponding tone mapping curve is generated based on the curve shape. When the data type is characterized as a mixed mode, it is determined whether the curve control parameter meets a preset valid condition. If the curve control parameter meets the preset valid condition, a tone mapping curve is generated based on the curve control parameter. The preset valid condition is that the curve control parameter is valid, or the current display device supports displaying the curve control parameter. If the curve control parameters do not meet the preset valid conditions, a tone mapping curve is generated based on the brightness distribution statistics.

6. The tone mapping method for high dynamic range images according to any one of claims 1 to 5, characterized in that, The step of determining the target display conditions corresponding to the tone mapping curve based on the target display condition information, and obtaining the display capability parameters of the current display device, and performing display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode to obtain the display-adapted tone mapping curve, includes: The display capability parameters corresponding to the current display device are determined, and the target display conditions corresponding to the tone mapping curve are determined based on the target display condition information; the target display condition information includes target peak brightness, black level information, transfer function information, and color gamut information. The difference between the target display conditions and the display capability parameters is determined, and the tone mapping curve is displayed and adapted based on the difference and the curve source mode to obtain the display-adapted mapping curve.

7. The tone mapping method for high dynamic range images according to claim 6, characterized in that, The step of performing display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode to obtain the display-adapted mapping curve includes: When the curve source mode is the reference fitting mode, the tone mapping curve is subjected to constraints on high-conservation light compression characteristics, brightness change trends and visual style to obtain the first display adaptation mapping curve. When the curve source mode is automatic generation mode, the display effect corresponding to the current display device is determined based on the display capability parameters, and the display device capability is optimized based on the display effect to obtain the second display adaptation mapping curve.

8. A tone mapping device for high dynamic range images, characterized in that, include: The dynamic metadata generation module is used to determine the dynamic metadata corresponding to the high dynamic range image to be processed. The dynamic metadata includes curve source indication information, data form indication information, and target display condition information; The curve source mode determination module is used to determine the curve source mode corresponding to the high dynamic range image to be processed based on the curve source indication information; the curve source mode includes an automatic generation mode and a reference fitting mode. A tone mapping curve generation module is used to determine the corresponding curve generation data using the data form indication information, and to generate a tone mapping curve based on the curve generation data; The curve display adaptation module is used to determine the target display conditions corresponding to the tone mapping curve based on the target display condition information, and to obtain the display capability parameters of the current display device, so as to perform display adaptation processing on the tone mapping curve based on the display capability parameters and the curve source mode, and obtain the display-adapted mapping curve. The image adjustment module is used to adjust the brightness of the high dynamic range image to be processed using the display adaptation mapping curve to obtain an output image adapted to the current display device.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the tone mapping method for high dynamic range images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the tone mapping method for high dynamic range images as described in any one of claims 1 to 7.

Images