A data processing method and apparatus
By carrying base layer data and enhancement layer data in the bitstream, the dynamic range of the image is adjusted to match the display capabilities of the display device. This solves the problems of the display device not fully utilizing its display capabilities and high power consumption in the image processing flow, achieving more efficient display effects and reduced power consumption.
Patent Information
- Application Number
- CN202510622264.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-14
- Publication Date
- 2026-11-17
AI Technical Summary
In existing image processing workflows, the decoded image is mismatched with the display capabilities of the display device, resulting in the display device failing to fully utilize its display capabilities or consuming excessive power.
By carrying the basic layer data, first enhancement layer data, and metadata of the original image in the bitstream, the dynamic range of the image is adjusted to adapt to the display capabilities of the display device. During the decoding process, the supported brightness of the display device is taken into account, and a secondary adjustment of the dynamic range is performed to ensure that the dynamic range of the target image matches the display capabilities of the display device.
It improves the utilization rate of display capabilities, reduces power consumption, and enhances image display, achieving consistent visual effects across different display devices.
Smart Images

Figure CN123123568A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data processing method and apparatus. Background Technology
[0002] Currently, image processing generally includes front-end processing (such as image generation and encoding), intermediate processing (such as transcoding and re-encoding), and back-end processing (such as decoding and display). However, the images obtained from current image processing workflows often do not match the display capabilities of display devices. Therefore, improving the compatibility between decoded images and display devices is an urgent problem to be solved. Summary of the Invention
[0003] This application provides a data processing method and apparatus that decodes image streams based on the display capabilities of a display device. This can better utilize the display capabilities of the display device while reducing the power consumption of the display device and improving the image display effect.
[0004] In a first aspect, this application provides a data processing method, the method comprising: acquiring a bitstream, the bitstream including information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image; decoding the bitstream to obtain a target image, the target image being an image with a second dynamic range obtained based on the first image and the first information, the first image being obtained according to the information indicating the first dynamic range of the first image, and the target image being a reconstructed image of the original image.
[0005] Considering that the first dynamic range of the first image obtained based on the information indicating the first dynamic range of the first image may not match the display capability of the display device, thus causing the display device to fail to fully utilize its display capability or to consume high power when displaying the first image, the bitstream of this application embodiment also includes first information indicating the second enhancement layer data of the original image, so that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream is an image with a second dynamic range obtained based on the first image and the aforementioned first information. In this way, the target image with the second dynamic range is obtained based on the first image with the first dynamic range and the second enhancement layer data, enabling the second dynamic range of the target image to adapt to the display capability of the display device, and reducing the power consumption of the display device while better utilizing its display capability, thereby improving the image display effect.
[0006] In some possible implementations, the information indicating the first image with the first dynamic range includes base layer data of the original image, first enhancement layer data of the original image, and metadata of the original image, wherein the first image is an image with the first dynamic range obtained based on the base layer data, the first enhancement layer data, and the metadata.
[0007] For example, this metadata is the first segment of the metadata.
[0008] In this way, by carrying the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image in the bitstream, the first image with the first dynamic range can be obtained based on the base layer data, the first enhancement layer data, and the metadata of the original image carried in the bitstream, so that the first image meets the initial adjustment requirements of the dynamic range. Thus, the target image can be obtained based on the first image and the second enhancement layer data with the first dynamic range.
[0009] In some possible implementations, the bitstream includes metadata of the original image, and the target image is an image obtained based on the first image, the first information, the metadata of the original image, and the display brightness supported by the display device.
[0010] For example, this metadata is the second segment of the metadata. However, both the second segment here and the first segment mentioned above are metadata of the original image.
[0011] In this embodiment, during the process of adjusting the dynamic range of the first image based on the second enhancement layer data, the display brightness supported by the display device is also considered. This ensures that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream is an image with a second dynamic range, obtained based on the second enhancement layer data, the aforementioned first information, and the display brightness supported by the display device. Thus, the target image with the second dynamic range is obtained by performing a secondary adjustment of the dynamic range of the first image with the first dynamic range based on the display brightness supported by the display device. This allows the second dynamic range of the obtained target image to adapt to the display brightness of the display device, and further, it enables better utilization of the display device's display capabilities while reducing the power consumption of the display device and improving the image display effect.
[0012] In some possible implementations, the display brightness supported by the display device is the maximum display brightness supported by the display device.
[0013] In this embodiment, during the process of adjusting the dynamic range of the first image based on the second enhancement layer data, the degree of adjustment of the dynamic range of the first image by the second enhancement layer data is determined based on the maximum display brightness of the display device. This ensures that the dynamic range of the obtained target image is consistent with the maximum display brightness of the display device, avoiding the problem of high power consumption caused by the dynamic range of the target image being greater than the maximum display brightness of the display device, and also avoiding the problem of not fully utilizing the display capability of the display device and poor display effect when the dynamic range of the target image is less than the maximum display brightness of the display device.
[0014] In some possible implementations, the metadata includes a first headroom of the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data. The second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
[0015] In this embodiment, the metadata includes the first headroom of the basic data layer, the degree of adjustment of the dynamic range of the first enhancement layer data to the basic layer data (indicated by the second information), and the degree of adjustment of the first dynamic range of the second enhancement layer data to the first image (indicated by the third information). The dynamic range can be controlled layer by layer, so that each dynamic range can be precisely controlled, avoiding overexposure or loss of detail in the image corresponding to each dynamic range, and making the determined target image more accurate.
[0016] In some possible implementations, the target image is an image obtained based on a first image, first information, a second headroom, a third headroom, and display brightness.
[0017] In this embodiment, the target image is obtained based on a first image, first information, a second headroom, a third headroom, and the display brightness of the display device. Thus, the headroom of the first image whose dynamic range is to be adjusted (i.e., the second headroom) and the headroom of the image obtained by adjusting the first image to its maximum (or minimum) dynamic range using the second enhancement layer data (hereinafter referred to as the third image) (i.e., the third headroom) can be referenced. In other words, the obtained target image is obtained by flexibly adjusting the degree of dynamic range of the first image based on the second enhancement layer data, referencing the source headroom, the target headroom, and the display brightness of the display device. This allows for precise control of the target image's dynamic range, avoiding overexposure or loss of detail in the target image obtained after dynamic range adjustment, resulting in better image quality. Furthermore, the target image obtained by this method is obtained by adaptively changing the adjustment degree with reference to the display brightness of the display device. The degree of adjustment of the brightness range of the target image obtained by adjusting the first image is determined based on the display capabilities of the current display device, allowing the target image's dynamic range to better adapt to different display devices and achieve a more consistent visual effect across different display devices.
[0018] In some possible implementations, the target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data, wherein the weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness.
[0019] In this embodiment, the target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data. The weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness of the display device. During the adjustment of the dynamic range of the first image, the weights of the second enhancement layer data are referenced, and the degree of dynamic range adjustment is flexibly determined based on these weights. Since the weights are based on the second headroom, the third headroom, and the display brightness of the display device, the degree of dynamic range adjustment is flexibly determined based on the display brightness of the display device. This allows for precise control of the target image's dynamic range, avoiding overexposure or loss of detail in the target image obtained after dynamic range adjustment. This results in better image quality and allows the target image's dynamic range to better adapt to different display devices, achieving a more consistent visual effect across different display devices.
[0020] In some possible implementations, the first image is an image with a first dynamic range obtained based on base layer data, first enhancement layer data, first headroom, and second headroom.
[0021] In this way, the degree of adjustment of the dynamic range of the base layer data can be obtained based on the first headroom and the first enhancement layer data of the base layer data, which can better adjust the base layer data to obtain a first image that meets the expected first dynamic range.
[0022] In some possible implementations, the second information is the second headroom or the ratio between the second headroom and the first headroom.
[0023] In this embodiment, the encoder can select to transmit a second headroom or a ratio between the second headroom and the first headroom according to different scenario requirements (such as decoding complexity and encoding / decoding efficiency requirements). This allows the decoder to determine the second headroom based on the most suitable method for the scenario, improving the flexibility and adaptability of determining the second headroom.
[0024] For example, the second headroom is determined based on the proportional relationship between the second headroom and the first headroom. It can adaptively adjust to changes in the first headroom, improving the flexibility and adaptability of determining the second headroom. Furthermore, carrying this proportional relationship in the bitstream reduces the amount of data compared to carrying the second headroom itself, thereby improving encoding and decoding efficiency. In addition, it helps improve interoperability and consistency between different codecs. Simultaneously, since the second headroom is determined based on a proportional relationship, compared to setting different second headrooms for different scenarios, it reduces the number of second headrooms that need to be maintained, simplifying the configuration process and reducing complexity.
[0025] For example, if the encoding end directly transmits the second headroom, the decoding end can directly obtain the second headroom, which improves the accuracy of obtaining the second headroom. At the same time, it does not require determining the second headroom through additional proportional relationships, which simplifies the process of determining the second headroom and reduces the decoding complexity.
[0026] In some possible implementations, the third information is the third headroom or the proportional relationship between the third headroom and the second headroom.
[0027] In this embodiment, the encoder can select to transmit the third headroom or the ratio between the third headroom and the second headroom according to different scenario requirements (such as decoding complexity and encoding / decoding efficiency requirements). In this way, the decoder can determine the third headroom based on the most suitable method for the scenario in different scenarios, which improves the flexibility and adaptability of determining the third headroom.
[0028] For example, the third headroom is determined based on the ratio between the third and second headrooms. It can adaptively adjust to changes in the second headroom, improving the flexibility and adaptability of its determination. Furthermore, carrying this ratio in the bitstream reduces the amount of data compared to carrying the third headroom itself, thus improving encoding and decoding efficiency. Additionally, it helps improve interoperability and consistency between different codecs. Moreover, since the third headroom is determined based on a ratio, compared to setting different third headrooms for different scenarios, it reduces the number of third headrooms that need to be maintained, simplifying the configuration process and reducing complexity.
[0029] For example, the encoding end can directly transmit the third headroom, and the decoding end can directly obtain the third headroom, which improves the accuracy of the obtained third headroom. At the same time, it does not require additional proportional relationships to determine the third headroom, which simplifies the process of determining the third headroom and reduces the decoding complexity.
[0030] In some possible implementations, the target image is an image with a second dynamic range obtained based on a first image converted to the same color gamut and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0031] In this way, the dynamic range of the first image can be adjusted (including changing and keeping the dynamic range constant) within the same color gamut to obtain the target image with the second dynamic range. Because the first image and the second enhancement layer data used to obtain the target image are in the same color gamut, it can be ensured that the dynamic range of the first image can be adjusted using the second enhancement layer data, ensuring the accuracy of the adjustment and improving the visual effect of the target image.
[0032] In some possible implementations, the target image is an image with a second dynamic range obtained based on a first image converted to the target resolution and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0033] This ensures that the size and proportion of the first image and the second enhancement layer data are consistent, and visual misalignment caused by size mismatch can be avoided during the synthesis process.
[0034] In some possible implementations, the target resolution differs from the resolution of the base layer data.
[0035] In this embodiment, unlike the prior art which uses the resolution of the base layer as a benchmark and upsamples the enhancement layer data to obtain a target image with the same resolution as the base layer data, the resolution of the reconstructed image displayed by the display device is still the resolution of the base layer data, which fails to make better use of the display capabilities of the display device, in this embodiment, the resolution of the obtained target image can be a second resolution with the same resolution as the first enhancement layer data (e.g., a second resolution), or the resolution of the obtained target image can also be a third resolution with the same resolution as the second enhancement layer data (a third resolution). The greater diversity of the obtained target image resolution enhances the adaptability of the target image to different display devices.
[0036] Furthermore, with the second and third resolutions being higher than the resolution of the base layer data, higher quality and sharper target images can be obtained, making better use of the display capabilities of the display device and improving the user's viewing experience.
[0037] In some possible implementations, the first dynamic range is the same as the second dynamic range, and the pixel values of corresponding pixels in the first image and the target image are different.
[0038] In this embodiment, the bitstream includes first information indicating the second enhancement layer data of the original image. When the first dynamic range is the expected dynamic range of the reconstructed image (i.e., the target image) of the original image, or when the first dynamic range is compatible with the display capability of the display device, there is no need to modify the data processing procedure of this application (e.g., the processing logic of the decoder) or change the data structure of the bitstream. Instead, the first information indicating the second enhancement layer data can continue to be used to adjust the dynamic range in combination with the first image, so that the second dynamic range of the adjusted target image is the same as the first dynamic range of the first image, thereby improving the compatibility of the encoding and decoding system and reducing the complexity of data maintenance.
[0039] In some possible implementations, the bitstream further includes fourth information indicating the third enhancement layer data, the target image being an image with a second dynamic range obtained based on the second image and the fourth information, and the second image being an image with a third dynamic range obtained based on the first image and the first information. To further improve the adaptability of the target image to the display capabilities of the display device, the bitstream in this embodiment may include three or more enhancement layer data, so that the decoding end can generate the target image according to the display capabilities of the display device. For example, the bitstream in this embodiment may also include fourth information indicating the third enhancement layer data, so that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream in this application is an image with a second dynamic range obtained based on the third enhancement layer data and the aforementioned fourth information. In this way, the target image with the second dynamic range is obtained based on the second image and the third enhancement layer data with the third dynamic range, enabling the second dynamic range of the obtained target image to adapt to the display capabilities of the display device, and reducing the power consumption of the display device while better utilizing the display capabilities of the display device, thus improving the image display effect.
[0040] In some possible implementations, the target image is an image obtained based on the second image, the fourth information, metadata, and the display brightness supported by the display device.
[0041] In this embodiment, during the process of adjusting the dynamic range of the first image based on the third enhancement data, the display brightness supported by the display device is also considered. This ensures that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream is an image with a second dynamic range, obtained based on the third enhancement layer data, the aforementioned fourth information, and the display brightness supported by the display device. Thus, the target image with the second dynamic range is obtained by readjusting the dynamic range of the second image with the third dynamic range based on the display brightness supported by the display device. This allows the second dynamic range of the obtained target image to adapt to the display brightness of the display device, and also enables better utilization of the display device's display capabilities while reducing the power consumption of the display device and improving the image display effect.
[0042] Secondly, this application provides a data processing method, which includes: acquiring an original image; generating a bitstream corresponding to the original image, the bitstream including information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image, the bitstream being used by a decoding end to generate a reconstructed image of the second dynamic range of the original image.
[0043] In this embodiment, the encoding end acquires the original image and generates a bitstream of the original image. The bitstream may include information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image. This allows the bitstream to include information indicating the first image with a first dynamic range and information indicating the second enhancement layer data, resulting in a smaller amount of data transmitted in the bitstream. This eliminates the need to transmit a large amount of enhancement layer data, enabling the decoding end to generate a reconstructed image of the original image based on the first image and the second enhancement layer data according to the display requirements of the original image. For example, the reconstructed image of the original image may be obtained by adjusting the first image based on the second enhancement layer data according to the display requirements, resulting in a better reconstruction effect of the original image and improving the accuracy and image quality of the generated reconstructed image.
[0044] In some possible implementations, the information indicating the first image of the first dynamic range includes the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image.
[0045] In this embodiment, the bitstream may include base layer data of the original image, first enhancement layer data of the original image, first information indicating second enhancement layer data of the original image, and metadata of the original image. Thus, the bitstream may include information about multiple enhancement layer data, enabling the decoding side to dynamically adjust the base layer data multiple times using multiple enhancement layers according to the display requirements of the original image to obtain a reconstructed image of the original image. For example, the reconstructed image of the original image is obtained by adjusting the base layer data based on the first enhancement layer data of the original image according to display requirements, and then further adjusting the result of adjusting the first enhancement layer data based on the second enhancement layer data. This results in a better reconstruction effect of the original image, improving the accuracy and image quality of the generated reconstructed image.
[0046] In some possible implementations, the metadata includes a first luminance headroom of the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data, wherein the second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range, wherein the third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
[0047] Thirdly, this application provides a data processing apparatus, comprising: an acquisition module for acquiring a bitstream, the bitstream including information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image; and a decoding module for decoding the bitstream to obtain a target image, the target image being an image with a second dynamic range obtained based on the first image and the first information, the first image being obtained according to the information indicating the first dynamic range of the first image, and the target image being a reconstructed image of the original image.
[0048] In some possible implementations, the information indicating the first image with the first dynamic range includes base layer data of the original image, first enhancement layer data of the original image, and metadata of the original image, wherein the first image is an image with the first dynamic range obtained based on the base layer data, the first enhancement layer data, and the metadata.
[0049] In some possible implementations, the bitstream includes metadata of the original image, and the target image is an image obtained based on the first image, the first information, the metadata of the original image, and the display brightness supported by the display device.
[0050] In some possible implementations, the display brightness supported by the display device is the maximum display brightness supported by the display device.
[0051] In some possible implementations, the metadata includes a first headroom for the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data. The second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
[0052] In some possible implementations, the target image is an image obtained based on a first image, first information, a second headroom, a third headroom, and display brightness.
[0053] In some possible implementations, the target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data, wherein the weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness.
[0054] In some possible implementations, the first image is an image with a first dynamic range obtained based on base layer data, first enhancement layer data, first headroom, and second headroom.
[0055] In some possible implementations, the second information is the second headroom or the ratio between the second headroom and the first headroom.
[0056] In some possible implementations, the third information is the third headroom or the proportional relationship between the third headroom and the second headroom.
[0057] In some possible implementations, the target image is an image with a second dynamic range obtained based on a first image converted to the same color gamut and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0058] In some possible implementations, the target image is an image with a second dynamic range obtained based on a first image converted to the target resolution and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0059] In some possible implementations, the target resolution differs from the resolution of the base layer data.
[0060] In some possible implementations, the first dynamic range is the same as the second dynamic range, and the pixel values of corresponding pixels in the first image and the target image are different.
[0061] In some possible implementations, the bitstream also includes fourth information indicating the third enhancement layer data, the target image being an image with a second dynamic range obtained based on the second image and the fourth information, and the second image being an image with a third dynamic range obtained based on the first image and the first information.
[0062] In some possible implementations, the target image is an image obtained based on the second image, the fourth information, metadata, and the display brightness supported by the display device.
[0063] Fourthly, this application provides a data processing apparatus, which includes: an acquisition module for acquiring an original image; and an encoding module for generating a bitstream corresponding to the original image, the bitstream including information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image, the bitstream being used by a decoding end to generate a reconstructed image of the second dynamic range of the original image.
[0064] In some possible implementations, the information indicating the first image of the first dynamic range includes the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image.
[0065] Fifthly, this application provides a decoder, which includes a memory and a processor, the memory being coupled to the processor; the memory stores program instructions that, when executed by the processor, cause the processor to implement the method described in the first aspect or any embodiment of the first aspect.
[0066] In a sixth aspect, an encoder is provided, the encoder including a memory and a processor, the memory being coupled to the processor; the memory storing program instructions that, when executed by the processor, cause the processor to implement the method described in the second aspect or any embodiment of the second aspect.
[0067] In a seventh aspect, a decoder is provided. The decoder includes processing circuitry that implements the method described in the first aspect or any embodiment of the first aspect.
[0068] Eighthly, an encoder is provided. The encoder includes a processing circuit that implements the method described in the second aspect or any embodiment of the second aspect.
[0069] Ninthly, this application provides a computer-readable storage medium including a computer program that, when executed on a device, causes the device to perform the method described in the first aspect or any embodiment of the first aspect, or causes the device to perform the method described in the second aspect.
[0070] In a tenth aspect, this application provides a computer program that, when executed by a device, performs the method described in the first aspect or any embodiment of the first aspect, or performs the method described in the second aspect.
[0071] In one aspect, this application provides a computer program product comprising computer program code that, when executed on a device, causes the device to perform the method described in the first aspect or any embodiment of the first aspect, or to perform the method described in the second aspect.
[0072] In a twelfth aspect, this application provides a bitstream generated according to the method described in the second aspect above.
[0073] In a thirteenth aspect, this application provides a computer-readable storage medium storing a bitstream generated according to the method described in the second aspect above.
[0074] In a fourteenth aspect, an apparatus for storing a bitstream is provided, the apparatus comprising: a transceiver unit and a storage unit, the transceiver unit being configured to receive a bitstream generated according to the method described in the second aspect above, and the storage unit being configured to store the bitstream.
[0075] In a fifteenth aspect, an apparatus for transmitting a bitstream is provided, the apparatus comprising: a storage unit and a transceiver unit, the storage unit being used to store a bitstream generated according to the method described in the second aspect above, and the transceiver unit being used to transmit the bitstream.
[0076] The beneficial effects described in aspects three through fifteen above can be referred to the description of any of the embodiments in aspect one or two, and will not be repeated here. Based on the embodiments provided in the above aspects, this application can also make further combinations to provide more embodiments. Attached Figure Description
[0077] Figure 1 A schematic block diagram of an encoding / decoding system provided in this application embodiment;
[0078] Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this application;
[0079] Figure 3 A flowchart illustrating another data processing method provided in an embodiment of this application;
[0080] Figure 4 This is a flowchart illustrating another data processing method provided in an embodiment of this application;
[0081] Figure 5 A schematic diagram of a data processing apparatus provided in an embodiment of this application;
[0082] Figure 6 This is a schematic diagram of another data processing apparatus provided in an embodiment of this application. Detailed Implementation
[0083] To facilitate understanding of the embodiments of this application, some concepts or terms involved in the embodiments of this application will be explained.
[0084] Metadata: Data carried in a video signal that describes information related to the video.
[0085] Dynamic range: In the field of digital imaging, dynamic range represents the ratio of the maximum to the minimum grayscale value of an image pixel. Grayscale value can also be understood as brightness value, and the unit is nits (cd / m²). 2 ), 1 nit = 1 cd / m 2 .
[0086] Typically, the dynamic range of nature is quite large, and the brightness of a night scene under the stars is approximately 10. -3 cd / m 2The sun itself has a brightness of up to 10. 9 cd / m 2 This dynamic range reaches 10. 9 / 10 -3 =10 12 In real-world natural scenes, the brightness of the sun and starlight cannot be obtained simultaneously; therefore, the dynamic range of a real natural environment may not reach 10. 12 The dynamic range is on the order of magnitude; for real-world natural environments, it is typically around 10. -3 Up to 10 6 Within the range.
[0087] Dynamic range is divided into high dynamic range (HDR) or standard dynamic range (SDR), with standard dynamic range also known as low dynamic range. Generally, the brightness range is around 10... -3 Up to 10 6 Images with a brightness range of 0 to 255 are called high dynamic range (HDR) images, while those with a brightness range of 0 to 255 are called low dynamic range (LR) images. Currently, in most color digital images, each pixel in the R, G, and B channels is represented by one byte (8 bits). This means that the brightness range of each channel's pixels is 0 to 255, which is the standard dynamic range of the image. The imaging process of a digital camera is essentially a mapping from the dynamic range of the real world to the dynamic range supported by the display device; this is often a non-linear process.
[0088] Gamma correction: Gamma correction is a non-linear tonal editing method for images. It detects the dark and light parts of an image signal and increases their proportion, thereby improving image contrast. The photoelectric conversion characteristics of current displays, photographic film, and many electronic cameras are non-linear. The relationship between the output and input of these non-linear components can be represented by a power function: Output = (Input) γ .
[0089] The non-linear conversion of color values output by the device stems from the fact that the human visual system is not linear; humans perceive visual stimuli through comparison. When external stimuli are amplified at a certain ratio, the increase in stimulus appears uniform to humans. Therefore, for human perception, a physical quantity increasing in a geometric progression is uniform. To display the input color according to the laws of human vision, a non-linear conversion in the form of a power function is required, transforming linear color values into non-linear color values. The value of gamma (γ) can be determined based on the photoelectric conversion curve of the color space.
[0090] Color space: Color can be the eye's different perceptions of light of different frequencies, or it can represent objectively existing light of different frequencies. A color space is a range of colors defined by a coordinate system established to represent color. A color gamut mode, together with a color model, defines a color space. A color model is an abstract mathematical model that represents color using a set of color components. Color models can include, for example, the RGB (red, green, blue) model and the CMYK (cyan, magenta, yellow, key plate) model. The color gamut refers to the total number of colors a system can produce. For example, Adobe RGB and sRGB are different color spaces based on the RGB model. In this application, a color model can also be referred to as a color mode.
[0091] RGB color space refers to a color space that uses the luminance of the three primary colors—red, green, and blue—to quantitatively represent video or image signals. YCC color space represents a color space where luminance and chrominance are separated. The three components of a YCC color space video or image signal represent luminance-chrominance-chrominance, respectively. Common YCC color space video signals include YUV, YCbCr, and ICtCp. In this application, luminance and chrominance separation can be abbreviated as luminance-chrominance separation, and non-separation of luminance and chrominance can be abbreviated as luminance-chrominance non-separation.
[0092] Each device, such as a monitor or printer, has its own color space and can only generate colors within its color gamut. When an image is moved from one device to another, the colors of the image may change on different devices because each device converts and displays RGB or CMYK images according to its own color space.
[0093] Encoder: Also known as encoding side, encoding device, or encoding end. The image obtained by the encoder is the bitstream.
[0094] Decoder: Also known as the decoding side, decoding device, or decoding end. The decoder decodes the received bitstream to obtain the image.
[0095] Baseline data: also known as baseline image, base layer, base data, base image, fundamental image, base data, base layer, base layer data, or base image. Baseline data can be an SDR image or an HDR image.
[0096] Enhancement layer data: also known as enhancement layer image (gain map), enhancement layer, enhancement image, gain layer, gain layer data, gain layer image, or gain image. Enhancement layer data can include some detailed information about the image. Thus, by using the image detail information contained in the enhancement layer data to supplement the base layer data, an image with a higher or lower dynamic range can be synthesized.
[0097] In this application, there are multiple enhancement layer data corresponding to one image, which may include first enhancement layer data, second enhancement layer data, and optionally third enhancement layer data, so that there are two or more enhancement layer data.
[0098] For ease of description, the terms "SDR object" and "HDR object" will be used in some embodiments of this application. An SDR object can be a static SDR image (also called an SDR image, SDR photograph, SDR picture, etc.), an SDR video or other types of dynamic SDR image, or a frame from an SDR video or dynamic SDR image. Similarly, an HDR object can be a static HDR image (also called an HDR image, HDR photograph, HDR picture, etc.), an HDR video or other types of dynamic HDR image, or a frame from an HDR video or dynamic HDR image.
[0099] It is understood that, for ease of description, some embodiments of this application are described using HDR images as examples to illustrate the technical solutions of this application. However, it is understood that these embodiments can be applied not only to HDR images, but also to other HDR objects, such as HDR videos, dynamic HDR images, or a frame of an HDR video or dynamic HDR image.
[0100] To make the objectives, technical solutions, and advantages of this application clearer, the application will now be described in further detail with reference to the accompanying drawings.
[0101] Figure 1 This is a schematic block diagram of an encoding / decoding system used in this application, such as the video encoding / decoding system 1000 (hereinafter referred to as encoding / decoding system 1000) of this application. The video encoder 1015 (or simply encoder 1015) and video decoder 1022 (or simply decoder 1022) in the video encoding / decoding system 1000 represent devices, etc., that can be used to perform various technologies according to the various examples described in this application.
[0102] like Figure 1 As shown, the encoding / decoding system 1000 includes a source device 1010, which provides a bitstream to a target device 1020 for decoding the bitstream.
[0103] The source device 1010 includes an encoder 1015, and optionally may also include an image source 1011, an image preprocessor 1013 (or a preprocessing unit), a communication interface 1017, or a communication unit 1017.
[0104] Image source 1011 may include or may be any type of image capture device for capturing real-world images, and / or any type of image generation device, such as a computer graphics processor for generating computer animation images or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage device storing any of the images described above.
[0105] To distinguish the processing performed by the preprocessor 1013 or the preprocessing unit 1013, the image or image data 1012 may also be referred to as the raw image or raw image data 1012.
[0106] The preprocessor 1013 receives (raw) image data 1012 and preprocesses the image data 1012 to obtain a preprocessed image 1014 or preprocessed image data 1014. For example, the preprocessing performed by the preprocessor 1013 may include cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It is understood that the preprocessing unit 1013 is an optional component.
[0107] The encoder 1015 is used to receive preprocessed image data 1014 and provide encoded bitstream 1016.
[0108] The communication interface 1017 in the source device 1010 receives the bit stream 1016 and sends the bit stream 1016 (or other processed version) to another device such as the target device 1020 or any other device through the communication channel 1030, so as to store or reconstruct the original image from the bit stream 1016.
[0109] The target device 1020 includes a decoder 1022 (e.g., a video decoder 1022). Optionally, the target device 1020 may include a communication interface 1021 or a communication unit 1021, a post-processor 1024 (or a post-processing unit 1024), and a display device 1026.
[0110] The communication interface 1021 in the target device 1020 directly receives the bitstream 1016 (or any other processed version) from the source device 1010 or any other source device such as a storage device, and provides the bitstream 1016 to the decoder 1022. For example, the storage device is an encoded image data storage device.
[0111] Communication interfaces 1017 and 1021 transmit or receive bitstream 1016 or encoded data through a communication link or network between source device 1010 and target device 1020. The communication link can be, for example, a direct wired or wireless connection, and the communication network can be any type of network, such as a wired network, a wireless network, or any combination thereof, or any type of private network and public network, or any combination thereof.
[0112] For example, the communication interface 1017 can encapsulate the bitstream 1016 into a suitable format such as a message, and / or process the bitstream using any type of transmission encoding or processing, so as to transmit it on a communication link or communication network.
[0113] Communication interface 1021 corresponds to communication interface 1017. For example, it can receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain bitstream 1016.
[0114] Both communication interface 1017 and communication interface 1021 can be configured as follows: Figure 1 The arrow pointing from the source device 1010 to the corresponding communication channel 1030 of the target device 1020 indicates a one-way communication interface or a two-way communication interface, and can be used to send and receive messages, etc., to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as code stream transmission, etc.
[0115] Decoder 1022 is used to receive bitstream 1016 and provide decoded image data 1023 or decoded image 1023.
[0116] The post-processor 1024 in the target device 1020 is used to post-process the decoded image 1023 or the decoded image data 1023 (also known as reconstructed image data or reconstructed image) to obtain the post-processed image 1025 or the post-processed image data 1025. The post-processing performed by the post-processor 1024 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, trimming, or resampling.
[0117] The display device 1026 in the target device 1020 is used to receive the post-processed image data 1025 to display the image to a user or viewer. The display device 1026 can be or includes any type of display for displaying the reconstructed image, such as an integrated or external display screen or monitor. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.
[0118] although Figure 1 The source device 1010 and the target device 1020 are independent devices, but the device embodiments may also simultaneously include the functions of the source device 1010 and the target device 1020, or simultaneously include the functions of the source device 1010 or the corresponding functions and the target device 1020 or the corresponding functions. In these embodiments, the source device 1010 or the corresponding functions and the target device 1020 or the corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.
[0119] According to the description, Figure 1 The presence and division of different units or functions in the source device 1010 and / or target device 1020 shown may vary depending on the actual device and application, which is obvious to those skilled in the art.
[0120] The following is combined with Figure 1 The system shown here is an introduction to the coding scheme of this application.
[0121] In some embodiments, the data processing method of this application may include, but is not limited to, the following steps: Step 1 and Step 2.
[0122] Step 1: Obtain the original image.
[0123] In some embodiments, the original image can be acquired through a camera acquisition path, or the original image can be acquired through an acquisition device that acquires high dynamic range images.
[0124] Step 2: Generate the bitstream corresponding to the original image.
[0125] The bitstream includes information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image. This bitstream is used by the decoding end to generate a reconstructed image of the second dynamic range of the original image.
[0126] In some embodiments, the information indicating the first image of the first dynamic range is the first image.
[0127] For example, in a specific embodiment, the encoder can generate a bitstream including base layer data of the original image, first enhancement layer data of the original image, metadata of the original image, and first information indicating second enhancement layer data, and send the bitstream to a transcoding device or application (APP). After receiving the bitstream, the transcoding device or APP can obtain a first image with a first dynamic range based on the base layer data, the first enhancement layer data, and the metadata. Then, the transcoding device or APP encodes the first image and the first information indicating the second enhancement layer data to generate a bitstream corresponding to the original image, and sends the bitstream including the first image and the first information indicating the second enhancement layer data to the decoder. The method of this application embodiment can also be applied to the transcoding device or the APP.
[0128] In other embodiments, the information indicating the first image of the first dynamic range includes base layer data of the original image, first enhancement layer data of the original image, and metadata of the original image.
[0129] In a specific embodiment, the encoder encodes the base layer data of the original image, the first enhancement layer data of the original image, the first information indicating the second enhancement layer data of the original image, and the metadata of the original image to generate a bitstream corresponding to the original image, and sends the bitstream to the decoder.
[0130] The bitstream generated by the encoder includes the base layer data of the original image, the first enhancement layer data of the original image, the metadata of the original image, and the first information indicating the second enhancement layer data. The following section will discuss this in conjunction with... Figure 2 Please provide a detailed explanation.
[0131] Figure 2 This is a flowchart illustrating a data processing method provided in this application. The method is illustrated using an encoder as an example. In other embodiments, the executing entity may also be an electronic device or a server, etc., without limitation.
[0132] like Figure 2 As shown, the method includes the following steps S201 and S202.
[0133] S201. Obtain the original image.
[0134] In some embodiments, the original image can be acquired through a camera acquisition path. For example, multiple low dynamic range (LDR) images with different exposure values can be acquired through a camera, and these LDR images can be combined to obtain a high dynamic range (HDR) original image.
[0135] In other embodiments, high dynamic range (HMR) images can be acquired using an acquisition device to obtain a raw HMR image, and the raw HMR image can be sent to an encoder, which then acquires the raw HMR image acquired by the acquisition device.
[0136] This application does not limit the dynamic range of the original image.
[0137] S202. Generate the bitstream corresponding to the original image.
[0138] The bitstream includes base layer data of the original image, first enhancement layer data of the original image, first information indicating second enhancement layer data of the original image, and metadata of the original image. This bitstream is used by the decoding end to generate a reconstructed image of the original image.
[0139] In an optional embodiment, the encoder encodes the base layer data of the original image, the first enhancement layer data of the original image, the first information indicating the second enhancement layer data of the original image, and the metadata of the original image to generate a bitstream corresponding to the original image.
[0140] In some embodiments, low dynamic range (LPL) base layer data can be obtained by acquiring images using an acquisition device that acquires LLP images.
[0141] In some embodiments, a first image may be acquired, and first enhancement layer data may be obtained based on the first image and the base layer data.
[0142] The first image has a first dynamic range, and the original image has an original dynamic range. This application does not limit the size relationship between the first dynamic range and the original dynamic range.
[0143] In some embodiments, the original image may be acquired, and second enhancement layer data may be obtained based on the original image and the first image.
[0144] In an optional embodiment, the encoder may determine, based on the original image and the base layer data, first enhancement layer data of the original image, first information indicating second enhancement layer data of the original image, and metadata of the original image.
[0145] For example, the first information can be second enhancement layer data (e.g., an enhancement map), or it can be a curve or transformation information (e.g., a table) used to fit the transformation relationship between the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data, without considering the display brightness of the display device. This third dynamic range is the dynamic range of the second image 'described later.
[0146] For example, the original dynamic range can be the third dynamic range.
[0147] The above description uses a bitstream containing two enhancement layer data as an example. The method is the same when the bitstream contains three or more enhancement layer data.
[0148] For example, in some embodiments, the bitstream may also include fourth information indicating the third enhancement layer data. The encoder adds the encoded result of the fourth information indicating the third enhancement layer data to the bitstream corresponding to the original image.
[0149] In an optional embodiment, a second image can be acquired, and third enhancement layer data can be obtained based on the second image and the first image.
[0150] The second image has a third dynamic range, and the original image has an original dynamic range. This application does not limit the size relationship between the original dynamic range and the third dynamic range.
[0151] For example, the fourth information can be third enhancement layer data (e.g., an enhancement map), or it can be a curve or transformation information (e.g., a table) used to fit the transformation relationship between the fourth dynamic range and the third dynamic range. The fourth dynamic range is the maximum or minimum dynamic range to which the second image ' can be adjusted from the third dynamic range using the third enhancement layer data. This is such that the second dynamic range is the same as the fourth dynamic range, or the second dynamic range is smaller than the fourth dynamic range. The fourth dynamic range is the dynamic range of the third image '. The third image ' can be the image obtained by adjusting the second image ' from the third dynamic range to the fourth dynamic range using the third enhancement layer data.
[0152] For example, the original dynamic range could be the fourth dynamic range.
[0153] The metadata of the original image can be divided into two parts: one part is the metadata about the basic layer, and the other part is the metadata about each enhancement layer.
[0154] For example, metadata can carry transformation information. In some embodiments, metadata can also directly contain the specific content of the transformation information (in which case the metadata does not need to have a syntax structure corresponding to the transformation information).
[0155] In one possible scenario, the conversion information includes at least one of the following: color mode information, color gamut information, and resolution information.
[0156] Color mode information includes, but is not limited to, at least one of the following: color mode information of the base layer data, color mode information of the first enhancement layer data, color mode information in the transformation and processing corresponding to the base layer data and the first enhancement layer data (also referred to as the color mode information of the first image), color mode information of the second enhancement layer, and color mode information in the transformation and processing corresponding to the first image and the second enhancement layer data (also referred to as the color mode information of the target image). The target image is a reconstructed image of the original image described above.
[0157] The color mode information of the first image refers to the color mode information of the first image to be synthesized using the base layer data and the first enhancement layer data.
[0158] The color mode information of the target image refers to the color mode information of the target image to be synthesized using the data from the first image and the second enhancement layer.
[0159] Color gamut information includes, but is not limited to, at least one of the following: color gamut information of the base layer data, color gamut information of the first enhancement layer data, color gamut information in the transformation and processing corresponding to the base layer data and the first enhancement layer data (also known as color gamut information of the first image), color gamut information of the second enhancement layer, and color gamut information in the transformation and processing corresponding to the first image and the second enhancement layer data (also known as color gamut information of the target image).
[0160] The color gamut information of the first image refers to the color gamut information of the first image to be synthesized using the base layer data and the first enhancement layer data.
[0161] The color gamut information of the target image refers to the color gamut information of the target image to be synthesized using the data from the first image and the second enhancement layer.
[0162] The resolution information includes, but is not limited to, at least one of the following: resolution information of the base layer data, resolution information of the first enhancement layer data, resolution information in the transformation and processing corresponding to the base layer data and the first enhancement layer data (also known as the resolution information of the first image), resolution information of the second enhancement layer, and resolution information in the transformation and processing corresponding to the first image and the second enhancement layer data (also known as the resolution information of the target image).
[0163] The resolution of the first image refers to the resolution of the first image to be synthesized using the base layer data and the first enhancement layer data.
[0164] The resolution of the target image refers to the resolution of the target image (e.g., the second image) to be synthesized using the data from the first image and the second enhancement layer.
[0165] In one possible scenario, the conversion information may also include: conversion object information. The conversion object information indicates the conversion object for which the conversion scheme needs to be performed. The conversion object includes, but is not limited to, at least one of the following: base layer data, first enhancement layer data, an image obtained using the first enhancement layer data (e.g., the first image mentioned later on the decoding side), and second enhancement layer data.
[0166] In this way, by encoding the above-mentioned conversion information into the bitstream, the decoding side can use the metadata in the bitstream to synthesize various images to obtain a reconstructed image of the original image that is adapted to the display capabilities of the display device.
[0167] In this embodiment, an original image can be acquired and a bitstream of the original image can be generated. The bitstream may include information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image. Thus, the bitstream may include information about multiple enhancement layer data, enabling the reconstructed image generated on the decoding side to be obtained by repeatedly adjusting the dynamic range of the base layer data using multiple enhancement layers, based on the display requirements of the original image. For example, the reconstructed image of the original image may be obtained by adjusting the base layer data based on the first enhancement layer data of the original image according to actual needs, and then further adjusting the first enhancement layer data based on the second enhancement layer data. This results in a better reconstruction effect of the original image, improving the accuracy and image quality of the generated reconstructed image.
[0168] Optionally combined with the above Figure 1 The following section introduces a data processing method applicable to the decoding side, which can be implemented using any decoder (such as a high-efficiency video coding (HEVC) decoder, a joint photographic experts group (JPEG) decoder, a ProRes decoder, a high-efficiency image format (HEIF) decoder, etc.).
[0169] In some embodiments, the data processing method may include, but is not limited to, the following steps: Step 1 and Step 2.
[0170] Step 1, obtain the bitstream.
[0171] In some embodiments, the bit width of the data stream can be 8 bits, 10 bits, or 12 bits, etc., without limitation.
[0172] In some embodiments, the bitstream may include, but is not limited to: information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image.
[0173] Step 2: Decode the bitstream to obtain the target image.
[0174] In some embodiments, the target image is a reconstructed image of the original image, and the decoder can decode the bitstream to obtain the reconstructed image of the original image.
[0175] In a specific embodiment, the target image is an image with a second dynamic range obtained based on a first image and first information, wherein the first image is obtained based on information from the first image indicating the first dynamic range.
[0176] In some embodiments, the information indicating the first image of the first dynamic range may be the first image itself, and the bitstream may include the first image and first information indicating the second enhancement layer data. The decoder may decode the bitstream to obtain the first image and the first information indicating the second enhancement layer data, and based on the first image and the first information, the decoder obtains a target image with the second dynamic range.
[0177] In a specific embodiment, the forwarding device or APP sends a bitstream including a first image and first information indicating the second enhancement layer data, and the decoder receives the bitstream including the first image and the first information indicating the second enhancement layer data.
[0178] In this way, the bitstream obtained by the decoding end can include the first image, and the decoding end does not need to decode the first image based on the bitstream, which can reduce decoding complexity and improve decoding efficiency.
[0179] In other embodiments, the information indicating the first image of the first dynamic range includes base layer data of the original image, first enhancement layer data of the original image, and metadata of the original image. The bitstream may include the base layer data of the original image, the first enhancement layer data of the original image, metadata of the original image, and first information indicating the second enhancement layer data. After receiving the bitstream, the decoder can decode the bitstream to obtain the target image. The target image of the second dynamic range may be obtained based on the first image and the first information, and the first image may be obtained based on the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image.
[0180] The information about the first image in the bitstream that indicates the first dynamic range includes the base layer data of the original image, the first enhancement layer data of the original image, and the number of elements in the original image. This will be discussed below in conjunction with... Figure 3 Please provide a detailed explanation.
[0181] Figure 3A schematic diagram of a data processing flow provided in this application, such as Figure 3 As shown, the method flow may include, but is not limited to, the following steps:
[0182] S301. Obtain the bitstream.
[0183] In some embodiments, the bitstream may include, but is not limited to: base layer data of the original image, first enhancement layer data of the original image, first information indicating second enhancement layer data of the original image, and metadata of the original image.
[0184] In optional embodiments, metadata can be obtained from the metadata data structure in the bitstream, or from the Supplemental Enhancement Information (SEI) of HEVC or Versatile Video Coding (VVC), user-defined Network Abstraction Layer (NAL) units, or reserved packet units. It can also be obtained from APP extension information encapsulated in JPEG File Interchange Forma (JFIF), or from data segments encapsulated in MP4 (a multimedia container format). Alternatively, the metadata can be obtained from other file locations, such as after the end of image (EOI) syntax element in a complete JPEG file, or from other preset methods.
[0185] In this embodiment, the bitstream may include multiple enhancement layer data or information related to multiple enhancement layer data. The multiple enhancement layer data may include first enhancement layer data, second enhancement layer data (e.g., indicated by first information), and optionally, may also include third enhancement layer data.
[0186] Each enhancement layer data in the multiple enhancement layer data can be: enhancement layer data corresponding to a full-resolution image (e.g., the first image mentioned above).
[0187] As described in the encoding section above, in some embodiments, the metadata may include conversion information, which includes at least one of color mode information, color gamut information, and resolution information. The conversion information may also include conversion object information. For details, please refer to the description in the encoding section; further elaboration is unnecessary.
[0188] The basic layer data and multiple enhancement layer data obtained in this application can be image data in any color space (i.e., color mode), such as RGB, YUV, or any other color space. They can also be image data of any resolution or image data of any color gamut, without any limitations.
[0189] S302. Decode the bitstream to obtain the target image.
[0190] The target image is a reconstructed image of the original image. The decoder can decode the bitstream to obtain a reconstructed image of the original image.
[0191] In some embodiments, the target image is an image with a second dynamic range obtained based on a first image and first information indicating the second enhancement layer data of the original image.
[0192] In some embodiments, the first information may be second enhancement layer data (e.g., enhancement graph), or a curve or transformation information (e.g., table) used to fit the transformation relationship between the second dynamic range and the first dynamic range.
[0193] In some embodiments, the first information can be directly used to adjust the dynamic range of the first image (including increasing or decreasing the dynamic range, or keeping the dynamic range unchanged) to obtain a target image with a second dynamic range.
[0194] In some embodiments, the first image is an image with a first dynamic range obtained based on base layer data, first enhancement layer data, and metadata.
[0195] As described in the encoding method, the metadata can include two parts: metadata about the base layer and metadata about each enhancement layer. These two parts of the metadata can be divided into different segments depending on the application scenario. For example, the metadata about the base layer data and the metadata about the first enhancement layer data can be used as the first segment, and the first image can be an image with a first dynamic range obtained based on the base layer data, the first enhancement layer data, and the first segment of the metadata.
[0196] In some embodiments, the first image may be generated by the method of this application embodiment; alternatively, the first image may also be generated by other devices or software (i.e., the first image is obtained based on base layer data, first enhancement layer data, and metadata). The method of this application embodiment can obtain the first image and obtain the target image based on the first image and first information. The device or software that generates the first image may be a transcoding device or transcoding software, etc. The transcoding device can generate the first image and then send the first image to the execution subject of the method of this application embodiment.
[0197] In some embodiments, the target image may be generated by the method of this application embodiment; alternatively, the target image may be generated by other devices or software (i.e., the target image is obtained based on the first image and first information), and then the method of this application embodiment can acquire the target image. Thus, the device or software that generates the target image can receive the bitstream and send the target image to the execution entity of the method of this application embodiment.
[0198] In an optional embodiment, the target image may be an image obtained by adjusting the dynamic range of the first image to a full or partial extent based on the second enhancement layer data of the original image.
[0199] For example, the decoder can fully enhance the first image based on the second enhancement layer data (e.g., if the second enhancement layer data is 5 times, then the dynamic range of the first image is enhanced by 5 times) to obtain the target image; or, for another example, the decoder can partially enhance the first image based on the second enhancement layer data (e.g., if the second enhancement layer data is enhanced by 5 times, then the dynamic range of the first image is enhanced by only 3 times) to obtain the target image.
[0200] For example, the decoder can completely attenuate the first image based on the second enhancement layer data (for example, if the second enhancement layer data is 1 / 5, then the dynamic range of the first image is reduced to 1 / 5 of the original dynamic range of the first image) to obtain the target image; or, for another example, the decoder can partially attenuate the first image based on the second enhancement layer data (for example, if the second enhancement layer data is 3 / 5, then only the first image is attenuated to 2 / 5 of the original dynamic range of the first image) to obtain the target image.
[0201] The following is combined Figure 3 The specific embodiments are used to describe the generation process of the first image and the target image.
[0202] Continue to refer to Figure 3 The method may optionally also include steps S303, S304, and S305.
[0203] S303. Synthesize a first image with a first dynamic range based on the base layer data, the first enhancement layer data, and the metadata.
[0204] In a specific embodiment, the decoder can synthesize a first image with a first dynamic range based on the base layer data, the first enhancement layer data, and a first segment of metadata. The first segment of metadata includes metadata about the base layer data and metadata about the first enhancement layer data.
[0205] S304. Synthesize a second image with a second dynamic range based on the first image and first information (e.g., second enhancement layer data).
[0206] In some embodiments, when synthesizing the second image, if the color gamut, resolution, and color space of the first image and the second enhancement layer data are consistent, and when the first information is used to adjust the dynamic range of the first image, the first information is used to adjust the dynamic range to the full extent, then the second image can be synthesized without using metadata.
[0207] In other embodiments, when synthesizing the second image, if the first image and the second enhancement layer data differ in at least one of the following (e.g., color gamut, resolution, color space), then when synthesizing the second image, it is necessary not only to base it on the first image and the first information, but also on other factors. Figure 3 As indicated by the dashed arrow, a second segment of metadata is also required. This second segment may include, for example, metadata about the second enhancement layer data of the original image. Specifically, this metadata may include the conversion information mentioned above on the encoding side. This conversion information details how the second enhancement layer data is used to convert the first image to the original image. This conversion information may include at least one of color mode information, color gamut information, and resolution information. Thus, by utilizing this second segment of metadata, the first image and the first information (e.g., the second enhancement layer data) can be converted to the same color gamut, resolution, and color space before being used in subsequent synthesis to obtain the target image.
[0208] S305. Determine the target image based on the second image.
[0209] In an optional embodiment, the decoder may identify the second image as the target image.
[0210] In an optional embodiment, the decoder may process the second image to obtain the target image. For example, this processing may be any processing such as dynamic range adjustment or resolution adjustment, without limitation.
[0211] For example, the decoder can perform resolution conversion on the second image to obtain a second image with converted resolution, and then determine the second image with converted resolution as the target image.
[0212] In this embodiment, the relationship between the first dynamic range and the second dynamic range is not limited.
[0213] In this embodiment, considering that the first dynamic range of the first image, obtained based on the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image, may not match the display capability of the display device, thus causing the display device to fail to fully utilize its display capability or to consume high power when displaying the first image, the bitstream of this embodiment may further include first information indicating the second enhancement layer data of the original image. This ensures that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream is an image with a second dynamic range obtained based on the first image and the aforementioned first information. Thus, the target image with the second dynamic range is obtained based on the first image and the second enhancement layer data with the first dynamic range. This allows the second dynamic range of the target image to be adapted to the display capability of the display device, and enables better utilization of the display capability while reducing the power consumption of the display device and improving the image display effect.
[0214] For example, if the first dynamic range is less than the brightness range or maximum brightness value within the display device's display capabilities, directly using the first image as the reconstructed target image would prevent the display device from fully utilizing its display capabilities when displaying the target image. Therefore, the second dynamic range can be higher than the first dynamic range. For instance, if the first image is an SDR image and the target image is an HDR image, the method in this embodiment can enhance the dynamic range of the first image based on the second enhancement layer data, increasing it from the first dynamic range to the second dynamic range. This allows the second dynamic range to adapt to the display device's capabilities. For example, the second dynamic range can be within the dynamic range of the display device's maximum brightness (also known as peak brightness), or it can be close to but lower than that peak brightness. Thus, in scenarios where the display device has strong display performance, the method in this embodiment can display images with a higher dynamic range within the display device's capabilities, better utilizing the display device's display capabilities and improving the image display effect.
[0215] For example, if the first dynamic range of the first image is greater than the brightness range or maximum brightness of the display device, directly displaying the first image on the display device would result in high power consumption. Therefore, the second dynamic range can be lower than the first dynamic range; for example, the first image might be an HDR image, and the target image might be an SDR image. Thus, the method of this embodiment can reduce the dynamic range of the first image based on the second enhancement layer data, decreasing it from the first dynamic range to the second dynamic range. This allows the second dynamic range to adapt to the display device's capabilities; for example, the second dynamic range could be the dynamic range of the display device's maximum brightness (also known as peak brightness), or it could be a dynamic range close to but lower than that peak brightness. Thus, in scenarios where the display device's performance is weak, the method of this embodiment enables the display device to display images within its display capabilities, reducing the display device's power consumption.
[0216] For example, the first dynamic range of the first image and the second dynamic range of the target image can also be the same. In this way, although the bitstream of the embodiments of this application includes two layers of enhancement layer data, or three or more layers of enhancement layer data, the method of the embodiments of this application supports the transmission of multi-layer enhancement layer data with unchanged dynamic range, without requiring the dynamic range of the first image to be changed when using the method of the embodiments of this application for data processing, thereby improving the universal applicability of the method of this application.
[0217] In some embodiments, the data about the enhancement layer obtained by decoding the bitstream by the above method is not limited to the first enhancement layer data and the second enhancement layer data, but may also include the third enhancement layer data. That is, the enhancement layer data may be two or more layers. The function and usage of the third enhancement layer data and subsequent enhancement layer data are similar to the usage of the second enhancement layer data, and will not be described in detail.
[0218] It should be noted that when obtaining the target image by referencing the display brightness supported by the display device, the target image is obtained based on the last enhancement layer data used (e.g., the third enhancement layer) and the last generated image (e.g., the second image generated using the second enhancement layer data and the first image). However, when generating the second image using the second enhancement layer data and the first image, it is not necessary to refer to the display brightness supported by the display device. The specific process will be described in detail below.
[0219] In some embodiments, the method of this application can perform different processing on the base layer data of the original image and multiple enhancement layer data of the original image to obtain the target image. The following describes different cases using the example of multiple enhancement layer data including first enhancement layer data and second enhancement layer data. Of course, in other implementations, the multiple enhancement layer data can be two or more enhancement layer data. For example, the multiple enhancement layer data may also include third enhancement layer data. The implementation principle of the method of this application is similar and is not limited.
[0220] Whether obtaining the first image or the target image, the corresponding data or images to be synthesized need to be converted to the same dimension, such as the same color gamut, the same resolution, and the same color space, before they can be merged to obtain an image with adjusted dynamic range.
[0221] In some embodiments, before synthesizing the first image based on the base layer data and the first enhancement layer data, at least one of the base layer data and the first enhancement layer data may be transformed. For example, the transformation operation may be performed using a first segment of the aforementioned metadata, such that the transformed base layer data and the first enhancement layer data are in the same dimension.
[0222] In some embodiments, before synthesizing the second image based on the first image and the second enhancement layer data, at least one of the first image and the second enhancement layer data may be transformed based on metadata (e.g., a second segment of metadata) so that the transformed first image and the second enhancement layer data (or first information indicating the second enhancement layer data) are in the same dimension. For example, the second segment of the aforementioned metadata (e.g., transformation information) may be used to perform this transformation operation.
[0223] The conversion operation may include at least one of color gamut conversion, color mode conversion, and resolution conversion.
[0224] This transformation operation aims to convert the first image and the second enhancement layer data to the same dimension before merging them to obtain the second image.
[0225] As mentioned above, the same dimension can be at least one of the following: the same color gamut, the same color mode, and the same resolution. The following describes the corresponding conversion process for each of these three items.
[0226] (1) Color gamut conversion:
[0227] Color gamuts include, but are not limited to, any of the following: BT.2020, DCI-P3, Rec.709, NTSC, Rec.2020, etc.
[0228] In some embodiments, the target image is an image with a second dynamic range obtained based on a first image converted to the same color gamut and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0229] During color gamut conversion, the decoder can obtain the second enhancement layer data of the image format based on the first information, and then color gamut conversion can be performed in the image mode.
[0230] In an optional embodiment, the decoder may convert the color gamut of the first image, or it may convert the color gamut of the second enhancement layer data, to obtain the first image and the second enhancement layer data in the same color gamut. Then, the decoder may synthesize the first image and the second enhancement layer data in the same color gamut to obtain a target image with a second dynamic range.
[0231] In a specific embodiment, the second segment of the metadata may include color gamut information (also known as the color gamut information of the target image) in the transformation and processing corresponding to the first image and the second enhancement layer data. The decoder performs color gamut conversion on at least one of the first image and the second enhancement layer data based on the color gamut information of the target image to obtain the first image and the second enhancement layer data under the same color gamut.
[0232] For example, if the color gamut of the first image is color gamut 1, the color gamut of the second enhancement layer data is color gamut 2, and the color gamut of the target image can be color gamut 1, color gamut 2, or color gamut 3, the following will describe the different cases:
[0233] A. The first image and the target image have the same color gamut, both being color gamut 1.
[0234] In this case, the decoder can convert the color gamut of the second enhancement layer data from color gamut 2 to color gamut 1. In this way, the color gamut of the first image is the same as the color gamut of the second enhancement layer data after the color gamut conversion. The second image (e.g., the target image) synthesized from the first image and the second enhancement layer data after the color gamut conversion under color gamut 1 also has the color gamut 1.
[0235] B. The second enhancement layer data and the target image have the same color gamut, both being color gamut 2.
[0236] In this case, the decoder can convert the color gamut of the first image from color gamut 1 to color gamut 2. In this way, the color gamut of the second enhancement layer data is the same as that of the first image after the color gamut conversion. The second image (e.g., the target image) synthesized from the first image and the second enhancement layer data after the color gamut conversion is also in color gamut 2.
[0237] C. The color gamuts of the first image and the second enhancement layer data are different from those of the target image, which has a color gamut of 3.
[0238] In this case, the decoder can convert the color gamut of the first image from color gamut 1 to color gamut 3, and convert the color gamut of the second enhancement layer data from color gamut 2 to color gamut 3. In this way, the color gamut of the first image after color gamut conversion is the same as the color gamut of the second enhancement layer data after color gamut conversion. The second image (e.g., the target image) synthesized from the first image after color gamut conversion and the second enhancement layer data after color gamut conversion under color gamut 3 also has a color gamut of color gamut 3.
[0239] In an optional embodiment, the second segment of the metadata may further include conversion parameters used during color gamut conversion. These conversion parameters include at least one of a compensation matrix, a conversion matrix, a conversion function, and a compensation value.
[0240] In this embodiment, the target image is an image with a second dynamic range obtained based on a first image converted to the same color gamut and second enhancement layer data. This allows for adjustment of the dynamic range of the first image (including changing and maintaining the dynamic range) within the same color gamut to obtain the target image with the second dynamic range. Because the first image and second enhancement layer data used to obtain the target image are in the same color gamut, it ensures that the dynamic range of the first image can be adjusted using the second enhancement layer data, ensuring accurate adjustment and improving the visual effect of the target image.
[0241] (2) Color mode conversion:
[0242] Color modes include, but are not limited to, any of the following: YUV mode, Lab mode, HSV mode, YCbCr mode, ICtCp mode, RGB mode, RGBA mode, CMYK mode, etc.
[0243] In some embodiments, the target image is an image with a second dynamic range obtained based on a first image converted to the same color space and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0244] In an optional embodiment, the decoder may convert the color mode of the first image, or it may convert the color mode of the second enhancement layer data, to obtain the first image and the second enhancement layer data in the same color space. Then, the decoder may synthesize the first image and the second enhancement layer data in the same color space to obtain a target image with a second dynamic range.
[0245] In a specific embodiment, the second segment of the metadata may further include color mode information (also known as the color mode information of the target image) in the transformation and processing corresponding to the first image and the second enhancement layer data. The decoder performs color mode conversion on at least one of the first image and the second enhancement layer data based on the color mode information of the target image to obtain the first image and the second enhancement layer data under the same color mode.
[0246] For example, the color mode of the first image is color mode 1, the color mode of the second enhancement layer data is color mode 2, and the color mode of the target image can be color mode 1, color mode 2, or color mode 3. The following describes the different cases:
[0247] A. The first image and the target image have the same color mode, both being color mode 1.
[0248] In this case, the decoder can convert the color mode of the second enhancement layer data from color mode 2 to color mode 1. In this way, the color mode of the first image is the same as the color mode of the second enhancement layer data after the color mode conversion. The second image (e.g., the target image) synthesized from the first image and the second enhancement layer data after the color mode conversion under color mode 1 is also in color mode 1.
[0249] B. The second enhancement layer data and the target image have the same color mode, which is color mode 2.
[0250] In this case, the decoder can convert the color mode of the first image from color mode 1 to color mode 2. In this way, the color mode of the second enhancement layer data is the same as the color mode of the first image after the color mode conversion. The second image (e.g., the target image) synthesized from the first image and the second enhancement layer data after the color mode conversion under color mode 2 also has the color mode 2.
[0251] C. The color modes of the first image and the second enhancement layer data are different from those of the target image, which is color mode 3.
[0252] In this case, the decoder can convert the color mode of the first image from color mode 1 to color mode 3, and convert the color mode of the second enhancement layer data from color mode 2 to color mode 3. In this way, the color mode of the first image after color mode conversion is the same as the color mode of the second enhancement layer data after color mode conversion. The second image (e.g., the target image) synthesized from the first image after color mode conversion and the second enhancement layer data after color mode conversion under color mode 3 also has the color mode 3.
[0253] In an optional embodiment, the metadata may further include conversion parameters used when performing color mode conversion. These conversion parameters include at least one of: a compensation matrix, a conversion matrix, a conversion function, and compensation values.
[0254] (3) Resolution conversion:
[0255] In some embodiments, the target image is an image with a second dynamic range obtained based on a first image converted to a target resolution and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0256] In an optional embodiment, the decoder can convert at least one of the resolution of the first image and the resolution of the second enhancement layer data into a target resolution, and the decoder synthesizes the first image and the second enhancement layer data after conversion to the target resolution to obtain a target image with a second dynamic range and a resolution of the target resolution.
[0257] For example, the resolution of the base layer data is a first resolution, the resolution of the first enhancement layer data is a second resolution, and the resolution of the second enhancement layer data is a third resolution.
[0258] In some embodiments, the target resolution is the same as the resolution of the base layer data.
[0259] A. The first image and the target image have the same resolution, both being the first resolution.
[0260] The decoder can convert the resolution of the first enhancement layer data to the same first resolution as the base layer data. In this way, the resolution of the base layer data is the same as the resolution of the first enhancement layer data after the resolution conversion. The decoder synthesizes the base layer data and the first enhancement layer data after the resolution conversion to obtain a first image with the first resolution.
[0261] Next, the decoder can convert the resolution of the second enhancement layer data to a first resolution, which is the same as the resolution of the first image. In this way, the resolution of the first image is the same as the resolution of the second enhancement layer data after the resolution conversion. The decoder then synthesizes the first image and the second enhancement layer data after the resolution conversion to obtain a second image (e.g., the target image) with the first resolution.
[0262] In other embodiments, the target resolution is different from the resolution of the base layer data.
[0263] B. The first image and the target image have the same resolution, both being the second resolution.
[0264] The decoder can convert the resolution of the base layer data to a second resolution, which is the same as the resolution of the first enhancement layer data. In this way, the resolution of the first enhancement layer data is the same as the resolution of the base layer data after the conversion. The decoder synthesizes the first enhancement layer data and the base layer data after the conversion to obtain a first image with a second resolution.
[0265] Next, the decoder can convert the resolution of the second enhancement layer data to a second resolution that is the same as the resolution of the first image. In this way, the resolution of the first image is the same as the resolution of the second enhancement layer data after the resolution conversion. The decoder synthesizes the first image and the second enhancement layer data after the resolution conversion to obtain a second image (e.g., the target image) with a second resolution.
[0266] C. The resolution of the second enhancement layer data and the target image is the same, both being the third resolution.
[0267] In some embodiments, the resolution of the first image can be any one of 720P, 1080P, 2K, 4K, or 8K.
[0268] In some embodiments, the resolution of the second enhancement layer data can be any one of 720P, 1080P, 2K, 4K, or 8K.
[0269] In some embodiments, the resolution of the first image can be the same as the resolution of the second enhancement layer data. For example, if both the first image and the second enhancement layer data have a resolution of 4K, the decoder can directly synthesize the second enhancement layer data and the first image to obtain a second image (e.g., the target image) with a resolution of 4K.
[0270] In other embodiments, the resolution of the first image and the resolution of the second enhancement layer data may be different. For example, in some embodiments, the resolution of the first image may be greater than the resolution of the second enhancement layer data. For example, the resolution of the first image may be 4K, and the resolution of the second enhancement layer data may be 2K or 1080P. In other embodiments, the resolution of the first image may be less than the resolution of the second enhancement layer data. For example, the resolution of the first image may be 1080P or 2K, and the resolution of the second enhancement layer data may be 4K.
[0271] The decoder can convert the resolution of the first image to a third resolution, which is the same as the resolution of the second enhancement layer data. In this way, the resolution of the second enhancement layer data is the same as the resolution of the first image after the resolution conversion. The decoder synthesizes the second enhancement layer data and the first image after the resolution conversion to obtain a second image (e.g., the target image) with a third resolution.
[0272] Unlike existing technologies that use the resolution of the base layer as a benchmark and upsample the enhancement layer data to obtain a target image with the same resolution as the base layer data, the resolution of the reconstructed image displayed by the display device is still the resolution of the base layer data, which fails to make better use of the display capabilities of the display device. In this embodiment, the resolution of the obtained target image can be a second resolution with the same resolution as the first enhancement layer data (e.g., a second resolution), or the resolution of the obtained target image can be a third resolution with the same resolution as the second enhancement layer data (a third resolution). The greater diversity of the obtained target image resolution enhances the adaptability of the target image to different display devices.
[0273] Furthermore, with the second and third resolutions being higher than the resolution of the base layer data, higher quality and sharper target images can be obtained, making better use of the display capabilities of the display device and improving the user's viewing experience.
[0274] In an optional embodiment, the target resolution is the same as the resolution of the base layer data. After the decoder obtains the first image at the first resolution, it can convert the resolution of the second enhancement layer data to the same first resolution as the first image. In this way, the resolution of the first image is the same as the resolution of the second enhancement layer data after the resolution conversion. The decoder synthesizes the first image and the second enhancement layer data after the resolution conversion to obtain a second image (e.g., the target image) with the first resolution.
[0275] In this embodiment of the application, the target image is an image with a second dynamic range obtained based on the first image and the second enhancement layer data after being converted to the target resolution. This can ensure that the size and ratio of the first image and the second enhancement layer data are consistent, so that visual misalignment caused by size mismatch can be avoided during the synthesis process.
[0276] Furthermore, directly merging first image and second enhancement layer data of different resolutions may result in a blurry, distorted, or otherwise quality-loss-prone image. Pre-converting the first and second enhancement images to the target resolution allows for a better selection of merging strategies based on that target resolution, thereby improving the clarity of the target image and revealing its details.
[0277] In an optional embodiment, the first dynamic range and the second dynamic range described above may be the same or different.
[0278] In some embodiments, the first dynamic range is the same as the second dynamic range, and the pixel values of corresponding pixels in the first image and the target image are different.
[0279] For example, if the pixel value of each pixel in the second enhancement layer data is "1", it means that the second enhancement layer data adjusts the first dynamic range of the first image by a factor of 1. In other words, the dynamic range of the second image (or target image) obtained by adjusting the first image based on the second enhancement layer data is still the first dynamic range. Thus, after the decoder uses the second enhancement layer data "1" to adjust the dynamic range of the first image with the first dynamic range, the resulting second dynamic range is the first dynamic range.
[0280] Although the dynamic range of the target image is the same as that of the first image, the dynamic range of the first image is not adjusted when the target image is obtained using the data of the first image and the second enhancement layer. However, the pixel values of the target image with the second dynamic range may have changed compared to the corresponding pixel values of the first image.
[0281] Corresponding pixel values between two images refer to pixels 'a' and 'a' at the same position in the two images being mutually corresponding pixels. For example, in a 2x2 image, pixels 1 and 1' at the top left corner of both images are mutually corresponding pixels; pixels 2 and 2' at the top right corner of both images are mutually corresponding pixels; pixels 3 and 3' at the bottom left corner of both images are mutually corresponding pixels; and pixels 4 and 4' at the bottom right corner of both images are mutually corresponding pixels. In these four pairs of corresponding pixels—pixels 1 and 1', 2 and 2', 3 and 3', and 4 and 4'—the pixel values of the two pixels in each pair are different.
[0282] As an example, the target image is the first image after a filter has been applied.
[0283] As another example, the color gamut of the target image with the second dynamic range differs from that of the first image. For instance, the decoder can convert the color gamut of the first image to a second color gamut identical to that of the second enhancement layer data. The decoder then synthesizes the first image and the color-gamut-converted second enhancement layer data to obtain the target image with the second color gamut. In this way, although the dynamic range of the target image and the first image is the same, their color gamuts are different.
[0284] In this embodiment, the bitstream includes first information indicating the second enhancement layer data of the original image. When the first dynamic range is the expected dynamic range of the reconstructed image (i.e., the target image) of the original image, or when the first dynamic range is compatible with the display capability of the display device, there is no need to modify the data processing procedure (e.g., the processing logic of the decoder) of this application, nor is there a need to change the data structure of the bitstream. Instead, the first information indicating the second enhancement layer data can continue to be used to adjust the dynamic range in combination with the first image, so that the second dynamic range of the adjusted target image is the same as the first dynamic range of the first image, thereby improving the compatibility of the encoding and decoding system and reducing the complexity of data maintenance.
[0285] In other embodiments, the first dynamic range is different from the second dynamic range.
[0286] In an optional embodiment, the first dynamic range is smaller than the second dynamic range of the target image determined based on the first image and the second enhancement layer data.
[0287] For example, in some embodiments, the first image may be an SDR image, and the target image may be an HDR image. The HDR image and the SDR image have different photoelectric conversion functions, or color gamut, or dynamic range, at least one of these differences.
[0288] For example, in some other embodiments, the first image may be an HDR image, but the dynamic range of the HDR image is smaller than that of the target image.
[0289] For example, suppose the first dynamic range is DRx1 to DRx2, and the dynamic range of the target image is DRy1 to DRy2. In some embodiments, DRy1 is less than DRx1, and DRy2 is greater than DRx2. In other embodiments, DRy1 is less than DRx1, and DRy2 is equal to DRx2. In still other embodiments, DRy1 is equal to DRx1, and DRy2 is greater than DRx2. In other words, the target image can be considered as an image whose dynamic range is greater than that of the first image.
[0290] In this embodiment of the application, when the display device fails to fully utilize its display capabilities, the first image can be enhanced based on the data of the second enhancement layer, so that the second dynamic range of the adjusted target image is greater than the dynamic range of the first image. In this way, the second dynamic range is adapted to the display capabilities of the display device, and the display device can display images with a higher dynamic range within the display capability range of the display device, thus making better use of the display capabilities of the display device.
[0291] In an optional embodiment, the first dynamic range is greater than the second dynamic range of the target image determined based on the first image and the second enhancement layer data.
[0292] For example, in some embodiments, the target image may be an SDR image, and the first image may be an HDR image.
[0293] For example, in some other embodiments, the first image may be an SDR image, but the dynamic range of the SDR image is greater than that of the target image.
[0294] For example, suppose the first dynamic range is DRx1 to DRx2, and the dynamic range of the target image is DRy1 to DRy2. In some embodiments, DRx1 is less than DRy1, and DRx2 is greater than DRy2. In other embodiments, DRx1 is less than DRy1, and DRx2 is equal to DRy2. In still other embodiments, DRx1 is equal to DRy1, and DRx2 is greater than DRy2. In other words, the target image can be considered as an image with a dynamic range smaller than that of the first image.
[0295] In this embodiment of the application, when the power consumption of the display device is high when displaying the first image, the first image can be weakened based on the data of the second enhancement layer, so that the second dynamic range of the adjusted target image is smaller than the dynamic range of the first image. In this way, the second dynamic range is adapted to the display capability of the display device, and the display device can display images within the display capability range of the display device, thereby reducing the power consumption of the display device.
[0296] The above describes how to transform at least one of the first image and the second enhancement layer data to convert them to the same dimension so that they can be merged to obtain the target image with the second dynamic range.
[0297] In some embodiments, the decoder may further perform transformation processing on the first image, the second enhancement layer data, the transformed first image, and the transformed second enhancement layer data, and then merge the corresponding data after the transformation processing to obtain the target image.
[0298] The above conversion refers to at least one of the resolution conversion, color gamut conversion, and color space conversion mentioned above.
[0299] The above transformation processing methods may include, but are not limited to: normalization, inverse normalization, increasing offset, decreasing offset, multiplying by a coefficient, etc.
[0300] Taking the decoder processing the data to obtain the processed dataAfter as an example, the following transformation processing methods are explained. This data is used to indicate at least one of the first image, the second enhancement layer data, the transformed first image, and the transformed second enhancement layer data.
[0301] Method 1: Identity Transformation. The decoder can perform an identity transformation on the data using formula (1).
[0302] dataAfter[i]=data[i] formula (1)
[0303] Here, dataAfter[i] refers to the result obtained after processing, and data[i] refers to the image before processing. In methods two to seven below, the meanings of dataAfter[i] and data[i] are the same as their physical meanings in method one, and will not be explained or repeated here.
[0304] Method 2: If the metadata carries the upper limit THH and lower limit THL of data, the decoder can process the data using formula (2).
[0305] dataAfter[i]=data[i]*THH+(A-data[i])*THL formula (2)
[0306] Where A is the maximum value of the stored data. When the data is normalized to between 0 and 1.0, A is 1.0.
[0307] Method 3: If the metadata carries the upper limit THH and lower limit THL of data, and data can be non-linear encoded data such as PQ / HLG / LOG / Gamma, first convert data[i] to the linear domain, and then use formula (2) to process the converted data[i].
[0308] Method 4: If the metadata carries the lower bound THL of data, the decoder can process data using formula (3).
[0309] dataAfter[i]=data[i]+THL formula (3)
[0310] Method 5: If the metadata carries the upper limit THH of data, the decoder can use formula (4) to process data.
[0311] dataAfter[i]=THH+A-data[i] Formula (4)
[0312] Where A is the maximum value of the stored data. When the data is normalized to between 0 and 1.0, A is 1.0.
[0313] Method 6: If the metadata contains processing parameters, the decoder can obtain the processing method TME() based on the parameters and process the data using formula (5).
[0314] dataAfter[i]=TME(data[i]) Formula (5)
[0315] The processing method TME() can take various forms, such as Sigmoid, cubic spline, gamma, straight line, piecewise curve, or its inverse function, etc., and this application does not limit it. For example, the processing method TME() is the curve represented by formula (6) or its inverse curve.
[0316]
[0317] Where L and L' can be normalized optical or electrical signals, a, b, p, and m are photoelectric transfer coefficients, with a equal to 1.12762, b equal to -0.12762, p equal to 2.3, m equal to 0.14, and n is a set parameter. The processing method can also be a neural network transformation. In this case, the decoder can construct a neural network based on historical data (such as historical enhancement layer data, historical transformed enhancement layer data, historical base layer data, historical transformed base layer data, etc.), and use this neural network to obtain rules for processing enhancement layer data, transformed enhancement layer data, base layer data, transformed base layer data, etc.
[0318] Method 7: If the metadata contains processing parameters, the decoder can obtain the processing method TME() based on the parameters and process the data using formula (7).
[0319] dataAfter[i]=TME()*data[i] Formula (7)
[0320] In the above methods one through seven, 'i' can be a coordinate point. For example, for any coordinate point A(x, y) in a two-dimensional image, 'i' is the coordinate point of A in the one-dimensional image after the two-dimensional image is transformed into a one-dimensional image.
[0321] The above embodiments illustrate the data processing method of this application by taking as an example that at least one of the first image and the second enhancement layer data is first converted and then the above transformation processing is performed. In other embodiments, the above transformation processing can also be performed first and then the above conversion processing, the principle is similar, and will not be described in detail.
[0322] In some embodiments, after the first image and the second enhancement layer data are in the same dimension (optionally, after the above transformation), the first image and the second enhancement layer data after being in the same dimension can be merged to obtain a second image (e.g., the target image).
[0323] There are several ways to merge data. These methods can be preset or carried by metadata. Several possible merging methods are given below.
[0324] Method 1: The channels of the first image and the channels of the second enhancement layer data are directly multiplied.
[0325] In this case, the decoder can use formula (8) to merge the channels of the first image and the channels of the second enhancement layer data.
[0326] newPicture[i]=Picture1[i]*Enhance2[i] Formula (8)
[0327] Wherein, newPictue[i] refers to the merged image (e.g., the second image or the target image). Picture1[i] refers to the first image. Enhance2[i] refers to the second enhancement layer data. In the following formulas (9) to (13), the meanings of newPictue[i], Picture1[i], and Enhance2[i] are the same as their physical meanings in formula (8), and will not be explained or elaborated further.
[0328] Method 2: The channels of the first image and the channels of the second enhancement layer data are directly added together.
[0329] In this case, the decoder can use formula (9) to merge the channels of the first image and the channels of the second enhancement layer data.
[0330] newPictue[i]=Picture1[i]+Enhance2[i] Formula (9)
[0331] Where newPictue[i] refers to the merged image. Picture1[i] refers to the first image. Enhance2[i] refers to the data of the second enhancement layer.
[0332] Method 3: The channels of the first image and the channels of the processed second enhancement layer data are directly multiplied together.
[0333] In this case, the decoder can use formula (10) to merge the channels of the first image and the channels of the second enhancement layer data.
[0334] newPicture[i]=Picture1[i]*f(Enhance2[i]) Formula (10)
[0335] Where newPictue[i] refers to the merged image. Picture1[i] refers to the first image. Enhance2[i] refers to the data of the second enhancement layer. f() is the numerical domain transformation function.
[0336] Method 4: The channels of the first image and the channels of the processed second enhancement layer data are directly added together.
[0337] In this case, the decoder can use formula (11) to merge the channels of the first image and the channels of the second enhancement layer data.
[0338] newPicture[i]=Picture1[i]+f(Enhance2[i]) Formula (11)
[0339] Where newPictue[i] refers to the merged image. Picture1[i] refers to the first image. Enhance2[i] refers to the data of the second enhancement layer. f() is the numerical domain transformation function.
[0340] Method 5: The channels of the first image and the channels of the second enhancement layer data after the first processing are directly multiplied, and the second processing is performed on the result.
[0341] In this case, the decoder can use formula (12) to merge the channels of the first image and the channels of the second enhancement layer data.
[0342] newPictue[i]=g(Picture1[i]*f(Enhance2[i])) Formula (12)
[0343] Where newPictue[i] refers to the merged image. Picture1[i] refers to the first image. Enhance2[i] refers to the data of the second enhancement layer. f() is the numerical domain transformation function, and g() is the mapping function.
[0344] Method 6: The channels of the first image and the channels of the second enhancement layer data after the first processing are directly added together, and the second processing is performed on the result.
[0345] In this case, the decoder can use formula (13) to merge the channels of the first image and the channels of the second enhancement layer data.
[0346] newPictue[i]=g(Picture1[i]+f(Enhance2[i])) Formula (13)
[0347] Where newPictue[i] refers to the merged image. Picture1[i] refers to the first image. Enhance2[i] refers to the data of the second enhancement layer. f() is the numerical domain transformation function, and g() is the mapping function.
[0348] Depending on the needs of practical applications, the numerical domain transformation function f() and the mapping function g() in methods one to six of channel merging can be any function, and this application does not limit them, such as f(x) = x, g(x) = x.
[0349] Other merging methods can also be used for channel merging. The appropriate channel merging method should be selected according to the actual application needs. This application does not limit this method.
[0350] In some embodiments, the target image is an image obtained based on a first image, first information indicating the second enhancement layer data of the original image, metadata, and the display brightness supported by the display device.
[0351] In an optional embodiment, the first information may be second enhancement layer data (e.g., enhancement map), or it may be a curve or transformation information (e.g., a table) used to fit the transformation relationship between the second dynamic range and the first dynamic range.
[0352] The display brightness supported by the display device can be the range of brightness supported by the display device, the current display brightness of the display device, or the maximum display brightness supported by the display device.
[0353] In some embodiments, the target image is an image obtained based on a first image, first information indicating second enhancement layer data of the original image, the metadata (e.g., a second segment of the metadata) obtained from the decoded bitstream, and the display brightness supported by the display device.
[0354] The metadata used in this embodiment when obtaining the target image is different from the metadata used when obtaining the first image in the above embodiments, but these metadata are all metadata of the original image.
[0355] In some embodiments, the decoder may obtain the target image based on the first image, the first information, a second segment of metadata, and the display brightness supported by the display device.
[0356] In this embodiment, during the process of adjusting the dynamic range of the first image based on the second enhancement layer data, the display brightness supported by the display device is also considered. Thus, the target image with the second dynamic range is obtained by performing a secondary adjustment of the dynamic range of the first image with the first dynamic range based on the display brightness supported by the display device. This allows the second dynamic range of the target image to adapt to the display brightness of the display device. Furthermore, it can better utilize the display capabilities of the display device while reducing the power consumption of the display device and improving the display effect of the image.
[0357] In an optional embodiment, the aforementioned metadata (e.g., the second segment of metadata) may include a first headroom of the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data.
[0358] For example, headroom can also be referred to as peak margin.
[0359] From a physical perspective, diffuse white corresponds to the average brightness of white objects in a real-world scene, while HDR peak brightness simulates highlights through headroom. For example, a white sheet of paper is approximately 203 nits in sunlight and can be considered diffuse white. If the peak brightness of the base layer data is 406 nits, then the headroom of the base layer data is 406 nits / 203 nits = 2; if the peak brightness of the base layer data is 203 nits, then the headroom of the base layer data is 1.
[0360] headroom is a specific value that is calculated using a specific function, such as a ratio function (i.e., a multiple relationship) or a log function.
[0361] The second headroom is determined based on the first dynamic range and the dynamic range of the base layer data.
[0362] Taking this specific function as an example of a ratio function, for instance, the second headroom is the ratio of the peak brightness of the first dynamic range to the peak brightness of the base layer data.
[0363] The aforementioned third headroom is determined based on the third dynamic range and the first dynamic range.
[0364] Taking this specific function as a ratio function as an example, the third headroom is the ratio of the peak brightness of the third dynamic range to the peak brightness of the first dynamic range. Here, the third dynamic range is the maximum dynamic range (e.g., the second enhancement layer data is an integer, such as 5 times) or the minimum dynamic range (e.g., the second enhancement layer data is a decimal, such as 1 / 5 times) that the decoding side can adjust from the first dynamic range using the second enhancement layer data.
[0365] For example, if each pixel value in the second enhancement layer data is 5, then in a scenario where the dynamic range of the first image is enhanced to the full extent, the dynamic range of the first image can be enhanced by 5 times. The dynamic range of the first image obtained by enhancing the first dynamic range by 5 times is the third dynamic range. The second dynamic range can be the third dynamic range (i.e., using the second enhancement layer data to make full adjustments), or it can be smaller than the third dynamic range (i.e., using the second enhancement layer data to make partial adjustments, such as enhancing the brightness of the first image by 3 times to obtain the second dynamic range).
[0366] In this embodiment of the application, the metadata includes the first headroom of the basic data layer, the degree of adjustment of the dynamic range of the first enhancement layer data to the basic layer data (indicated by the second information), and the degree of adjustment of the first dynamic range of the second enhancement layer data to the first image (indicated by the third information). The dynamic range can be controlled layer by layer, so that each dynamic range can be precisely controlled, avoiding overexposure or loss of detail in the image corresponding to each dynamic range, and making the determined target image more accurate.
[0367] For example, the decoder can determine a second headroom based on the second information, and a third headroom based on the third information.
[0368] In some embodiments, the second information may be the second headroom, or the second information may be the ratio between the second headroom and the first headroom.
[0369] For example, the encoder may determine the second headroom as the second information, or the encoder may determine the ratio of the second headroom to the first headroom as the second information.
[0370] Correspondingly, if the second information is the second headroom, the decoder determines the second information as the second headroom. If the second information is the ratio of the second headroom to the first headroom, the decoder determines the second headroom as the product of the second information and the first headroom.
[0371] In the embodiments of this application, the encoder can select to transmit a second headroom or a ratio between the second headroom and the first headroom according to different scenario requirements (such as decoding complexity and encoding / decoding efficiency requirements). In this way, the decoder can determine the second headroom based on the most suitable method for the scenario in different scenarios, which improves the flexibility and adaptability of determining the second headroom.
[0372] For example, the second headroom is determined based on the proportional relationship between the second headroom and the first headroom. The encoding end can adaptively adjust the second headroom according to changes in the first headroom, improving the flexibility and adaptability in determining the second headroom. Furthermore, carrying this proportional relationship in the bitstream reduces the amount of data compared to carrying the second headroom itself, thereby improving encoding and decoding efficiency. In addition, it helps improve interoperability and consistency between different codecs. Simultaneously, since the second headroom is determined based on a proportional relationship, compared to setting different second headrooms for different scenarios, the number of second headrooms that need to be maintained is reduced, simplifying the configuration process and reducing complexity.
[0373] For example, if the encoding end directly transmits the second headroom, the decoding end can directly obtain the second headroom, which improves the accuracy of obtaining the second headroom. At the same time, it does not require determining the second headroom through additional proportional relationships, which simplifies the process of determining the second headroom and reduces the decoding complexity.
[0374] In a specific embodiment, the third information is the third headroom, or the third information is the ratio between the third headroom and the second headroom.
[0375] For example, the encoder may determine the third headroom as the third information, or the encoder may determine the ratio of the third headroom to the second headroom as the third information.
[0376] Correspondingly, if the third information is the third headroom, the decoder will determine the third information as the third headroom. If the third information is the ratio of the third headroom to the second headroom, the decoder will determine the third headroom as the product of the third information and the second headroom.
[0377] In the embodiments of this application, the encoder can select to transmit the third headroom or the ratio between the third headroom and the second headroom according to different scenario requirements (such as decoding complexity and encoding / decoding efficiency requirements). In this way, the decoder can determine the third headroom based on the most suitable method for the scenario in different scenarios, which improves the flexibility and adaptability of determining the third headroom.
[0378] For example, the third headroom is determined based on the ratio between the third and second headrooms. It can adaptively adjust to changes in the second headroom, improving the flexibility and adaptability of its determination. Furthermore, carrying this ratio in the bitstream reduces the amount of data compared to carrying the third headroom itself, thus improving encoding and decoding efficiency. Additionally, it helps improve interoperability and consistency between different codecs. Moreover, since the third headroom is determined based on a ratio, compared to setting different third headrooms for different scenarios, it reduces the number of third headrooms that need to be maintained, simplifying the configuration process and reducing complexity.
[0379] For example, if the encoding end directly transmits the third headroom, the decoding end can directly obtain the third headroom, which improves the accuracy of the obtained third headroom. At the same time, it does not require additional proportional relationships to determine the third headroom, simplifying the process of determining the third headroom and reducing the decoding complexity.
[0380] In an optional embodiment, the target image is an image obtained based on the first image, the first information, the second headroom, the third headroom, and the display brightness of the aforementioned display device.
[0381] In a specific embodiment, the decoder can determine the target image based on the first image, the first information, the second headroom, the third headroom, and the display brightness.
[0382] For example, the second segment of the metadata in this application embodiment may include a second headroom and a third headroom. The decoder may also obtain the display brightness of the display device on which the target image is to be displayed, such as the maximum display brightness, or the range of display brightness, or the current display brightness, without limitation. Thus, using the second headroom, the third headroom, and the display brightness of the display device, the second enhancement layer data is used to enhance the dynamic range of the first image to a full or partial extent.
[0383] In this embodiment, the target image is obtained based on a first image, first information, a second headroom, a third headroom, and the display brightness of the display device. Thus, the headroom of the first image whose dynamic range is to be adjusted (i.e., the second headroom) and the headroom of the image obtained by adjusting the first image to its maximum (or minimum) dynamic range using the second enhancement layer data (hereinafter referred to as the third image) (i.e., the third headroom) can be referenced. In other words, the obtained target image is obtained by flexibly adjusting the degree of dynamic range of the first image based on the second enhancement layer data, referencing the source headroom, the target headroom, and the display brightness of the display device. This allows for precise control of the target image's dynamic range, avoiding overexposure or loss of detail in the target image obtained after dynamic range adjustment, resulting in better image quality. Furthermore, the target image obtained by this method is obtained by adaptively changing the adjustment degree with reference to the display brightness of the display device. The degree of adjustment of the brightness range of the target image obtained by adjusting the first image is determined based on the display capabilities of the current display device, allowing the target image's dynamic range to better adapt to different display devices and achieve a more consistent visual effect across different display devices.
[0384] In an optional embodiment, the target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data. The weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness of the aforementioned display device.
[0385] In a specific embodiment, the decoder can determine the weight of the second enhancement layer data based on the second headroom, the third headroom, and the display brightness of the aforementioned display device, and the decoder can determine the target image based on the first image, the first information, and the weight of the second enhancement layer.
[0386] In some embodiments, the decoder may multiply the second enhancement layer data by the aforementioned weights, and then merge the weighted second enhancement layer data with the first image to obtain a merged image. The decoder uses the merged image (e.g., the second image) to obtain the target image.
[0387] For example, the decoder can determine the merged image by multiplying the first image and the second enhancement layer data; or, the decoder can determine the merged image by summing the first image and the second enhancement layer data; or, the decoder can determine the merged image by the ratio of the first image to the second enhancement layer data.
[0388] For example, the decoder can determine the weights of the second enhancement layer data through a mapping function. For instance, the weights of the second enhancement layer data can be defined as f(second headroom, third headroom, display brightness), where f() is the mapping function, and the second headroom, third headroom, and display brightness are the parameters of the mapping function f(). Depending on the specific application requirements, this application can use any function for this mapping function, and it does not impose any restrictions on the specific logic of the mapping function.
[0389] Formulas (8) to (13) above exemplify several possible merging methods between the second enhancement layer data and the first image. Further possible merging methods are listed below:
[0390] For example, newPictue[i] = Picture1[i] * enhancement2 * w, formula (14), where w = w1 = f(headroom_src, headroom_dst, max_light);
[0391] Alternatively, newPictue[i] = (Picture1[i] + enhancement2) * w, formula (15), w = w1, the explanation of w1 is the same as formula (14);
[0392] Alternatively, newPictue[i] = (Picture1[i] / enhance2) * w, formula (16), w = w1, the explanation of w1 is the same as formula (14).
[0393] Where Picture1 is the first image, enhancement2 is the second enhancement layer data, f(headroom_src, headroom_dst, max_light) is the weight w1 of the second enhancement layer data, headroom_src is the source headroom (here, the second headroom), headroom_dst is the target headroom (here, the third headroom), max_light is the display brightness of the display device (here, the maximum display brightness), and newPictue is the target image or the second image.
[0394] In the above formulas, 'i' represents the i-th pixel in the image.
[0395] The "*" in formulas (14) to (16) above can also be superimposed or replaced with "addition, i.e. +" or "division, i.e. / ".
[0396] In this embodiment, the target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data. The weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness of the display device. During the adjustment of the dynamic range of the first image, the weights of the second enhancement layer data are referenced, and the degree of dynamic range adjustment is flexibly determined based on these weights. Since the weights are based on the second headroom, the third headroom, and the display brightness of the display device, the degree of dynamic range adjustment is flexibly determined based on the display brightness of the display device. This allows for precise control of the target image's dynamic range, avoiding overexposure or loss of detail in the target image obtained after dynamic range adjustment. This results in better image quality and allows the target image's dynamic range to better adapt to different display devices, achieving a more consistent visual effect across different display devices.
[0397] In an optional embodiment, the display brightness supported by the display device may be a brightness range supported by the display device or a specific brightness value supported by the display device. For example, the specific brightness value may be the maximum display brightness (also known as peak brightness) or the minimum display brightness.
[0398] For example, the decoder can use the maximum display brightness supported by the display device as the display brightness of the display device referenced when obtaining the target image.
[0399] When the display brightness is at its maximum, the decoder determines the weights of the second enhancement layer data as w1 = f(headroom_src, headroom_dst, max_light).
[0400] When the display brightness is within a certain range, the decoder determines the weight of the second enhancement layer data based on any display brightness within that range. For example, the decoder determines the weight of the second enhancement layer data as w1 = (headroom_src, headroom_dst, light_X), where light_X is the display brightness of the display device (here, any display brightness within the brightness range of the display device).
[0401] For example, if the display device supports a maximum display brightness of 1000, and the user adjusts the backlight of the display device to make the current display brightness of the display device 800, then the decoder can respond to the user's backlight adjustment operation to determine the weight w1 of the second enhancement layer data based on the current display brightness "800" and the aforementioned second headroom and third headroom.
[0402] For example, headroom can refer to the ratio between the maximum display brightness a display device can handle and diffuse white light without causing overexposure or loss of detail. In other words, headroom indicates the extent to which a display device can accurately reproduce very bright light sources or reflected light. If a white sheet of paper has a brightness of 200 nits in sunlight, and the display device's maximum display brightness is 1000 nits, then the headroom ratio is 5:1 (i.e., 1000 divided by 200). This means that the display device is capable of displaying the brightest parts with a brightness five times higher than diffuse white light, thus better simulating the effects of direct sunlight or other highlight conditions.
[0403] In this embodiment, during the process of adjusting the dynamic range of the first image based on the second enhancement layer data, the degree of adjustment of the dynamic range of the first image by the second enhancement layer data is determined based on the maximum display brightness of the display device. This ensures that the dynamic range of the obtained target image is consistent with the maximum display brightness of the display device, avoiding high power consumption caused by the dynamic range of the target image being greater than the maximum display brightness of the display device, and also avoiding the problem of not fully utilizing the display capability of the display device and poor display effect when the dynamic range of the target image is less than the maximum display brightness of the display device.
[0404] The process of acquiring the first image is described below.
[0405] In an optional embodiment, the first image is an image with a first dynamic range obtained based on base layer data, first enhancement layer data, first headroom, and second headroom.
[0406] In a specific embodiment, the decoder can determine a first image with a first dynamic range based on the base layer data, the first enhancement layer data, the first headroom, and the second headroom.
[0407] For example, the decoder may determine the weight w2 of the first enhancement layer data based on the first headroom and the second headroom, and the decoder may determine the first image based on the base layer data, the first enhancement layer data, and the weight of the first enhancement layer.
[0408] In some embodiments, when obtaining the first image, formulas (14) to (16) above can also be used, but w is w2 = f(headroom_src, headroom_dst), where headroom_src represents the headroom of the basic layer data (i.e., the first headroom mentioned above), and headroom_dst is the second headroom mentioned above. newPictue[i] represents the first image, Picture1[i] represents the basic layer data, and enhancement2 here represents the first enhancement layer data. The physical meaning of i can be referred to the description following formula (7) above, and will not be repeated here.
[0409] In this embodiment, a first image is determined based on the base layer data, the first enhancement layer data, the first headroom, and the second headroom. The adjustment of the dynamic range of the base layer data by the first headroom of the base layer data and the first enhancement layer data is taken into account, so that the base layer data can be better adjusted to obtain a first image that meets the expected first dynamic range.
[0410] The following describes possible implementations of the decoder obtaining the second and third headrooms.
[0411] In an optional embodiment, the second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
[0412] Second headroom: The encoder can determine the second headroom based on the first dynamic range and the dynamic range of the base layer data.
[0413] In an optional embodiment, the encoder may determine the second headroom based on a functional relationship between the first dynamic range and the dynamic range of the base layer data. This functional relationship may be a ratio function, a logarithmic function, an exponential function, etc.
[0414] For example, the encoder performs a calculation with the first dynamic range as the numerator of the ratio function and the dynamic range of the base layer data as the denominator of the ratio function, and determines the result as the second headroom. For example, the first dynamic range is dynamic range A, the dynamic range of the base layer data is dynamic range B, and the second headroom is "dynamic range A / dynamic range B".
[0415] Alternatively, the encoder can perform a ratio function with the dynamic range of the base layer data as the numerator and the first dynamic range as the denominator, and determine the result as the second headroom. For example, if the first dynamic range is dynamic range A, the dynamic range of the base layer data is dynamic range B, and the second headroom is "dynamic range B / dynamic range A". The symbol " / " represents division.
[0416] For example, the encoder performs an operation with the first dynamic range as the base of an exponential function and the dynamic range of the base layer data as the exponent of the exponential function. The result of this operation is determined as the second headroom. For example, the second headroom is "dynamic range A". 动态范围B ".
[0417] Alternatively, the encoder performs calculations using the dynamic range of the base layer data as the base of an exponential function and the first dynamic range as the exponent of the exponential function. The result of this calculation is then defined as the second headroom. For example, the second headroom might be "Dynamic Range B". 动态范围A ".
[0418] For example, the encoder performs a calculation using the first dynamic range as the base of the log function and the dynamic range of the base layer data as the exponent of the log function. The result of this calculation is then used as the second headroom. For instance, the second headroom is "log". 动态范围A Dynamic range B.
[0419] Alternatively, the encoder uses the dynamic range of the base layer data as the base of the log function and the first dynamic range as the exponent of the log function, performs the calculation, and determines the result as the second headroom. For example, the second headroom is "log". 动态范围B Dynamic range A.
[0420] Third headroom: The encoder can determine the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the data from the second enhancement layer as the third dynamic range. Then, the encoder can determine the third headroom based on the third dynamic range and the first dynamic range.
[0421] In an optional embodiment, the encoder may determine the third headroom based on the functional relationship between the first dynamic range and the third dynamic range. This functional relationship may be a fractional function, a logarithmic function, an exponential function, etc.
[0422] For example, the encoder performs an operation with the first dynamic range as the numerator of the fractional function and the third dynamic range as the denominator of the fractional function, and determines the result of the operation as the third headroom. For example, the first dynamic range is dynamic range A, the third dynamic range is dynamic range C, and the third headroom is "dynamic range A / dynamic range C".
[0423] Alternatively, the encoder performs calculations using the third dynamic range as the numerator of a fractional function and the first dynamic range as the denominator, and determines the result as the third headroom. For example, the first dynamic range is dynamic range A, the third dynamic range is dynamic range C, and the third headroom is "dynamic range C / dynamic range A".
[0424] For example, the encoder performs an operation with the first dynamic range as the base of an exponential function and the third dynamic range as the exponent of the exponential function, and determines the result as the third headroom. For example, the third headroom is "Dynamic Range A". 动态范围C ".
[0425] Alternatively, the encoder performs calculations using the third dynamic range as the base of the exponential function and the first dynamic range as the exponent, and determines the result as the third headroom. For example, the third headroom is "dynamic range C". 动态范围A ".
[0426] For example, the encoder performs calculations using the first dynamic range as the base of the log function and the third dynamic range as the exponent of the log function, and determines the result as the third headroom. For instance, the third headroom is "log". 动态范围A Dynamic range C.
[0427] Alternatively, the encoder performs the operation using the third dynamic range as the base of the log function and the first dynamic range as the exponent, and determines the result as the third headroom. For example, the third headroom is "log". 动态范围C Dynamic range A.
[0428] For example, when the weight w1 of the second enhancement layer data is equal to 1, that is, when the first image is adjusted to the full extent of its dynamic range (e.g., enhanced) based on the second enhancement layer data, a target image with a second dynamic range can be obtained, and the functional relationship between the second dynamic range and the first dynamic range is the third headroom; when the weight of the second enhancement layer data is less than 1, that is, when the first image is adjusted to a partial extent of its dynamic range (e.g., enhanced) based on the second enhancement layer data, the functional relationship between the second dynamic range and the first dynamic range is not the third headroom.
[0429] The above embodiments use one enhancement layer data (second enhancement layer data) to enhance the first image to obtain the target image. This application also provides another embodiment, which can use two or more enhancement layer data (e.g., second enhancement layer data and third enhancement layer data) to enhance the first image to obtain the target image.
[0430] The following is combined Figure 4 The implementation process of another data processing method in this embodiment will be introduced.
[0431] exist Figure 4 In the text, regarding the process of acquiring the bitstream and obtaining the first image (e.g.) Figure 4 S301 and S303 shown are both related to Figure 3 The implementation processes shown (such as S301 and S303) are based on the same principle and achieve similar technical effects, so the same steps will not be repeated here.
[0432] Specifically, such as Figure 4 As shown, the method flow may also include, but is not limited to, the following steps:
[0433] Following S301, S306 decodes the bitstream to obtain information from it.
[0434] In some optional embodiments, the bitstream further includes fourth information indicating the third enhancement layer data, wherein the target image is an image with a second dynamic range obtained based on the second image' and the fourth information, and the second image' is an image with a third dynamic range obtained based on the first image and the first information indicating the second enhancement layer data of the original image. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data. The dynamic range of the second image' is this maximum or minimum dynamic range.
[0435] In optional embodiments, the fourth information may be third enhancement layer data (e.g., enhancement map), or a curve or conversion information (e.g., a table) used to fit the conversion relationship between the fourth dynamic range and the third dynamic range. The fourth dynamic range is the maximum or minimum dynamic range to which the second image can be adjusted from the third dynamic range using the third enhancement layer data. The fourth dynamic range is the dynamic range of the third image. The third image may be the image obtained after adjusting the second image from the third dynamic range to the fourth dynamic range using the third enhancement layer data. This ensures that the second dynamic range adjusted from the third dynamic range to the fourth dynamic range based on the display brightness of the display device is the same as the fourth dynamic range, or that the second dynamic range is smaller than the fourth dynamic range.
[0436] Continue to refer to Figure 4 Optionally, S303 may be included after S306, see details below. Figure 3 The introduction will not be repeated here.
[0437] Optionally, the method process also includes steps S307 and S308.
[0438] S307. Synthesize a second image based on the first image, the second enhancement layer data, and metadata (e.g., a second segment of the metadata).
[0439] Different from Figure 3 In the second image synthesized in the embodiment, there is no need to refer to the display brightness of the display device when synthesizing the second image. The other processes are the same as those for synthesizing the second image, and will not be described in detail.
[0440] In an optional embodiment, the decoder may adjust the first dynamic range of the first image based on the second enhancement layer data to obtain the second image.
[0441] S308. Based on the second image and the third enhancement layer data, optionally combine the metadata of the original image (e.g., the part of the metadata related to the third enhancement layer data, referred to as the third segment) to synthesize the third image.
[0442] In an optional embodiment, the decoder may enhance the second image based on the third enhancement layer data to obtain the third image.
[0443] The principle of obtaining the third image in step S308 is the same as described above. Figure 3The principle of obtaining the second image in the embodiment is the same. Referring to the above formulas (14) to (16), when determining the weight w3 of the third enhancement layer data, where w = w3, it is necessary not only to refer to the source headroom (here, the headroom of the second image) and the target headroom (here, the fourth headroom corresponding to the third enhancement layer data), but also to refer to the display brightness of the display device, which will not be elaborated further.
[0444] In an optional embodiment, the third image is an image obtained based on the second image, the fourth information, metadata (e.g., the third segment of the metadata, i.e. the part of the metadata related to the third enhancement layer data), and the display brightness supported by the display device.
[0445] In a specific embodiment, the decoder can obtain a third image with a second dynamic range based on the second image, the fourth information, metadata, and the display brightness supported by the display device.
[0446] For example, the decoder can adjust the dynamic range of the second image based on the fourth information and the display brightness of the display device, so that its dynamic range is adjusted from the third dynamic range to the second dynamic range.
[0447] In some embodiments, the metadata also includes fifth information of the fourth headroom corresponding to the third enhancement layer data. The decoder can determine the weight of the third enhancement layer data based on the third headroom, the fourth headroom, and the display brightness of the display device. The decoder can determine the third image based on the second image, the fourth information, and the weight w3 of the third enhancement layer data.
[0448] The aforementioned fourth headroom is determined based on the fourth dynamic range and the third dynamic range. The fourth dynamic range is the dynamic range of the aforementioned third image ', and the third dynamic range is the dynamic range of the aforementioned second image '.
[0449] Taking this specific function as a ratio function as an example, for instance, the fourth headroom is the ratio of the peak brightness of the fourth dynamic range to the peak brightness of the third dynamic range. Here, the fourth dynamic range is the maximum dynamic range (e.g., the third enhancement layer data is an integer, such as 5 times) or the minimum dynamic range (e.g., the third enhancement layer data is a decimal, such as 1 / 5 times) to which the second image can be adjusted from the third dynamic range using the third enhancement layer data on the decoding side.
[0450] The fifth piece of information can be the fourth headroom, or it can be the proportional relationship between the fourth headroom and the aforementioned third headroom.
[0451] For example, when using the principle of any one of the above formulas (14) to (16) to obtain the third image, newPictue[i] in the above formulas (14) to (16) is the third image, Picture1[i] is the second image, enhancement2 is the third enhancement layer data, and w is w3.
[0452] w3=f(headroom_src, headroom_dst, max_light).
[0453] Where w3 is the weight of the third enhancement layer data, headroom_src is the source headroom (here, the third headroom), headroom_dst is the target headroom (here, the fourth headroom), and max_light is the display brightness of the display device (here, the maximum display brightness).
[0454] S305. Determine the target image based on the third image.
[0455] In an optional embodiment, the decoder may identify the third image as the target image.
[0456] In an optional embodiment, the decoder may process the third image to obtain the target image. For example, the decoder may perform resolution conversion on the third image to obtain a third image with converted resolution, and determine the third image with converted resolution as the target image.
[0457] For example, the principle of obtaining the third image in S305 is the same as that of obtaining the third image in S308, and will not be elaborated here.
[0458] In this embodiment, during the adjustment of the dynamic range of the first image based on the third enhancement data, the display brightness supported by the display device is also considered. This ensures that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream is an image with a second dynamic range, obtained based on the third enhancement layer data, the aforementioned fourth information, and the display brightness supported by the display device. Thus, the target image with the second dynamic range is obtained by readjusting the dynamic range of the second image with the third dynamic range based on the display brightness supported by the display device. This allows the second dynamic range of the target image to adapt to the display brightness of the display device, and also enables better utilization of the display device's display capabilities while reducing the power consumption of the display device and improving the image display effect.
[0459] In this embodiment, considering that the third dynamic range of the second image, obtained based on the first image, the second enhancement layer data, and metadata, may not match the display capabilities of the display device, resulting in the display device failing to fully utilize its display capabilities or experiencing high power consumption when displaying the second image, the bitstream in this embodiment may include three or more enhancement layer data to further improve the adaptability of the target image to the display capabilities of the display device. This allows the decoding end to generate the target image according to the display capabilities of the display device. For example, the bitstream in this embodiment may also include fourth information indicating the third enhancement layer data, so that the reconstructed image (i.e., the target image) of the original image obtained by decoding the bitstream is an image with a second dynamic range obtained based on the third enhancement layer data and the aforementioned fourth information. In this way, the target image with the second dynamic range is obtained based on the second image with the third dynamic range and the third enhancement layer data, enabling the second dynamic range of the obtained target image to adapt to the display capabilities of the display device. Furthermore, it can better utilize the display capabilities of the display device while reducing the power consumption of the display device and improving the image display effect.
[0460] For example, if the third dynamic range is smaller than the brightness range or maximum brightness of the display device's display capability, and the second image is directly used as the reconstructed image of the target image, the display device cannot fully utilize its display capability when displaying the target image. Therefore, the third dynamic range can be higher than the first dynamic range. For example, if the second image is an SDR image and the target image is an HDR image, the method of this application embodiment can enhance the dynamic range of the second image based on the third enhancement layer data, increasing it from the third dynamic range to the second dynamic range. In this way, the second dynamic range can be adapted to the display capability of the display device. For example, the second dynamic range can be the dynamic range that matches the maximum brightness (also called peak brightness) of the display device, or it can be a dynamic range close to but lower than the peak brightness. Thus, in scenarios where the display device has strong display performance, the method of this application embodiment can display images with a higher dynamic range within the display capability of the display device, better utilizing the display capability of the display device and improving the image display effect.
[0461] For example, if the third dynamic range is greater than the brightness range or maximum brightness value of the display device's display capability, directly displaying the second image using the display device would result in high power consumption, leading to high power consumption when displaying the second image. Therefore, the second dynamic range can be lower than the third dynamic range; for example, the second image might be an HDR image, and the target image might be an SDR image. Thus, the method of this embodiment can reduce the dynamic range of the second image based on the third enhancement layer data, reducing it from the third dynamic range to the second dynamic range. This allows the second dynamic range to adapt to the display device's display capability. For example, the second dynamic range could be within the dynamic range of the display device's maximum brightness (also known as peak brightness), or it could be close to but lower than that peak brightness. In scenarios where the display device's display performance is weak, the method of this embodiment can enable the display device to display images within its display capability range, reducing the display device's power consumption.
[0462] Based on the same inventive concept as the above method, such as Figure 5 As shown, this embodiment of the invention also provides a data transmission device 500, which includes an acquisition module 501 and a decoding module 502.
[0463] The acquisition module 501 is used to acquire a bitstream, which includes information indicating a first image with a first dynamic range and first information indicating second enhancement layer data of the original image; the decoding module 502 is used to decode the bitstream to obtain a target image, which is an image with a second dynamic range obtained based on the first image and the first information, the first image being obtained according to the information of the first image indicating the first dynamic range, and the target image being a reconstructed image of the original image.
[0464] In some possible implementations, the information indicating the first image with the first dynamic range includes base layer data of the original image, first enhancement layer data of the original image, and metadata of the original image, wherein the first image is an image with the first dynamic range obtained based on the base layer data, the first enhancement layer data, and the metadata.
[0465] In some possible implementations, the bitstream includes metadata of the original image, and the target image is an image obtained based on the first image, the first information, the metadata of the original image, and the display brightness supported by the display device.
[0466] In some possible implementations, the display brightness supported by the display device is the maximum display brightness supported by the display device.
[0467] In some possible implementations, the metadata includes a first headroom for the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data. The second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
[0468] In some possible implementations, the target image is an image obtained based on a first image, first information, a second headroom, a third headroom, and display brightness.
[0469] In some possible implementations, the target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data, wherein the weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness.
[0470] In some possible implementations, the first image is an image with a first dynamic range obtained based on base layer data, first enhancement layer data, first headroom, and second headroom.
[0471] In some possible implementations, the second information is the second headroom or the ratio between the second headroom and the first headroom.
[0472] In some possible implementations, the third information is the third headroom or the proportional relationship between the third headroom and the second headroom.
[0473] In some possible implementations, the target image is an image with a second dynamic range obtained based on a first image converted to the same color gamut and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0474] In some possible implementations, the target image is an image with a second dynamic range obtained based on a first image converted to the target resolution and second enhancement layer data, the second enhancement layer data being obtained based on the first information.
[0475] In some possible implementations, the target resolution differs from the resolution of the base layer data.
[0476] In some possible implementations, the first dynamic range is the same as the second dynamic range, and the pixel values of corresponding pixels in the first image and the target image are different.
[0477] In some possible implementations, the bitstream also includes fourth information indicating the third enhancement layer data, the target image being an image with a second dynamic range obtained based on the second image and the fourth information, and the second image being an image with a third dynamic range obtained based on the first image and the first information.
[0478] In some possible implementations, the target image is an image obtained based on the second image, the fourth information, metadata, and the display brightness supported by the display device.
[0479] This application embodiment also provides a data processing device 600, such as... Figure 6 As shown, the data processing device 600 includes an acquisition module 601 and an encoding module 602.
[0480] The acquisition module 601 is used to acquire the original image; the encoding module 602 is used to generate a bitstream corresponding to the original image. The bitstream includes the basic layer data of the original image, the first enhancement layer data of the original image, the first information indicating the second enhancement layer data of the original image, and the metadata of the original image. The bitstream is used by the decoding end to generate a reconstructed image of the original image.
[0481] This application also provides a chip, including one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the above-described related method steps are executed to achieve the method in the above embodiments. This embodiment also provides a computer-readable storage medium storing computer instructions, which, when executed on an electronic device, cause the electronic device to execute the above-described related method steps to achieve the method in the above embodiments. Exemplarily, the computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0482] This embodiment also provides a computer program product comprising computer instructions that, when executed by a computer or processor, cause the computer to perform the aforementioned steps to implement the methods described in the above embodiments. Exemplarily, the computer program product may be stored in a computer-readable storage medium.
[0483] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the methods in the above-described method embodiments.
[0484] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
[0485] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.
[0486] For example, the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0487] It should be understood that the memory or storage medium mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be ROM, programmable read-only memory (PROM), EPROM, EEPROM, or flash memory. Volatile memory can be RAM, which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0488] When the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.
[0489] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.
[0490] This application also provides a computer-readable storage medium, including a program or instructions that, when run on a computer, cause the encoding / decoding method in the method embodiments described above to be implemented.
[0491] In the embodiments of this application, the term "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where any one of a, b, c, ab, ac, bc, or abc can include a single a, a single b, a single c, or multiple a, multiple b, multiple c.
[0492] Furthermore, unless otherwise stated, the ordinal numbers such as "first" and "second" mentioned in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, sequence, priority, or importance of multiple objects. For example, the first priority criterion and the second priority criterion are only used to distinguish different criteria, and do not indicate that the content, priority, or importance of these two criteria are different.
[0493] Furthermore, the terms "comprising" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules / units is not limited to the listed steps or modules, but may also include steps or modules / units not listed.
[0494] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0495] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0496] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0497] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0498] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A data processing method, characterized in that, The method includes: Acquire a bitstream, the bitstream including information indicating a first image of a first dynamic range and first information indicating second enhancement layer data of the original image; The bitstream is decoded to obtain a target image, which is an image with a second dynamic range obtained based on the first image and the first information. The first image is obtained based on the information of the first image indicating the first dynamic range, and the target image is a reconstructed image of the original image.
2. The method according to claim 1, characterized in that, The information indicating the first image with the first dynamic range includes the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image. The first image is an image with the first dynamic range obtained based on the base layer data, the first enhancement layer data, and the metadata.
3. The method according to claim 1 or 2, characterized in that, The bitstream includes metadata of the original image, and the target image is an image obtained based on the first image, the first information, the metadata of the original image, and the display brightness supported by the display device.
4. The method according to claim 3, characterized in that, The display brightness supported by the display device is the maximum display brightness supported by the display device.
5. The method according to claim 3 or 4, characterized in that, The metadata includes a first luminance headroom of the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data. The second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
6. The method according to claim 5, characterized in that, The target image is an image obtained based on the first image, the first information, the second headroom, the third headroom, and the display brightness.
7. The method according to claim 5 or 6, characterized in that, The target image is an image obtained based on the weights of the first image, the first information, and the second enhancement layer data. The weights of the second enhancement layer data are obtained based on the second headroom, the third headroom, and the display brightness.
8. The method according to any one of claims 5 to 7, characterized in that, The first image is an image with a first dynamic range obtained based on the base layer data, the first enhancement layer data, the first headroom, and the second headroom.
9. The method according to any one of claims 5 to 8, characterized in that, The second information is the second headroom or the ratio between the second headroom and the first headroom.
10. The method according to any one of claims 5 to 9, characterized in that, The third information is the third headroom or the ratio between the third headroom and the second headroom.
11. The method according to any one of claims 1 to 10, characterized in that, The target image is an image with the second dynamic range obtained based on the first image after conversion to the same color gamut and the second enhancement layer data, wherein the second enhancement layer data is obtained based on the first information.
12. The method according to any one of claims 1 to 11, characterized in that, The target image is an image with the second dynamic range obtained based on the first image converted to the target resolution and the second enhancement layer data, wherein the second enhancement layer data is obtained based on the first information.
13. The method according to claim 12, characterized in that, The target resolution is different from the resolution of the base layer data.
14. The method according to any one of claims 1 to 13, characterized in that, The first dynamic range is the same as the second dynamic range, and the pixel values of corresponding pixels in the first image and the target image are different.
15. The method according to claim 1 or 2, characterized in that, The bitstream also includes fourth information indicating the third enhancement layer data, the target image is an image with a second dynamic range obtained based on the second image and the fourth information, and the second image is an image with a third dynamic range obtained based on the first image and the first information.
16. The method according to claim 15, characterized in that, The target image is an image obtained based on the second image, the fourth information, the metadata, and the display brightness supported by the display device.
17. A data processing method, characterized in that, The method includes: Obtain the original image; A bitstream corresponding to the original image is generated. The bitstream includes information indicating a first image with a first dynamic range and first information indicating the second enhancement layer data of the original image. The bitstream is used by the decoding end to generate a reconstructed image of the second dynamic range of the original image.
18. The method according to claim 17, characterized in that, The information of the first image indicating the first dynamic range includes the base layer data of the original image, the first enhancement layer data of the original image, and the metadata of the original image.
19. The method according to claim 18, characterized in that, The metadata includes a first luminance headroom of the base layer data, second information indicating a second headroom corresponding to the first enhancement layer, and third information indicating a third headroom corresponding to the second enhancement layer data. The second headroom is determined based on the first dynamic range and the dynamic range of the base layer data, and the third headroom is determined based on the third dynamic range and the first dynamic range. The third dynamic range is the maximum or minimum dynamic range to which the first image can be adjusted from the first dynamic range using the second enhancement layer data.
20. An encoder, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 17-19.
21. A decoder, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 16.
22. An encoder, characterized in that, include: A processing circuit that implements the method as described in any one of claims 17-19.
23. A decoder, characterized in that, include: A processing circuit that implements the method as described in any one of claims 1 to 16.
24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 16, or causes the computer or processor to perform the method as described in any one of claims 17 to 19.
25. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a computer or processor, cause the method as described in any one of claims 1 to 16 to be performed, or cause the method as described in any one of claims 17 to 19 to be performed.
26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a bitstream generated according to the method described in any one of claims 17-19.
