Tone mapping method and apparatus

Abstract

The present application provides a tone mapping method and apparatus, in which tone mapping is performed on an image to be displayed based on reference information and corresponding types of terminal device information. The reference information includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution. The terminal device information is a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution. picture In an embodiment of the present application, the tone mapping is performed on the image to flexibly adapt to various environments and display devices, and the terminal display device resolution is at least one of the following: size, user viewing distance, user viewing angle, ambient light brightness, or terminal display device resolution. picture Other information such as size and user viewing distance is also taken into consideration, so that the display adaptation effect is effectively improved.

Description

[Technical Field]

[0001] This application relates to the field of display technology, and in particular to a tone mapping method and apparatus. [Background technology]

[0002] Dynamic range is the ratio of the maximum value to the minimum value of a variable in many fields. In a digital image, the dynamic range is the ratio of the minimum to the maximum luminance value in the displayable range of the image. In the same real-world scene, the dynamic range is typically 10 -3 Candela per square meter (cd / m 2 )~10 6 cd / m 2 High dynamic range (HDR) video technology expands the displayable luminance range of an image, capturing information across a wider range of real-world luminance, allowing more detail in bright and dark areas to be displayed within the image. However, the luminance of the video or image typically does not match the displayable luminance range of a standard dynamic range (SDR) display device or many other existing HDR display devices.

[0003] Therefore, when a video or image is displayed on an existing display device, the brightness of the video or image needs to be processed based on the display capability of the display device so that the brightness of the video or image fits into the displayable brightness range of the display device. This process is called tone mapping (TM).

[0004] However, in existing solutions, only the display luminance range of the display screen is considered during tone mapping, and the optimal display adaptation effect cannot be achieved. Summary of the Invention

[0005] The present application provides a tone mapping method and apparatus, and when tone mapping is performed on an image, other information such as terminal display device size and user viewing distance can also be taken into account, thereby improving the display adaptation effect. [Means for solving the problem]

[0006] According to a first aspect, an embodiment of the present application provides a tone mapping method, the method including: obtaining a display target image; obtaining reference information, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; obtaining terminal device information, the terminal device information including at least one of a terminal display device size, a user viewing distance, a user viewing angle, an ambient light luminance, or a terminal display device resolution; and performing tone mapping on the display target image based on the reference information and a corresponding type of terminal device information.

[0007] In this embodiment of the present application, when performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information, the reference display device size corresponds to the terminal display device size, the reference viewing distance corresponds to the user viewing distance, the reference viewing angle corresponds to the user viewing angle, the reference ambient light brightness corresponds to the ambient light brightness, and the reference display device resolution corresponds to the terminal display device resolution.

[0008] In this embodiment of the present application, when tone mapping is performed on an image, other information such as the terminal display device size and the user viewing distance is also taken into consideration, thus improving the display adaptation effect.

[0009] In one possible implementation, the step of performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information includes the steps of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information, and performing tone mapping on the image to be displayed based on the tone mapping parameters.

[0010] In one possible implementation, the step of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information includes a step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters.

[0011] In this implementation of the present application, the initial tone mapping parameters of the image to be displayed are adjusted based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters. In this way, in order to flexibly adapt to various environments and display devices, the tone mapping parameters are varied according to the terminal display device size, user viewing distance, user viewing angle, ambient light brightness, or terminal display device resolution, thereby effectively improving the display adaptation effect and improving the tone mapping effect and display effect of the image. In addition, the user's visual perception is more consistent in various application scenarios, thereby improving the user's viewing experience.

[0012] In one possible implementation, the step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain tone mapping parameters includes: adjusting the initial tone mapping parameters of the image to be displayed in a first manner to obtain the tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a first condition; and adjusting the initial tone mapping parameters of the image to be displayed in a second manner to obtain the tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a second condition. The initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve.

[0013] In this implementation, the method for adjusting the initial tone mapping parameters corresponds to the relationship between the reference information and the terminal device information, so that different adjustment methods are used for different application scenarios to obtain different adjusted tone mapping parameters, thereby meeting the user requirements of different application scenarios, adapting to changing application scenarios, and improving the user experience.

[0014] In one possible implementation, the first condition includes at least one of the following: the terminal display screen size is smaller than the reference display device size; the user viewing distance is larger than the reference viewing distance; the user viewing angle deviates from the reference viewing angle; the ambient light luminance is larger than the reference ambient light luminance; or the terminal display device resolution is smaller than the reference display device resolution.

[0015] In this implementation, if the terminal display screen size is smaller than the reference display device size, the image visually perceived by the user will be dark or have insufficient contrast. Similarly, if the user viewing distance is larger than the reference viewing distance, the user viewing angle is deviated from the reference viewing angle, the ambient light luminance is larger than the reference ambient light luminance, or the terminal display device resolution is smaller than the reference display device resolution, the image visually perceived by the user will be dark or blurred, affecting the user's viewing experience. Therefore, adjusting the tone mapping parameters in the first manner helps improve the overall brightness and detail definition of the image and can improve the image tone mapping effect in various application scenarios, thereby meeting the user requirements of various application scenarios and improving the user's visual experience.

[0016] In one possible implementation, the step of adjusting initial tone mapping parameters of the image to be displayed in a first manner includes the steps of obtaining one or more first luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained mapped values ​​of the adjusted tone mapping curve at the one or more first luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at the one or more first luminance feature values.

[0017] In this implementation, to adapt to application scenarios covered by the first condition, improve the display adaptation effect, and enhance the user's overall luminance perception of the image, the mapped values ​​of the adjusted tone mapping curve for one or more first luminance feature values ​​are increased, thereby increasing all or part of the mapped values ​​of the adjusted tone mapping curve and enhancing all or part of the luminance of the image obtained by tone mapping on the image based on the tone mapping parameters. In this way, the user's luminance perception is consistent in various application scenarios, improving the user experience. For example, if the terminal display screen size is smaller than the reference display device size, the mapped values ​​of the tone mapping curve are increased to improve the luminance of the image. In this way, the user's visual luminance perception on a small display screen is the same as that on a large display screen, improving the user's visual comfort.

[0018] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner further includes the steps of obtaining one or more second luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained slope of the adjusted tone mapping curve at the one or more second luminance feature values ​​is greater than the slope of the initial tone mapping curve at the one or more second luminance feature values.

[0019] In this implementation, to accommodate application scenarios covered by the first condition, the slope of the adjusted tone mapping curve is increased for one or more second luminance feature values, thereby increasing all or part of the slope of the adjusted tone mapping curve and improving all or part of the contrast and luminance of the image obtained by tone mapping on the image based on the adjusted tone mapping curve. In this way, the user's perception of contrast and image detail is consistent across various application scenarios, improving the user experience and display adaptation effects. For example, if the terminal display screen size is smaller than the reference display device size, the slope of the tone mapping curve is increased to improve the image contrast and luminance. In this way, the user's perception of image definition and luminance on a small display screen is the same as that on a large display screen, improving the user's visual comfort.

[0020] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in a first manner further includes the steps of obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a first feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the enhancement layer is a second feature value, and adjusting the initial tone mapping parameters of the any of the enhancement layers so that the absolute value of the value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value.

[0021] In this implementation, the first and second feature values ​​may be positive or negative values, and when the value obtained by mapping the first feature value based on the adjusted tone mapping curve is compared with the second feature value, the absolute values ​​of the two values ​​are compared.

[0022] In this implementation, after the base layer is mixed with one or more adjusted enhancement layers, the absolute values ​​of the mapped values ​​corresponding to one or more enhancement layers in the image are increased, thereby emphasizing the features of the enhancement layers, ensuring more high-frequency components, and improving local contrast or brightness. The enhancement layers include an edge layer and a texture layer. Therefore, to accommodate the application scenarios covered by the first condition above and improve the display adaptation effect, the corresponding image feature enhancement includes edge sharpness enhancement and higher texture definition, so that complex images have better tone mapping and display effects. For example, if the terminal display screen size is smaller than the reference display device size, the mapped values ​​of the enhancement layer tone mapping curve are increased to improve the local contrast, detail definition, or brightness of the image. In this way, the definition and detail of the image viewed by a user on a small display screen are the same as those on a large display screen, improving the user's visual comfort.

[0023] In one possible implementation, the second condition includes at least one of the following: the terminal display screen size is larger than the reference display device size; the user viewing distance is smaller than the reference viewing distance; the ambient light luminance is smaller than the reference ambient light luminance; or the terminal display device resolution is larger than the reference display device resolution.

[0024] In this implementation, if the terminal display screen size is larger than the reference display device size, the image visually perceived by the user will be bright, dazzling, or have excessively high contrast, causing visual discomfort. Similarly, if the user viewing distance is smaller than the reference viewing distance, the ambient light luminance is smaller than the reference ambient light luminance, or the terminal display device resolution is larger than the reference display device resolution, the image visually perceived by the user will be bright, dazzling, causing eye harm or excessively high contrast, causing visual discomfort and affecting the user's viewing experience. Therefore, by adjusting the tone mapping curve in the second manner to reduce the overall brightness and contrast of the image, the tone mapping effect and display effect of the image can be improved in various application scenarios, thereby meeting the user requirements of various application scenarios and improving the user's visual experience.

[0025] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes the steps of obtaining one or more third luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained mapped values ​​of the adjusted tone mapping curve at the one or more third luminance feature values ​​are smaller than the mapped values ​​of the initial tone mapping curve at the one or more third luminance feature values.

[0026] In this implementation, to accommodate application scenarios covered by the second condition, the mapped values ​​of the adjusted tone mapping curve for one or more third luminance feature values ​​are reduced, thereby reducing all or part of the mapped values ​​of the adjusted tone mapping curve and reducing all or part of the luminance of the image obtained by tone mapping on the image based on the adjusted tone mapping curve. In this way, the user's luminance perception is consistent across various application scenarios, improving the display adaptation effect and user experience. For example, if the terminal display screen size is larger than the reference display device size, the mapped values ​​of the tone mapping curve are reduced to reduce the luminance of the image. In this way, the user's visual luminance perception on a large display screen is the same as that on a small display screen, improving the user's visual comfort.

[0027] In one possible implementation, the one or more third luminance feature values ​​include one or more luminance feature values ​​of dark portions of the image to be displayed.

[0028] In this implementation, because the human eye is more sensitive to the luminance of dark areas, the luminance of dark areas of the image obtained by tone mapping on the image based on the adjusted tone mapping parameters is further reduced to avoid more noise in the application scenarios covered by the second condition. In this way, the user's luminance perception of dark areas is consistent in various application scenarios, improving the display adaptation effect and user experience. For example, if the terminal display screen size is larger than the reference display device size, the mapped value of the dark areas based on the tone mapping curve is reduced to further darken the dark areas of the image. In this way, the user's visual luminance perception on a large display screen is the same as that on a small display screen, improving the user's visual comfort.

[0029] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner further includes the steps of obtaining one or more fourth luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained slope of the adjusted tone mapping curve at the one or more fourth luminance feature values ​​is smaller than the slope of the initial tone mapping curve at the one or more fourth luminance feature values.

[0030] In this implementation, to prevent excessively high contrast that causes discomfort to the user, adapt to application scenarios covered by the second condition, and improve display adaptation effects and user experience, the slope of the adjusted tone mapping curve is reduced for one or more fourth luminance feature values. This reduces all or part of the slope of the adjusted tone mapping curve, thereby reducing all or part of the contrast and luminance of the image obtained by tone mapping on the image based on the adjusted tone mapping curve. For example, if the terminal display screen size is larger than the reference display device size, the slope of the tone mapping curve is reduced to reduce the contrast and luminance of the image. In this way, the contrast of the image viewed by the user on a large display screen matches the perceived contrast and luminance on a small display screen, preventing excessively high contrast and resulting user discomfort due to the large display screen and improving the user's visual comfort.

[0031] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner further includes the steps of obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a third feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the enhancement layer is a fourth feature value, and adjusting the initial tone mapping parameters of the any of the enhancement layers so that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is smaller than the absolute value of the fourth feature value.

[0032] In this implementation, the third and fourth feature values ​​may be positive or negative values, and when the value obtained by mapping the third feature value based on the adjusted tone mapping curve is compared with the fourth feature value, the absolute values ​​of the two values ​​are compared.

[0033] In this implementation, after the base layer is mixed with one or more adjusted enhancement layers, the mapped values ​​corresponding to one or more enhancement layers in the image are reduced, thereby weakening the features of the enhancement layers, ensuring fewer high-frequency components, and reducing local contrast or brightness, i.e., reducing pixel gradients. The enhancement layers include edge layers and texture layers. Therefore, to accommodate the application scenarios covered by the second condition, avoid the poor display effect caused by excessively high contrast and its impact on the user's viewing experience, and improve display adaptation effects, the image feature weakening includes weakening edge sharpness and weakening texture details, so that complex images have better tone mapping and display effects. For example, if the terminal display screen size is larger than the reference display device size, the mapped values ​​of the enhancement layer tone mapping curve are reduced to reduce the local contrast, detail definition, and brightness of the image. In this way, the definition of the image viewed by the user on the large display screen matches the viewing definition and perception of detailed information on the small display screen, preventing excessively high contrast caused by the large display screen and the resulting discomfort to the user, and improving the user's visual comfort.

[0034] In one possible implementation, a reference tone mapping parameter of the image to be displayed is obtained. A display capability of the terminal display device in the terminal device information is obtained. The reference tone mapping parameter of the image to be displayed is adjusted based on the reference tone mapping parameter and the display capability of the terminal display device to obtain an initial tone mapping parameter.

[0035] In this implementation, the reference tone mapping parameters of the image to be displayed are adjusted based on the reference tone mapping parameters and the display capability information of the display screen, so that the resulting initial tone mapping parameters are adapted to the terminal display device in terms of display capability and are used as a basis for further adjusting the tone mapping parameters by referring to other information of the terminal display device.

[0036] In one possible implementation, the step of obtaining the reference information includes the steps of obtaining the reference information from metadata of the image to be displayed, obtaining an index of the reference information from the metadata, and selecting the reference information from a plurality of predefined reference information based on the index, or using the predefined reference information as the reference information.

[0037] In one possible implementation, the step of obtaining the terminal device information includes a step of obtaining the terminal device information by measurement, a step of selecting the terminal device information from a plurality of predefined terminal information based on a predefined index, or a step of using the predefined terminal device information as the terminal device information.

[0038] According to a second aspect, an embodiment of the present application provides an image encoding method, the method including the steps of: writing an image to be displayed into a bitstream; and writing reference information of the image to be displayed into the bitstream, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution, and the reference information is for tone mapping for the image to be displayed.

[0039] In this embodiment of the present application, in order to flexibly adapt to different environments and display devices, tone mapping is performed on the image to be displayed based on the reference information together with the corresponding type of terminal device information of this decoder, so as to effectively improve the display adaptation effect, and improve the tone mapping effect and display effect of the image. In addition, the user's visual perception is more consistent in different application scenarios, so as to improve the user's viewing experience.

[0040] In one possible implementation, the method further includes writing reference tone mapping parameters of the image to be displayed into the bitstream, the reference tone mapping parameters being for obtaining initial tone mapping parameters, and the reference information being used by the decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, the tone mapping parameters being for tone mapping for the image to be displayed.

[0041] In this implementation, the reference tone mapping parameters of the image to be displayed are adjusted based on the reference tone mapping parameters in conjunction with the display capability information of the decoder's display screen, so that the resulting initial tone mapping parameters are adapted to the terminal display device in terms of display capability and are used as a basis for the decoder to again adjust the tone mapping parameters by referring to other information of the terminal display device.

[0042] In one possible implementation, writing reference information for the image to be displayed into the bitstream comprises writing the reference information into the bitstream or writing an index of the reference information into the bitstream.

[0043] In one possible implementation, the method further includes writing one or more luma feature values ​​of the image to be displayed into the bitstream, the one or more luma feature values ​​being used by the decoder to adjust the initial tone mapping parameters to obtain the tone mapping parameters.

[0044] In this implementation, the tone mapping parameters adjusted by the decoder based on one or more luminance feature values ​​can be flexibly adapted to different environments and display devices, thereby effectively improving the display adaptation effect and improving the tone mapping effect and display effect of the image. In addition, the user's visual perception is more consistent in different application scenarios, thereby improving the user's viewing experience.

[0045] According to a third aspect, an embodiment of the present application provides a tone mapping apparatus including a module configured to perform a method according to the first aspect or any one of the possible implementations of the first aspect.

[0046] For example, the present device Obtain an image to be displayed; obtaining reference information, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; and a receiving module configured to obtain terminal device information, the terminal device information including at least one of a terminal display device size, a user viewing distance, a user viewing angle, an ambient light brightness, or a terminal display device resolution; a tone mapping module configured to perform tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information; may include:

[0047] In this embodiment of the present application, when performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information, the reference display device size corresponds to the terminal display device size, the reference viewing distance corresponds to the user viewing distance, the reference viewing angle corresponds to the user viewing angle, the reference ambient light brightness corresponds to the ambient light brightness, and the reference display device resolution corresponds to the terminal display device resolution.

[0048] In one possible implementation, the step of performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information includes the steps of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information, and performing tone mapping on the image to be displayed based on the tone mapping parameters.

[0049] In one possible implementation, the step of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information includes a step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters.

[0050] In one possible implementation, the step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain tone mapping parameters includes: adjusting the initial tone mapping parameters of the image to be displayed in a first manner to obtain adjusted tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a first condition; and adjusting the initial tone mapping parameters of the image to be displayed in a second manner to obtain tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a second condition. The initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve.

[0051] In one possible implementation, the first condition includes at least one of the following: the terminal display screen size is smaller than the reference display device size; the user viewing distance is larger than the reference viewing distance; the user viewing angle deviates from the reference viewing angle; the ambient light luminance is larger than the reference ambient light luminance; or the terminal display device resolution is smaller than the reference display device resolution.

[0052] In one possible implementation, the step of adjusting initial tone mapping parameters of the image to be displayed in a first manner includes the steps of obtaining one or more first luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained mapped values ​​of the adjusted tone mapping curve at the one or more first luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at the one or more first luminance feature values.

[0053] In one possible implementation, the step of adjusting the initial tone mapping curve of the image to be displayed in the first manner further includes the steps of obtaining one or more second luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping curve so that the obtained slope of the adjusted tone mapping curve at the one or more second luminance feature values ​​is greater than the slope of the initial tone mapping curve at the one or more second luminance feature values.

[0054] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in a first manner further includes the steps of obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a first feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the enhancement layer is a second feature value, and adjusting the initial tone mapping parameters of the any of the enhancement layers so that the absolute value of the value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value.

[0055] In this implementation, the first and second feature values ​​may be positive or negative values, and when the value obtained by mapping the first feature value based on the adjusted tone mapping curve is compared with the second feature value, the absolute values ​​of the two values ​​are compared.

[0056] In one possible implementation, the second condition includes at least one of the following: the terminal display screen size is larger than the reference display device size; the user viewing distance is smaller than the reference viewing distance; the ambient light luminance is smaller than the reference ambient light luminance; or the terminal display device resolution is larger than the reference display device resolution.

[0057] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes the steps of obtaining one or more third luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained mapped values ​​of the adjusted tone mapping curve at the one or more third luminance feature values ​​are smaller than the mapped values ​​of the initial tone mapping curve at the one or more third luminance feature values.

[0058] In one possible implementation, the one or more third luminance feature values ​​include one or more luminance feature values ​​of dark portions of the image to be displayed.

[0059] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner further includes the steps of obtaining one or more fourth luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained slope of the adjusted tone mapping curve at the one or more fourth luminance feature values ​​is smaller than the slope of the initial tone mapping curve at the one or more fourth luminance feature values.

[0060] In one possible implementation, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner further includes the steps of obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a third feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the enhancement layer is a fourth feature value, and adjusting the initial tone mapping parameters of the any of the enhancement layers so that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is smaller than the absolute value of the fourth feature value.

[0061] In this implementation, the third and fourth feature values ​​may be positive or negative values, and when the value obtained by mapping the third feature value based on the adjusted tone mapping curve is compared with the fourth feature value, the absolute values ​​of the two values ​​are compared.

[0062] In one possible implementation, the tone mapping module is further configured to obtain a reference tone mapping of the image to be displayed, obtain a terminal display device display capability in the terminal device information, and adjust the reference tone mapping parameter of the image to be displayed based on the reference tone mapping parameter and the terminal display device display capability to obtain an initial tone mapping parameter.

[0063] In one possible implementation, the step of obtaining the reference information includes the steps of obtaining the reference information from metadata of the image to be displayed, obtaining an index of the reference information from the metadata, and selecting the reference information from a plurality of predefined reference information based on the index, or using the predefined reference information as the reference information.

[0064] In one possible implementation, the step of obtaining the terminal device information includes a step of obtaining the terminal device information by measurement, a step of selecting the terminal device information from a plurality of predefined terminal information based on a predefined index, or a step of using the predefined terminal device information as the terminal device information.

[0065] It should be understood that the third aspect corresponds to the technical solution of the first aspect, and therefore the beneficial effects obtained in the possible implementations are similar, and the details will not be described again here.

[0066] According to a fourth aspect, an embodiment of the present application provides an image encoding device, the device including modules configured to perform a method according to the second aspect or any one of the possible implementations of the second aspect.

[0067] For example, the present device an encoding module configured to write a to-be-displayed image into a bitstream and to write reference information for the to-be-displayed image into the bitstream, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution, and the reference information is for tone mapping for the to-be-displayed image; a transmitting module configured to transmit the bitstream; may include:

[0068] In one possible implementation, the encoding module is further configured to write reference tone mapping parameters of the image to be displayed to the bitstream, the reference tone mapping parameters being for obtaining initial tone mapping parameters, and the reference information being used by the decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, the tone mapping parameters being for tone mapping for the image to be displayed.

[0069] In one possible implementation, writing reference information for the image to be displayed into the bitstream comprises writing the reference information into the bitstream or writing an index of the reference information into the bitstream.

[0070] In one possible implementation, the encoding module is further configured to write one or more luma feature values ​​of the image to be displayed to the bitstream, the one or more luma feature values ​​being used by the decoder to adjust the initial tone mapping parameters to obtain the tone mapping parameters.

[0071] According to a fifth aspect, an embodiment of the present application provides an electronic device including a memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform the method of the first aspect or any one of the possible implementations of the first aspect.

[0072] It should be understood that for details of the technical effects that may be achieved by the fifth aspect, reference should be made to the description of the technical effects provided by the first aspect or any one of the possible implementations of the first aspect, and the details will not be described again here.

[0073] According to a sixth aspect, an embodiment of the present application provides an electronic device including a memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform the method of the second aspect or any one of the possible implementations of the second aspect.

[0074] It should be understood that for details of the technical effects that can be achieved in the sixth aspect, reference should be made to the description of the technical effects provided by the second aspect or any one of the possible implementations of the second aspect, and the details will not be described again here.

[0075] According to a seventh aspect, an embodiment of the present application provides a tone mapping system including a tone mapping apparatus according to the third aspect and any one of the possible implementations of the third aspect, and an image coding apparatus according to the fourth aspect and any one of the possible implementations of the fourth aspect, or including an electronic device according to the fifth aspect and an electronic device according to the sixth aspect.

[0076] It should be understood that for details of the technical effects that may be achieved by the seventh aspect, reference should be made to the description of the technical effects provided by the first aspect or any one of the possible implementations of the first aspect, and the description of the technical effects provided by the second aspect or any one of the possible implementations of the second aspect, and the details will not be described again here.

[0077] According to an eighth aspect, an embodiment of the present application provides a bitstream structure. The bitstream includes coded data of an image and parameters necessary for processing the image, and the parameters necessary for processing the image include reference information according to any one of the first aspect, possible implementations of the first aspect, the second aspect, and possible implementations of the second aspect. The reference information includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution.

[0078] In one possible implementation, the bitstream further includes reference tone mapping parameters, where the reference tone mapping curve parameters are for deriving initial tone mapping parameters, and the reference information is used by the decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, where the tone mapping parameters are for tone mapping for the image to be displayed.

[0079] In one possible implementation, the bitstream includes an index of the reference information.

[0080] In one possible implementation, the bitstream includes one or more luminance feature values.

[0081] According to a ninth aspect, an embodiment of the present application provides a computer-readable storage medium comprising a program or instructions, which, when executed on a computer, perform the method of the first aspect or any one of possible implementations of the first aspect, or perform the method of the second aspect or any one of possible implementations of the second aspect.

[0082] It should be understood that for details of the technical effects that may be achieved by the seventh aspect, reference should be made to the description of the technical effects provided by the first aspect or any one of the possible implementations of the first aspect, or the second aspect or any one of the possible implementations of the second aspect, and the details will not be described again here.

[0083] According to a tenth aspect, an embodiment of the present application provides a computer program product including a program or instructions, which, when executed on a computer, perform the method of the first aspect or any one of possible implementations of the first aspect, or perform the method of the second aspect or any one of possible implementations of the second aspect.

[0084] It should be understood that for details of the technical effects that may be achieved by the eighth aspect, reference should be made to the description of the technical effects provided by the first aspect or any one of the possible implementations of the first aspect, or the second aspect or any one of the possible implementations of the second aspect, and the details will not be described again here. [Brief explanation of the drawings]

[0085] [Figure 1] 1 is a block diagram of a codec system to which an embodiment of the present application is applied; [Figure 2A] 1 is a block diagram of a content supply system for implementing a content distribution service to which an embodiment of the present application applies; [Figure 2B] 2B is a diagram of an exemplary configuration of the terminal device 2106 of FIG. 2A. [Figure 3A] 1 is a work flowchart of a streaming media system to which an embodiment of the present application is applied; [Figure 3B] 1 is a diagram of the architecture of a streaming media system to which an embodiment of the present application applies; [Figure 4A] FIG. 1 is a diagram of a possible system architecture to which an embodiment of the present application is applicable. [Figure 4B] 1 is a diagram of the structure of an image processing system according to an embodiment of the present application; [Figure 5] 1 is a schematic flowchart of a tone mapping method according to an embodiment of the present application; [Figure 6] FIG. 10 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to an embodiment of the present application. [Figure 7] FIG. 10 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to an embodiment of the present application. [Figure 8] FIG. 10 is an exemplary diagram of an initial tone mapping parameter adjustment scheme for an enhancement layer according to an embodiment of the present application. [Figure 9] FIG. 10 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to an embodiment of the present application. [Figure 10] FIG. 10 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to an embodiment of the present application. [Figure 11] 1 is a schematic flowchart of an image encoding method according to an embodiment of the present application; [Figure 12] FIG. 1 is an exemplary diagram of a tone mapping device according to an embodiment of the present application. [Figure 13] 1 is an exemplary diagram of an image encoding device according to an embodiment of the present application; [Figure 14] 1 is an exemplary diagram of an electronic device according to an embodiment of the present application. [Figure 15] 1 is an exemplary diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0086] In the embodiments of the present application, the term "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes a relationship between related objects and indicates that three relationships may exist. For example, A and / or B can refer to the following three cases: when only A is present, when both A and B are present, and when only B is present, and A and B may be singular or plural. The character " / " generally indicates an "or" relationship between related objects. "At least one of the following" or similar expressions refers to a singular item or any combination of these items, including any combination of multiple items. For example, at least one of a, b, or c can refer to a, b, c, ab, ac, bc, or abc, and any one of a, b, c, ab, ac, bc, or abc can include a single a, a single b, and a single c, or can include multiple a, multiple b, and multiple c.

[0087] Additionally, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of the present application are used to distinguish between multiple objects, but are not intended to limit the order, chronological order, priority, or importance of the multiple objects. For example, a first priority criterion and a second priority criterion are simply used to distinguish between different criteria, and are not used to indicate different contents, priorities, or importance of the two types of criteria.

[0088] In addition, the terms "comprise" and "have" in the embodiments, claims, and accompanying drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules / units is not limited to the enumerated steps or modules, and may further include unenumerated steps or modules / units.

[0089] For ease of understanding, the technical terms used in the embodiments of the present application will be explained first.

[0090] (1) Dynamic range is the ratio of maximum to minimum luminance in a video or image signal.

[0091] (2) A high dynamic range (HDR) image is an image with a dynamic range of 0.001 nit to 10,000 nit, where nit is a unit of illuminance.

[0092] (3) Standard dynamic range (SDR) images, also known as low dynamic range images, are images with a dynamic range typically between 1 nit and 100 nit.

[0093] (4) Metadata is used to record key information about an image in a scene or frame. In an embodiment of the present application, the metadata may further include reference tone mapping parameters representing a reference tone mapping curve and reference information. The reference information includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution, and is used for tone mapping for the image. The metadata may further include dynamic metadata and static metadata. Dynamic metadata may be understood as data associated with each frame of an image (hereinafter also referred to as attribute information), and this data may change for each individual video. Static metadata may be understood as data associated with an image sequence, and this data does not change within the image sequence.

[0094] (5) Tone mapping is the process of mapping the luminance of an image to the displayable luminance range of a target display device. Specifically, tone mapping can be classified into high-dynamic-to-low-dynamic mapping and low-dynamic-to-high-dynamic mapping. For example, an HDR image contains a 4000-nit illumination signal, but the target display device (TV or iPad) only has a 500-nit HDR display capability. Therefore, the 4000-nit signal needs to be mapped to a 500-nit device, i.e., a high-to-low tone mapping process. As another example, an SDR image contains a 100-nit SDR signal, and the target display device has a 2000-nit display capability. Therefore, the 100-nit signal needs to be mapped to a 2000-nit device, i.e., a low-to-high tone mapping process.

[0095] (6) Image contrast is a measure of the individual luminance levels between the brightest and darkest feature values ​​in the bright and dark areas of an image, i.e., the grayscale difference of an image. A larger difference indicates higher contrast, while a smaller difference indicates lower contrast. A good contrast ratio of 120:1 can easily display vivid and rich colors, and a contrast ratio of 300:1 can support all levels of color.

[0096] (7) The mapping curve is a function curve for mapping the brightness of the image. The mapping curve in the embodiment of the present application may include a Sigmoid function curve, a cubic spline function curve, or a Gamma function curve.

[0097] (8) Display adaptation is the processing of tones on a video or image signal to adapt to the display characteristics of the target display device.

[0098] When a video or image is displayed on an existing display device, the brightness of the video or image needs to be processed based on the display capability of the display device so that the brightness of the video or image is adapted to the displayable brightness range of the display device. This process is called tone mapping (TM). However, currently, there are various display devices, and there are differences in other aspects, such as the displayable brightness range of the display screen and the display size. Therefore, in existing solutions, the mechanism of matching the maximum and minimum brightness values ​​of the image to the maximum and minimum brightness values ​​of the display screen alone cannot achieve an optimal display adaptation effect. Therefore, this application provides a tone mapping method and apparatus. When tone mapping is performed on an image, other information, such as the terminal display device size and the user viewing distance, is also taken into account, thereby improving the display adaptation effect.

[0099] A codec system to which the present application is applied will be described below with reference to Fig. 1. Fig. 1 is a block diagram of a codec system to which an embodiment of the present application is applied. For example, a video codec system 10 (or codec system 10 for short) to which the technology of the present application can be applied. A video encoder 20 (or encoder 20 for short) and a video decoder 30 (or decoder 30 for short) of the video codec system 10 correspond to devices that can be configured to perform techniques according to various examples described in the present application.

[0100] 1, codec system 10 includes a source device 12. Source device 12 is configured to provide encoded data 21, such as an encoded image, to a destination device 14 for decoding the encoded data.

[0101] The source device 12 includes an encoder 20 and may optionally include an image source 16 , an image pre-processing unit 18 (or pre-processing unit), and a communication interface or unit 22 .

[0102] Image source 16 may include or be, for example, any type of image capture device that captures real-world images, and / or any type of image generation device, such as a computer graphics processing unit that generates computer-animated images, or any type of other device that obtains and / or provides real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images)). Image source may also be any type of memory or storage device that stores any of the foregoing images.

[0103] To distinguish the processing performed by the preprocessor 18 or preprocessing unit 18 , the image or image data 17 is sometimes referred to as the original image or original image data 17 .

[0104] The pre-processor 18 is configured to receive the (original) image data 17 and pre-process the image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. For example, the pre-processing performed by the pre-processor 18 may include cropping, color format conversion (e.g., RGB to YCbCr), color correction, or noise removal. It will be understood that the pre-processing unit 18 may be an optional component.

[0105] Video encoder 20 is configured to receive pre-processed image data 19 and provide encoded image data 21 .

[0106] A communications interface 22 in the source device 12 may be configured to receive the encoded image data 21 and transmit the encoded image data 21 (or any further processed version thereof) via a communications channel 13 to another device, such as a destination device 14, or to any other device, for storage or direct reconstruction.

[0107] Destination device 14 includes a decoder 30 (e.g., a video decoder 30), and may further, i.e., optionally, include a communications interface or communications unit 28, a post-processor 32 (or post-processing unit 32), and a display device 34.

[0108] The communications interface 28 in the destination device 14 is configured to receive the encoded image data 21 (or any further processed version thereof) directly from the source device 12 or from any other source device, such as a storage device, such as an encoded image data storage device, and to provide the encoded image data 21 to a decoder 30.

[0109] The communication interface 22 and the communication interface 28 may be configured to transmit or receive the encoded image data 21 or encoded data via a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or via any type of network, such as a wired or wireless network or any combination thereof, or any type of private and public network or any combination thereof.

[0110] The communications interface 22 may be configured, for example, to package the encoded image data 21 into an appropriate format, e.g., into packets, and / or to process the encoded image data using any type of transmission coding or processing for transmission over a communications link or network.

[0111] The communications interface 28, which forms the counterpart of the communications interface 22, may be configured to receive the transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or depackaging to obtain the encoded image data 21, for example.

[0112] Both communication interface 22 and communication interface 28 may be configured as unidirectional communication interfaces, as indicated by the arrows in communication channel 13 in FIG. 1 pointing from source device 12 to destination device 14, or as bidirectional communication interfaces, e.g., to send and receive messages, e.g., to set up connections, acknowledge, and exchange any other information related to the communication link and / or data transmission, e.g., related to the transmission of encoded image data.

[0113] The decoder 30 is configured to receive the encoded image data 21 and to provide decoded image data 31 or a decoded image 31 .

[0114] A post-processor 32 in the destination device 14 is configured to post-process the decoded image data 31 (also called reconstructed image data), e.g., the decoded image 31, to obtain post-processed image data 33, e.g., a post-processed image 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., YCbCr to RGB), color correction, cropping, or resampling, or any other processing that prepares the decoded image data 31 for display by, for example, a display device 34.

[0115] A display device 34 within destination device 14 is configured to receive the post-processed image data 33, for example, to display the image to a user or viewer. Display device 34 may be or include any type of display, such as an integrated or external display screen or display, for presenting the reconstructed image. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0116] 1 depicts source device 12 and destination device 14 as separate devices, embodiments of the devices may instead include both or both functionality, i.e., source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such embodiments, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0117] As will be apparent to those skilled in the art based on the description, the presence and (specific) functional division of individual units or functions in source device 12 and / or destination device 14 shown in FIG. 1 may vary depending on the actual device and application.

[0118] A content supply system for a content distribution service to which the present application is applied will be described below with reference to FIG. 2A . FIG. 2A is a block diagram of a content supply system for implementing a content distribution service to which an embodiment of the present application is applied. The content supply system 2100 includes a capture device 2102, a terminal device 2106, and (optionally) a display 2126. The capture device 2102 communicates with the terminal device 2106 via a communication link 2104. The communication link may include the aforementioned communication channel 13. The communication link 2104 may include, but is not limited to, WiFi, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0119] The capture device 2102 can generate data and encode it using the encoding method described in the above embodiment. Alternatively, the capture device 2102 can deliver the data to a streaming server (not shown), which then encodes the data and transmits the encoded data to the terminal device 2106. The capture device 2102 can include, but is not limited to, a camera, a smartphone or tablet computer, a computer or laptop computer, a videoconferencing system, a PDA, an in-vehicle device, or any combination thereof. For example, the capture device 2102 can include the aforementioned source device 12. If the data includes video, a video encoder 20 in the capture device 2102 can actually perform the video encoding process. If the data includes audio (i.e., voice), an audio encoder 20 in the capture device 2102 can actually perform the audio encoding process. In some practical scenarios, the capture device 2102 delivers the encoded video data and the encoded audio data by multiplexing them together. In other practical scenarios, for example, in a videoconferencing system, the encoded audio data and the encoded video data are not multiplexed. The capture device 2102 delivers the encoded audio data and the encoded video data separately to the terminal device 2106 .

[0120] A terminal device 2106 of the content delivery system 2100 receives and reproduces the encoded data. The terminal device 2106 may be a device capable of receiving and restoring data, such as a smartphone or tablet computer 2108, a computer or laptop computer 2110, a network video recorder (NVR) / digital video recorder (DVR) 2112, a TV 2114, a set-top box (STB) 2116, a video conferencing system 2118, a video surveillance system 2120, a personal digital assistant (PDA) 2122, an in-vehicle device 2124, or any combination thereof, capable of decoding the encoded data. For example, the terminal device 2106 may include the destination device 14 described above. If the encoded data includes video, the video decoder 30 in the terminal device is prioritized to perform video decoding. If the encoded data includes audio, the audio decoder in the terminal device is prioritized to perform audio decoding. The terminal device 2106 may be a video playback application, a streaming media playback application, a streaming media playback platform, a live streaming platform, etc. running on the terminal device.

[0121] In the case of a terminal device having a display, such as a smartphone or tablet computer 2108, a computer or laptop computer 2110, an NVR / DVR 2112, a TV 2114, a PDA 2122, or an in-vehicle device 2124, the terminal device can feed the decoded data to its display. In the case of a terminal device without a display, such as an STB 2116, a video conferencing system 3118, or a video surveillance system 2120, an external display 2126 is connected to the device to receive and display the decoded data.

[0122] When each device in this system performs encoding or decoding, it can use an image encoding device or an image decoding device as shown in the above embodiments.

[0123] 2B is a diagram of an example configuration of the terminal device 2106 of FIG. 2A. After the terminal device 2106 receives the bitstream from the capture device 2102, a protocol processing unit 2202 analyzes the transmission protocol of the bitstream. The protocol may include, but is not limited to, Real Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof.

[0124] After the protocol processing unit 2202 processes the stream, a stream file is generated. This file is output to the demultiplexing unit 2204. The demultiplexing unit 2204 can separate the multiplexed data into coded audio data and coded video data. As mentioned above, in other practical scenarios, for example, in a video conferencing system, coded audio data and coded video data are not multiplexed. In this situation, the coded data is sent to the video decoder 3206 and the audio decoder 2208 without passing through the demultiplexing unit 2204.

[0125] This demultiplexing process generates a video elementary stream (ES), an audio ES, and optionally subtitles. A video decoder 2206, which includes the video decoder 30 described in the previous embodiment, decodes the video ES using the decoding method shown in the previous embodiment to generate video frames, and sends the data to a synchronization unit 2212. An audio decoder 2208 decodes the audio ES to generate audio frames, and sends the data to a synchronization unit 2212. Alternatively, the video frames may be stored in a buffer (not shown) before being sent to the synchronization unit 2212. Similarly, the audio frames may be stored in a buffer (not shown) before being sent to the synchronization unit 2212.

[0126] The synchronization unit 2212 synchronizes the video and audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 2212 synchronizes the presentation of video and audio information. The information may be encoded in a syntax using timestamps for the presentation of the encoded audio and visual data and timestamps for the delivery of the data stream.

[0127] If the stream contains subtitles, the subtitle decoder 2210 decodes the subtitles, synchronizes them with the video and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 2216 .

[0128] The present invention is not limited to the above-described system, and either the image encoding device or the image decoding device of the above-described embodiments can be incorporated into other systems, for example, car systems.

[0129] A streaming media system to which an embodiment of the present application can be applied will be described below with reference to Fig. 3A, which is a work flowchart of the streaming media system to which an embodiment of the present application can be applied.

[0130] A streaming media system includes a content creation module that generates necessary content data, such as video or audio. The streaming media system further includes a video encoding module that encodes the generated content with an encoder. The streaming media system further includes a video stream transmission module that transmits the encoded video in the form of a bitstream. Optionally, the format of the video stream may be converted to a bitstream format of a transport protocol commonly used by over-the-top (OTT) devices. For example, the protocol may include, but is not limited to, Real Time Streaming Protocol (RTSP), Hypertext Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof. Optionally, for ease of use, video stream storage may be performed to store the video stream in its original format and / or multiple converted bitstream formats. Additionally, the streaming media system further includes a video stream packaging module configured to package the video stream to generate a packaged video stream. The packaged video stream may be referred to as a video streaming media packet.For example, the video streaming media packets may be generated based on a transcoded video stream or a stored video stream. In addition, the streaming media system further includes a content distribution network (CDN), which is configured to distribute the video streaming media packets to multiple OTT devices, such as mobile phones, computers, tablets, and home projectors.

[0131] It should be noted that the video encoding, video stream transmission, video stream transcoding, video stream storage, video streaming media packet generation, and content delivery network may all be implemented on cloud servers.

[0132] An exemplary architecture of the streaming media system in the present application will be described below with reference to Figure 3B. The architecture of the streaming media system in the present application includes a client device, a content delivery network, and a cloud server.

[0133] A user on a client device sends a play or playback request to the cloud platform, and optionally the content of the request sent may be the title of the movie or television program to be played.

[0134] The cloud platform makes a decision, responds to the client, and sends the client the address of the content requested by the client on the CDN. Optionally, the content sent to the client may be a uniform resource locator (URL). Specifically, the playback application service in the cloud platform checks user permissions and privileges, and then determines which specific files are needed to process the playback request by considering the characteristics of each client and the current network conditions. Note that the content delivery network (CDN) periodically reports its operational status, learned routes, and available content (files) to the cache control service on the cloud platform.

[0135] The client then sends a request to the CDN to play the content based on the address, and the CDN provides the content for the client and finally completes the client's request.

[0136] The following describes a system architecture to which an embodiment of the present application is applicable, with reference to Figure 4A. Figure 4A is a diagram of a possible system architecture to which an embodiment of the present application is applicable. The system architecture in this embodiment of the present application includes a front-end device, a transmission link, and a terminal display device.

[0137] The front-end device is configured to collect or create HDR / SDR content (e.g., HDR / SDR video or images).

[0138] In one possible embodiment, the front-end device may be further configured to extract corresponding metadata from the HDR content. This metadata may include global mapping information, local mapping information, and dynamic and static metadata corresponding to the HDR content. The front-end device can transmit the HDR content and the metadata to the terminal display device via a transmission link. Specifically, the HDR content and the metadata may be transmitted in the form of one data packet or may be transmitted separately in two data packets. This is not specifically limited in this embodiment of the present application.

[0139] Optionally, the terminal display device may be configured to receive the metadata and the HDR content, obtain mapping curves for global and local tone mapping for the HDR content based on global and local mapping information included in the corresponding metadata extracted from the HDR content and information related to the terminal display device, convert the HDR content into display content suitable for an HDR display device or an SDR display device within the terminal display device, and display the display content. It should be understood that in various embodiments, the terminal display device may include a display device capable of displaying a dynamic range lower or higher than the HDR content generated by the front-end device. This is not a limitation of the present application.

[0140] Optionally, in this application, the front-end device and the terminal display device may be separate physical devices. For example, the front-end device may be a video capture device or a video production device. The video capture device may be a device such as a camera, a camera, or an image rendering machine. The terminal display device may be any device with video playback capabilities, such as virtual reality (VR) glasses, a mobile phone, a tablet, a TV, or a projector.

[0141] Optionally, the transmission link between the front-end device and the terminal display device may be a wireless connection or a wired connection. The wireless connection may use technologies such as long-term evolution (LTE), 5th generation (5G), and future mobile communications. The wireless connection may further include technologies such as wireless fidelity (WiFi), Bluetooth, and near field communication (NFC). The wired connection may include an Ethernet connection, a local area network connection, etc. This is not specifically limited.

[0142] In the present application, the functions of the front-end device and the terminal display device may alternatively be integrated into the same physical device, for example, a terminal device with video capture capabilities such as a mobile phone or a tablet. In the present application, part of the functions of the front-end device and part of the functions of the terminal display device may alternatively be integrated into the same physical device. This is not specifically limited.

[0143] An end-to-end image processing system provided in an embodiment of the present application will be described below with reference to Fig. 4B. This system can be applied to the system architecture shown in Fig. 4A. Fig. 4B is a diagram of the structure of an image processing system according to an embodiment of the present application. In Fig. 4B, for example, HDR video is used as an example of HDR / SDR content. The image processing system includes an HDR pre-processing module, an HDR video encoding module, an HDR video decoding module, and a tone mapping module.

[0144] The HDR pre-processing module and the HDR video encoding module may be located in the front-end device shown in FIG. 4A, and the HDR video decoding module and the tone mapping module may be located in the terminal display device shown in FIG. 4A.

[0145] The HDR preprocessing module is configured to extract dynamic metadata (e.g., maximum, minimum, average, and variation range of luminance) from the HDR video, determine mapping curve parameters based on the dynamic metadata and the display capability of a target display device, write the mapping curve parameters into the dynamic metadata to obtain HDR metadata, and transmit the HDR metadata. The HDR video may be captured or may be HDR video processed by a colorist. The display capability of the target display device is the displayable luminance range of the target display device.

[0146] The HDR video encoding module is configured to perform video encoding on the HDR video and HDR metadata (e.g., embed the HDR metadata in a user-specified portion of the bitstream) in accordance with a video compression standard (e.g., an AVS or HEVC standard) to output a corresponding bitstream (an AVS or HEVC bitstream).

[0147] The HDR video decoding module is configured to decode the generated bitstream (AVS bitstream or HEVC bitstream) according to a standard corresponding to the bitstream format, and output decoded HDR video and HDR metadata.

[0148] The tone mapping module is configured to generate a mapping curve based on the mapping curve parameters in the decoded HDR metadata, perform tone mapping (i.e., HDR adaptation or SDR adaptation) on the decoded HDR video, and transmit the HDR-adapted video obtained by the tone mapping to an HDR display terminal for display, or transmit the SDR-adapted video to an SDR display terminal for display.

[0149] For example, the HDR pre-processing module may reside within a video capture device or a video production device.

[0150] For example, the HDR video encoding module may reside within a video capture device or a video production device.

[0151] For example, HDR video decoding modules may reside in set-top boxes, TV display devices, mobile terminal display devices, and video conversion devices such as live streaming or network video applications.

[0152] For example, the tone mapping module may exist in a video conversion device such as a set-top box, a TV display device, a mobile terminal display device, and a live streaming or network video application. More specifically, the tone mapping module may exist in the form of a chip or a software program in a set-top box, a TV display, and a mobile terminal display, and may also exist in the form of a software program in a video conversion device such as a live streaming or network video application.

[0153] In one possible embodiment, when both the tone mapping module and the HDR video decoding module are present in a set-top box, the set-top box can complete the functions of receiving, decoding, and tone mapping the video bitstream, and then transmit the decoded video data to a display device through a high definition multimedia interface (HDMI) for display, allowing users to enjoy the video content.

[0154] The following describes the process by which a terminal display device performs tone mapping with reference to specific accompanying drawings.

[0155] Please refer to Figure 5. Figure 5 is a schematic flowchart of a tone mapping method according to an embodiment of the present application. The method may be performed by a terminal display device, and specifically includes the following steps:

[0156] Step S501: An image to be displayed is obtained.

[0157] In this embodiment of the present application, the image to be displayed may be an HDR image or video, or may be an SDR image or video. The image to be displayed may be an optical signal (i.e., a linear signal), or an electrical signal (i.e., a nonlinear signal). This is not limited in the present application. If the image to be displayed is an electrical signal, the electrical signal may be converted into an optical signal based on an optical-electronic transfer function.

[0158] In one possible embodiment, the image to be displayed is transmitted in the form of a bitstream, and the terminal display device is able to receive the bitstream from the front-end device and decode the bitstream to obtain the data of the image to be displayed.

[0159] Optionally, the data for the image to be displayed includes image data, e.g., pixel data. It should be understood that this embodiment of the present application does not limit the format of the image. The image data may be in any color space, e.g., RGB or YUV, and the image data may be any bit width, e.g., 8 bits, 10 bits, or 12 bits.

[0160] Optionally, the data of the image to be displayed includes metadata. It should be understood that the format of the metadata is not limited in this embodiment of the present application. The metadata includes data such as an image or video data format, area division information, area traversal order information, image features, or mapping curve parameters, and one or more metadata information units. The metadata information units include data such as coordinate information, image features, or curve parameters. For example, the metadata may include reference information of the image to be displayed, such as a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution, and the metadata may further include tone mapping information, such as a tone mapping curve.

[0161] Optionally, the decoding device (e.g., a set-top box) receives a bitstream from the front-end device, decodes the bitstream to obtain data for an image to be displayed, and then transmits the decoded data for the image to be displayed to the terminal display device.

[0162] Optionally, the process of decoding the bitstream includes decoding the received bitstream with a decoder to obtain data for the image to be displayed, for example, the decoder may be an HEVC decoder, a JPEG decoder, a ProRes decoder, or a VVC decoder.

[0163] In another possible embodiment, the image to be displayed is an image on which software processing can be performed, including but not limited to tone mapping, dehazing, deblurring, zooming in / out, color gamut conversion, and neural network based graphics processing.

[0164] In another possible embodiment, the image to be displayed may be an image on which hardware processing is performed, including but not limited to tone mapping, dehazing, deblurring, zooming in / out, color gamut conversion, and neural network-based graphics processing.

[0165] Step S502: Obtain reference information, where the reference information includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution.

[0166] In this embodiment of the present application, reference information of a display target image is obtained. The reference information is for tone mapping of the display target image to improve a user's viewing effect, and includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light brightness, or a reference display device resolution. Optionally, the reference information may be used to adjust a reference tone mapping curve in conjunction with terminal device information.

[0167] Optionally, the reference display device size may be the size of the display device used when the image to be displayed is created. Optionally, the reference display device size includes at least one of the length, width, or diagonal length of the display device. This is not a limitation of the present application. Optionally, the reference display device size may be obtained by a transformation based on at least one of the length, width, or diagonal length of the display device. This is not a limitation of the present application. Optionally, the reason for needing to perform the transformation may be for aesthetic reasons, such as bending or folding, or for visual reasons, such as a change in display size caused by visual correction performed by a VR device. Optionally, the reference display device size may be obtained from metadata of the image to be displayed, and an index of the reference display device size may be obtained from the metadata, and the reference device size may be selected from multiple predefined reference display device sizes based on the index, or the predefined reference display device size may be directly used as the reference display device size. The reference display device size index may correspond to various standards to correspond to display device sizes specified in various standards, such as 1920x1080 HD, 1280x720 HD 720P, and 3840x2160 Ultra HD.

[0168] Optionally, the reference viewing distance may be the viewing distance of a reference display device used when the image to be displayed is created. Optionally, the reference viewing distance may be the linear distance between the user and the center point of the display device, the perpendicular distance between the user and the image plane of the display device, or the linear distance between the user and a predetermined position on the image plane of the display device. This is not a limitation of the present application. Optionally, the reference viewing distance may be obtained from metadata of the image to be displayed, an index of the reference viewing distance may be obtained from the metadata, and the reference viewing distance may be selected from a plurality of predetermined reference viewing distances based on the index, or the predetermined reference viewing distance may be directly used as the reference viewing distance.

[0169] Optionally, the reference viewing angle may be a viewing angle range of a reference display device used when the image to be displayed is created. Optionally, the reference viewing angle may be a range of angles of deviation from a perpendicular line perpendicular to the display device at the center point position, and this range of angles allows a user to accurately observe the image to be displayed. This is not a limitation of the present application. Optionally, the reference viewing angle may be obtained from metadata of the image to be displayed, or an index of the reference viewing angle may be obtained from the metadata, and the reference viewing angle may be selected from a plurality of predefined reference viewing angles based on the index, or the predefined reference viewing angle may be directly used as the reference viewing angle.

[0170] Optionally, the reference ambient light intensity may be the ambient light intensity when the image to be displayed is created, or optionally, the reference ambient light intensity may be obtained from metadata of the image to be displayed, an index of the reference ambient light intensity may be obtained from the metadata, and the reference ambient light intensity may be selected from a plurality of predefined reference ambient light intensities based on the index, or the predefined reference ambient light intensity may be used directly as the reference ambient light intensity.

[0171] Optionally, the reference display device resolution may be the resolution of the reference display device used when the image to be displayed is created. Optionally, the reference display device resolution value may be obtained from metadata of the image to be displayed, an index of the reference display device resolution may be obtained from the metadata, and the reference display device resolution may be selected from a plurality of predefined reference display device resolutions based on the index, or the predefined reference display device resolution may be directly used as the reference display device resolution. The index of the reference display device resolution may correspond to various standards to correspond to display device resolutions specified in various standards, for example, 1920×1080 HD, 1280×720 HD 720P, and 3840×2160 Ultra HD standards.

[0172] Step S503: Obtain terminal device information, where the terminal device information includes at least one of a terminal display device size, a user viewing distance, a user viewing angle, ambient light brightness, or a terminal display device resolution.

[0173] In this embodiment of the present application, terminal device information is obtained and used in conjunction with the reference information to adjust the reference tone mapping curve.

[0174] Optionally, the terminal display device size includes at least one of the length, width, or diagonal length of the terminal display device. This is not a limitation of the present application. Optionally, the terminal display device size may be obtained by a transformation based on at least one of the length, width, or diagonal length of the display device. This is not a limitation of the present application. Optionally, the reason for needing to perform the transformation may be for appearance reasons, such as bending or folding, or for visual reasons, such as a change in display size caused by visual correction performed by a VR device. Optionally, the terminal device size may be obtained by actually measuring the terminal device, or may be selected from multiple predefined terminal display device sizes based on a predefined index. Alternatively, the predefined terminal display device size may be directly used as the terminal display device size. The index may correspond to various standards to correspond to display device sizes specified in various standards, such as 1920×1080 HD, 1280×720 HD 720P, and 3840×2160 Ultra HD.

[0175] Optionally, the user viewing distance may be the linear distance between the user and the center point of the terminal display device, the perpendicular distance between the user and the image plane of the terminal display device, or the linear distance between the user and a predetermined position on the image plane of the terminal display device. This is not limited in the present application. Optionally, a single user viewing distance may be obtained by actual measurement, or multiple user viewing distances may be obtained by actual measurement, and then a representative user viewing distance may be selected by calculation. For example, the center of the multiple user viewing distances may be selected as the user viewing distance. Optionally, the user viewing distance may be selected from multiple default user viewing distances based on a default index, or the default user viewing distance may be directly used as the user viewing distance. Optionally, if a single user exists, a single user viewing distance may be obtained by actual measurement, or if multiple users exist, the default user viewing distance may be directly used as the user viewing distance.

[0176] Optionally, the user viewing angle may be a range of angles of deviation from a perpendicular line perpendicular to the display device at the center point position. Optionally, a single user viewing distance may be obtained by actual measurement, or multiple user viewing distances may be obtained by actual measurement, and then a representative user viewing angle may be selected by calculation. For example, an average angle of multiple user viewing angles may be selected as the user viewing angle. Optionally, a user viewing angle may be selected from multiple predefined user viewing angles based on a predefined index, or the predefined user viewing angle may be directly used as the user viewing angle. Optionally, if a single user is present, a single user viewing angle may be obtained by actual measurement, or if multiple users are present, the predefined user viewing angle may be directly used as the user viewing angle.

[0177] Optionally, the ambient light intensity may be obtained by actual measurement by a sensor, the ambient light intensity may be selected from a plurality of predefined ambient light intensities based on a predefined index, or the predefined ambient light intensity may be used directly as the ambient light intensity.

[0178] Optionally, the terminal display device resolution may be the actual value of the terminal display device resolution, or may be selected from multiple default terminal display device resolutions based on the index. Alternatively, the default terminal display device resolution may be directly used as the terminal display device resolution. The terminal display device resolution index may correspond to various standards to correspond to display device resolutions specified in various standards, such as 1920 x 1080 HD, 1280 x 720 HD 720P, and 3840 x 2160 Ultra HD.

[0179] In one possible embodiment, the tone mapping method further comprises adjusting the reference tone mapping parameters of the image to be displayed based on the reference tone mapping parameters and the display capabilities of the terminal display device to obtain initial tone mapping parameters.

[0180] In this embodiment of the present application, the reference tone mapping parameters represent a reference tone mapping curve, and the initial tone mapping parameters represent an initial tone mapping curve.

[0181] First, reference tone mapping parameters of the image to be displayed are obtained. Optionally, the reference tone mapping parameters may be obtained from metadata of the image to be displayed. For example, the metadata may include display capability information of multiple reference display devices, such as peak luminance and initial global tone mapping curve parameters for tone mapping to peak luminance. Alternatively, the metadata may include histogram information and tone mapping curve parameter information of the ST2094-40 standard, for example. Alternatively, the metadata may include tone mapping curve parameter information of the ST2094-10 standard, for example. Optionally, an index of the reference tone mapping parameter may be obtained from the metadata, and the reference tone mapping parameter may be selected from multiple predefined reference tone mapping curve parameters based on the index, or the predefined reference tone mapping parameter may be directly used as the reference tone mapping parameter. The reference tone mapping curve parameter may be obtained from an SEI, a user-specified NAL unit, or a reserved packet unit packaged by HEVC or VVC, from APP extension information packaged in JFIF, or from a data segment packaged in MP4. This is not a limitation in this application.

[0182] Then, the display capability of the terminal display device in the terminal device information is obtained, for example, the peak brightness and valley brightness of the terminal display device. Optionally, the display screen display capability of the terminal display device may be obtained and used as the terminal display device display capability, or the terminal display device display capability may be selected from a plurality of default terminal display device display capabilities based on a default index, or the default terminal display device display capability may be directly used as the terminal display device display capability.

[0183] Optionally, reference display device display capabilities may be obtained from the reference information to adjust the reference tone mapping parameters. The reference display device display capabilities may be the display capabilities of the reference display device used when the image to be displayed is created, such as the peak luminance and valley luminance of the reference display device. Optionally, the reference display device display capabilities may be obtained from metadata of the image to be displayed, and an index of the reference display device display capabilities may be obtained from the metadata, and the reference display device display capability may be selected from a plurality of default reference display device display capabilities based on the index, or the default reference display device display capability may be directly used as the reference display device display capability.

[0184] Finally, the reference tone mapping parameters of the image to be displayed are adjusted based on the reference tone mapping parameters and the display capabilities of the terminal display device to obtain initial tone mapping parameters.

[0185] In one possible embodiment, parameter information of an initial tone mapping curve may be obtained based on parameter information of a reference tone mapping curve and display capability information of a terminal display device.

[0186] Optionally, after the terminal display device display capability information is obtained, one or more appropriate default display device information and one or more corresponding reference tone mapping curve parameters are selected based on the terminal display device display capability, and then initial tone mapping curve parameters are obtained based on the one or more corresponding reference tone mapping curve parameters to adapt to the terminal display device display capability information of the terminal device.

[0187] Optionally, the metadata may include multiple default display device information, such as peak luminance and initial tone mapping curve parameters for tone mapping to the peak luminance. The format of the metadata is not limited in this application. For example, the metadata may include histogram information and tone mapping curve parameter information of the ST2094-40 standard. Alternatively, the metadata may include tone mapping curve parameter information of the ST2094-10 standard.

[0188] Optionally, multiple default display device information, such as peak luminance and reference tone mapping curve parameters for tone mapping to the peak luminance, may be locally pre-set default values.

[0189] Optionally, once the reference tone mapping curve parameters are obtained, parameter adjustment may be performed on the reference tone mapping curve parameters to adapt to terminal display device information.

[0190] Optionally, once multiple reference tone mapping curve parameters are obtained, initial tone mapping curve parameters may be obtained in a weighted averaging manner based on the relationship between the terminal display device information and multiple predefined display device information corresponding to the multiple reference tone mapping curves.

[0191] Optionally, the tone mapping curve may be calculated based on part of the parameter information of the reference tone mapping curve and terminal display device information, where part of the parameter information of the reference tone mapping curve may be carried in metadata or may be a locally pre-set default value.

[0192] In another possible embodiment, parameter information of the initial tone mapping curve may be obtained based on parameter information of the reference tone mapping curve, characteristic information of the image itself, and terminal device information. The characteristic information of the image itself includes content characteristic information, such as maximum, minimum, and average luminance of the content, and characteristic information related to a default percentage distribution of the image. The parameter information of the reference tone mapping curve may be carried in metadata or may be a locally preset default value. The characteristic information of the image itself may be carried in metadata.

[0193] In another possible embodiment, the initial tone mapping curve parameters may be calculated based on content feature information of the image to be displayed in the metadata. Optionally, some of the parameter values ​​of the initial tone mapping curve may be pre-defined and do not need to be calculated.

[0194] It should be noted that the tone mapping curve may take various forms, such as a Sigmoid function curve, a cubic spline function curve, and a Gamma function, which is not limited in the present invention.

[0195] For example, the following function may be used to obtain the initial tone mapping curve:

number

[0196] Based on the curve, a mapping relationship between normalized HDR / SDR source data and normalized HDR display data may be obtained. L may be R, G, or B in the RGB color gamut, or the maximum value max(R,G,B) of R, G, and B, or the luminance Y in the YUV color gamut, or the luminance component L in the Lab color gamut. L and L' may be normalized optical signals or electrical signals, which is not limited in the present invention.

[0197] For example, in this function, m may be preset to 2.4, n may be preset to 1, and b may be preset to the valley brightness of the terminal display device.

[0198] The process of obtaining the parameter p of the function based on the feature information of the image and the predetermined relationship is as follows: first, the feature information of the image is obtained from metadata including the maximum value maxL, the minimum value minL, and the average value avgL, and then p is obtained based on the feature information of the image and the predetermined relationship.

[0199] Optionally, p may be obtained based on avgL and a predetermined relationship.

number

[0200] P valueH0 , P valueL0 , TPH0, and TPL0 are all default parameters. valueH0 may be set to 3.5, P valueL0 may be set to 4.0, TPH0 may be set to 0.6, and TPL0 may be set to 0.3.

[0201] Optionally, p may be derived based on maxL and a predetermined relationship.

number

[0202] PdeltaH1 , P deltaL1 , TPH1, and TPL1 are all default parameters. deltaH1 may be set to 3.5, P deltaL1 may be set to 4.0, TPH1 may be set to 0.6, and TPL1 may be set to 0.3.

[0203] Optionally, p may instead be adjusted based on a predetermined relationship with maxL.

number

number

[0204] The peak luminance of the terminal display device is MaxDisplay, the valley luminance is MinDisplay, and the maximum value of the image feature information obtained from the metadata is maxL.

[0205] Step S504: Perform tone mapping on the image to be displayed according to the reference information and corresponding terminal device information. In this embodiment of the present application, when performing tone mapping on the image to be displayed according to the reference information and corresponding type of terminal device information, the reference display device size corresponds to the terminal display device size, the reference viewing distance corresponds to the user viewing distance, the reference viewing angle corresponds to the user viewing angle, the reference ambient light luminance corresponds to the ambient light luminance, and the reference display device resolution corresponds to the terminal display device resolution.

[0206] In this embodiment of the present application, when tone mapping is performed on an image, other information such as the terminal display device size and the user viewing distance is also taken into consideration, thus improving the display adaptation effect.

[0207] In one possible embodiment, the step of performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information includes the steps of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information, and performing tone mapping on the image to be displayed based on the tone mapping parameters.

[0208] In one possible embodiment, the step of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information includes a step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters.

[0209] In this embodiment of the present application, the step of adjusting the initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain adjusted tone mapping parameters includes the step of adjusting the initial tone mapping parameters of the image to be displayed in various manners to obtain adjusted tone mapping parameters when the relationship between the reference information and the terminal device information satisfies various conditions.

[0210] In one possible embodiment, the step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain tone mapping parameters includes: adjusting the initial tone mapping parameters of the image to be displayed in a first manner to obtain the tone mapping parameters when a relationship between the reference information and the terminal device information satisfies a first condition; and adjusting the initial tone mapping parameters of the image to be displayed in a second manner to obtain the tone mapping parameters when a relationship between the reference information and the corresponding type of terminal device information satisfies a second condition. The initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve.

[0211] Optionally, the first condition may include at least one of the following: the terminal display screen size is smaller than the reference display device size; the user viewing distance is greater than the reference viewing distance; the user viewing angle is offset from the reference viewing angle; the ambient light luminance is greater than the reference ambient light luminance; or the terminal display device resolution is smaller than the reference display device resolution. If the relationship between the reference information and the terminal device information satisfies the first condition, the initial tone mapping parameters need to be adjusted. As a result, the overall luminance of the image obtained by tone mapping based on the adjusted tone mapping parameters may be improved, and higher detail definition may be achieved, such as improved local contrast, enhanced edge definition, more high-frequency components, or improved pixel gradients. This results in a more consistent user perception and an improved user experience.

[0212] When the relationship between the reference information and the terminal device information satisfies the first condition, there are three methods for adjusting the initial tone mapping parameters of the image to be displayed in the first method, namely, Method 1, Method 2, and Method 3. In the present application, one or more of Method 1, Method 2, and Method 3 can be selected and executed. This is not limited in the present application. In addition, when multiple methods are executed, the order in which each method is executed is not limited in the present application. Specific adjustment methods include:

[0213] Method 1: Adjust the initial tone mapping parameters so that the resulting mapped values ​​of the adjusted tone mapping curve at one or more first luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at one or more first luminance feature values.

[0214] In this embodiment of the present application, one or more first luminance feature values ​​of the image to be displayed need to be obtained first. Optionally, the one or more first luminance feature values ​​may be an average luminance value of the entire frame of the image to be displayed, an average luminance value of one or more areas of interest in the image to be displayed, a weighted average luminance value of multiple areas in the image to be displayed, or an average luminance value of one or more luminance ranges among multiple luminance ranges obtained by dividing the luminance histogram of the image to be displayed. This is not limited in this application. The one or more areas of interest in the image to be displayed may be selected from characteristic areas of the image to be displayed, such as faces, green plants, red flags, areas with high complexity, areas with obvious noise, and uneven areas. The method of selecting the characteristic areas is not limited in this application. Optionally, the method of selecting the areas may be obtained from metadata, according to a predetermined rule, or directly based on the characteristics of the image to be displayed. Optionally, the method of calculating the average value may be an arithmetic mean value, a geometric mean value, or the like. This is not limited in this application.

[0215] For example, the average value of the luminance of the entire frame of the image to be displayed may be selected as the first luminance feature value, and the calculation formula is as follows:

number

[0216] L avg is the average brightness of the entire frame of the image to be displayed, and L n,m is an arbitrary luminance value of the luminance of the entire frame of the image to be displayed, width and height are the width and height of the image to be displayed, and n and m are pixel indices in the width and height directions of the image to be displayed, respectively. n,mmay be R, G, or B in the RGB color gamut, or the maximum value of R, G, and B, max(R,G,B), or the luminance Y in the YUV color gamut, or the luminance component L in the Lab color gamut. L and L' may be normalized optical signals or electrical signals, which is not limited in the present invention.

[0217] The initial tone mapping parameters are then adjusted so that the resulting mapped values ​​of the adjusted tone mapping curve at one or more first luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at one or more first luminance feature values.

[0218] In one possible embodiment, the process of adjusting the initial tone mapping parameters first obtains a mapped value of the initial tone mapping curve for one or more first luminance feature values. For example, if the initial tone mapping curve is LY=h(L), L may be R, G, or B in the RGB color gamut, or the maximum value of R, G, and B, max(R,G,B), or the luminance Y in the YUV color gamut, or the luminance component L in the Lab color gamut. L and L' may be normalized optical signals or electrical signals, which is not limited in the present invention. LY is a mapped value obtained by mapping L based on the tone mapping curve, and is the average luminance L of the entire frame of the displayed image. avg The mapped value of the curve at LY avg =h(L avg Optionally, one or more parameters may be adjusted directly to increase the mapped value of the adjusted tone mapping curve at one or more first luminance feature values.

[0219] Optionally, new mapped values ​​for one or more first luminance feature values ​​may be further obtained according to a predetermined algorithm based on the reference information and the terminal display device information, and adjusted tone mapping parameters may be obtained based on the obtained one or more new mapped values. Optionally, the predetermined algorithm may be a predetermined pre-function, and the gain coefficient P corresponding to the reference information and the terminal display device information may be obtained based on the predetermined pre-function. In this case, the new mapped values ​​LYN avg =P*LY avg Optionally, in the form of a predefined pre-function, the gain factor P may be obtained from a predefined look-up table based on the reference information and terminal display device information.

[0220] Optionally, a tone mapping parameter equation or set of equations may be established based on one or more new mapped values ​​to obtain one or more parameters of the adjusted tone mapping curve in combination with other predetermined parameters to obtain all parameters of the adjusted tone mapping curve. Optionally, establishing a tone mapping curve equation or set of equations based on one or more new mapped values ​​may be based on a tone mapping curve function, for example, if the initial tone mapping curve function is LY=h(L), the equation may be avg =h(L avg ), or may be based on an inverse tone mapping curve function, for example, the inverse initial tone mapping curve function is L=h -1 (LY), then the formula is L avg =h -1 (LYN avg Optionally, one or more parameters may instead be adjusted directly so that the mapped values ​​of the adjusted tone mapping curve for one or more first luminance feature values ​​become one or more new mapped values.

[0221] For example, see FIG. 6 for the tone mapping curve adjusted by Scheme 1 and the initial tone mapping curve. FIG. 6 is an exemplary diagram of the initial tone mapping parameter adjustment scheme according to one embodiment of the present application. The initial tone mapping curve represented by the initial tone mapping curve parameters is curve 601. The average luminance of the entire image, 6, is selected as the first luminance feature value. After the parameters are updated, the adjusted tone mapping curve is 602, and the mapped value of the adjusted tone mapping curve 602 at the first luminance feature value 6 is greater than the mapped value of the initial tone mapping curve 601 at the first luminance feature value 6. In addition, the overall mapped value of the adjusted tone mapping curve 602 is greater than that of the initial tone mapping curve 601. In this way, the overall brightness of the image obtained by tone mapping based on the adjusted tone mapping parameters is improved, so that when the terminal display screen size is smaller than the reference display device size, the user viewing distance is larger than the reference viewing distance, the user viewing angle deviates from the reference viewing angle, the ambient light brightness is larger than the reference ambient light brightness, or the terminal display device resolution is smaller than the reference display device resolution, the user's perception is more consistent and the display adaptation effect is better.

[0222] Method 2: Adjust the initial tone mapping parameters so that the resulting slope of the adjusted tone mapping curve at one or more second luminance feature values ​​is greater than the slope of the initial tone mapping curve at one or more second luminance feature values.

[0223] In this embodiment of the present application, one or more second luminance feature values ​​of the image to be displayed need to be obtained first. The method for selecting the one or more second luminance feature values ​​is similar to the method for selecting the one or more first luminance feature values ​​in Scheme 1. Details will not be described again here. Optionally, the one or more second luminance feature values ​​may be partially or completely the same as the one or more first luminance feature values, or may be completely different.

[0224] Then, parameters of the initial tone mapping curve are adjusted so that the slope of the adjusted tone mapping curve at the one or more second luminance feature values ​​is greater than the slope of the initial tone mapping curve at the one or more second luminance feature values. Optionally, one or more parameters may be adjusted directly to increase the slope of the adjusted tone mapping curve at the one or more second luminance feature values.

[0225] For example, first, the first derivative d(LY) / dL=h'(L) is obtained based on the initial tone mapping curve function LY=h(L). The curve parameters are P curve and the second luminance feature value L avg The corresponding slope value is LY' avg =h'(L avg ) Then, based on the reference information and the terminal display device information, a second luminance feature value L avg New curve parameter P1 in curve The new curve parameter P1 curve is h'(P1 curve ,L avg )>h'(P curve ,L avg ) in the default algorithm. avg To analyze the relationship between avg =h'(L avg ) may be used as variables, and the appropriate curve parameters may be adjusted to the parameter P curve is selected as.

[0226] For example, if the initial tone mapping curve is the following function curve:

number

[0227] L is luminance, L' is a mapped value obtained by mapping based on a tone mapping curve, a, p, m, n, and b are parameters of the tone mapping curve, and L may be R, G, or B in the RGB color gamut, or the maximum value max(R,G,B) of R, G, and B, or the luminance Y in the YUV color gamut, or the luminance component L in the Lab color gamut. L and L' may be normalized optical signals or electrical signals, which is not limited in the present invention.

[0228] To obtain adjusted tone mapping parameters, the slope value can be increased by decreasing p based on the current values ​​of the curve parameters p and a, and by increasing a.

[0229] Optionally, a new slope for one or more first luminance feature values ​​may be obtained according to a predetermined algorithm based on the reference information and the terminal display device information, and an equation or set of equations for tone mapping curve parameters may be established based on the one or more new slopes, and one or more parameters of the adjusted tone mapping curve may be obtained by solving the equation or equations in combination with other predetermined parameters to obtain all parameters of the adjusted tone mapping curve. Optionally, the predetermined algorithm may be a predetermined prior function, and a gain coefficient Q corresponding to the reference information and the terminal display device information may be obtained based on the predetermined prior function. In this case, the new slope LYN' avg =Q*LY' avg Optionally, in the form of a predefined pre-function, the gain factor Q may be obtained from a predefined look-up table based on the reference information and the terminal display device information. For example, if the initial tone mapping curve function is LY=h(L), then the first derivative is d(LY) / dL=h'(L), and the formula is LYN' avg =h'(L avg) Alternatively, the formula or set of formulas may be derived based on the inverse of the first derivative of the tone mapping curve function, which is not a limitation herein. Optionally, one or more parameters may instead be adjusted directly, such that the slope of the adjusted tone mapping curve at one or more first luminance feature values ​​becomes one or more new slopes.

[0230] For example, see FIG. 7 for the tone mapping curve adjusted by Scheme 2 and the initial tone mapping curve. FIG. 7 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to one embodiment of the present application. The initial tone mapping curve is curve 701. An average luminance of 5 for the entire image is selected as the second luminance feature value. After the parameters are updated, the adjusted tone mapping curve is 702, and the slope of the adjusted tone mapping curve 702 at the second luminance feature value 5 is greater than the slope of the initial tone mapping curve 701 at the second luminance feature value 5. In addition, the slopes of multiple portions of the adjusted tone mapping curve 702 are greater than the overall slope of the initial tone mapping curve 701. In this way, the overall contrast of the image obtained by tone mapping based on the adjusted tone mapping parameters is improved, and its pixel gradient is improved, so that higher detail definition is achieved, so that when the terminal display screen size is smaller than the reference display device size, when the user viewing distance is larger than the reference viewing distance, when the user viewing angle deviates from the reference viewing angle, when the ambient light brightness is larger than the reference ambient light brightness, or when the terminal display device resolution is smaller than the reference display device resolution, the user's perception is more consistent, the display adaptation effect is better, and the display effect of complex images is especially improved.

[0231] Method 3: Obtain a base layer and one or more enhancement layers of an image to be displayed, wherein a value obtained by mapping a first feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the any of the one or more enhancement layers is a second feature value, and adjust the initial tone mapping curve of the any of the enhancement layers so that the absolute value of the value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value.

[0232] In this embodiment, the first feature value and the second feature value may be positive or negative values, and when the value obtained by mapping the first feature value based on the adjusted tone mapping curve is compared with the second feature value, the absolute values ​​of the two values ​​are compared.

[0233] In this embodiment of the present application, a base layer and one or more enhancement layers of an image to be displayed are first obtained. Optionally, the enhancement layer includes at least one of an edge layer or a texture layer. Optionally, the base layer and the one or more enhancement layers may be obtained by filtering with a filter, or the base layer and the one or more enhancement layers may be obtained by combining image segmentation and filtering. Optionally, the filter may be a bilateral filter, a multilateral filter, or the like, as is not limited in the present application. Any value of any of the one or more enhancement layers is a first feature value, and a value obtained by mapping the any enhancement layer to the first feature value based on an initial tone mapping curve is a second feature value.

[0234] Then, the initial tone mapping parameters of the any enhancement layer are adjusted so that the absolute value of the value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value. Optionally, the value obtained by mapping based on the adjusted tone mapping curve may be increased by N times the second feature value, where N is an integer, fraction, or decimal greater than 1. This is not a limitation in the present application.

[0235] For example, the image may be filtered with a spatial filter, a temporal filter, or a combination thereof to extract data for individual layers of the image.

[0236] Any luminance value L1[m] of the base layer L1 may be:

number

[0237] L is the luminance value of the original image, filter[n] is the filter coefficient at any position in the filter, and filterL is the length of the filter.

[0238] An arbitrary luminance value L2[m] (first feature value) in the enhancement layer L2 may be as follows: L2[m]=L[m]-L1[m]

[0239] L is the luminance value of the original image, and L may be R, G, or B in the RGB gamut, or the maximum value of R, G, and B max(R,G,B), or the luminance Y in the YUV gamut, or the luminance component L in the Lab gamut.

[0240] Then, based on the tone mapping parameter information transferred by the predetermined information or metadata, the tone mapping curves to be used by the multiple layers respectively are obtained, and tone mapping is performed separately for each layer.

[0241] For example, if the tone mapping curve obtained for the base layer is g(), then any mapped value of the base layer L1 obtained by mapping is L1'[m]. L1'[m]=g(L1[m])

[0242] Before adjustment, for the tone mapping curve f() obtained for the enhancement layer, any mapped value of the enhancement layer L2 obtained by mapping is L2'[m] (the second feature value). L2'[m]=f(L2[m])

[0243] When the mapped value is increased by N times, L2'[m] is increased by N times to L2''[m], where N is an integer or fraction greater than 1.

[0244] L2''[m]=N*L2'[m] The final mapped value L'[m] of the image is obtained by mixing the mapped value L1'[m] of the base layer obtained by tone mapping with the mapped value L2''[m] of the enhancement layer obtained by tone mapping based on a gain coefficient gain[n] that is set in advance or sent with the metadata. L'[m]=gain[1]*L1'[m]+gain[2]*L2''[m]=gain[1]*L1'[m]+gain[1]*N*L2'[m]

[0245] The mapped value of the enhancement layer in the final mapped value of the image is N times larger than that obtained before the adjustment. The brightness of the enhancement layer in the image is improved, which improves the local contrast and detail definition of the image, and improves the user's viewing experience.

[0246] Optionally, if L is R, G, and B in the RGB gamut, then L1' corresponds to RP[0], GP[0], and BP[0], and L2'' corresponds to RP[1], GP[1], and BP[1].

[0247] Finally, RP[n], GP[n], and BP[n] are combined with a gain factor gain[n], which is preset or sent with the metadata, for multiple layers to obtain the final mapped values ​​Rtm, Gtm, and Btm, where n indicates the index of the multiple layers. Rtm=ΣRP[n]*gain[n] Gtm=ΣGP[n]*gain[n] and Btm=ΣBP[n]*gain[n]

[0248] For example, see FIG. 8 for the tone mapping curve and initial tone mapping curve of the enhancement layer adjusted by Scheme 3. FIG. 8 is an exemplary diagram of an initial tone mapping parameter adjustment scheme for the enhancement layer according to one embodiment of the present application. The tone mapping curve of the base layer is 801, and the initial tone mapping curve of the enhancement layer is curve 802. The parameters are adjusted to double the mapped value of curve 802, resulting in an adjusted tone mapping curve of the enhancement layer, 803. Compared with that of curve 802, the overall mapped value of curve 803 is increased by two times. Therefore, in the image obtained by blending the base layer and the enhancement layer tone-mapped based on the adjusted tone mapping parameters of the enhancement layer, more high-frequency components are preserved, the features of the enhancement layer are more distinct, and local contrast is improved, thereby achieving higher detail definition. For example, if the enhancement layer is an edge layer, the sharpness of the edges is enhanced, and if the enhancement layer is a texture layer, the texture details are more distinct. Therefore, when the terminal display screen size is smaller than the reference display device size, when the user viewing distance is larger than the reference viewing distance, when the user viewing angle deviates from the reference viewing angle, when the ambient light brightness is larger than the reference ambient light brightness, or when the terminal display device resolution is smaller than the reference display device resolution, the user's perception is more consistent and the display adaptation effect is better, especially the display effect of complex images is improved.

[0249] In one possible embodiment, adjusting the initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters includes adjusting the initial tone mapping parameters of the image to be displayed in a second manner to obtain the tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a second condition, wherein the initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve.

[0250] Optionally, the second condition includes at least one of the following: the terminal display screen size is larger than the reference display device size; the user viewing distance is smaller than the reference viewing distance; the ambient light luminance is smaller than the reference ambient light luminance; or the terminal display device resolution is larger than the reference display device resolution. If the relationship between the reference information and the terminal device information satisfies the second condition, it should be understood that the initial tone mapping curve parameters need to be adjusted, and as a result, the average luminance of the image obtained by tone mapping based on the adjusted tone mapping curve parameters will be reduced or dark areas will become darker. Therefore, the user's perception will be more consistent, and the user experience will be improved.

[0251] If the relationship between the reference information and the corresponding type of terminal device information satisfies the second condition, there are three methods for adjusting the initial tone mapping parameters of the image to be displayed in the second method, namely, Method 4, Method 5, and Method 6. In the present application, one or more of Method 4, Method 5, and Method 6 can be selected and executed. This is not limited in the present application. In addition, when multiple methods are executed, the order in which each method is executed is not limited in the present application. Specific adjustment methods include:

[0252] Method 4: Adjust the initial tone mapping parameters so that the resulting mapped values ​​of the adjusted tone mapping curve at one or more third luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at one or more third luminance feature values.

[0253] In this embodiment of the present application, one or more third luminance feature values ​​of the image to be displayed need to be obtained first. The method for selecting one or more third luminance feature values ​​is similar to the method for selecting one or more first luminance feature values ​​in Scheme 1. Details will not be described again here.

[0254] Then, the parameters of the initial tone mapping curve are adjusted so that the resulting mapped values ​​of the adjusted tone mapping curve at one or more third luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at one or more third luminance feature values.

[0255] In one possible embodiment, the process of adjusting the initial tone mapping parameters first obtains mapped values ​​of the initial tone mapping curve at one or more first luminance feature values. For example, if the initial tone mapping curve function is LY=h(L), the average luminance value L of the entire frame of the image to be displayed is obtained. avg The mapped value of the curve at LY avg =h(L avg Optionally, one or more parameters may be adjusted directly to reduce the mapped values ​​of the adjusted tone mapping curve at one or more third luminance feature values.

[0256] Optionally, new mapped values ​​for one or more first luminance feature values ​​may be further obtained according to a predetermined algorithm based on the reference information and the terminal display device information, and parameters of the adjusted tone mapping curve may be obtained based on the obtained one or more new mapped values. Optionally, the predetermined algorithm may be a predetermined pre-function, and the attenuation coefficient U corresponding to the reference information and the terminal display device information may be obtained based on the pre-defined pre-function. In this case, the new mapped value LYN avg =U*LY avg Optionally, in the form of a predefined prior function, the attenuation coefficient U may be obtained from a predefined look-up table based on the reference information and terminal display device information.

[0257] Optionally, a formula or set of formulas for the parameters of the tone mapping curve may be established based on one or more new mapped values ​​to obtain one or more parameters of the adjusted tone mapping curve in combination with other predetermined parameters to obtain all parameters of the adjusted tone mapping curve. Optionally, establishing a formula or set of formulas for the tone mapping curve parameters based on one or more new mapped values ​​may be based on the tone mapping curve, for example, if the initial tone mapping curve function is LY=h(L), the formula may be avg =h(L avg ), or may be based on an inverse tone mapping curve function, for example, the inverse initial tone mapping curve function is L=h -1 (LY), then the formula is L avg =h -1 (LYN avg Optionally, one or more parameters may instead be adjusted directly so that the mapped values ​​of the adjusted tone mapping curve for one or more first luminance feature values ​​become one or more new mapped values.

[0258] Optionally, the one or more third luminance feature values ​​include one or more luminance feature values ​​of a dark portion of the image to be displayed, or one or more luminance feature values ​​of the entire image to be displayed. The dark portion may be a luminance range of less than 5 nits, which is below the minimum luminance distinguishable by the human eye, or may be a darker portion of the image obtained by dividing the image or dividing the luminance histogram. This is not a limitation of the present application.

[0259] For example, if the third luminance feature value is the average luminance value of the dark parts of the image, see FIG. 9 for the tone mapping curve adjusted by Scheme 4 and the initial tone mapping curve. FIG. 9 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to an embodiment of the present application. The initial tone mapping curve represented by the initial tone mapping parameters is curve 901. An average luminance of 4.5 in the dark parts of the image is selected as the third luminance feature value. After the parameters are updated, the adjusted tone mapping curve is 902, and the mapped value of the adjusted tone mapping curve 902 at the third luminance feature value of 4.5 is smaller than the mapped value of the initial tone mapping curve 901 at the third luminance feature value of 4.5. In addition, the overall mapped value of the dark parts of the adjusted tone mapping curve 902 is smaller than that of the initial tone mapping curve 901. Thus, the dark parts of the image obtained by tone mapping based on the adjusted tone mapping parameters become darker. Since the human eye is more sensitive to dark areas, if the terminal display screen size is larger than the reference display device size, if the user viewing distance is smaller than the reference viewing distance, if the ambient light luminance is smaller than the reference ambient light luminance, or if the terminal display device resolution is larger than the reference display device resolution, the human eye may see noise in the dark areas. Therefore, the dark areas can be made darker to adapt to changing application scenarios, so that the user's visual perception is more consistent, and the image display adaptation effect and display effect are improved.

[0260] For example, when the third luminance feature value is the average luminance value of the entire image, see FIG. 10 for the tone mapping curve adjusted by Scheme 4 and the initial tone mapping curve. FIG. 10 is an exemplary diagram of an initial tone mapping parameter adjustment scheme according to an embodiment of the present application. The initial tone mapping curve is curve 1001. The average luminance of 6 of the entire image is selected as the third luminance feature value. After the parameters are updated, the adjusted tone mapping curve is 1002, and the mapped value of the adjusted tone mapping curve 1002 at the average luminance of 6 is smaller than the mapped value of the initial tone mapping curve 1001 at the average luminance of 6. In addition, the overall mapped value of the adjusted tone mapping curve 1002 is smaller than that of the initial tone mapping curve 1001. In this way, the overall brightness of the image obtained by tone mapping based on the adjusted tone mapping parameters is reduced, so that when the terminal display screen size is larger than the reference display device size, when the user viewing distance is smaller than the reference viewing distance, when the ambient light brightness is smaller than the reference ambient light brightness, or when the terminal display device resolution is larger than the reference display device resolution, the user's visual perception is more consistent, the display adaptation effect is better, and the user's eyes can be protected to a certain extent.

[0261] Method 5: Adjust the initial tone mapping parameters so that the resulting slope of the adjusted tone mapping curve at one or more fourth luminance feature values ​​is smaller than the slope of the initial tone mapping curve at one or more fourth luminance feature values.

[0262] In this embodiment of the present application, one or more fourth luminance feature values ​​of the image to be displayed need to be obtained first. The method for selecting the one or more fourth luminance feature values ​​is similar to the method for selecting the one or more first luminance feature values ​​in Scheme 1. Details will not be described again here. Optionally, the one or more fourth luminance feature values ​​may be partially or completely the same as the one or more third luminance feature values, or may be completely different.

[0263] Then, parameters of the initial tone mapping curve are adjusted so that the slope of the adjusted tone mapping curve at the one or more fourth luminance feature values ​​is less than the slope of the initial tone mapping curve at the one or more fourth luminance feature values. Optionally, one or more parameters may be adjusted directly to reduce the slope of the adjusted tone mapping curve at the one or more fourth luminance feature values.

[0264] For example, first, the first derivative d(LY) / dL=h'(L) is obtained based on the initial tone mapping curve function LY=h(L). The curve parameters are P curve and the second luminance feature value L avg The corresponding slope value is LY' avg =h'(L avg ) Then, based on the reference information and the terminal display device information, a second luminance feature value L avg New curve parameter Q1 in curve The new curve parameter P1 curve is h'(P1 curve ,L avg ) <h’(P curve ,L avg ) in the default algorithm. avg To analyze the relationship between avg =h'(L avg ) may be used as variables, and the appropriate curve parameters may be adjusted to the parameter P curve is selected as.

[0265] For example, if the initial tone mapping curve is the following function curve:

number

[0266] L is the luminance, L' is the mapped value obtained by mapping based on the tone mapping curve, and a, p, m, n, and b are parameters of the tone mapping curve. L may be R, G, or B in the RGB gamut, or the maximum value of R, G, and B max(R,G,B), or the luminance Y in the YUV gamut, or the luminance component L in the Lab gamut.

[0267] To obtain adjusted tone mapping curve parameters, the slope value can be reduced by reducing a based on the current values ​​of the curve parameters p and a, and by increasing p.

[0268] Optionally, a new slope for one or more first luminance feature values ​​may be obtained according to a predetermined algorithm based on the reference information and the terminal display device information, and an equation or set of equations for tone mapping curve parameters may be established based on the one or more new slopes, and one or more parameters of the adjusted tone mapping curve may be obtained by solving the equation or equations in combination with other predetermined parameters to obtain all parameters of the adjusted tone mapping curve. Optionally, the predetermined algorithm may be a predetermined prior function, and the attenuation coefficient V corresponding to the reference information and the terminal display device information may be obtained based on the predetermined prior function. In this case, the new slope LYN' avg =V*LY' avg Optionally, in the form of a predefined prior function, the attenuation coefficient V may be obtained from a predefined lookup table based on the reference information and the terminal display device information. For example, if the initial tone mapping curve function is LY=h(L), then the first derivative is d(LY) / dL=h'(L), and the formula is LYN'avg=h'(L avg) Alternatively, the formula or set of formulas may be derived based on the inverse of the first derivative of the tone mapping curve function, which is not a limitation herein. Optionally, one or more parameters may instead be adjusted directly, such that the slope of the adjusted tone mapping curve at one or more first luminance feature values ​​becomes one or more new slopes.

[0269] In this embodiment of the present application, the overall contrast of the image obtained by tone mapping based on the adjusted tone mapping curve parameters is reduced and its pixel gradient is improved, thereby achieving lower detail definition, so that when the terminal display screen size is larger than the reference display device size, when the user viewing distance is smaller than the reference viewing distance, when the ambient light brightness is smaller than the reference ambient light brightness, or when the terminal display device resolution is larger than the reference display device resolution, poor image display caused by excessively high contrast is prevented, and the display adaptation effect and the display effect of complex images are improved.

[0270] Method 6: Obtain a base layer and one or more enhancement layers of an image to be displayed, wherein a value obtained by mapping a third feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the any of the enhancement layers is a fourth feature value, and adjust an initial tone mapping parameter of the any of the enhancement layers such that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is smaller than the absolute value of the fourth feature value.

[0271] In this implementation, the third and fourth feature values ​​may be positive or negative values, and when the value obtained by mapping the third feature value based on the adjusted tone mapping curve is compared with the fourth feature value, the absolute values ​​of the two values ​​are compared.

[0272] In this embodiment of the present application, a base layer and one or more enhancement layers of an image to be displayed are first obtained. Optionally, the enhancement layer includes at least one of an edge layer or a texture layer. Optionally, the base layer and the one or more enhancement layers may be obtained by filtering with a filter, or the base layer and the one or more enhancement layers may be obtained by combining image segmentation and filtering. Optionally, the filter may be a bilateral filter, a multilateral filter, or the like, as is not limited in the present application. Any value of any of the one or more enhancement layers is a third feature value, and a value obtained by mapping the any enhancement layer to the third feature value based on an initial tone mapping curve is a fourth feature value.

[0273] Then, the initial tone mapping parameters of the given enhancement layer are adjusted so that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is greater than the absolute value of the fourth feature value.

[0274] The value obtained by mapping based on the adjusted tone mapping curve may be reduced by M times the fourth feature value, where M is a fraction or decimal number greater than 0 and less than 1. This is not limited in the present application.

[0275] For example, the image may be filtered with a spatial filter, a temporal filter, or a combination thereof to extract data for individual layers of the image.

[0276] Any luminance value L1[m] of the base layer L1 may be:

number

[0277] L is the luminance value of the original image, filter[n] is the filter coefficient at any position in the filter, and filterL is the length of the filter.

[0278] An arbitrary luminance value L2[m] (third feature value) in the enhancement layer L2 may be as follows: L2[m]=L[m]-L1[m]

[0279] L is the luminance value of the original image, and may be R, G, or B in the RGB gamut, or the maximum value of R, G, and B max(R,G,B), or the luminance Y in the YUV gamut, or the luminance component L in the Lab gamut.

[0280] Then, based on the tone mapping parameter information transferred by the predetermined information or metadata, the tone mapping curves to be used by the multiple layers respectively are obtained, and tone mapping is performed separately for each layer.

[0281] For example, if the tone mapping curve obtained for the base layer is g(), then any mapped value of the base layer L1 obtained by mapping is L1'[m]. L1'[m]=g(L1[m])

[0282] Before adjustment, for the tone mapping curve f() obtained for the enhancement layer, an arbitrary mapped value of the enhancement layer L2 obtained by mapping is L2'[m] (the fourth feature value). L2'[m]=f(L2[m])

[0283] The mapped value is reduced by a factor of M, and L2'[m] is reduced by a factor of M to L2''[m], where M is an integer or fraction greater than 0 and less than 1. L2''[m]=M*L2'[m]

[0284] The final mapped value L'[m] of the image is obtained by mixing the mapped value L1'[m] of the base layer obtained by tone mapping with the mapped value L2''[m] of the enhancement layer obtained by tone mapping based on a gain coefficient gain[n] that is preset or transmitted with the metadata. L'[m]=gain[1]*L1'[m]+gain[2]*L2''[m]=gain[1]*L1'[m]+gain[1]*M*L2'[m]

[0285] The mapped value of the enhancement layer in the final mapped value of the image is M times smaller than that before adjustment. The luminance of the enhancement layer in the image is reduced, which reduces the local contrast and detail definition of the image and prevents the user from experiencing visual discomfort caused by excessively high contrast.

[0286] Optionally, if L is R, G, and B in the RGB gamut, then L1` corresponds to RP[0], GP[0], and BP[0], and L2`` corresponds to RP[1], GP[1], and BP[1].

[0287] Finally, RP[n], GP[n], and BP[n] are combined with a gain factor gain[n], which is preset or sent with the metadata, for multiple layers to obtain the final mapped values ​​Rtm, Gtm, and Btm, where n indicates the index of the multiple layers. Rtm=ΣRP[n]*gain[n] Gtm=ΣGP[n]*gain[n] and Btm=ΣBP[n]*gain[n]

[0288] In this embodiment of the present application, in the image obtained by mixing the base layer and the enhancement layer that is tone-mapped based on the adjusted tone mapping curve parameters of the enhancement layer, the mapped values ​​of the enhancement layer are reduced, the features of the enhancement layer are weakened, fewer high-frequency components are ensured, and local contrast is reduced, thereby reducing detail definition. For example, if the enhancement layer is an edge layer, the edge definition is weakened, and if the enhancement layer is a texture layer, the texture details are weakened. Therefore, when the terminal display screen size is smaller than the reference display device size, when the user viewing distance is greater than the reference viewing distance, when the user viewing angle is deviated from the reference viewing angle, when the ambient light brightness is greater than the reference ambient light brightness, or when the terminal display device resolution is smaller than the reference display device resolution, it is possible to prevent user discomfort caused by poor image display and excessively high contrast, resulting in a better display adaptation effect, especially an improved display effect for complex images, and an improved user viewing experience.

[0289] In one possible embodiment, tone mapping is performed on the image to be displayed based on the parameters of the adjusted tone mapping curve, ie, the tone mapping parameters.

[0290] In this embodiment of the present application, tone mapping is performed on the image to be displayed based on the adjusted tone mapping parameters. The method and curve used in the tone mapping process are not limited in this application. Optionally, the method used in the tone mapping process can be global tone mapping or local tone mapping, which is not limited in this application.

[0291] For example, a reference tone mapping implementation is as follows:

[0292] First, a tone mapping coefficient lookup table lut is obtained based on the adjusted tone mapping parameters, and the lookup table lut is used to obtain a tone mapping coefficient Ratio.

[0293] Next, the luminance value lum of each pixel (R, G, B) is obtained, and the tone mapping coefficient Ratio corresponding to the luminance value lum is obtained from the lookup table lut: Ratio=lut[lum]

[0294] It should be noted that lum is not limited in this application and other luminances may be used in the calculation, such as R, G, or B in the RGB gamut, or the luminance component L in the Lab gamut, or I in the ICtCp format, or the luminance Y in the YUV gamut.

[0295] When Lum is calculated based on R, G, and B of the RGB color gamut, the method for calculating the luminance value Lum of each pixel (R, G, B) is not limited in this application, for example, Lum=max(R, G, B) or Lum=a*R+b*G+c*B, where a, b, and c are weighting coefficients.

[0296] Finally, tone mapping is performed by multiplying each pixel (R, G, B) by the tone mapping coefficient Ratio. TM , G TM , B TM ) R TM =R*Ratio, G TM =G*Ratio, and B TM =B*Ratio.

[0297] Alternatively, [YUV]=M*[RGB], YTM=Y*Ratio, and [RGB]=M -1 *[YUV], where M is selected according to the BT.2020 or BT.709 standard.

[0298] Optionally, when tone mapping is performed on the image to be displayed based on the reference information and the corresponding type of terminal device information, the reference information and the corresponding type of terminal device information may be input to an AI neural network, and the AI ​​neural network outputs an image obtained by tone mapping on the image to be displayed.

[0299] Optionally, the saturation of each pixel (R, G, B) in the image obtained by tone mapping may be adjusted. It should be understood that the metadata may include color attribute information (e.g., color saturation adjustment intensity) of the at least one frame of image. The color information of the at least one frame of image obtained by mapping may be corrected based on color attribute information and luminance information of each pixel in the image to be displayed obtained before and after tone mapping.

[0300] For example, pixel A in an image to be displayed is used as an example. A ratio between a first luminance value obtained by tone mapping for pixel A and a second luminance value obtained before tone mapping of pixel A can be determined. A saturation adjustment coefficient is determined based on the ratio, the maximum display capability of the terminal display device, the maximum display capability of a reference display device (e.g., a video production device) in the front-end device, color attribute information of at least one frame of image, and the second luminance value before tone mapping of pixel A. The color information of pixel A is adjusted based on the saturation adjustment coefficient. The maximum display capability can be understood as a maximum display luminance value.

[0301] It should be understood that the color information may be a YUV signal or a color signal of another color gamut, which is not specifically limited in this embodiment of the present application. The Y in "YUV" represents the luminance information of the image, and UV represents the color information of the image.

[0302] In one possible implementation, the color information is a YUV signal, and the color information may be corrected by multiplying the UV components of each pixel in at least one frame of image obtained by tone mapping by a saturation adjustment coefficient.

[0303] For example, this saturation adjustment process may be as follows: The ratioc for color adjustment is obtained based on the luminance value lum of each pixel and the tone mapping coefficient Ratio, and the ratioc is expressed as: ratioc=(ratio) N etc.

[0304] [YUV]=M*[RGB], UTM=U*ratioc, and VTM=V*ratioc, where M is selected according to the BT.2020 or BT.709 standard.

[0305] It should be noted that the present invention is not limited to saturation adjustment algorithms, also called color adjustment algorithms, and may be implemented in various color gamuts, for example, RGB, IctCp, HSV, and Lab.

[0306] Optionally, gamut mapping may be added after tone mapping or saturation adjustment to convert from the current color gamut to a target color gamut, including but not limited to the color gamuts specified in the BT.2020 standard, the BT.709 standard, the DCI-P3 standard, and the SRGB standard.

[0307] The following describes the process of image encoding by the front-end device with reference to specific accompanying drawings.

[0308] Please refer to Figure 11. Figure 11 is a schematic flowchart of an image encoding method according to an embodiment of the present application. The method may be performed by a front-end device, and specifically includes the following steps:

[0309] Step S1101: The image to be displayed is written into the bitstream.

[0310] It should be understood that the image to be displayed may be one or more frames of an image. When the image to be displayed is an image of multiple consecutive frames, the image to be displayed may be understood as a video. Optionally, the image to be displayed may be an HDR image or video, or an SDR image or video.

[0311] Step S1102: Write reference information of the image to be displayed into the bitstream, where the reference information includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution.

[0312] In this embodiment of the present application, the reference information is the same as the reference information in step S502, and the details will not be described again here. The reference information is for tone mapping to the image to be displayed.

[0313] An embodiment of the present application provides a bitstream structure, where the bitstream includes at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution.

[0314] Optionally, the method further includes writing reference tone mapping parameters of the to-be-displayed image into the bitstream, the reference tone mapping parameters being for deriving initial tone mapping parameters, the reference information being used by the decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, the tone mapping parameters being for tone mapping for the to-be-displayed image.

[0315] Optionally, writing the reference information of the image to be displayed into the bitstream includes writing the reference information into the bitstream or writing an index of the reference information into the bitstream. The index of the reference information is the same as the index of the reference information in step S502. Details will not be described again here.

[0316] Optionally, the method further includes writing one or more luma feature values ​​of the image to be displayed into the bitstream. The one or more luma feature values ​​are used by the decoder to adjust the initial tone mapping parameters to obtain adjusted tone mapping parameters. The one or more luma feature values ​​are similar to the one or more first luma feature values ​​of Scheme 1. Details will not be described again here.

[0317] An embodiment of the present application further provides a tone mapping device 1200. Please refer to Fig. 12. Fig. 12 is an exemplary diagram of a tone mapping device according to an embodiment of the present application. 1200 has a function of implementing the terminal display device of Fig. 4A. The functions, units, or means may be implemented using software, hardware, or by hardware executing corresponding software.

[0318] The receiving module 1201 receives the image to be displayed, obtaining reference information, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; and Obtaining terminal device information, the terminal device information including at least one of a terminal display device size, a user viewing distance, a user viewing angle, an ambient light brightness, or a terminal display device resolution; It is configured as follows.

[0319] The tone mapping module 1202 is configured to perform tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information.

[0320] In this embodiment of the present application, when performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information, the reference display device size corresponds to the terminal display device size, the reference viewing distance corresponds to the user viewing distance, the reference viewing angle corresponds to the user viewing angle, the reference ambient light brightness corresponds to the ambient light brightness, and the reference display device resolution corresponds to the terminal display device resolution.

[0321] In one possible embodiment, the step of performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information includes the steps of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information, and performing tone mapping on the image to be displayed based on the tone mapping parameters.

[0322] In one possible embodiment, the step of obtaining tone mapping parameters based on the reference information and the corresponding type of terminal device information includes a step of adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters.

[0323] In one possible embodiment, adjusting the initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters includes adjusting the initial tone mapping parameters of the image to be displayed in a first manner to obtain the adjusted tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a first condition, wherein the initial tone mapping parameters represent an initial tone mapping curve and the tone mapping parameters represent an adjusted tone mapping curve.

[0324] In one possible embodiment, the first condition includes at least one of the following: the terminal display screen size is smaller than the reference display device size; the user viewing distance is larger than the reference viewing distance; the user viewing angle deviates from the reference viewing angle; the ambient light luminance is larger than the reference ambient light luminance; or the terminal display device resolution is smaller than the reference display device resolution.

[0325] In one possible embodiment, the step of adjusting initial tone mapping parameters of the image to be displayed in a first manner includes the steps of obtaining one or more first luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained mapped values ​​of the adjusted tone mapping curve at the one or more first luminance feature values ​​are greater than the mapped values ​​of the initial tone mapping curve at the one or more first luminance feature values.

[0326] In one possible embodiment, the step of adjusting the initial tone mapping curve of the image to be displayed in the first manner further includes the steps of obtaining one or more second luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping curve so that the obtained slope of the adjusted tone mapping curve at the one or more second luminance feature values ​​is greater than the slope of the initial tone mapping curve at the one or more second luminance feature values.

[0327] In one possible embodiment, the step of adjusting the initial tone mapping parameters of the image to be displayed in a first manner further includes the steps of obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a first feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the any of the one or more enhancement layers is a second feature value, and adjusting the initial tone mapping parameters of the any of the enhancement layers so that the absolute value of the value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value.

[0328] In this embodiment, the first feature value and the second feature value may be positive or negative values, and when the value obtained by mapping the first feature value based on the adjusted tone mapping curve is compared with the second feature value, the absolute values ​​of the two values ​​are compared.

[0329] In one possible embodiment, adjusting the initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters includes adjusting the initial tone mapping parameters of the image to be displayed in a second manner to obtain the tone mapping parameters when the relationship between the reference information and the terminal device information satisfies a second condition, wherein the initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve.

[0330] In one possible embodiment, the second condition includes at least one of the following: the terminal display screen size is larger than the reference display device size; the user viewing distance is smaller than the reference viewing distance; the ambient light luminance is smaller than the reference ambient light luminance; or the terminal display device resolution is larger than the reference display device resolution.

[0331] In one possible embodiment, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes the steps of obtaining one or more third luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained mapped values ​​of the adjusted tone mapping curve at the one or more third luminance feature values ​​are smaller than the mapped values ​​of the initial tone mapping curve at the one or more third luminance feature values.

[0332] In one possible embodiment, the one or more third luminance feature values ​​include one or more luminance feature values ​​of dark portions of the image to be displayed.

[0333] In one possible embodiment, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner further includes the steps of obtaining one or more fourth luminance feature values ​​of the image to be displayed, and adjusting the initial tone mapping parameters so that the obtained slope of the adjusted tone mapping curve at the one or more fourth luminance feature values ​​is smaller than the slope of the initial tone mapping curve at the one or more fourth luminance feature values.

[0334] In one possible embodiment, the step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner further includes the steps of obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a third feature value of any of the one or more enhancement layers based on an initial tone mapping curve of the any of the one or more enhancement layers is a fourth feature value, and adjusting the initial tone mapping parameters of the any of the enhancement layers so that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is smaller than the absolute value of the fourth feature value.

[0335] In this implementation, the third and fourth feature values ​​may be positive or negative values, and when the value obtained by mapping the third feature value based on the adjusted tone mapping curve is compared with the fourth feature value, the absolute values ​​of the two values ​​are compared.

[0336] In one possible embodiment, the tone mapping module is further configured to obtain a reference tone mapping of the image to be displayed, obtain a terminal display device display capability in the terminal device information, and adjust the reference tone mapping parameter of the image to be displayed based on the reference tone mapping parameter and the terminal display device display capability to obtain an initial tone mapping parameter.

[0337] In one possible embodiment, the step of obtaining the reference information includes the step of obtaining the reference information from metadata of the image to be displayed, the step of obtaining an index of the reference information from the metadata, and the step of selecting the reference information from a plurality of predefined reference information based on the index, or the step of using the predefined reference information as the reference information.

[0338] In one possible embodiment, the step of obtaining the terminal device information includes a step of obtaining the terminal device information by measurement, a step of selecting the terminal device information from a plurality of predefined terminal information based on a predefined index, or a step of using the predefined terminal device information as the terminal device information.

[0339] It should be understood that the third aspect corresponds to the technical solution of the first aspect, and therefore the beneficial effects obtained in the possible implementations are similar, and the details will not be described again here.

[0340] It should be understood that all relevant contents of the steps in the foregoing embodiments of the tone mapping method can be cited in the functional descriptions of the corresponding functional modules, and will not be described in detail again here.

[0341] An embodiment of the present application further provides an image encoding device 1300. Please refer to Fig. 13. Fig. 13 is an exemplary diagram of an image encoding device according to an embodiment of the present application. The device 1300 has a function of implementing the front-end device of Fig. 4A. For example, a function, unit, or means may be implemented using software, hardware, or by hardware executing corresponding software.

[0342] See, for example, Figure 13. The device 1300 includes: an encoding module 1301 configured to write a to-be-displayed image into a bitstream and to write reference information for the to-be-displayed image into the bitstream, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution, and the reference information is for tone mapping for the to-be-displayed image; a transmitting module 1302 configured to transmit the bitstream; may include:

[0343] In one possible embodiment, the encoding module is further configured to write reference tone mapping parameters of the image to be displayed into the bitstream, the reference tone mapping parameters being for deriving initial tone mapping parameters, and the reference information being used by the decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, the tone mapping parameters being for tone mapping for the image to be displayed.

[0344] In one possible embodiment, writing reference information for the image to be displayed into the bitstream comprises writing the reference information into the bitstream or writing an index of the reference information into the bitstream.

[0345] In one possible embodiment, the encoding module is further configured to write one or more luma feature values ​​of the image to be displayed into the bitstream, the one or more luma feature values ​​being used by the decoder to adjust the initial tone mapping parameters to obtain the tone mapping parameters.

[0346] It should be understood that all relevant contents of the steps in the above embodiment of the image encoding method can be cited as the functional description of the corresponding functional modules, and the details will not be described again here.

[0347] An embodiment of the present application further provides an electronic device 1400. See Fig. 14. Fig. 14 is an exemplary diagram of an electronic device according to an embodiment of the present application. The electronic device is configured to implement the tone mapping method of the embodiment shown in Fig. 5.

[0348] 14, electronic device 1400 may include a processor 1401 configured to execute programs or instructions stored in memory 1402. When the programs or instructions stored in memory 1402 are executed, the processor is configured to perform the tone mapping method of the embodiment shown in FIG.

[0349] Optionally, electronic device 1400 may further include a communication interface 1403. Figure 14 indicates with dashed lines that communication interface 1403 is optional for electronic device 1400.

[0350] The number of processors 1401, the number of memories 1402, and the number of communication interfaces 1403 do not constitute limitations on the embodiments of the present application, and may be randomly configured based on service requirements in specific implementations.

[0351] Optionally, the memory 1402 is located external to the electronic device 1400 .

[0352] Optionally, electronic device 1400 includes memory 1402. Memory 1402 is coupled to at least one processor 1401, and memory 1402 stores instructions that can be executed by at least one processor 1401. Figure 14 indicates with dashed lines that memory 142 is optional for electronic device 1400.

[0353] The processor 1401 and the memory 1402 may be coupled through an interface circuit or may be integrated, which is not limited herein.

[0354] The specific connection medium between the processor 1401, the memory 1402, and the communication interface 1403 is not limited in this embodiment of the present application. In this embodiment of the present application, the processor 1401, the memory 1402, and the communication interface 1403 are connected to each other through the bus 1404 in FIG. 14. The bus is represented by a bold line in FIG. 14. The connection manner between the other components is merely an example for explanation, and the specific connection is not limited thereto. The bus may be classified into an address bus, a data bus, a control bus, etc. For ease of representation, only one bold line is used to represent the bus in FIG. 14, but this does not mean that there is only one bus or only one type of bus.

[0355] An embodiment of the present application further provides an electronic device 1500. See Fig. 15. Fig. 15 is an exemplary diagram of an electronic device according to an embodiment of the present application, configured to implement the image encoding method of the embodiment shown in Fig. 11.

[0356] 15, electronic device 1500 may include a processor 1501 configured to execute programs or instructions stored in memory 1502. When the programs or instructions stored in memory 1502 are executed, processor 1501 is configured to perform the image encoding method of the embodiment shown in FIG.

[0357] Optionally, electronic device 1500 may further include a communication interface 1503. Figure 15 indicates with dashed lines that communication interface 1503 is optional for electronic device 1500.

[0358] The number of processors 1501, the number of memories 1502, and the number of communication interfaces 1603 do not constitute limitations on the embodiments of the present application, and may be randomly configured based on service requirements in specific implementations.

[0359] Optionally, memory 1502 is located external to electronic device 1500 .

[0360] Optionally, electronic device 1500 includes memory 1502. Memory 1502 is coupled to at least one processor 1501, and memory 1502 stores instructions that can be executed by at least one processor 1501. Figure 15 indicates with dashed lines that memory 1502 is optional for electronic device 1600.

[0361] The processor 1501 and the memory 1502 may be coupled through an interface circuit or may be integrated, which is not limited herein.

[0362] The specific connection medium between the processor 1501, the memory 1502, and the communication interface 1503 is not limited in this embodiment of the present application. In this embodiment of the present application, the processor 1501, the memory 1502, and the communication interface 1503 are connected to each other through the bus 1504 in FIG. 15. The bus is represented by a bold line in FIG. 15. The connection manner between the other components is merely an example for explanation, and the specific connection is not limited thereto. The bus may be classified into an address bus, a data bus, a control bus, etc. For ease of representation, only one bold line is used to represent the bus in FIG. 15, but this does not mean that there is only one bus or only one type of bus.

[0363] It should be understood that the processors referred to in the embodiments of the present application may be implemented by hardware or software. When the processor is implemented using hardware, the processor may be a logic circuit, an integrated circuit, etc. When the processor is implemented using software, the processor may be a general-purpose processor and is implemented by reading software code stored in a memory.

[0364] For example, a processor may be a central processing unit (CPU), or other general-purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0365] It should be understood that the memory referred to in the embodiments of this application may be volatile or nonvolatile, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM may be used, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus random access memory (direct rambus RAM, DR RAM).

[0366] It should be noted that the memory (storage module) may be integrated into the processor when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0367] Note that memory as described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0368] An embodiment of the present application further provides a computer-readable storage medium containing a program or instructions, which, when executed on a computer, performs the tone mapping method of the embodiment shown in Figure 4 or the image encoding method of the embodiment shown in Figure 12.

[0369] Those skilled in the art will understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Thus, the present application may use the form of a hardware-only embodiment, a software-only embodiment, or an embodiment having a combination of software and hardware. Furthermore, the present application may use the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROM, and optical memory) containing computer-usable program code.

[0370] The present application has been described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It will be understood that each process and / or each block in the flowcharts and / or block diagrams, and combinations of the processes and / or blocks in the flowcharts and / or block diagrams, can be implemented using computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or any other programmable data processing device to generate a machine, such that the instructions, when executed by the processor of the computer or any other programmable data processing device, generate an apparatus that performs the specified functions in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0371] These computer program instructions may be stored in a computer-readable memory that can instruct a computer or any other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable memory create an artifact that includes an instruction apparatus that implements a particular function in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0372] The computer program instructions may in turn be loaded into a computer or other programmable data processing device, such that a sequence of operations and steps are performed on the computer or other programmable device, thereby generating a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing a particular function in one or more procedures in the flowcharts and / or one or more blocks in the block diagrams.

[0373] It is apparent that those skilled in the art can make various modifications and changes to the present application without departing from the spirit and scope of the present application. In this case, if the modifications and changes made to the present application are within the scope of the claims of the present application and its equivalent technology, the present application intends to cover these modifications and changes. [Explanation of symbols]

[0374] 10 video codec system or codec system, 20 video encoder or encoder, 30 video decoder or decoder, 12 source device, 14 destination device, 16 image source, 18 image pre-processing unit or pre-processor, 22 communication interface or communication unit, 17 image or image data, original image or original image data, 19 pre-processed image or pre-processed image data, 21 encoded image data, 13 communication channel, 28 communication interface or communication unit, 32 post-processor or post-processing unit, 34 display device, 31 decoded image data or decoded image, 33 post-processed image data, 2100 content supply system, 2102 capture device, 2106 terminal device, 2126 display, 2104 communication link, 2108 smartphone or tablet computer, 2110 computer or laptop computer, 2112 network video recorder / digital video recorder, 2114 TV, 2116 set-top box, 2118 Video conferencing system, 2120 video surveillance system, 2122 personal digital assistant, 2124 in-vehicle device, 2202 protocol processing unit, 2204 demultiplexing unit, 2206 video decoder, 2212 synchronization unit, 2208 audio decoder, 3214 video / audio display, 2210 subtitle decoder, 2216 video / audio / subtitle display, 601 initial tone mapping curve, 602 adjusted tone mapping curve, 701 initial tone mapping curve, 702 adjusted tone mapping curve, 801 base layer tone mapping curve, 802 enhancement layer initial tone mapping curve, 90 initial tone mapping curve, 902 adjusted tone mapping curve, 1001 initial tone mapping curve, 1002 adjusted tone mapping curve, 1200 tone mapping device, 1201 receiving module, 1202 tone mapping module, 1300 image coding device, 1301 Encoding module, 1302, transmitting module, 1400, electronic device, 1402, memory, 1401, processor, 1403Communication interface, 1404 bus, 1500 electronic device, 1502 memory, 1501 processor, 1503 communication interface, 1504 bus

Claims

1. 1. A tone mapping method, said method comprising: obtaining an image to be displayed; obtaining reference information, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; Obtaining terminal device information, the terminal device information including at least one of a terminal display device size, a user viewing distance, a user viewing angle, an ambient light brightness, or a terminal display device resolution; performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information; A method comprising:

2. The step of performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information includes: obtaining a tone mapping parameter based on the reference information and the corresponding type of the terminal device information; performing tone mapping on the image to be displayed based on the tone mapping parameters; 2. The method of claim 1, comprising:

3. The step of obtaining the tone mapping parameters based on the reference information and the corresponding type of the terminal device information includes: adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters; 3. The method of claim 2, comprising:

4. adjusting the initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters, adjusting the initial tone mapping parameter of the image to be displayed in a first manner to obtain the tone mapping parameter when the relationship between the reference information and the terminal device information of the corresponding type satisfies a first condition; or adjusting the initial tone mapping parameter of the image to be displayed in a second manner to obtain the tone mapping parameter when a relationship between the reference information and the terminal device information of the corresponding type satisfies a second condition; Including, the initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve; The method of claim 3.

5. The first condition is: At least one of the following: the terminal display screen size is smaller than the reference display device size; the user viewing distance is larger than the reference viewing distance; the user viewing angle is deviated from the reference viewing angle; the ambient light luminance is larger than the reference ambient light luminance; or the terminal display device resolution is smaller than the reference display device resolution.

5. The method of claim 4, comprising:

6. The step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner includes: obtaining one or more first luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting mapped value of the adjusted tone mapping curve at the one or more first luminance feature values ​​is greater than a mapped value of the initial tone mapping curve at the one or more first luminance feature values; 6. The method of claim 4 or 5, comprising:

7. The step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner includes: obtaining one or more second luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting slope of the adjusted tone mapping curve at the one or more second luminance feature values ​​is greater than a slope of the initial tone mapping curve at the one or more second luminance feature values; 7. The method of claim 4, further comprising:

8. The step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner includes: obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a first feature value of any of the one or more enhancement layers based on the initial tone mapping curve of the any of the enhancement layers is a second feature value; adjusting the initial tone mapping parameters of the any enhancement layer such that the absolute value of a value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value; 8. The method of claim 4, further comprising:

9. The second condition is: At least one of the following: the terminal display screen size is larger than the reference display device size; the user viewing distance is smaller than the reference viewing distance; the ambient light luminance is smaller than the reference ambient light luminance; or the terminal display device resolution is larger than the reference display device resolution.

5. The method of claim 4, comprising:

10. The step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes: obtaining one or more third luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting mapped value of the adjusted tone mapping curve at the one or more third luminance feature values ​​is smaller than a mapped value of the initial tone mapping curve at the one or more third luminance feature values; 10. The method of claim 4 or 9, comprising:

11. The one or more third luminance feature values ​​are: one or more luminance feature values ​​of dark portions of the image to be displayed 11. The method of claim 4, 9, or 10, comprising:

12. The step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes: obtaining one or more fourth luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting slope of the adjusted tone mapping curve at the one or more fourth luminance feature values ​​is less than a slope of the initial tone mapping curve at the one or more fourth luminance feature values; 12. The method of any one of claims 4 or 9 to 11, further comprising:

13. The step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes: obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a third feature value of any of the enhancement layers among the one or more enhancement layers based on the initial tone mapping curve is a fourth feature value; adjusting the initial tone mapping parameters of the any enhancement layer such that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is smaller than the absolute value of the fourth feature value; 13. The method of any one of claims 4 or 9 to 12, further comprising:

14. The method comprises: obtaining reference tone mapping parameters for the image to be displayed; obtaining a terminal display device display capability in the terminal device information; adjusting the reference tone mapping parameters of the image to be displayed based on the reference tone mapping parameters and the display capabilities of the terminal display device to obtain the initial tone mapping parameters; 4. The method of claim 3, further comprising:

15. The step of obtaining the reference information includes: obtaining the reference information from metadata of the image to be displayed; obtaining an index of the reference information from the metadata, and selecting the reference information from a plurality of predefined reference information based on the index; or using a predetermined reference as the reference; 15. The method of any one of claims 1 to 14, comprising:

16. The step of obtaining terminal device information includes: obtaining the terminal device information by measurement; selecting the terminal device information from a plurality of predefined terminal information based on a predefined index; or using default terminal device information as the terminal device information 16. The method of any one of claims 1 to 15, comprising:

17. 1. A method for encoding an image, the method comprising: writing an image to be displayed into a bitstream; writing reference information for the image to be displayed into the bitstream, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; the reference information is for tone mapping of the image to be displayed; A method comprising:

18. The method comprises: writing reference tone mapping parameters of the image to be displayed into the bitstream, the reference tone mapping parameters being for obtaining initial tone mapping parameters, and the reference information being used by a decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, the tone mapping parameters being for tone mapping for the image to be displayed; 18. The method of claim 17, further comprising:

19. The step of writing the reference information of the image to be displayed into the bitstream includes: writing said reference information into said bitstream; or writing an index of said reference information into said bitstream.

19. The method of claim 17 or 18, comprising:

20. The method comprises: writing one or more luminance feature values ​​of the image to be displayed into the bitstream, the one or more luminance feature values ​​are used by the decoder to adjust the initial tone mapping parameters to obtain the tone mapping parameters.

20. The method of any one of claims 17 to 19, further comprising:

21. 1. A tone mapping apparatus, comprising: Obtain an image to be displayed; obtaining reference information, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; and a receiving module configured to obtain terminal device information, the terminal device information including at least one of a terminal display device size, a user viewing distance, a user viewing angle, an ambient light brightness, or a terminal display device resolution; a tone mapping module configured to perform tone mapping on the image to be displayed based on the reference information and the corresponding type of the terminal device information; An apparatus comprising:

22. The step of performing tone mapping on the image to be displayed based on the reference information and the corresponding type of terminal device information includes: obtaining a tone mapping parameter based on the reference information and the corresponding type of the terminal device information; performing tone mapping on the image to be displayed based on the tone mapping parameters; 22. The apparatus of claim 21, comprising:

23. The step of obtaining the tone mapping parameters based on the reference information and the corresponding type of the terminal device information includes: adjusting initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters; 23. The apparatus of claim 22, comprising:

24. adjusting the initial tone mapping parameters of the image to be displayed based on the reference information and the corresponding type of terminal device information to obtain the tone mapping parameters, When the relationship between the reference information and the terminal device information of the corresponding type satisfies a first condition, adjusting the initial tone mapping parameter of the image to be displayed in a first manner to obtain the tone mapping parameter; When the relationship between the reference information and the terminal device information of the corresponding type satisfies a second condition, adjusting the initial tone mapping parameter of the image to be displayed in a second manner to obtain the tone mapping parameter; Including, the initial tone mapping parameters represent an initial tone mapping curve, and the tone mapping parameters represent an adjusted tone mapping curve; 24. The apparatus of claim 23.

25. The first condition is: At least one of the following: the terminal display screen size is smaller than the reference display device size; the user viewing distance is larger than the reference viewing distance; the user viewing angle is deviated from the reference viewing angle; the ambient light luminance is larger than the reference ambient light luminance; or the terminal display device resolution is smaller than the reference display device resolution.

25. The apparatus of claim 24, comprising:

26. The step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner includes: obtaining one or more first luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting mapped value of the adjusted tone mapping curve at the one or more first luminance feature values ​​is greater than a mapped value of the initial tone mapping curve at the one or more first luminance feature values; 26. The apparatus of claim 24 or 25, comprising:

27. The step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner includes: obtaining one or more second luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting slope of the adjusted tone mapping curve at the one or more second luminance feature values ​​is greater than a slope of the initial tone mapping curve at the one or more second luminance feature values; 27. The apparatus of any one of claims 24 to 26, further comprising:

28. The step of adjusting the initial tone mapping parameters of the image to be displayed in the first manner includes: obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a first feature value of any of the one or more enhancement layers based on the initial tone mapping curve of the any of the enhancement layers is a second feature value; adjusting the initial tone mapping parameters of the any enhancement layer such that the absolute value of a value obtained by mapping the first feature value based on the adjusted tone mapping curve is greater than the absolute value of the second feature value; 28. The apparatus of any one of claims 24 to 27, further comprising:

29. The second condition is: At least one of the following: the terminal display screen size is larger than the reference display device size; the user viewing distance is smaller than the reference viewing distance; the ambient light luminance is smaller than the reference ambient light luminance; or the terminal display device resolution is larger than the reference display device resolution.

25. The apparatus of claim 24, comprising:

30. The step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes: obtaining one or more third luminance feature values ​​of the image to be displayed; adjusting the initial tone mapping parameters such that a resulting mapped value of the adjusted tone mapping curve at the one or more third luminance feature values ​​is smaller than a mapped value of the initial tone mapping curve at the one or more third luminance feature values; 30. The apparatus of claim 24 or 29, comprising:

31. The one or more third luminance feature values ​​are: one or more luminance feature values ​​of dark portions of the image to be displayed 31. The apparatus of claim 30, comprising:

32. The step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes: obtaining one or more fourth luminance feature values ​​of the image to be displayed; 32. The apparatus of claim 24, further comprising: adjusting the initial tone mapping parameters such that a resulting slope of the adjusted tone mapping curve at the one or more fourth luminance feature values ​​is less than a slope of the initial tone mapping curve at the one or more fourth luminance feature values.

33. The step of adjusting the initial tone mapping parameters of the image to be displayed in the second manner includes: obtaining a base layer and one or more enhancement layers of the image to be displayed, wherein a value obtained by mapping a third feature value of any of the enhancement layers among the one or more enhancement layers based on the initial tone mapping curve is a fourth feature value; adjusting the initial tone mapping parameters of the any enhancement layer such that the absolute value of the value obtained by mapping the third feature value based on the adjusted tone mapping curve is smaller than the absolute value of the fourth feature value; 33. The apparatus of any one of claims 24 or 29 to 32, further comprising:

34. The tone mapping module: obtaining reference tone mapping parameters for the image to be displayed; Obtaining terminal display device display capabilities in the terminal device information; and adjusting the reference tone mapping parameters of the image to be displayed based on the reference tone mapping parameters and the display capabilities of the terminal display device to obtain the initial tone mapping parameters; 24. The apparatus of claim 23, further configured to:

35. The step of obtaining the reference information includes: obtaining the reference information from metadata of the image to be displayed; obtaining an index of the reference information from the metadata, and selecting the reference information from a plurality of predefined reference information based on the index; or using a predetermined reference as the reference; 35. The apparatus of any one of claims 21 to 34, comprising:

36. The step of obtaining terminal device information includes: obtaining the terminal device information by measurement; selecting the terminal device information from a plurality of predefined terminal information based on a predefined index; or using default terminal device information as the terminal device information 36. The apparatus of any one of claims 21 to 35, comprising:

37. 1. An image encoding apparatus, comprising: Write the image to be displayed to the bitstream, and an encoding module configured to write reference information for the image to be displayed into the bitstream, the reference information including at least one of a reference display device size, a reference viewing distance, a reference viewing angle, a reference ambient light luminance, or a reference display device resolution; an encoding module, wherein the reference information is for tone mapping of the image to be displayed; a transmission module configured to transmit the bitstream; An apparatus comprising:

38. The encoding module: writing reference tone mapping parameters of the image to be displayed into the bitstream, the reference tone mapping parameters being for deriving the tone mapping parameters, the reference information being used by a decoder to adjust the initial tone mapping parameters to obtain tone mapping parameters, the tone mapping parameters being for tone mapping for the image to be displayed; 38. The apparatus of claim 37, further configured to:

39. The step of writing the reference information of the image to be displayed into the bitstream includes: writing said reference information into said bitstream; or writing an index of said reference information into said bitstream.

39. The apparatus of claim 37 or 38, comprising:

40. The encoding module: writing one or more luma feature values ​​of the image to be displayed into the bitstream, the one or more luma feature values ​​being used by a decoder to adjust the initial tone mapping parameters to obtain the tone mapping parameters; 40. The apparatus of any one of claims 37 to 39, further configured to:

41. An electronic device comprising a memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to perform the method of any one of claims 1 to 16 or claims 17 to 20.

42. 21. A computer-readable storage medium storing a computer program which, when run on a computer or processor, performs the method of any one of claims 1 to 16 or claims 17 to 20.

43. A computer program product comprising a program or instructions, which, when run on a computer or processor, enable the computer or processor to carry out a method according to any one of claims 1 to 16 or 17 to 20.

Images