An image signal conversion and processing method, apparatus, and terminal device

Abstract

This application provides an image signal conversion processing method, apparatus, and terminal device. The method includes: acquiring a primary color component of a pixel to be processed; converting a first value of the primary color component into a second value according to a preset dynamic range conversion model; wherein the dynamic range conversion model is: where L is the first value, L′ is the second value, and k1, k2, k3, a, b, m, n, and p are model parameters.

Description

[0001] This application is a divisional application. The original application has the application number 201910999368.6 and the original application date is October 18, 2019. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of image processing technology, and in particular to an image signal conversion and processing method, apparatus, and terminal device. Background Technology

[0003] In digital imaging, dynamic range refers to the ratio between the maximum and minimum grayscale values ​​within the displayable range of an image. For real-world natural scenes, a brightness of 10... -3 nits to 10 4 Within the nits range, the dynamic range of the real world can reach 10. 7 This is called high dynamic range (HDR). Currently, in most color digital images, each of the R, G, and B channels uses one byte (8 bits) for storage. This means that each channel represents a grayscale range of 0–255, which is the dynamic range of the color digital image, also known as low dynamic range (LDR). Therefore, the imaging process of a digital camera is essentially a mapping from the high dynamic range of the real world to the low dynamic range of the image.

[0004] A typical HDR video processing workflow includes preprocessing, encoding, decoding, and post-processing. The HDR video input undergoes photoelectric conversion, color space conversion, floating-point to fixed-point conversion, 4:4:4 to 4:2:0 downsampling, and 4:2:0 encoder encoding to obtain the bitstream. This bitstream is then decoded by a 4:2:0 decoder, upsampled from 4:2:0 to 4:4:4, fixed-point to floating-point conversion, color space conversion, and photoelectric conversion to obtain the final output HDR video. The conversion function performs a non-linear transformation on the HDR source. Since HDR video ultimately needs to be quantized into integer bits of data before encoding, and considering that the dynamic range of HDR is much larger than integer bits, direct linear quantization would result in significant information loss from the HDR source. Therefore, the main purpose of the conversion function is to protect the luminance range that requires special attention through non-linear transformation.

[0005] The primary color signal, representing the light signal of the real world, can be represented by "L" or "E". It typically records the values ​​corresponding to specific color components (such as R, G, B, or Y), and is usually proportional to light intensity. The primary color signal of an image can be represented by its true brightness (e.g., 10000 nits) or by normalized brightness, such as normalizing a maximum brightness of 10000 nits to a maximum brightness of 1. The processed image information obtained after converting the primary color signal is the digital representation of the image, normalized to [0,1]. It can be represented by "L′" or "E′", typically used to represent non-linear color values. The processed image information is obtained by photoelectric conversion of the primary color signal (normalized to [0,1]), which can include primary colors such as R, G, B, and Y. The output primary color signal is obtained by electro-optic conversion of the input image information, which is a restored real-world light signal. The optical-electro transfer function (OETF) is sometimes also called the photoelectric transfer function or photoelectric transfer curve; the electro-optical transfer function (EOTF) is also sometimes called the electro-optical transfer function or electro-optical transfer curve. Floating-point to fixed-point conversion is sometimes called quantization, and fixed-point to floating-point conversion is sometimes called dequantization. In one existing technology, the photoelectric transfer function is proposed based on a human eye brightness perception model. This photoelectric transfer function can be:

[0006] R′=PQ_TF(max(0,min(R / 10000,1)))

[0007] G′=PQ_TF(max(0,min(G / 10000,1)))

[0008] B′=PQ_TF(max(0,min(B / 10000,1)))

[0009] Where m1 = 0.1593017578125, m2 = 78.84375, c1 = 0.8359375, c2 = 18.8515625, c3 = 18.6875. Summary of the Invention

[0010] This application provides an image signal conversion and processing method, apparatus, and terminal device, which can improve the quality of mutual conversion between images with different dynamic ranges to better adapt to display devices and obtain better quality display images.

[0011] In a first aspect of this application, an image signal conversion processing method is provided, the method comprising: acquiring a primary color component of a pixel to be processed; converting a first value of the primary color component into a second value according to a preset dynamic range conversion model; wherein the dynamic range conversion model is:

[0012]

[0013] Where L is the first value, L′ is the second value, and k1, k2, k3, a, b, m, n and p are model parameters.

[0014] This application provides a novel dynamic range conversion model that offers greater freedom for matching richer image content. By configuring the model parameters appropriately, the quality of image dynamic range conversion can be improved.

[0015] In one feasible implementation, before converting the first value of the primary color component into the second value according to the preset dynamic range conversion model, the method of obtaining the model parameters is determined according to the identification information carried by the metadata, wherein the metadata includes static metadata and / or dynamic metadata.

[0016] This application provides multiple methods for obtaining model parameters and enables the display end to know them through the identification information in the metadata, providing multiple choices between different implementation complexities and effects.

[0017] In one feasible implementation, the metadata and the pixel to be processed are obtained by decoding the bitstream associated with the source image signal.

[0018] In one feasible implementation, the model parameters are obtained by: decoding the numerical values ​​of the model parameters carried in the metadata; or, using a preset value as the numerical value of the model parameters; or, calculating the numerical values ​​of the model parameters based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

[0019] For example, this application provides three possible methods for obtaining model parameters. It should be understood that the methods for obtaining the values ​​of multiple model parameters may be different; that is, the methods for obtaining different model parameters in a set of model parameters represented by the same identifier may be the same or different.

[0020] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is obtained by decoding the numerical values ​​of the model parameters carried by the metadata.

[0021] It should be understood that carrying all the model parameter values ​​in the metadata is a simple implementation method, and it is also more reliable when the video content generation end already knows the display device's parameters.

[0022] In one feasible implementation, at least one of k1, k2, k3, m, and n is obtained according to a preset value, wherein 0 <n≤8,0<m≤8。

[0023] In one feasible implementation, k1 is 1; in one feasible implementation, k2 is 1; in one feasible implementation; in one feasible implementation, k3 is 1; in one feasible implementation, m is 2.4 or 1; in one feasible implementation, n is 1 or 5 / 12.

[0024] It should be understood that obtaining model parameters through preset values ​​is the simplest method and does not require the transmission of additional information. For example, embodiments of this application provide the values ​​of k1, k2, k3, m, and n.

[0025] In one feasible implementation, b is the minimum brightness value of the display device after normalization.

[0026] In one feasible implementation, k3 is the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization.

[0027] In one feasible implementation, the maximum brightness value of a pixel in the frame to be processed of the source image signal includes: the maximum value among the brightness components of the pixel in the frame to be processed; or, the maximum value among the maximum primary color components of the pixel in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among multiple primary color components of a pixel.

[0028] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is calculated based on information from the source image signal and / or information from the display device.

[0029] In this embodiment, the model parameters are obtained through the source image signal or the properties of the display device, and the resulting dynamic range conversion processing is more in line with the characteristics of actual applications, resulting in better performance. Several specific implementation methods are provided below as examples.

[0030] In one feasible implementation, the information of the source image signal includes the average brightness value of the pixels in the frame to be processed of the source image signal, p is obtained according to the following formula:

[0031]

[0032] Where avgL is the average brightness value of pixels in the frame to be processed of the source image signal after normalization, TPH, TPL, p valueH and p valueL All are preset values, 0≤TPL, TPH≤1, and g() is a monotonically increasing function in the interval from 0 to 1.

[0033] In one feasible implementation, TPL is 0.5, TPH is 0.8, and p valueL It is 3.5, p valueH The value is 5.0, and g(x) = x, where x is a rational number in the interval from 0 to 1.

[0034] In one feasible implementation, the average brightness value of the pixels in the frame to be processed of the source image signal includes: the average value of the brightness components of the pixels in the frame to be processed; or, the average value of the maximum primary color components of the pixels in the frame to be processed.

[0035] In one feasible implementation, the information of the source image signal includes the maximum brightness value of the pixels in the frame to be processed of the source image signal and the minimum brightness value of the pixels in the frame to be processed of the source image signal, and 'a' is obtained according to the following formula:

[0036] a=(MaxDisplay-MinDisplay) / (H(MaxSource)-H(MinSource))

[0037] Wherein, MaxSource is the maximum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MinSource is the minimum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MaxDisplay is the maximum brightness value of the display device after normalization, and MinDisplay is the minimum brightness value of the display device after normalization.

[0038] In one feasible implementation, the minimum brightness value of a pixel in the frame to be processed of the source image signal includes: the minimum value of the brightness component of the pixel in the frame to be processed; or, the minimum value of the maximum primary color component of the pixel in the frame to be processed.

[0039] In one feasible implementation, a, p, and m are obtained by solving the following system of equations:

[0040]

[0041] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of the pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0042] In some embodiments, L3 is MaxSource.

[0043] It should be understood that when any one of the parameters a, p, and m has been obtained in advance through other means, the values ​​of the other two unknown parameters can be obtained by using any two equations in the system of equations.

[0044] In one feasible implementation, a and p are obtained by solving the following system of equations:

[0045]

[0046] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of the pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0047] When L2 is less than the normalized value of 5 nits, M1 equals L1 and N1 equals F1; when L2 is greater than or equal to the normalized value of 5 nits, M1 equals L2 and N1 equals F2. It should be understood that, for example, 5 nits can be used as the threshold, or other values ​​can be used, without limitation.

[0048] In one feasible implementation, the mapping value is obtained according to the following formula:

[0049]

[0050] in,

[0051]

[0052] y(i) is the mapping value, f(i) is the maximum brightness value of the i-th pixel in the frame to be processed of the source image signal, MaxDisplay and MinDisplay are the maximum and minimum brightness values ​​of the display device, and N frameis the number of pixels in the frame to be processed, and LR is the perceived brightness compensation factor, 0 < LR ≤ 2.

[0053] In one feasible implementation, LR is 0.8, 1, or 1.2.

[0054] In one feasible implementation, the perceived brightness value satisfies: J(Lp)-J(1nit)=(J(L0)-J(1nit))*Rate, where Lp is the perceived brightness value, L0 is the preset brightness value, Rate is the preset ratio, and J(x) is the proportion of pixels with brightness less than x nits in the frame to be processed of the source image signal.

[0055] In one feasible implementation, the preset brightness value is 5 nits, and the preset ratio is 10%, 20%, or 30%.

[0056] In one feasible implementation, the perceived brightness value is obtained by decoding the metadata.

[0057] In one feasible implementation, 'a' is the difference between the normalized maximum brightness value and the normalized minimum brightness value of the display device.

[0058] In one feasible implementation, p is obtained according to the following formula: p = c * v + d, where c and d are preset values.

[0059] In one feasible implementation, c is 7 and d is 3.

[0060] In one feasible implementation, v is obtained according to the following formula: v = J(Tq) - J(Tp).

[0061] Wherein, J(x) is the proportion of pixels with a brightness less than x nit in the frame to be processed of the source image signal, Tp is the value after normalization of 1 nit, and Tq is the ratio of the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization to the maximum value of the display device after normalization, multiplied by Tp.

[0062] In one feasible implementation, the normalization process includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in the ST 2084, 2014-SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in the Rec.2100-ITU standard.

[0063] In one feasible implementation, the method further includes: decoding the maximum brightness value of a pixel in the frame to be processed of the source image signal from the metadata; correcting the maximum brightness value of the pixel in the frame to be processed of the source image signal; correspondingly, at least one of k1, k2, k3, a, b, m, n, and p is calculated based on the information of the source image signal and / or the information of the display device, including: at least one of k1, k2, k3, a, b, m, n, and p is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

[0064] In a second aspect of this application, an image signal conversion and processing apparatus is provided, the apparatus comprising: an acquisition module for acquiring a primary color component of a pixel to be processed; and a conversion module for converting a first value of the primary color component into a second value according to a preset dynamic range conversion model; wherein the dynamic range conversion model is:

[0065]

[0066] Where L is the first value, L′ is the second value, and k1, k2, k3, a, b, m, n and p are model parameters.

[0067] In one feasible implementation, before converting the first value of the primary color component into the second value according to the preset dynamic range conversion model, the acquisition module is further configured to: determine the acquisition method of the model parameters according to the identification information carried by the metadata, wherein the metadata includes static metadata and / or dynamic metadata.

[0068] In one feasible implementation, the metadata and the pixel to be processed are obtained by decoding the bitstream associated with the source image signal.

[0069] In one feasible implementation, the model parameters are obtained by: decoding the numerical values ​​of the model parameters carried in the metadata; or, using a preset value as the numerical value of the model parameters; or, calculating the numerical values ​​of the model parameters based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

[0070] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is obtained by decoding the numerical values ​​of the model parameters carried by the metadata.

[0071] In one feasible implementation, at least one of k1, k2, k3, m and n is obtained according to a preset value, wherein 0 < n ≤ 8 and 0 < m ≤ 8.

[0072] In one feasible implementation, k1 is 1; in one feasible implementation, k2 is 1; in one feasible implementation; in one feasible implementation, k3 is 1; in one feasible implementation, m is 2.4 or 1; in one feasible implementation, n is 1 or 5 / 12.

[0073] In one feasible implementation, b is the minimum brightness value of the display device after normalization.

[0074] In one feasible implementation, k3 is the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization.

[0075] In one feasible implementation, the maximum brightness value of a pixel in the frame to be processed of the source image signal includes: the maximum value among the brightness components of the pixel in the frame to be processed; or, the maximum value among the maximum primary color components of the pixel in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among multiple primary color components of a pixel.

[0076] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is calculated based on information from the source image signal and / or information from the display device.

[0077] In one feasible implementation, the information of the source image signal includes the average brightness value of the pixels in the frame to be processed of the source image signal, p is obtained according to the following formula:

[0078]

[0079] Where avgL is the average brightness value of pixels in the frame to be processed of the source image signal after normalization, TPH, TPL, p valueH and p valueL All are preset values, 0≤TPL, TPH≤1, and g() is a monotonically increasing function in the interval from 0 to 1.

[0080] In one feasible implementation, TPL is 0.5, TPH is 0.8, and p valueL It is 3.5, p valueH The value is 5.0, and g(x) = x, where x is a rational number in the interval from 0 to 1.

[0081] In one feasible implementation, the average brightness value of the pixels in the frame to be processed of the source image signal includes: the average value of the brightness components of the pixels in the frame to be processed; or, the average value of the maximum primary color components of the pixels in the frame to be processed.

[0082] In one feasible implementation, the information of the source image signal includes the maximum brightness value of the pixels in the frame to be processed of the source image signal and the minimum brightness value of the pixels in the frame to be processed of the source image signal, and 'a' is obtained according to the following formula:

[0083] a=(MaxDisplay-MinDisplay) / (H(MaxSource)-H(MinSource))

[0084] Wherein, MaxSource is the maximum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MinSource is the minimum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MaxDisplay is the maximum brightness value of the display device after normalization, and MinDisplay is the minimum brightness value of the display device after normalization.

[0085] In one feasible implementation, the minimum brightness value of a pixel in the frame to be processed of the source image signal includes: the minimum value of the brightness component of the pixel in the frame to be processed; or, the minimum value of the maximum primary color component of the pixel in the frame to be processed.

[0086] In one feasible implementation, a, p, and m are obtained by solving the following system of equations:

[0087]

[0088] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of the pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0089] In some embodiments, L3 is MaxSource.

[0090] In one feasible implementation, a and p are obtained by solving the following system of equations:

[0091]

[0092] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of the pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0093] When L2 is less than the normalized value of 5 nits, M1 equals L1 and N1 equals F1. When L2 is greater than or equal to the normalized value of 5 nits, M1 equals L2 and N1 equals F2.

[0094] In one feasible implementation, the mapping value is obtained according to the following formula:

[0095]

[0096] in,

[0097]

[0098] y(i) is the mapping value, f(i) is the maximum brightness value of the i-th pixel in the frame to be processed of the source image signal, MaxDisplay and MinDisplay are the maximum and minimum brightness values ​​of the display device, and N frame is the number of pixels in the frame to be processed, and LR is the perceived brightness compensation factor, 0 < LR ≤ 2.

[0099] In one feasible implementation, LR is 0.8, 1, or 1.2.

[0100] In one feasible implementation, the perceived brightness value satisfies: J(Lp)-J(1nit)=(J(L0)-J(1nit))*Rate, where Lp is the perceived brightness value, L0 is the preset brightness value, Rate is the preset ratio, and J(x) is the proportion of pixels with brightness less than x nits in the frame to be processed of the source image signal.

[0101] In one feasible implementation, the preset brightness value is 5 nits, and the preset ratio is 10%, 20%, or 30%.

[0102] In one feasible implementation, the perceived brightness value is obtained by decoding the metadata.

[0103] In one feasible implementation, 'a' is the difference between the normalized maximum brightness value and the normalized minimum brightness value of the display device.

[0104] In one feasible implementation, p is obtained according to the following formula: p = c * v + d, where c and d are preset values.

[0105] In one feasible implementation, c is 7 and d is 3.

[0106] In one feasible implementation, v is obtained according to the following formula: v = J(Tq) - J(Tp).

[0107] Wherein, J(x) is the proportion of pixels with a brightness less than x nit in the frame to be processed of the source image signal, Tp is the value after normalization of 1 nit, and Tq is the ratio of the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization to the maximum value of the display device after normalization, multiplied by Tp.

[0108] In one feasible implementation, the normalization process includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in the ST 2084, 2014-SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in the Rec.2100-ITU standard.

[0109] In one feasible implementation, the acquisition module is further configured to: decode the maximum brightness value of a pixel in the frame to be processed of the source image signal from the metadata; correct the maximum brightness value of the pixel in the frame to be processed of the source image signal; correspondingly, at least one of k1, k2, k3, a, b, m, n and p is calculated based on the information of the source image signal and / or the information of the display device, including: at least one of k1, k2, k3, a, b, m, n and p is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

[0110] In a third aspect of this application, an image processing apparatus is provided, comprising: a non-volatile memory and a processor coupled to each other, the processor calling program code stored in the memory to perform the method as described in the first aspect.

[0111] In a fourth aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, which, when executed on a processor, implements the method described in the first aspect.

[0112] It should be understood that the second to fourth aspects of this application are consistent with the technical solutions of the first aspect of this application, and the beneficial effects obtained by each feasible implementation are similar, so they will not be described again. Attached Figure Description

[0113] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0114] Figure 1 This is a schematic diagram of an image signal conversion and processing system provided in an embodiment of this application;

[0115] Figure 2 This is a flowchart illustrating an image signal conversion processing method provided in an embodiment of this application;

[0116] Figure 3 This is a flowchart illustrating another image signal conversion processing method provided in the embodiments of this application;

[0117] Figure 4 This is a schematic diagram of a system architecture for applying image signal conversion provided in an embodiment of this application;

[0118] Figure 5 This is a flowchart illustrating another image signal conversion processing method provided in the embodiments of this application;

[0119] Figure 6 This is a schematic diagram illustrating the relationship between the brightness of an image scene and parameter m provided in the embodiments of this application;

[0120] Figure 7 This is a schematic diagram illustrating the relationship between the brightness of an image scene and parameter p provided in the embodiments of this application;

[0121] Figure 8 This is a schematic diagram of the structure of an image signal conversion and processing device provided in an embodiment of this application;

[0122] Figure 9 This is a schematic diagram of the structure of a terminal device provided in the embodiments of this application. Detailed Implementation

[0123] The technical solutions of the present invention will be clearly described below with reference to the accompanying drawings.

[0124] Before introducing the specific implementation methods, the terminology that may be used in the embodiments of this application will be explained first:

[0125] The maximum display brightness value (max_display_mastering_luminance): a 16-bit unsigned integer. It represents the maximum display brightness of the device. (In units of 1 cd / m²) 2The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0126] The minimum display brightness value (min_display_mastering_luminance) of the display device: a 16-bit unsigned integer. It represents the minimum display brightness of the display device. It is expressed in terms of 0.0001 cd / m². 2 The units range from 0.0001 cd / m 2 Up to 6.5535 cd / m 2 The value of max_display_mastering_luminance should be greater than the value of min_display_mastering_luminance.

[0127] The minimum value of the maximum primary color component of the pixel in the source image signal (minimum_maxrgb), also known as the minimum value among the maximum values ​​of the RGB components: take the maximum value of the three RGB components of each pixel in a scene or a frame of an image, and find the minimum value among these maximum values.

[0128] The average value of the maximum primary color component of the pixel in the source image signal (average_maxrgb), also known as the average value of the maximum values ​​of the RGB components: take the maximum value of the three RGB components of each pixel in a scene or a frame of an image, and calculate the average of these maximum values.

[0129] The maximum value of the primary color component of the pixel in the source image signal (maximum_maxrgb), also known as the maximum value among the maximum values ​​of the RGB components: take the maximum value among the three RGB components of each pixel in a scene or a frame of an image, and take the largest value among these maximum values.

[0130] Metadata: Records key information about images in a video, scene, or frame.

[0131] Static metadata: Metadata that remains unchanged within a sequence.

[0132] Dynamic metadata: Metadata that changes as the image or scene changes.

[0133] PQ HDR video: HDR video sampled and transmitted in PQ format.

[0134] HLG HDR video: HDR video sampled and transmitted in HLG format.

[0135] HDR non-linear editing video: Video processed by non-linear editing software.

[0136] It should be understood that in the embodiments of the present application, there are: when A > B, perform a step C or present a state C; when A ≤ B, perform another step D or present another state D. It should be understood that when A = B, in some feasible embodiments, it may also be satisfied that a step C is performed or a state C is presented, that is, when A ≥ B, a step C is performed or a state C is presented; when A < B, another step D is performed or another state D is presented. The embodiments of the present application all include the above two feasible embodiments, which will not be elaborated hereinafter.

[0137] Figure 1 is a schematic diagram of an image signal conversion processing system provided in the embodiments of the present application. As Figure 1 shown, a general HDR end-to-end processing system is generally divided into four parts: HDR production, HDR transmission, HDR decoding, and HDR display.

[0138] HDR production includes editing HDR materials, color grading processing, outputting HDR non-linear edited videos, and extracting dynamic metadata of HDR non-linear edited videos through HDR preprocessing.

[0139] HDR transmission includes video encoding of the HDR non-linear edited video and HDR dynamic metadata after HDR production in accordance with video compression standards (for example, AVS or HEVC standards), and outputting corresponding bitstreams (AVS or HEVC bitstreams).

[0140] HDR decoding includes decoding the generated bitstream according to the standard corresponding to the bitstream format, and outputting HDR decoded videos and HDR dynamic metadata.

[0141] HDR display or SDR display: HDR display is to process the HDR decoded video (HDR adaptation processing) by combining HDR dynamic metadata and target display terminal parameters, and display the processed HDR video (HDR adapted video) on the HDR display terminal, where the target display terminal parameters are the brightness range of the target display terminal. SDR display is to process the HDR video (SDR adaptation processing) by combining HDR dynamic metadata and target display terminal parameters, and display the processed HDR video (SDR adapted video) on the SDR display terminal, where the target display terminal parameters are the brightness range of the target display terminal.

[0142] Figure 2 is a schematic flowchart of an image signal conversion processing method provided in the embodiments of the present application. As shown in the figure, the image signal conversion processing method in the embodiments of the present invention may at least include:

[0143] S201, the first terminal device performs photoelectric conversion on the input primary color signal through a preset photoelectric conversion function to obtain the photoelectric converted image information.

[0144] The terminal device can perform photoelectric conversion on the input primary color signal using a preset photoelectric conversion function to obtain the converted image information. The terminal device can be a satellite, a personal computer (PC), or a smartphone, etc.

[0145] S202, the first terminal device converts the photoelectric converted image information from RGB space to YCbCr space through a preset first color space conversion function to obtain the space-converted image information.

[0146] S203, the first terminal device performs floating-point to fixed-point conversion on the spatially converted image information in YCbCr space to obtain the image information after floating-point to fixed-point conversion.

[0147] S204, the first terminal device downsamples the image information after converting from floating point to fixed point to obtain downsampled image information.

[0148] S205, the first terminal device encodes the downsampled image information to obtain the encoded image information.

[0149] S206, the first terminal device sends the encoded image information to the second terminal device.

[0150] S207, the second terminal device decodes the encoded image information to obtain the decoded image information.

[0151] S208, the second terminal device upsamples the decoded image information to obtain the upsampled image information.

[0152] S209, the second terminal device performs fixed-point to floating-point conversion on the upsampled image information to obtain the image information after fixed-point to floating-point conversion.

[0153] S210, the second terminal device converts the fixed-point to floating-point image information from YCbCr space to RGB space through a preset second color space conversion function to obtain the space-converted image information.

[0154] S211, the second terminal device performs electro-optic conversion on the spatially converted image information using a preset electro-optic conversion function to obtain the output primary color signal.

[0155] S212, the second terminal device outputs the primary color signal.

[0156] exist Figure 2In the image signal conversion and processing method shown, the first terminal device performs photoelectric conversion on the input primary color signal using a preset photoelectric conversion function to obtain photoelectric converted image information. Then, it converts the photoelectric converted image information from RGB space to YCbCr space using a preset first color space conversion function to obtain space-converted image information. Next, it performs floating-point to fixed-point conversion on the space-converted image information in YCbCr space to obtain floating-point to fixed-point image information. Finally, it downsamples the floating-point to fixed-point image information to obtain downsampled image information. The downsampled image information is then encoded and sent to the second terminal device. The terminal device decodes the encoded image information to obtain decoded image information, upsamples the decoded image information to obtain upsampled image information, converts the upsampled image information to fixed-point to floating-point, and then converts the fixed-point to floating-point image information from YCbCr space to RGB space using a preset second color space conversion function to obtain space-converted image information. Finally, it performs electro-optic conversion on the space-converted image information using a preset electro-optic conversion function to obtain the output primary color signal, which improves quantization quality and increases resource utilization.

[0157] Figure 3 This is a flowchart illustrating an image signal conversion processing method provided in another embodiment of this application. Figure 3 The image signal conversion processing method in the illustrated embodiments may include at least:

[0158] S301, the terminal device performs photoelectric conversion on the input primary color signal through a preset photoelectric conversion function to obtain the photoelectric converted image information.

[0159] The terminal device can perform photoelectric conversion on the input primary color signal using a preset photoelectric conversion function to obtain the converted image information. The terminal device can be a smartphone, camera, or tablet computer, etc. The image can be captured by a camera or pre-stored locally.

[0160] S302, the terminal device converts the photoelectric converted image information from RGB space to YCbCr space through a preset first color space conversion function to obtain the space-converted image information.

[0161] S303, the terminal device performs floating-point to fixed-point conversion on the image information in the YCbCr space to obtain the image information after floating-point to fixed-point conversion.

[0162] S304, the terminal device downsamples the image information after converting from floating point to fixed point to obtain downsampled image information.

[0163] S305, the terminal device upsamples the downsampled image information to obtain the upsampled image information.

[0164] S306, the terminal device performs fixed-point to floating-point conversion on the upsampled image information to obtain the image information after fixed-point to floating-point conversion.

[0165] S307, the terminal device converts the fixed-point to floating-point image information from YCbCr space to RGB space through a preset second color space conversion function to obtain the space-converted image information.

[0166] S308, the terminal device performs electro-optic conversion on the spatially converted image information through a preset electro-optic conversion function to obtain the output primary color signal.

[0167] S309, the terminal device outputs the primary color signal.

[0168] exist Figure 3 In the image signal conversion and processing method shown, the terminal device performs photoelectric conversion on the input primary color signal using a preset photoelectric conversion function to obtain the photoelectric converted image information. Then, it converts the photoelectric converted image information from RGB space to YCbCr space using a preset first color space conversion function to obtain the space-converted image information. Next, it performs floating-point to fixed-point conversion on the space-converted image information in YCbCr space to obtain the floating-point to fixed-point image information. Finally, it downsamples the floating-point to fixed-point image information to obtain the downsampled image information. Then, it upsamples the downsampled image information to obtain the upsampled image information. Finally, it performs fixed-point to floating-point conversion on the upsampled image information to obtain the fixed-point to floating-point image information. Then, it converts the fixed-point to floating-point image information from YCbCr space to RGB space using a preset second color space conversion function to obtain the space-converted image information. Finally, it performs electro-optic conversion on the space-converted image information using a preset electro-optic conversion function to obtain the output primary color signal. This process improves quantization quality and increases resource utilization.

[0169] It should be understood that, for example, Figure 2 and Figure 3 The image signal conversion processing method described herein introduces the entire process of image signal conversion. This application mainly includes a dynamic range conversion module for image signals and corresponding processing methods, such as the photoelectric conversion process and the determination of the photoelectric conversion function in the above embodiments, which can be combined with... Figure 2 and Figure 3 This application does not limit the scope of other image signal conversion and processing, including those processes.

[0170] Dynamic range conversion (DRC) is primarily used for adapting front-end HDR signals to back-end HDR display devices. For example, the brightness range of the light signal captured by the front end might be 4000 nits, while the display capability of the back-end HDR display device (TV, iPad) is only 500 nits. Mapping the 4000-nit signal to a 500-nit display device is a process of high-to-low dynamic range mapping (tone-mapping). In another example, the brightness range of the SDR signal (Standard Dynamic Range signal, which, relative to HDR, belongs to LDR as mentioned above) captured by the front end might be 100 nits, while the display needs to show a 2000-nit TV signal. Displaying the 100-nit signal on a 2000-nit display device is a low-to-high tone-mapping process.

[0171] Figure 4 The system architecture to which the embodiments of this application are applied is illustrated by way of example. In one feasible implementation, the front-end device (including video acquisition and production equipment) completes the production of high dynamic range (HDR) content, and transmits the video source data (image information) and metadata (dynamic metadata and / or static metadata) to the display end through the transport layer. Depending on the display capability (SDR or HDR) of different display devices, the display end converts the received video source data (in some embodiments, combined with information provided by metadata) into a display image adapted to the display device. It should be understood that... Figure 4 In the terms "low dynamic range HDR" and "low dynamic range SDR," "low dynamic range" refers to content that is high dynamic range HDR content generated by the front end. It should also be understood that in different embodiments, the display device may have a higher dynamic range display capability than the HDR content generated by the front end, and the process of adapting the display can also be applied to this system architecture; this application does not limit it.

[0172] The dynamic range conversion module in this application embodiment, for example, can exist in video conversion devices such as set-top boxes, television displays, mobile terminal displays, and online live streaming and video applications. More specifically, the module can exist as a chip in set-top boxes, television displays, and mobile terminal displays, and as a software program in video conversion devices such as online live streaming and video applications.

[0173] Figure 5 This is a flowchart illustrating an image signal conversion processing method provided in another embodiment of this application. Figure 5 The image signal conversion processing method in the illustrated embodiments may include at least:

[0174] S501 receives a bitstream containing source image signal data and metadata.

[0175] Generally, both the source image signal data (HDR video signal) and metadata are transmitted in the form of a bitstream. This bitstream is obtained at the video content production end by encoding the image data and auxiliary information, and at the display end by decoding, lossily or losslessly restoring it to the source image signal data and metadata. The source image signal data can be pixel data, and the metadata can include the format of the video source data and various parameters related to HDR video processing, such as parameters in the dynamic range conversion model.

[0176] It should be understood that the format of the source image signal data is not limited in the embodiments of this application. For example, it can be data in YUV color space, data in RGB color space, 8-bit data, 10-bit data, or 12-bit data.

[0177] It should also be understood that the format of metadata is not limited in this application embodiment. For example, it can be standard ST2094-40 containing histogram information and tone-mapping curve parameter information, or it can be standard ST2094-10 containing tone-mapping curve parameter information. Metadata can be dynamic metadata, static metadata, or both static and dynamic metadata.

[0178] In the embodiments of this application, the received source image signal data can be an optical signal or an electrical signal, and this application does not limit it.

[0179] S502, optional, determines the method for obtaining the model parameters of the dynamic range transformation model.

[0180] In one feasible implementation, the model parameters are obtained in a preset manner, that is, the method of obtaining each model is determined according to a protocol known in advance by both the HDR production end and the HDR display end.

[0181] In another feasible implementation, the acquisition method of each model parameter can be adaptively determined based on information from the source image signal data, such as the distribution characteristics of brightness values ​​in the brightness histogram.

[0182] In another feasible implementation, multiple acquisition methods can be predefined. The HDR production end writes the selected acquisition method into the identifier information of the metadata, and the HDR display end learns the specific acquisition method by decoding the identifier information in the metadata.

[0183] For example, the method of obtaining model parameters may include: decoding the value of the model parameters carried by the metadata, that is, the value of the model parameters is directly written into the metadata at the HDR production end; or, using a preset value as the value of the model parameters; or, calculating the value of the model parameters based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

[0184] Furthermore, the information of the source image signal can be directly included in the metadata as syntax elements, or the syntax elements of the metadata can be parsed first, and then the information of the source image signal can be derived and calculated based on the parsed syntax elements. For example, the information of the source image signal includes the maximum brightness value of the pixels in the frame to be processed, the average brightness value of the pixels in the frame to be processed, and the minimum brightness value of the pixels in the frame to be processed.

[0185] It should be understood that, in order to handle the large frame-by-frame variations in bright areas such as the sun and to address the adverse effects on overall brightness estimation, it is necessary to correct the maximum brightness value of pixels in the frame to be processed from the source image signal. This eliminates interference from bright areas and allows for the calculation of more suitable model parameters. The correction value is less than or equal to the maximum value of the actual pixels in the frame to be processed. Specifically, it can be set to a value such that pixels greater than the darkest pixel but less than that value account for a preset proportion of the total pixels, such as 99%, 95%, or 90%. This value is the correction value. Alternatively, the correction value can be set to the average value of the total pixels plus (brightness 1 - brightness 2) / 2, where brightness 1 satisfies that pixels greater than the darkest pixel but less than brightness 1 account for 90% of the total pixels, and brightness 2 satisfies that pixels greater than the darkest pixel but less than brightness 2 account for 10% of the total pixels. In this embodiment, the maximum brightness value of pixels in the frame to be processed from the source image signal after correction is called the correction value of the maximum brightness value of pixels in the frame to be processed from the source image signal.

[0186] In some feasible implementations, the pixels in the frame to be processed can refer to all pixels in the frame or to valid pixels in the frame. The valid pixels are pixels within a preset range of the source image signal, or pixels within a range determined according to the metadata. For example, valid pixels can be source image signals within a dark threshold and a bright threshold range. The dark threshold can be a preset threshold, or it can be obtained by setting a preset proportion in the source image signal such that pixels smaller than the dark threshold account for a preset proportion of the total pixels in the source image signal, such as 0%, 0.5%, 1%, 5%, 10%, etc. The bright threshold can be a preset threshold, or it can be obtained by setting a preset proportion in the source image signal such that pixels larger than the bright threshold account for a preset proportion of the total pixels in the source image signal, such as 0%, 0.5%, 1%, 5%, 10%, etc.

[0187] It should be understood that there can be multiple model parameters. Identification information can indicate the same acquisition method used by a group of model parameters, or it can indicate a preset combination of acquisition methods, under which different model parameters can use the same or different acquisition methods.

[0188] It should also be understood that the image corresponding to the source image signal can contain different regions, and different regions can have their own different methods of obtaining model parameters, different values ​​of model parameters, and different transformation models, without limitation.

[0189] In one specific embodiment, the dynamic metadata syntax table includes:

[0190] for(w=0;w<num_windows;w++) tone_mapping_mode[w]

[0191] That is, the image corresponding to the source image signal can contain num_windows regions. For each region, a marker information tone_mapping_mode is decoded to determine how the model parameters of that region are obtained.

[0192] For example, the relationship between the value of tone_mapping_mode and the number of undetermined parameters in the model can be:

[0193]

[0194] For example, the relationship between the value of tone_mapping_mode and the method of obtaining model parameters can be:

[0195]

[0196] For example, acquisition process 1 includes: all parameters are set to preset values; acquisition process 2 includes: the first parameter is set to a preset value, the second parameter is obtained by parsing metadata, and the third parameter is calculated based on the information of the source image signal carried by the metadata and the information of the display device.

[0197] S503, optional, determines the effective range of the applied preset dynamic range conversion model.

[0198] It should be understood that in some embodiments, a dynamic range conversion model can be applied to pixels with all primary color value ranges, while in other embodiments, a dynamic range conversion model is only applicable to pixels with a portion of the primary color value ranges. Therefore, it is first necessary to determine the applicable range of different dynamic range conversion models, and then select a dynamic range conversion model for conversion based on the applicable range of the pixel to be processed. This application does not limit the determination of the applicable range of the dynamic range conversion model in the embodiments of this application.

[0199] S504, obtain a primary color component of the pixel to be processed.

[0200] A pixel to be processed is obtained from the received source image signal data. Let's assume that this pixel satisfies the effective range of the dynamic range conversion model in this embodiment.

[0201] Let's assume the pixel to be processed is in the RGB color gamut, and obtain the value of one of the R, G, and B components. In one feasible implementation, obtain the value of the component with the largest value among the R, G, and B components, and let's denote this value as Fmax.

[0202] S505, according to the preset dynamic range conversion model, convert the first value of the primary color component into the second value.

[0203] The dynamic range conversion model is as follows:

[0204]

[0205] Where L is the first value, that is, Fmax obtained in step S504, L′ is the second value, and k1, k2, k3, a, b, m, n and p are model parameters.

[0206] For example, the methods for obtaining various model parameters may include:

[0207] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is obtained by decoding the numerical values ​​of the model parameters carried in the metadata. For example, the numerical values ​​of all model parameters are determined by the HDR production end and directly encoded into the metadata before being transmitted to the HDR display end.

[0208] In another feasible implementation, the values of some parameters are determined according to preset values.

[0209] Exemplarily, at least one of k1, k2, and k3 can take the value of 1.

[0210] Exemplarily, m is a rational number greater than 0 and less than or equal to 8 (0 < m ≤ 8). Specifically, m can take the value of 2.4.

[0211] Exemplarily, n is a rational number greater than 0 and less than or equal to 8 (0 < n ≤ 8). Specifically, n can take the value of 1.

[0212] Exemplarily, m takes the value of 2.4 and n takes the value of 1, or m takes the value of 1 and n takes the value of 5 / 12 (for example, n is taken to one decimal place, i.e., 0.4).

[0213] In another feasible implementation, the values of some parameters are directly or obtained through calculation according to the information of the source image signal carried by the metadata and / or the information of the display device.

[0214] Exemplarily, b can be the minimum brightness value of the display device after normalization processing.

[0215] Exemplarily, k3 can be the maximum brightness value of the pixels in the to-be-processed frame of the source image signal after normalization processing.

[0216] Among them, exemplarily, the normalization processing includes: the process of converting the to-be-normalized signal into a non-linear PQ signal value according to the process specified by the ST 2084, 2014 - SMPTE standard (in some embodiments, the PQ(x) function is used to represent this normalization processing of x), or the process of converting the to-be-normalized signal into a non-linear HLG signal value according to the process specified by the Rec.2100 - ITU standard (in some embodiments, the HLG(x) function is used to represent this normalization processing of x).

[0217] The maximum brightness value of the pixels in the to-be-processed frame of the source image signal can be: the maximum value in the brightness components of the pixels in the to-be-processed frame. For pixels in the YUV color space, it is the maximum value in the Y component. For pixels in the RGB color space, the brightness value can be calculated through the values of R, G, and B; or it can be the maximum value in the maximum primary color components of the pixels in the to-be-processed frame. The maximum primary color component is the primary color component with the largest value among the multiple primary color components of a pixel, such as the largest one among R, G, and B.

[0218] It should be understood that the methods for obtaining the minimum brightness value and the brightness component of the average brightness value of pixels in the frame to be processed in the source image signal described below are the same as the methods for obtaining the brightness component of the maximum brightness value, and will not be repeated here.

[0219] For example, at least one of k1, k2, k3, a, b, m, n, and p is calculated based on information from the source image signal and / or information from the display device.

[0220] In some feasible implementations, before calculating model parameters using the maximum brightness value of the pixels in the frame to be processed of the source image signal, the method decodes the maximum brightness value of the pixels in the frame to be processed from the metadata, and further includes: correcting the maximum brightness value of the pixels in the frame to be processed of the source image signal; correspondingly, at least one of k1, k2, k3, a, b, m, n and p is calculated based on the information of the source image signal and / or the information of the display device, including: at least one of k1, k2, k3, a, b, m, n and p is obtained based on the correction value of the maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0221] In one embodiment, let's assume that this embodiment corresponds to a value of tone_mapping_mode, and the process of determining the values ​​of each model parameter includes:

[0222] (1) k1, k2, k3, m, n take preset values ​​of 1, 1, 1, 2.4, 1 respectively;

[0223] (2) b is the minimum brightness value of the display device after normalization, i.e., PQ (MinDisplay);

[0224] (3) p is obtained according to the following formula:

[0225]

[0226] Where avgL is the average brightness value of pixels in the frame to be processed of the source image signal after normalization, TPH, TPL, p valueH and p valueL All are preset values, 0≤TPL, TPH≤1, and g() is a monotonically increasing function in the interval from 0 to 1.

[0227] For example, TPL is 0.5, TPH is 0.8, and p valueL It is 3.5, p valueH The value is 5.0, and g(x) = x, where x is a rational number in the interval from 0 to 1.

[0228] The information of the source image signal includes the average brightness value of the pixels in the frame to be processed of the source image signal, i.e., avgL can be obtained by decoding the metadata.

[0229] In one feasible implementation, the average brightness value of the pixels in the frame to be processed of the source image signal includes: the average value of the brightness components of the pixels in the frame to be processed; or, the average value of the maximum primary color components of the pixels in the frame to be processed.

[0230] (4)a is obtained according to the following formula:

[0231] a=(MaxDisplay-MinDisplay) / (H(MaxSource)-H(MinSource))

[0232] Wherein, MaxSource is the maximum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MinSource is the minimum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MaxDisplay is the maximum brightness value of the display device after normalization, and MinDisplay is the minimum brightness value of the display device after normalization.

[0233] The information of the source image signal includes the maximum brightness value of the pixels in the frame to be processed of the source image signal and the minimum brightness value of the pixels in the frame to be processed of the source image signal, that is, MaxSource and MinSource can be obtained by decoding metadata.

[0234] It should be understood that, in some feasible implementations, MaxSource is a correction value for the maximum brightness value of the pixels in the frame to be processed of the source image signal after normalization.

[0235] In one feasible implementation, the minimum brightness value of a pixel in the frame to be processed of the source image signal includes: the minimum value of the brightness component of the pixel in the frame to be processed; or, the minimum value of the maximum primary color component of the pixel in the frame to be processed.

[0236] It should be understood that, depending on the different normalization standards, the maximum brightness value of the pixels in the frame to be processed of the source image signal after normalization is also called the maximum brightness value of the pixels in the frame to be processed of the source image signal in the PQ domain, or the maximum brightness value of the pixels in the frame to be processed of the source image signal in the HLG domain. The physical quantities after normalization also conform to the same situation, which will not be elaborated on later.

[0237] In another embodiment, let's assume that this embodiment corresponds to another value of tone_mapping_mode, and the process of determining the values ​​of each model parameter includes:

[0238] (1) k1, k2, k3 are set to preset values ​​of 1, 1, 1 respectively;

[0239] (2) b is the minimum brightness value of the display device after normalization;

[0240] (3) When n takes the preset value of 1, a, m, and p are obtained by solving the following system of equations:

[0241]

[0242] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of pixels in the frame to be processed of the source image signal. (In some embodiments, L3 is MaxSource.) The mapping value is obtained according to the following formula:

[0243]

[0244] in,

[0245]

[0246] y(i) is the mapping value, f(i) is the maximum brightness value of the i-th pixel in the frame to be processed of the source image signal, MaxDisplay and MinDisplay are the maximum and minimum brightness values ​​of the display device, Nframe is the number of pixels in the frame to be processed, and LR is the perceptual brightness compensation factor, 0 < LR ≤ 2. For example, LR can be 0.8, 1, or 1.2.

[0247] The perceived brightness value satisfies: J(Lp) - J(1nit) = (J(L0) - J(1nit)) * Rate, where Lp is the perceived brightness value, L0 is the preset brightness value, Rate is the preset ratio, and J(x) is the proportion of pixels with a brightness less than x nits in the frame to be processed of the source image signal. That is, Rate is the proportion of the number of pixels between 1 nit and Lp in the source image signal, after normalization, to the total number of pixels between 1 nit and L0. For example, the preset brightness value is 5 nits, and the preset ratio is 10%, 20%, or 30%. J(x) can be obtained by statistically accumulating a histogram of pixels in the frame to be processed of the source image signal.

[0248] In another feasible implementation, the perceived brightness value is obtained by decoding the metadata.

[0249] It should be understood that, in some feasible implementations, L3 is a correction value for the maximum brightness value of the pixels in the frame to be processed of the source image signal after normalization.

[0250] When m takes the preset value of 2.4, a, n, and p are obtained by solving the following system of equations:

[0251]

[0252] Furthermore, taking the preset value of n as 1 as an example, it should be understood that when any one of the unknown parameters a, m, p has been obtained through other means, the values ​​of the other two unknown parameters can be obtained by using only any two equations in the system of equations.

[0253] Specifically, in one embodiment, let's assume m takes a preset value of 2.4. Then a and p are obtained by solving the following system of equations:

[0254]

[0255] Where LM = L2 and FM = F2, or LM = L1 and FM = F1.

[0256] In another embodiment, when L2 is less than the normalized value of the preset brightness value, LM = L1 and FM = F1; when L2 is greater than or equal to the normalized value of the preset brightness value, LM = L2 and FM = F2. For example, the preset brightness value can be 2 nit, 5 nit, etc.

[0257] In the third embodiment, let's assume that this embodiment corresponds to the third value of tone_mapping_mode, and the process of determining the values ​​of each model parameter includes:

[0258] (1) k1, k2, m, n take preset values ​​of 1, 1, 2.4, 1 respectively;

[0259] (2) b is the minimum brightness value of the display device after normalization;

[0260] (3) k3 is the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization. For example, for the RGB color space, k3 is the maximum value of the maximum primary color component value of the pixel in the frame to be processed of the source image signal after normalization.

[0261] It should be understood that, in some feasible implementations, k3 is a correction value for the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization.

[0262] (4) a is the difference between the maximum brightness value of the display device after normalization and the minimum brightness value of the display device after normalization;

[0263] (5) p is obtained according to the following formula: p = c * v + d, where c and d are preset values. For example, c is 7, d is 3, and v is obtained according to the following formula: v = J(Tq) - J(Tp). Where J(x) is the proportion of pixels with brightness less than xnit in the frame to be processed of the source image signal, Tp is the value of init after normalization, and Tq is Tp times the ratio of the maximum brightness value of the pixels in the frame to be processed of the source image signal after normalization to the maximum value of the display device after normalization. J(x) can be obtained by statistically accumulating a histogram of pixels in the frame to be processed of the source image signal. In some feasible implementations, c and d can also be determined according to the information of the source image signal and / or the information of the display device, such as the maximum brightness value of the display device, the brightness range and distribution of the source image signal, etc.

[0264] In other implementations of this embodiment, v can be calculated in different ways, specifically:

[0265] In this embodiment, k3 = E src_max .

[0266] E src_max The following method will be used to determine this:

[0267] E src_max =max(E disp_max ,prctile(x))

[0268] Where prctile(x) represents the value at x percent of the maximum primary color component value of the pixel in the frame to be processed from the source image signal. prctile(x) is included in dynamic metadata, such as prctile(100), prctile(99), prctile(95), etc. The default value is prctile(100). The smaller the value of x here, the brighter the output image will be. Therefore, if you want to increase the brightness, you can use prctile(99) or prctile(95). The maximum brightness value of the display device (E) disp_max ) is E src_max The lower limit of the value. This means that for source image signals with a maximum brightness lower than the maximum brightness value of the display device, tone mapping will maintain the maximum brightness of the source image signal.

[0269] p can be determined using histogram information from metadata, based on the hue loss caused by tone mapping. The theoretical basis for this method is the limitation of human eye perception of contrast. When the brightness of a certain area of ​​an image is below the light threshold Tp (approximately 1 to 5 nits), the human eye has difficulty perceiving the contrast information in that area. The light threshold varies from person to person; for simplicity, Tp is set to 1 nit by default in this embodiment, assuming that the human eye can effectively perceive image areas above 1 nit, while the contrast perception of the human eye will significantly decrease in areas below 1 nit.

[0270] If a certain proportion of pixels in an image's histogram are above 1 nit before tone mapping but drop below 1 nit after tone mapping, then this proportion represents the tonal loss. The image histogram is statistically analyzed in the PQ domain. Before determining p, the dynamic range conversion curve cannot be determined; therefore, it's reasonable to consider E... src_max Corresponding to E disp_max The linear mapping is approximated by tone mapping, and this is used to estimate the tone loss caused by tone mapping. The approximate tone mapping is as follows:

[0271]

[0272] Where Lin is the input value and Lout is the output value, both being values ​​in the PQ domain. The hue loss v is calculated using the following formula:

[0273] v=J(Lout=0.15)-J(Lin=0.15)

[0274] Here, 0.15 represents the PQ domain luminance value corresponding to 1 nit (i.e., the normalized value after processing 1 nit). J(Lout = 0.15) represents the proportion of pixels below 1 nit after approximate tone mapping, and J(Lin = 0.15) represents the proportion of pixels below 1 nit in the frame to be processed from the source image signal. It should be understood that the cumulative histogram J is normalized, i.e., it satisfies J(E... src_max = 1. As the hue loss v increases, the parameter p increases accordingly.

[0275] It should be understood that in some other feasible embodiments, the model parameters can be determined dynamically. For example, the values ​​of p, n, and m can be determined by judging the brightness of the image scene. Figure 6This demonstrates the relationship between the brightness of an image scene and the parameter m. Within a first scene brightness range, m is directly proportional to the scene brightness; the greater the scene brightness, the larger the value of m. Let the upper bound of the first scene brightness range be a first threshold, and the lower bound be a second threshold. When the scene brightness exceeds the first scene brightness range, the value of m is constrained to the m value corresponding to either the first or second threshold. Figure 7 This demonstrates the relationship between the brightness of an image scene and the parameter p. Within a second scene brightness range, p and the scene brightness show an inversely proportional trend; the higher the scene brightness, the smaller the value of p. Let's assume the upper bound of the second scene brightness range is a third threshold and the lower bound is a fourth threshold. When the scene brightness exceeds the second scene brightness range, the value of p is constrained to the p value corresponding to either the third or fourth threshold. n is determined based on the brightness distribution in the brightness histogram of the scene brightness. Specifically, a brightness histogram can be created based on the scene brightness. The more the scene brightness values ​​are distributed at the two ends of the brightness distribution (bright and dark areas), and less distributed in the middle area, the closer the value of n is to 1. For example, after determining p, n, and m, parameters a and b can be calculated based on the maximum display capability of the display device, the minimum display capability of the display device, and the scene color range (or, the mapping value of the scene color range) corresponding to the color signal to be processed. The display capability of the display device can be obtained from the display device's parameters or the manufacturer's information. The maximum display capability of display devices can be around 570 nits, 800 nits, and 1000 nits, while the minimum display capability is usually 0 nits, but can also be 1 nit. Scene color gamut, including the maximum and minimum color values ​​of the scene, can be carried in dynamic metadata. The maximum color value can be the maximum value among the R, G, and B components, and the percentage of the maximum value (e.g., 90% of the maximum value) can also be determined based on the weighted values ​​of R, G, and B (e.g., the average value). The minimum color value can be 0, or it can be determined based on the weighted values ​​of R, G, and B.

[0276] Once the parameters of the dynamic range conversion model are determined, the conversion from the first value to the second value can be completed based on the dynamic range conversion model.

[0277] In one feasible implementation, the method in step S505 can be used to convert all primary color components of the pixel to be processed. In another feasible implementation, the above conversion can be performed only on the primary color component with the largest value among the three primary color components of the pixel to be processed, and the ratio of the second value after denormalization to the first value after denormalization can be calculated, let's say K. The K times the value of each primary color component of the pixel to be processed after denormalization is taken as the converted value.

[0278] S506, The converted image signal is displayed on the display device.

[0279] It should be understood that after obtaining the converted image signal and before displaying the image signal, one or more image processing procedures such as saturation adjustment, color gamut transformation, noise reduction, and sharpening can be performed on the converted image signal. This application embodiment does not limit this.

[0280] It should be understood that in the embodiments of this application, the dynamic range conversion model introduces a parameter n, which can reflect the characteristics of the brightness distribution in the scene corresponding to the color signal to be processed, providing greater flexibility for the adaptation of the dynamic range conversion model and the scene. With reasonable configuration of other model parameters, the conversion of image content with different dynamic ranges can achieve better display effects.

[0281] Figure 8 This is a schematic diagram of an image signal conversion and processing device provided in an embodiment of this application. The image signal conversion and processing device provided in this embodiment can be used to implement the present invention. Figure 2 , Figure 3 or Figure 5 The image signal conversion and processing method described herein includes some or all of the process steps. Figure 8 An image signal conversion and processing apparatus 800 is shown, comprising: an acquisition module 801 for acquiring a primary color component of a pixel to be processed; and a conversion module 802 for converting a first value of the primary color component into a second value according to a preset dynamic range conversion model; wherein the dynamic range conversion model is:

[0282]

[0283] Where L is the first value, L′ is the second value, and k1, k2, k3, a, b, m, n and p are model parameters.

[0284] In one feasible implementation, before converting the first value of the primary color component into the second value according to the preset dynamic range conversion model, the acquisition module 801 is further configured to: determine the acquisition method of the model parameters according to the identification information carried by the metadata, wherein the metadata includes static metadata and / or dynamic metadata.

[0285] In one feasible implementation, the metadata and the pixel to be processed are obtained by decoding the bitstream associated with the source image signal.

[0286] In one feasible implementation, the model parameters are obtained by: decoding the numerical values ​​of the model parameters carried in the metadata; or, using a preset value as the numerical value of the model parameters; or, calculating the numerical values ​​of the model parameters based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

[0287] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is obtained by decoding the numerical values ​​of the model parameters carried by the metadata.

[0288] In one feasible implementation, at least one of k1, k2, k3, m and n is obtained according to a preset value, wherein 0 < n ≤ 8 and 0 < m ≤ 8.

[0289] In one feasible implementation, the model parameters shall have values ​​that satisfy at least one of the following: m is 2.4 or 1, n is 1 or 5 / 12, k1 is 1, k2 is 1, and k3 is 1.

[0290] In one feasible implementation, b is the minimum brightness value of the display device after normalization.

[0291] In one feasible implementation, k3 is the maximum brightness value of the pixel in the frame to be processed of the source image signal after normalization.

[0292] In one feasible implementation, the maximum brightness value of a pixel in the frame to be processed of the source image signal includes: the maximum value among the brightness components of the pixel in the frame to be processed; or, the maximum value among the maximum primary color components of the pixel in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among multiple primary color components of a pixel.

[0293] In one feasible implementation, at least one of k1, k2, k3, a, b, m, n, and p is calculated based on information from the source image signal and / or information from the display device.

[0294] In one feasible implementation, the information of the source image signal includes the average brightness value of the pixels in the frame to be processed of the source image signal, p is obtained according to the following formula:

[0295]

[0296] Where avgL is the average brightness value of pixels in the frame to be processed of the source image signal after normalization, TPH, TPL, p valueH and p valueL All are preset values, 0≤TPL, TPH≤1, and g() is a monotonically increasing function in the interval from 0 to 1.

[0297] In one feasible implementation, TPL is 0.5, TPH is 0.8, and p valueL It is 3.5, p valueH The value is 5.0, and g(x) = x, where x is a rational number in the interval from 0 to 1.

[0298] In one feasible implementation, the average brightness value of the pixels in the frame to be processed of the source image signal includes: the average value of the brightness components of the pixels in the frame to be processed; or, the average value of the maximum primary color components of the pixels in the frame to be processed.

[0299] In one feasible implementation, the information of the source image signal includes the maximum brightness value of the pixels in the frame to be processed of the source image signal and the minimum brightness value of the pixels in the frame to be processed of the source image signal, and 'a' is obtained according to the following formula:

[0300] a=(MaxDisplay-MinDisplay) / (H(MaxSource)-H(MinSource))

[0301] Wherein, MaxSource is the maximum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MinSource is the minimum brightness value of a pixel in the frame to be processed of the source image signal after normalization, MaxDisplay is the maximum brightness value of the display device after normalization, and MinDisplay is the minimum brightness value of the display device after normalization.

[0302] In one feasible implementation, the minimum brightness value of a pixel in the frame to be processed of the source image signal includes: the minimum value of the brightness component of the pixel in the frame to be processed; or, the minimum value of the maximum primary color component of the pixel in the frame to be processed.

[0303] In one feasible implementation, a, p, and m are obtained by solving the following system of equations:

[0304]

[0305] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of the pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0306] In one feasible implementation, a and p are obtained by solving the following system of equations:

[0307]

[0308] Wherein, F1 is the normalized value of 1 nit, F2 is the normalized mapping value of the source image signal on the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of the pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of the pixels in the frame to be processed of the source image signal.

[0309] When L2 is less than the normalized value of 5 nits, M1 equals L1 and N1 equals F1. When L2 is greater than or equal to the normalized value of 5 nits, M1 equals L2 and N1 equals F2.

[0310] In one feasible implementation, the mapping value is obtained according to the following formula:

[0311]

[0312] in,

[0313]

[0314] y(i) is the mapping value, f(i) is the maximum brightness value of the i-th pixel in the frame to be processed of the source image signal, MaxDisplay and MinDisplay are the maximum and minimum brightness values ​​of the display device, Nframe is the number of pixels in the frame to be processed, LR is the perceived brightness compensation factor, 0 < LR ≤ 2.

[0315] In one feasible implementation, LR is 0.8, 1, or 1.2.

[0316] In one feasible implementation, the perceived brightness value satisfies: J(Lp)-J(1nit)=(J(L0)-J(1nit))*Rate, where Lp is the perceived brightness value, L0 is the preset brightness value, Rate is the preset ratio, and J(x) is the proportion of pixels with brightness less than x nits in the frame to be processed of the source image signal.

[0317] In one feasible implementation, the preset brightness value is 5 nits, and the preset ratio is 10%, 20%, or 30%.

[0318] In one feasible implementation, the perceived brightness value is obtained by decoding the metadata.

[0319] In one feasible implementation, 'a' is the difference between the normalized maximum brightness value and the normalized minimum brightness value of the display device.

[0320] In one feasible implementation, p is obtained according to the following formula: p = c * v + d, where c and d are preset values.

[0321] In one feasible implementation, c is 7 and d is 3.

[0322] In one feasible implementation, v is obtained according to the following formula: v = J(Tq) - J(Tp), where J(x) is the proportion of pixels with a brightness less than x nits in the frame to be processed of the source image signal, Tp is the value after normalization of 1 nit, and Tq is the ratio of the maximum brightness value of the pixels in the frame to be processed of the source image signal after normalization to the maximum value of the display device after normalization, multiplied by Tp.

[0323] In one feasible implementation, the normalization process includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in the ST 2084, 2014-SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in the Rec.2100-ITU standard.

[0324] In one feasible implementation, the acquisition module 801 is further configured to: decode the maximum brightness value of a pixel in the frame to be processed of the source image signal from the metadata; correct the maximum brightness value of the pixel in the frame to be processed of the source image signal; correspondingly, at least one of k1, k2, k3, a, b, m, n and p is calculated based on the information of the source image signal and / or the information of the display device, including: at least one of k1, k2, k3, a, b, m, n and p is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

[0325] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. For example... Figure 9 As shown, the terminal device 900 may include a processor 901, a memory 902, an input device 903, and an output device 904. The processor 901 is connected to the memory 902, the input device 903, and the output device 904, for example, the processor 901 can be connected to the memory 902, the input device 903, and the output device 904 via a bus.

[0326] Among them, processor 901 can be a central processing unit (CPU), a network processor (NP), etc.

[0327] The memory 902 may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory may also include combinations of the above types of memory.

[0328] Processor 901 is used to execute Figure 2 , Figure 3 or Figure 5 The image signal conversion and processing method described herein includes some or all of the process steps.

[0329] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms are not necessarily directed at the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0330] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0331] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can include, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory, read-only memory, erasable and editable read-only memory, fiber optic device, and portable optical disc read-only memory. Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0332] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0333] Furthermore, the modules in the various embodiments of this invention can be implemented either in hardware or as software functional modules. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0334] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image signal conversion and processing method, characterized in that, The method includes: Obtain a primary color component of the pixel to be processed; According to a preset dynamic range conversion model, the first value of the primary color component is converted into a second value; wherein, the dynamic range conversion model is: in, The first value is... The second value, , , a, b, m, n, and p are model parameters, and a, p, and m are obtained by solving the following system of equations: Wherein, F1 is the normalized value of 1 nit, F2 is the normalized source image signal mapped onto the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of pixels in the frame to be processed of the source image signal.

2. The method according to claim 1, characterized in that, Before converting the first value of the primary color component to the second value according to the preset dynamic range conversion model, the method further includes: The method for obtaining the model parameters is determined based on the identification information carried by the metadata, which includes static metadata and / or dynamic metadata.

3. The method according to claim 2, characterized in that, The metadata and the pixels to be processed are obtained by decoding the bitstream associated with the source image signal.

4. The method according to claim 2, characterized in that, The methods for obtaining the model parameters include: Decode the numerical values ​​of the model parameters carried by the metadata; or, Use preset values ​​as the numerical values ​​of the model parameters; or, The values ​​of the model parameters are calculated based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

5. The method according to claim 2, characterized in that, , , At least one of b and n is obtained by decoding the numerical values ​​of the model parameters carried in the metadata.

6. The method according to claim 1, characterized in that, , , and at least one of n is obtained according to a preset value, where 0 n≤8.

7. The method according to claim 6, characterized in that, The model parameters shall have values ​​that satisfy at least one of the following: n is 1 or 5 / 12. k 1 is 1, k 2 is 1. k 3 is 1.

8. The method according to claim 1, characterized in that, b is the minimum brightness value of the display device after normalization.

9. The method according to claim 1, characterized in that, k 3 represents the maximum brightness value of the pixel in the frame to be processed from the source image signal after normalization.

10. The method according to claim 9, characterized in that, The maximum brightness value of the pixels in the frame to be processed from the source image signal includes: The maximum value of the luminance components of the pixels in the frame to be processed; or, The maximum value among the maximum primary color components of the pixels in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among the multiple primary color components of a pixel.

11. The method according to claim 4, characterized in that, , , At least one of b and n is calculated based on information from the source image signal and / or information from the display device.

12. The method according to any one of claims 1-11, characterized in that, The mapping value is obtained according to the following formula: y = ， in, y Let f(i) be the maximum brightness value of the i-th pixel in the frame to be processed of the source image signal, and MaxDisplay and MinDisplay be the maximum and minimum brightness values ​​of the display device, respectively. N frame Where is the number of pixels in the frame to be processed, and LR is the perceived brightness compensation factor, 0 LR≤2.

13. The method according to claim 12, characterized in that, LR is 0.8, 1, or 1.

2.

14. The method according to claim 12, characterized in that, The perceived brightness value satisfies: J(Lp)–J(1nit) = (J(L0)-J(1nit)) Rate, where Lp is the perceived brightness value, L0 is the preset brightness value, Rate is the preset ratio, and J(x) is the proportion of pixels with a brightness less than x nits in the frame to be processed of the source image signal.

15. The method according to claim 14, characterized in that, The preset brightness value is 5 nits, and the preset ratio is 10%, 20%, or 30%.

16. The method according to claim 12, characterized in that, The perceived brightness value is obtained from the decoded metadata.

17. The method according to any one of claims 1-11, characterized in that, Normalization processing includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in ST 2084, 2014 - SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in Rec. 2100 - ITU standard.

18. The method according to claim 11, characterized in that, Also includes: Decode the maximum brightness value of the pixels in the frame to be processed from the metadata of the source image signal; The maximum brightness value of the pixels in the frame to be processed of the source image signal is corrected; Correspondingly, the , , At least one of b and n is calculated based on information from the source image signal and / or information from the display device, including: , , At least one of b and n is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

19. An image signal conversion and processing method, characterized in that, The method includes: Obtain a primary color component of the pixel to be processed; According to a preset dynamic range conversion model, the first value of the primary color component is converted into a second value; wherein, the dynamic range conversion model is: in, The first value is... The second value, , , a, b, m, n, and p are model parameters, and a and p are obtained by solving the following system of equations: Wherein, F1 is the normalized value of 1 nit, F2 is the normalized source image signal mapped onto the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of pixels in the frame to be processed of the source image signal. When L2 is less than the normalized value of 5 nit, M1 equals L1 and N1 equals F1. When L2 is greater than or equal to the normalized value of 5 nit, M1 equals L2 and N1 equals F2.

20. The method according to claim 19, characterized in that, Before converting the first value of the primary color component to the second value according to the preset dynamic range conversion model, the method further includes: The method for obtaining the model parameters is determined based on the identification information carried by the metadata, which includes static metadata and / or dynamic metadata.

21. The method according to claim 20, characterized in that, The metadata and the pixels to be processed are obtained by decoding the bitstream associated with the source image signal.

22. The method according to claim 20, characterized in that, The methods for obtaining the model parameters include: Decode the numerical values ​​of the model parameters carried by the metadata; or, Use preset values ​​as the numerical values ​​of the model parameters; or, The values ​​of the model parameters are calculated based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

23. The method according to claim 20, characterized in that, , , At least one of b, m, and n is obtained by decoding the numerical values ​​of the model parameters carried in the metadata.

24. The method according to claim 19, characterized in that, , , At least one of m and n is obtained according to a preset value, where 0 n≤8, 0 m≤8.

25. The method according to claim 24, characterized in that, The model parameters shall have values ​​that satisfy at least one of the following: m is 2.4 or 1, n is 1 or 5 / 12. k 1 is 1, k 2 is 1. k 3 is 1.

26. The method according to claim 19, characterized in that, b is the minimum brightness value of the display device after normalization.

27. The method according to claim 19, characterized in that, k 3 represents the maximum brightness value of the pixel in the frame to be processed from the source image signal after normalization.

28. The method according to claim 27, characterized in that, The maximum brightness value of the pixels in the frame to be processed from the source image signal includes: The maximum value of the luminance components of the pixels in the frame to be processed; or, The maximum value among the maximum primary color components of the pixels in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among the multiple primary color components of a pixel.

29. The method according to claim 19, characterized in that, , , At least one of b, m, and n is calculated based on information from the source image signal and / or information from the display device.

30. The method according to any one of claims 19-29, characterized in that, Normalization processing includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in ST 2084, 2014 - SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in Rec. 2100 - ITU standard.

31. The method according to claim 29, characterized in that, Also includes: Decode the maximum brightness value of the pixels in the frame to be processed from the metadata of the source image signal; The maximum brightness value of the pixels in the frame to be processed of the source image signal is corrected; Correspondingly, the , , At least one of b, m, and n is calculated based on information from the source image signal and / or information from the display device, including: , , At least one of b, m, and n is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

32. An image signal conversion and processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a primary color component of the pixel to be processed; The conversion module is used to convert the first value of the primary color component into a second value according to a preset dynamic range conversion model; wherein the dynamic range conversion model is: in, The first value is... The second value, , , a, b, m, n, and p are model parameters, and a, p, and m are obtained by solving the following system of equations: Wherein, F1 is the normalized value of 1 nit, F2 is the normalized source image signal mapped onto the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of pixels in the frame to be processed of the source image signal.

33. The apparatus according to claim 32, characterized in that, Before converting the first value of the primary color component into the second value according to the preset dynamic range conversion model, the acquisition module is further configured to: The method for obtaining the model parameters is determined based on the identification information carried by the metadata, which includes static metadata and / or dynamic metadata.

34. The apparatus according to claim 33, characterized in that, The metadata and the pixels to be processed are obtained by decoding the bitstream associated with the source image signal.

35. The apparatus according to claim 33, characterized in that, The methods for obtaining the model parameters include: Decode the numerical values ​​of the model parameters carried by the metadata; or, Use preset values ​​as the numerical values ​​of the model parameters; or, The values ​​of the model parameters are calculated based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

36. The apparatus according to claim 33, characterized in that, , , At least one of b and n is obtained by decoding the numerical values ​​of the model parameters carried in the metadata.

37. The apparatus according to claim 32, characterized in that, , , and at least one of n is obtained according to a preset value, where 0 n≤8.

38. The apparatus according to claim 32, characterized in that, The model parameters shall have values ​​that satisfy at least one of the following: n is 1 or 5 / 12. k 1 is 1, k 2 is 1. k 3 is 1.

39. The apparatus according to claim 32, characterized in that, b is the minimum brightness value of the display device after normalization.

40. The apparatus according to claim 32, characterized in that, k 3 represents the maximum brightness value of the pixel in the frame to be processed from the source image signal after normalization.

41. The apparatus according to claim 40, characterized in that, The maximum brightness value of the pixels in the frame to be processed from the source image signal includes: The maximum value of the luminance components of the pixels in the frame to be processed; or, The maximum value among the maximum primary color components of the pixels in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among the multiple primary color components of a pixel.

42. The apparatus according to claim 35, characterized in that, , , At least one of b and n is calculated based on information from the source image signal and / or information from the display device.

43. The apparatus according to any one of claims 32-42, characterized in that, The mapping value is obtained according to the following formula: y = ， in, y Let f(i) be the maximum brightness value of the i-th pixel in the frame to be processed of the source image signal, and MaxDisplay and MinDisplay be the maximum and minimum brightness values ​​of the display device, respectively. N frame Where is the number of pixels in the frame to be processed, and LR is the perceived brightness compensation factor, 0 LR≤2.

44. The apparatus according to claim 43, characterized in that, LR is 0.8, 1, or 1.

2.

45. The apparatus according to claim 43, characterized in that, The perceived brightness value satisfies: J(Lp)–J(1nit) = (J(L0)-J(1nit)) Rate, where Lp is the perceived brightness value, L0 is the preset brightness value, Rate is the preset ratio, and J(x) is the proportion of pixels with a brightness less than x nits in the frame to be processed of the source image signal.

46. ​​The apparatus according to claim 45, characterized in that, The preset brightness value is 5 nits, and the preset ratio is 10%, 20%, or 30%.

47. The apparatus according to claim 43, characterized in that, The perceived brightness value is obtained from the decoded metadata.

48. The apparatus according to any one of claims 32-42, characterized in that, Normalization processing includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in ST 2084, 2014 - SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in Rec. 2100 - ITU standard.

49. The apparatus according to claim 42, characterized in that, The acquisition module is also used for: Decode the maximum brightness value of the pixels in the frame to be processed from the metadata of the source image signal; The maximum brightness value of the pixels in the frame to be processed of the source image signal is corrected; Correspondingly, the , , At least one of b and n is calculated based on information from the source image signal and / or information from the display device, including: , , At least one of b and n is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

50. An image signal conversion and processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a primary color component of the pixel to be processed; A conversion device is used to convert a first value of the primary color component into a second value according to a preset dynamic range conversion model; wherein the dynamic range conversion model is: in, The first value is... The second value, , , a, b, m, n, and p are model parameters, and a and p are obtained by solving the following system of equations: Wherein, F1 is the normalized value of 1 nit, F2 is the normalized source image signal mapped onto the display device, F3 is the normalized maximum brightness value of the display device, L1 is the normalized perceived brightness value of the source image signal, L2 is the normalized average brightness value of pixels in the frame to be processed of the source image signal, and L3 is the normalized maximum brightness value of pixels in the frame to be processed of the source image signal. When L2 is less than the normalized value of 5 nit, M1 equals L1 and N1 equals F1. When L2 is greater than or equal to the normalized value of 5 nit, M1 equals L2 and N1 equals F2.

51. The apparatus according to claim 50, characterized in that, Before converting the first value of the primary color component into the second value according to the preset dynamic range conversion model, the acquisition module further includes: The method for obtaining the model parameters is determined based on the identification information carried by the metadata, which includes static metadata and / or dynamic metadata.

52. The apparatus according to claim 51, characterized in that, The metadata and the pixels to be processed are obtained by decoding the bitstream associated with the source image signal.

53. The apparatus according to claim 51, characterized in that, The methods for obtaining the model parameters include: Decode the numerical values ​​of the model parameters carried by the metadata; or, Use preset values ​​as the numerical values ​​of the model parameters; or, The values ​​of the model parameters are calculated based on the information of the source image signal and / or the information of the display device, wherein the information of the source image signal is obtained by decoding the metadata.

54. The apparatus according to claim 51, characterized in that, , , At least one of b, m, and n is obtained by decoding the numerical values ​​of the model parameters carried in the metadata.

55. The apparatus according to claim 50, characterized in that, , , At least one of m and n is obtained according to a preset value, where 0 n≤8, 0 m≤8.

56. The apparatus according to claim 55, characterized in that, The model parameters shall have values ​​that satisfy at least one of the following: m is 2.4 or 1, n is 1 or 5 / 12. k 1 is 1, k 2 is 1. k 3 is 1.

57. The apparatus according to claim 50, characterized in that, b is the minimum brightness value of the display device after normalization.

58. The apparatus according to claim 50, characterized in that, k 3 represents the maximum brightness value of the pixel in the frame to be processed from the source image signal after normalization.

59. The apparatus according to claim 58, characterized in that, The maximum brightness value of the pixels in the frame to be processed from the source image signal includes: The maximum value of the luminance components of the pixels in the frame to be processed; or, The maximum value among the maximum primary color components of the pixels in the frame to be processed, wherein the maximum primary color component is the primary color component with the largest value among the multiple primary color components of a pixel.

60. The apparatus according to claim 53, characterized in that, , , At least one of b, m, and n is calculated based on information from the source image signal and / or information from the display device.

61. The apparatus according to any one of claims 50-60, characterized in that, Normalization processing includes: converting the signal to be normalized into a non-linear PQ signal value according to the process specified in ST 2084, 2014 - SMPTE standard, or converting the signal to be normalized into a non-linear HLG signal value according to the process specified in Rec. 2100 - ITU standard.

62. The apparatus according to claim 60, characterized in that, The acquisition module is also used for: Decode the maximum brightness value of the pixels in the frame to be processed from the metadata of the source image signal; The maximum brightness value of the pixels in the frame to be processed of the source image signal is corrected; Correspondingly, the , , At least one of b, m, and n is calculated based on information from the source image signal and / or information from the display device, including: , , At least one of b, m, and n is obtained based on the correction value of the maximum brightness value of the pixel in the frame to be processed of the source image signal.

63. An image processing device, characterized in that, include: A non-volatile memory and a processor are coupled together, the processor calling program code stored in the memory to perform the method as described in any one of claims 1-31.

64. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, implements the method described in any one of claims 1-31.

Images