Coding method, decoding method and related equipment

Abstract

The invention provides a coding method, a decoding method and related equipment. The method comprises the following steps: acquiring a first HDR image and N pieces of first base layer data corresponding to the first HDR image; determining N pieces of first enhancement layer data according to the first HDR image, the N pieces of first basic layer data and at least one coding scheme, N being a positive integer greater than or equal to 1; and encoding the N pieces of first enhancement layer data, the N pieces of first basic layer data and first metadata to obtain a code stream, the first metadata comprising first indication information, and the first indication information being used for indicating the at least one encoding scheme or a decoding scheme corresponding to the at least one encoding scheme. According to the technical scheme, flexible format conversion can be realized, and the coding efficiency and the final synthesis effect are improved.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on August 15, 2023, with application number 202311035466.0, and the priority of the Chinese patent application with the invention name “Encoding method and related equipment”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of coding and decoding, and more specifically, to a coding method, a decoding method and related devices. Background Art

[0003] Dynamic range is used in many fields to express the ratio of the maximum value to the minimum value of a variable. In digital images, the dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. Such a dynamic range reaches 1,000,000,000 / 0.001=1013. However, in real scenes in nature, the brightness of the sun and the brightness of the stars are not obtained at the same time. For natural scenes in the real world, the dynamic range is between 10 -3 to 10 6 In most color digital images, each channel of red (R), green (G), and blue (B) is stored in an 8-bit byte. That is, the range of each channel is 0 to 255 gray levels. Here, 0 to 255 is the dynamic range of the image. In the real world, the dynamic range of the same scene is between 10 -3 to 10 6 The dynamic range of an image or video is called a low dynamic range (LDR).

[0004] HDR images or HDR videos may need to be transcoded during the distribution or transmission process. The current HDR transcoding solution is to directly transcode HDR images or HDR videos. After decoding, the HDR image or HDR video obtains HDR pixel values ​​and HDR format identifiers and information. HDR pixel values ​​usually need to be processed (such as image up and down sampling, enhancement, etc.). The HDR format identifier and information, as well as the processed HDR pixel values ​​are then sent to the encoder for re-encoding. The re-encoded HDR image or HDR video is distributed to other devices. This solution of directly transcoding HDR images or HDR videos (which can be called a "single-layer transcoding solution") may cause abnormalities in the transcoded HDR images or HDR videos, affecting the user experience.

[0005] Therefore, how to provide a more flexible and effective encoding / decoding solution is a problem that needs to be solved urgently in this field. Summary of the invention

[0006] The present application provides an encoding method, a decoding method and related devices, which can realize flexible format conversion and improve encoding and decoding efficiency and final synthesis effect.

[0007] In a first aspect, an embodiment of the present application provides an encoding method, the method comprising: acquiring a first HDR image and N first base layer data corresponding to the first HDR image; determining N first enhancement layer data according to the first HDR image, the N first base layer data and at least one encoding scheme, where N is a positive integer greater than or equal to 1; encoding the N first enhancement layer data, the N first base layer data and first metadata to obtain a code stream, the first metadata comprising first indication information, the first indication information being used to indicate the at least one encoding scheme or a decoding scheme corresponding to the at least one encoding scheme.

[0008] In the above technical method, the information required for decoding is carried by the first metadata, and the first metadata and the data of the HDR image are encoded together to obtain a code stream. In this way, the decoding end device can determine the decoding scheme according to the first metadata, and then use the determined decoding scheme to decode the received data. Since the decoding end device can know the decoding scheme, the encoding end can flexibly select the encoding scheme as needed, thereby improving the encoding efficiency and the final synthesis effect.

[0009] In combination with the first aspect, in a possible implementation manner of the first aspect, determining N first enhancement layer data according to the first HDR image, the N first basic layer data and at least one encoding scheme includes: determining N intermediate enhancement data according to the first HDR image and the N first basic layer data; encoding the N intermediate enhancement data according to the at least one encoding scheme to obtain the N first enhancement layer data.

[0010] In combination with the first aspect, in a possible implementation manner of the first aspect, encoding the N intermediate enhancement data according to the at least one encoding scheme to obtain the N first enhancement layer data includes: using N i The i-th intermediate enhancement data in the N intermediate enhancement data is encoded by a coding scheme to obtain the i-th first enhancement layer data in the N first enhancement layer data, wherein the i-th first enhancement layer data includes N i area, use N i The encoding scheme for encoding the i-th intermediate enhanced data includes: using the N i The coding scheme has an effect on the N i Regions are encoded, N i is a positive integer greater than or equal to 2, i=1,…,N.

[0011] The above technical solution divides the enhancement layer data into multiple regions and encodes each region using a corresponding encoding scheme, so that a suitable encoding scheme can be selected according to the characteristics of different regions, thereby further improving the encoding efficiency and the final synthesis effect.

[0012] In combination with the first aspect, in a possible implementation manner of the first aspect, the at least one coding scheme includes M groups of coding schemes, and the i-th group of coding schemes in the M groups of coding schemes includes N i A coding scheme, M is a positive integer greater than or equal to 1.

[0013] In combination with the first aspect, in a possible implementation manner of the first aspect, the at least one encoding scheme includes M encoding schemes, and according to the at least one encoding scheme, the N intermediate enhancement data are encoded to obtain the N first enhancement layer data, including: using the i-th encoding scheme among the M encoding schemes, encoding the i-th intermediate enhancement data among the N intermediate enhancement data to obtain the i-th first enhancement layer data among the N first enhancement layer data, M is a positive integer greater than or equal to 1, i=1,…,N.

[0014] Through the above technical solution, a suitable encoding scheme can be selected for different intermediate enhancement layer data, thereby improving the encoding efficiency and the final synthesis result.

[0015] In combination with the first aspect, in a possible implementation manner of the first aspect, the determining N intermediate enhancement data according to the first HDR image and the N first basic layer data includes: determining N reference intermediate enhancement data according to the first HDR image and the N first basic layer data, wherein the reference intermediate enhancement data is multi-channel data; determining the i-th intermediate enhancement data among the N intermediate enhancement data according to the i-th reference intermediate enhancement data among the N reference intermediate enhancement data, wherein the intermediate enhancement data is single-channel data, i=1,…,N.

[0016] In combination with the first aspect, in a possible implementation manner of the first aspect, determining the i-th intermediate enhancement data among the N intermediate enhancement data based on the i-th reference intermediate enhancement data among the N reference intermediate enhancement data includes: determining that the i-th intermediate enhancement data includes data of a channel of the i-th reference intermediate enhancement data.

[0017] The above technical solution selects the encoding scheme based on the channel in the multi-channel data as the granularity, thereby improving the encoding efficiency and the final synthesis result.

[0018] In combination with the first aspect, in a possible implementation manner of the first aspect, determining, according to the i-th reference intermediate enhancement data among the N reference intermediate enhancement data, the i-th intermediate enhancement data among the N reference intermediate enhancement data includes: transforming multi-channel data of the i-th reference intermediate enhancement data respectively to obtain multiple transformation results, the multiple transformation results correspond to the multi-channel data one-to-one, and each transformation result of the multiple transformation results is obtained after the corresponding channel is transformed; and determining that the i-th intermediate enhancement data is one of the multiple transformation results.

[0019] By changing and then encoding the multi-channel data, the coding efficiency and the final synthesis result can be further improved.

[0020] In combination with the first aspect, in a possible implementation manner of the first aspect, determining the i-th intermediate enhancement data among the N intermediate enhancement data based on the i-th reference intermediate enhancement data among the N reference intermediate enhancement data includes: determining the i-th intermediate enhancement data based on a relationship function and multi-channel data of the i-th reference intermediate enhancement data.

[0021] In combination with the first aspect, in a possible implementation manner of the first aspect, the i-th intermediate enhancement data includes values ​​of multiple pixels, each pixel in the multiple pixels corresponds to multiple groups of pixels, the multiple groups of pixels respectively belong to multi-channel data of the i-th reference intermediate enhancement data, each group of pixels in the multiple groups of pixels includes at least one pixel, and the pixel value of each pixel in the multiple pixels is determined based on the pixel values ​​of the corresponding multiple groups of pixels and the relationship function.

[0022] In combination with the first aspect, in a possible implementation manner of the first aspect, the at least one coding scheme includes one or more of the following coding schemes: PQ coding, HLG coding, gamma coding, log coding, or arbitrary curve coding.

[0023] In combination with the first aspect, in a possible implementation of the first aspect, the first indication information is used to indicate: at least one encoding function; the at least one encoding function and the parameters of each encoding function in the at least one encoding function; the at least one decoding function; or, the at least one decoding function and the parameters of each decoding function in the at least one decoding function.

[0024] In a second aspect, an embodiment of the present application provides a decoding method, the method comprising: decoding an acquired code stream to obtain N first enhancement layer data, N first base layer data and first metadata, wherein the first metadata includes first indication information, the first indication information is used to determine at least one decoding scheme, N is a positive integer greater than or equal to 1; according to the at least one decoding scheme, the N first base layer data and the N first enhancement layer data, determine a second HDR image.

[0025] In the above technical method, the information required for decoding is carried by the first metadata, and the first metadata and the data of the HDR image are encoded together to obtain a code stream. In this way, the decoding end device can determine the decoding scheme according to the first metadata, and then use the determined decoding scheme to decode the received data. Since the decoding end device can know the decoding scheme, the encoding end can flexibly select the encoding scheme as needed, thereby improving the encoding efficiency and the final synthesis effect.

[0026] In combination with the second aspect, in a possible implementation manner of the second aspect, determining the second HDR image according to the at least one decoding scheme, the N first basic layer data and the N first enhancement layers, includes: decoding the N first enhancement layer data according to the at least one decoding scheme to obtain N second enhancement layer data; determining the second HDR image according to the N first basic layer data and the N second enhancement layer data.

[0027] In combination with the second aspect, in a possible implementation manner of the second aspect, the decoding of the N first enhancement layer data according to the at least one decoding scheme to obtain the N second enhancement layer data includes: respectively using N i A decoding scheme is used to decode the i-th first enhanced data among the N first enhanced data to obtain the i-th second enhanced layer data among the N second enhanced layer data, wherein the i-th first enhanced layer data includes N i area, use N iThe decoding scheme for decoding the i-th first enhanced data includes: using the N i The decoding scheme for the Nth first enhanced data i regions to decode, N i is a positive integer greater than or equal to 2, i=1,…,N.

[0028] The above technical solution divides the enhancement layer data into multiple regions and encodes each region using a corresponding encoding scheme, so that a suitable encoding scheme can be selected according to the characteristics of different regions, thereby further improving the encoding efficiency and the final synthesis effect.

[0029] In combination with the second aspect, in a possible implementation manner of the second aspect, the at least one decoding scheme includes M groups of decoding schemes, and the i-th group of decoding schemes in the M groups of decoding schemes includes the N i decoding schemes, and M is a positive integer greater than or equal to 1.

[0030] In combination with the second aspect, in a possible implementation of the second aspect, the at least one decoding scheme includes M decoding schemes, and according to the at least one decoding scheme, the N first enhancement layer data are decoded to obtain N second enhancement layer data, including: using the i-th decoding scheme among the M decoding schemes to decode the i-th first enhancement layer data among the N first enhancement layer data to obtain the i-th second enhancement layer data among the N second enhancement layer data, where M is a positive integer greater than or equal to 1, i=1,…,N.

[0031] Through the above technical solution, a suitable encoding scheme can be selected for different intermediate enhancement layer data, thereby improving the encoding efficiency and the final synthesis result.

[0032] In combination with the second aspect, in a possible implementation of the second aspect, the at least one decoding scheme includes a decoding scheme corresponding to one or more of the following encoding schemes: PQ encoding, HLG encoding, gamma encoding, log encoding, or arbitrary curve encoding.

[0033] In combination with the second aspect, in a possible implementation of the second aspect, the first indication information is used to indicate: at least one encoding function; the at least one encoding function and the parameters of each encoding function in the at least one encoding function; the at least one decoding function; or, the at least one decoding function and the parameters of each decoding function in the at least one decoding function.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, which includes a unit for implementing the first aspect or any possible implementation method of the first aspect.

[0035] Exemplarily, the electronic device may be an encoder.

[0036] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a unit for implementing the second aspect or any possible implementation method of the second aspect.

[0037] Exemplarily, the electronic device may be a decoder.

[0038] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor, wherein the processor is used to couple with a memory, read and execute instructions and / or program codes in the memory to execute the first aspect or any possible implementation method of the first aspect.

[0039] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a processor, wherein the processor is used to couple with a memory, read and execute instructions and / or program codes in the memory to execute the second aspect or any possible implementation method of the second aspect.

[0040] In the seventh aspect, an embodiment of the present application provides a chip system, which includes a logic circuit, which is used to couple with an input / output interface and transmit data through the input / output interface to execute the first aspect or any possible implementation method of the first aspect.

[0041] In an eighth aspect, an embodiment of the present application provides a chip system, which includes a logic circuit, which is used to couple with an input / output interface and transmit data through the input / output interface to execute the second aspect or any possible implementation method of the second aspect.

[0042] In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program code. When the program code stored in the computer storage medium is run on a computer, the computer executes the first aspect or any possible implementation of the first aspect.

[0043] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program code. When the program code stored in the computer storage medium is run on a computer, the computer executes the second aspect or any possible implementation of the second aspect.

[0044] In the eleventh aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, when the computer program code runs on a computer, enables the computer to execute the second aspect or any possible implementation of the second aspect.

[0045] In a twelfth aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, when the computer program code runs on a computer, enables the computer to execute the second aspect or any possible implementation of the second aspect.

[0046] In a thirteenth aspect, an embodiment of the present application provides a code stream, which is determined according to the first aspect or any possible implementation method of the first aspect.

[0047] In a fourteenth aspect, an embodiment of the present application provides a bitstream, which includes N base layer data, N enhancement layer data and first metadata corresponding to a first HDR image, and the first metadata is used to indicate at least one encoding scheme for encoding the N enhancement layer data or a decoding scheme corresponding to the at least one encoding scheme.

[0048] In a fifteenth aspect, an embodiment of the present application provides a device for storing a code stream, wherein the device is used to store the code stream of the thirteenth aspect or the fourteenth aspect.

[0049] In the sixteenth aspect, an embodiment of the present application provides a device for transmitting a code stream, the device comprising: a transmitter and a receiver, the receiver being used to receive a code stream generated by the encoding method described in the first aspect above, and the transmitter being used to send the code stream to a client device via a transmission medium.

[0050] In the seventeenth aspect, an embodiment of the present application provides a device for transmitting a code stream, the device comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store the code stream generated according to the encoding method described in the first aspect above, the transmitter being used to obtain the code stream from the storage medium and send the code stream to a client device via a transmission medium.

[0051] In an eighteenth aspect, an embodiment of the present application provides a system for distributing code streams, the system comprising: at least one storage medium for storing code streams generated according to the encoding method described in the first aspect above, a streaming media server for obtaining a target code stream from the at least one storage medium and sending the target code stream to a client device, wherein the streaming media server comprises a content server or a content distribution server. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic diagram of dynamic range mapping.

[0053] Figure 2 This is the image of the PQ photoelectric transfer function.

[0054] Figure 3 This is the image of the HLG photoelectric transfer function.

[0055] Figure 4 This is the image of the SLF photoelectric transfer function.

[0056] Figure 5 A schematic block diagram of a coding solution system used in an embodiment of the present application.

[0057] Figure 6 A block diagram of a content providing system for implementing content distribution services used in an embodiment of the present application.

[0058] Figure 7 A schematic diagram of a streaming media system workflow applicable to an embodiment of the present application.

[0059] Figure 8 A schematic diagram describing an exemplary end-to-end process of the present application is provided.

[0060] Fig. 9 An exemplary streaming media system architecture of the present application is described.

[0061] Fig.10 A schematic diagram of a possible system architecture applicable to an embodiment of the present application.

[0062] Fig.11 It is a schematic diagram of a method for processing digital signals provided according to an embodiment of the present application.

[0063] Fig.12 It is a schematic flowchart of an encoding method provided according to an embodiment of the present application.

[0064] Fig.13 It is a schematic flowchart of a decoding method provided according to an embodiment of the present application.

[0065] Fig.14 It is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application.

[0066] Fig.15 It is a schematic structural block diagram of another electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0068] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0069] First of all, in order to facilitate understanding of the embodiments of the present application, some concepts or terms involved in the embodiments of the present application are explained.

[0070] Color value: A numerical value corresponding to a specific image primary color component (such as R, G, B, or Y).

[0071] Digital code value: A digital expression of an image signal. Digital code value is used to represent nonlinear primary color values.

[0072] Linear color value: A linear color value that is proportional to the light intensity. In an optional case, its value should be normalized to [0,1], referred to as E.

[0073] Nonlinear primary color value: Nonlinear primary color value is the normalized digital expression value of image information, which is proportional to the digital coding value. In an optional case, its value should be normalized to [0,1], referred to as E′.

[0074] Electro-optical transfer function (EOTF): A conversion relationship from nonlinear primary color values ​​to linear primary color values.

[0075] Metadata: Data carried in a video signal that describes the video source information.

[0076] Dynamic metadata: Metadata associated with each frame of an image that changes from frame to frame.

[0077] Static metadata: Metadata associated with an image sequence that remains constant within the image sequence.

[0078] Luma: represents the combination of nonlinear primary color signals, symbolized by Y'.

[0079] Luminance mapping: The luminance of the source image is mapped to the luminance of the target system.

[0080] Display adaptation: Processing of video signals to adapt the display characteristics of the target display.

[0081] Source picture: The image input to the HDR pre-processing stage.

[0082] Mastering display: A reference display used when editing and producing video signals to determine the effects of video editing and production.

[0083] Linear scene light signal: In HDR video technology, the HDR video signal with content as scene light refers to the scene light captured by the camera / camera sensor, which is generally a relative value; the linear scene light signal is encoded by hybrid log-gamma (HLG) to obtain the HLG signal, which is a scene light signal and is nonlinear; the scene light signal generally needs to be converted into a display light signal through OOTF before being displayed on the display device.

[0084] Linear display light signal: In HDR video technology, the HDR video signal with content as display light refers to the display light emitted by the display device, which is generally an absolute value in nits. The linear display light signal is encoded by perceptual quantization (PQ) to obtain a PQ signal, which is a display light signal and a nonlinear signal. The display light signal is generally displayed on the display device according to its absolute brightness.

[0085] Opto-optical transfer function (OOTF): A curve used in video technology to convert one optical signal into another.

[0086] Dynamic range: The ratio of the maximum brightness to the minimum brightness in a video signal.

[0087] Luma-chroma-chroma (LCC), separates the three components of a video signal into luminance and chrominance.

[0088] The optical-electro transfer function (OETF) represents the conversion relationship between the linear signal and the nonlinear signal of an image pixel. The commonly used OETFs at this stage include the following three:

[0089] Perceptual quantizer (PQ) photoelectric transfer function, hybrid log-gamma (HLG) photoelectric transfer function and scene luminance fidelity (SLF) photoelectric transfer function. These three photoelectric transfer functions are photoelectric transfer functions specified by the audio video coding standard (AVS).

[0090] Dynamic range is used in many fields to express the ratio of the maximum value to the minimum value of a variable. In digital images, the dynamic range represents the ratio between the maximum grayscale value and the minimum grayscale value within the displayable range of the image. The dynamic range in nature is quite large. The brightness of a night scene under the starry sky is about 0.001cd / m2, and the brightness of the sun itself is as high as 1,000,000,000cd / m2. Such a dynamic range reaches 1,000,000,000 / 0.001=1013. However, in real scenes in nature, the brightness of the sun and the brightness of the stars are not obtained at the same time. For natural scenes in the real world, the dynamic range is between 10 -3 to 10 6 In most color digital images, each channel of R, G, and B is stored in one byte of 8 bits. That is to say, the range of each channel is 0 to 255 gray levels. Here, 0 to 255 is the dynamic range of the image. -3 to 10 6 We call it high dynamic range (HDR), while the dynamic range of ordinary pictures is called low dynamic range (LDR). The imaging process of digital cameras is actually the mapping of the high dynamic range of the real world to the low dynamic range of photos. This is often a nonlinear process.

[0091] Figure 1 is a schematic diagram of dynamic range mapping.

[0092] The PQ photoelectric transfer function is a perceptual quantization photoelectric transfer function proposed based on the brightness perception model of the human eye. Figure 2 , Figure 2is the image of the PQ photoelectric transfer function. The PQ photoelectric transfer function represents the conversion relationship from the linear signal value of the image pixel to the nonlinear signal value in the PQ domain.

[0093] The HLG photoelectric transfer function is an improvement on the traditional gamma curve. Figure 3 , Figure 3 This is an image of the HLG photoelectric transfer function. The HLG photoelectric transfer function uses the traditional gamma curve in the low segment and supplements the log curve in the high segment. The HLG photoelectric transfer function represents the conversion relationship between the linear signal value of the image pixel and the nonlinear signal value in the HLG domain.

[0094] The SLF photoelectric transfer function is the optimal curve obtained based on the brightness distribution of the HDR scene while satisfying the optical characteristics of the human eye. Figure 4 , Figure 4 is the image of SLF photoelectric transfer function. SLF photoelectric transfer curve represents the conversion relationship from linear signal value of image pixel to nonlinear signal value in SLF domain.

[0095] Linear space: In this application, linear space refers to the space where linear optical signals are located.

[0096] Nonlinear space: In this application, nonlinear space refers to the space where linear light signals are converted using nonlinear curves; nonlinear curves commonly used in HDR include PQ EOTF-1 curve, HLG OETF curve, etc., and nonlinear curves commonly used in SDR include gamma curves; it is generally believed that linear light signals are visually linear relative to the human eye after being encoded by the above nonlinear curves. It should be understood that nonlinear space can be considered as visual linear space.

[0097] Gamma correction: Gamma correction is a method of nonlinear color editing of images. It can detect the dark and light parts of the image signal and increase the ratio of the two, thereby improving the image contrast effect. The photoelectric conversion characteristics of current display screens, photographic films and many electronic cameras can be nonlinear. The relationship between the output and input of these nonlinear components can be expressed by a power function, that is, output = (input) γ.

[0098] The nonlinear conversion of the color values ​​output by the device is due to the fact that the human visual system is not linear, and humans perceive visual stimuli through comparison. The external world strengthens the stimulus in a certain proportion, and for humans, this stimulus increases evenly. Therefore, for human perception, the physical quantity that increases in geometric progression is uniform. In order to display the input color according to the laws of human vision, it is necessary to convert the linear color value into a nonlinear color value through the nonlinear conversion in the form of the above power function. The value of gamma γ can be determined according to the photoelectric conversion curve of the color space.

[0099] Color space: Color can be the different perceptions of the eye for light of different frequencies, or it can represent objectively existing light of different frequencies. Color space is a range of colors defined by a coordinate system established by people to represent colors. A color gamut and a color model together define a color space. Among them, a color model is an abstract mathematical model that represents color with a set of color components. A color model may include, for example, a three-primary color light model (red green blue, RGB) and a four-color printing model (cyan magenta yellow key plate, CMYK). A color gamut refers to the sum of colors that a system can produce. Exemplarily, Adobe RGB and sRGB are two different color spaces based on the RGB model. Each device, such as a monitor or printer, has its own color space and can only generate colors within its color gamut. When an image is moved from one device to another, the color of the image may change on different devices because each device converts and displays RGB or CMYK according to its own color space.

[0100] RGB space refers to a space that uses the brightness of the three primary colors of red, green, and blue to quantitatively represent video signals; YCC space is a color space that represents the separation of brightness and color. The three components of the YCC space video signal represent brightness, chromaticity, and chromaticity respectively. Common YCC space video signals include YUV, YCbCr, ICtCp, etc.

[0101] In order to obtain images with a higher dynamic range, the bit width of the image is usually greater than or equal to 10 bits. Common coding standards that support HDR include H.266, H.265 and high efficiency image format (HEIF). The common joint photographic experts group (JPEG) and H.264 only support 8-bit encoding, so they cannot support HDR video and HDR images well.

[0102] HDR images or HDR videos may need to be transcoded during distribution or transmission. The current HDR transcoding solution is to directly transcode HDR images or HDR videos. After decoding, HDR images or HDR videos obtain HDR pixel values ​​and HDR format identifiers and information. HDR pixel values ​​usually need to be processed (such as image up and down sampling, enhancement, etc.). HDR format identifiers and information, as well as processed HDR pixel values ​​are then sent to the encoder for re-encoding. The re-encoded HDR images or HDR videos are distributed to other devices. This solution of directly transcoding HDR images or HDR videos (which can be called a "single-layer transcoding solution") may cause abnormalities in the transcoded HDR images or HDR videos, affecting the user experience. Because when a single-layer coded HDR bitstream is transcoded (decoded and then re-encoded), it depends on the transcoding system to correctly send all the HDR information obtained by the decoder to the encoder. If the information is lost, the transcoding system will fail to re-encode in the correct format, resulting in an erroneous bitstream, which will result in abnormal visual effects when the user watches, seriously affecting the user experience. Because many of the distribution and transcoding systems on the market have not been upgraded to support the correct transmission of HDR information from decoding to encoding, mainstream mobile phone and application manufacturers have received many user complaints in this regard.

[0103] For ease of description, the term "HDR object" is used in some embodiments of the present application. An HDR object can be a static HDR image (also referred to as an HDR image, HDR photo, HDR picture, etc.), an HDR video or other types of dynamic HDR images, or a frame of an HDR video or a dynamic HDR image.

[0104] It is understandable that, for the sake of ease of description, some embodiments of the present application take HDR images as an example to introduce the technical solution of the present application. However, it is understandable that these embodiments can be applied not only to HDR images, but also to other HDR objects, such as HDR videos, dynamic HDR images, and a frame of an HDR video or dynamic HDR image.

[0105] The following combination Figure 5 Describe the coding and decoding system used in this application, Figure 5 A schematic block diagram of a coding and decoding system used in an embodiment of the present application is shown, for example, a video coding and decoding system 10 (or simply referred to as coding and decoding system 10) that can utilize the technology of the present application. The video encoder 20 (or simply referred to as encoder 20) and the video decoder 30 (or simply referred to as decoder 30) in the video coding and decoding system 10 represent devices that can be used to perform various technologies according to various examples described in the present application.

[0106] like Figure 5As shown, the codec system 10 includes a source device 12, which is used to provide coded data such as coded image data 21 to a target device 14 for decoding the coded data.

[0107] The source device 12 includes an encoder 20 . Optionally, the source device 12 may further include an image source 16 , a preprocessor 18 (or a preprocessing unit), and a communication interface or a communication unit 22 .

[0108] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0109] In order to distinguish the processing performed by the pre-processor 18 or the pre-processing unit 18 , the image or the image data 17 may also be referred to as a raw image or raw image data 17 .

[0110] The preprocessor 18 is used to receive (raw) image data 17 and preprocess the image data 17 to obtain preprocessed image data 19 or preprocessed image data 19. For example, the preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color adjustment, or denoising. It is understood that the preprocessing unit 18 may be an optional component.

[0111] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 .

[0112] The communication interface 22 in the source device 12 may be used to receive the encoded image data 21 and send the encoded image data 21 (or any other processed version thereof) to another device such as the target device 14 or any other device via the communication channel 13 for storage or reconstruction.

[0113] The target device 14 includes a decoder 30 (eg, a video decoder 30 ) and, in addition or alternatively, may include a communication interface or communication unit 28 , a post-processor 32 (or a post-processing unit 32 ), and a display device 34 .

[0114] The communication interface 28 in the target device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, the storage device is a encoded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0115] The communication interface 22 and the communication interface 28 can be used to send or receive the encoded image data 21 or the encoded data through a direct communication link between the source device 12 and the target device 14, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any combination thereof. For example, the communication interface 22 can be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or use any type of transmission coding or processing to process the encoded image data for transmission over a communication link or communication network. The communication interface 28 corresponds to the communication interface 22, for example, and can be used to receive the transmission data, and use any type of corresponding transmission decoding or processing and / or decapsulation to process the transmission data to obtain the encoded image data 21. Both the communication interface 22 and the communication interface 28 can be configured as follows: Figure 5 A unidirectional communication interface, or a bidirectional communication interface, is indicated by an arrow pointing from the source device 12 to the corresponding communication channel 13 of the target device 14, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0116] The decoder 30 is configured to receive the encoded image data 21 and to provide decoded image data 31 or decoded images 31 .

[0117] The post-processor 32 in the target device 14 is used to post-process the decoded image data 31 (also called reconstructed image data) such as the decoded image to obtain the post-processed image data 33 such as the post-processed image. The post-processing performed by the post-processor 32 may include, for example, color format conversion (e.g., conversion from YCbCr to RGB), color adjustment, cropping or resampling, or any other processing for generating the decoded image data 31 for display by the display device 34 or the like.

[0118] The display device 34 in the target device 14 is used to receive the post-processed image data 33 to display the image to a user or viewer, etc. The display device 34 can be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or display. For example, the display screen can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0119] although Figure 5 The source device 12 and the target device 14 are shown as independent devices, but the device embodiment may also include both the source device 12 and the target device 14 or the functions of both the source device 12 and the target device 14, that is, both the source device 12 or the corresponding functions and the target device 14 or the corresponding functions. In these embodiments, the source device 12 or the corresponding functions and the target device 14 or the corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0120] According to the description, Figure 5 The presence and (exact) division of different units or functions in the source device 12 and / or the target device 14 shown may vary depending on the actual device and application, as will be apparent to the skilled person.

[0121] Combine the following Figure 6 Describe the content providing system of the content distribution service applied by this application, Figure 6 A block diagram of a content providing system for implementing a content distribution service used in an embodiment of the present application. The content providing system 2100 includes a capture device 2102, a terminal device 2106, and an (optional) display 2126. The capture device 2102 communicates with the terminal device 2106 via a communication link 2104. The communication link may include the above-mentioned communication channel 13. The communication link 2104 includes, but is not limited to, WiFi, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0122] The capture device 2102 may encode the captured data using the encoding method shown in the embodiment of the present application. Alternatively, the capture device 2102 may distribute the captured data to a streaming media server (not shown in the figure), which encodes the data and transmits the encoded data to the terminal device 2106. The capture device 2102 includes but is not limited to a camera, a smart phone or tablet computer, a computer or laptop computer, a video conferencing system, a personal digital assistant (PDA), a vehicle-mounted device, or any combination thereof. In some embodiments, the capture device 2102 may include the above-mentioned source device 12.

[0123] The terminal device 2106 in the content providing system 2100 receives and regenerates the decoded data. The terminal device 2106 can be a device with data receiving and recovery capabilities, such as a smart phone or tablet computer, a computer / laptop 2110, a network video recorder (NVR) / digital video recorder (DVR) 2112, a television 2114, a set top box (STB) 2116, a video conferencing system 2118, a video surveillance system 2120, a personal digital assistant 2122, a vehicle-mounted device 2124 or any combination thereof, or such a device capable of decoding the above-mentioned encoded data. In some embodiments, the terminal device 2106 may include the above-mentioned target device 14. The terminal device 2106 can decode the received data. In some embodiments, the terminal device 2106 can be a video playback application, a streaming media playback application, a streaming media playback platform or a live broadcast platform running on the terminal device.

[0124] For terminal devices with a display, such as a smartphone or tablet, computer or laptop 2110, NVR / DVR 2112, TV 2114, PDA 2122, or vehicle-mounted device 2124, the terminal device can send the decoded data to its display. For terminal devices without a display, such as STB 2116, video conferencing system 3118, or video surveillance system 2120, connect the device to an external display 2126 to receive and display the decoded data.

[0125] When each device in the system performs encoding or decoding, the image encoding device or image decoding device shown in the embodiments of the present application may be used.

[0126] Combine the following Figure 7 A streaming media system to which the embodiments of the present application are applicable is described. Figure 7 A schematic diagram of a streaming media system workflow applicable to an embodiment of the present application.

[0127] The streaming media system includes a content creation module to generate the required content data, such as video or audio. The streaming media system also includes a video encoding module to encode the generated content through an encoder. The streaming media system also includes a video stream transmission module to transmit the encoded video in the form of a code stream. Optionally, the format of the video stream can be converted into a code stream format of a common transmission protocol for OTT (over-the-top) devices, for example, the protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG HTTP dynamic adaptive streaming media (MPEG Dynamic Adaptive Streaming over HTTP, MPEG-DASH), Real-time Transport Protocol (RTP), Real-time Messaging Protocol (RTMP) or any combination thereof. Optionally, the video stream can be stored to store the original format of the video stream and / or the converted multiple code stream formats for easy use. Furthermore, the streaming media system also includes a video stream encapsulation module for encapsulating the video stream to generate an encapsulated video stream, which can be called a video streaming media package. Exemplarily, the video streaming media package can be generated according to the transcoded video stream or the stored video stream. Furthermore, the streaming media system also includes a content distribution network (CDN), which is used to distribute the video streaming media package to multiple OTT devices, such as mobile phones, computers, tablets, home projectors, etc.

[0128] It should be noted that video encoding, video streaming transmission, video stream transcoding, video stream storage, video streaming package generation and content distribution network can all be implemented on cloud servers.

[0129] Combine the following Figure 8 A schematic diagram describing an exemplary end-to-end process of the present application is provided.

[0130] like Figure 8As shown, the original video file (also called a master file) is obtained through material production (such as shooting a video, making a computer graphics (CG) video, etc.), editing, color grading and other procedures. Then, based on the original video file, the corresponding dynamic metadata is obtained. After encoding the original video file and the dynamic metadata, a compressed video is obtained. The compressed video is distributed / transmitted to the terminal device (for example, a computer, a set-top box, a mobile phone, a tablet computer, etc.). The terminal device decodes the compressed video to obtain a decompressed video, and then displays it to the user through a display device (for example, a monitor, a TV, etc.).

[0131] Combine the following Fig. 9 An exemplary streaming media system architecture of the present application is described. The streaming media system architecture includes: a client device, a content distribution network, and a cloud server.

[0132] The user on the client device sends a play or playback request to the cloud platform.

[0133] The cloud platform makes a decision, responds to the client device, and sends the address of the client-requested content on the CDN to the client device.

[0134] The client device then requests the CDN to play the content based on the address, and the CDN provides the content to the client device, ultimately completing the request of the client device.

[0135] Combine the following Fig.10 Describe the system architecture applicable to the embodiments of the present application. Fig.10 This is a schematic diagram of a possible system architecture applicable to an embodiment of the present application. The system architecture of the embodiment of the present application includes: a front-end device, a transmission link, and a terminal display device.

[0136] Among them, the front-end device is used to capture or produce HDR / SDR content (for example, HDR / SDR video or image).

[0137] In a possible embodiment, the front-end device can also be used to extract corresponding metadata from the HDR content, and the metadata may include global mapping information, local mapping information, and dynamic metadata and static metadata corresponding to the HDR content. The front-end device can send the HDR content and metadata to the terminal display device through a transmission link. Specifically, the HDR content and metadata can be transmitted in the form of one data packet, or in two data packets respectively, which is not specifically limited in the embodiments of the present application.

[0138] Optionally, the terminal display device can be used to receive metadata and HDR content, and extract the global mapping information, local mapping information and the terminal display device information contained in the corresponding metadata according to the HDR content, obtain a mapping curve to perform global tone mapping and local tone mapping on the HDR content, convert it into display content adapted for the HDR display device or SDR device in the terminal display device, and display it. It should be understood that in different embodiments, the terminal display device may include a display device with a display capability of a lower dynamic range or a higher dynamic range than the HDR content generated by the front-end device, and this application is not limited thereto.

[0139] Optionally, the front-end device and the terminal display device in the present application may be independent and different physical devices. For example, the front-end device may be a video acquisition device or a video production device, wherein the video acquisition device may be a video camera, a camera, an image plotter, etc. The terminal display device may be a device with a video playback function, such as virtual reality (VR) glasses, a mobile phone, a tablet, a television, a projector, etc.

[0140] Optionally, the transmission link between the front-end device and the terminal display device can be a wireless connection or a wired connection, wherein the wireless connection can adopt technologies including, for example, long term evolution (LTE), fifth generation (5G) mobile communications, and future mobile communications. Wireless connections can also include technologies such as wireless fidelity (WiFi), Bluetooth, and near field communication (NFC). Wired connections can include Ethernet connections, local area network connections, etc. There is no specific limitation on this.

[0141] The present application may also integrate the functions of the front-end device and the functions of the terminal display device on the same physical device, for example, a terminal device such as a mobile phone or a tablet with a video shooting function. The present application may also integrate some functions of the front-end device and some functions of the terminal display device on the same physical device. There is no specific limitation on this.

[0142] Fig.11 Schematic diagram of a method for processing digital signals provided in accordance with an embodiment of the present application. Fig.11 As shown, the method for processing digital signals provided in the embodiment of the present application can be applied at the decoding end.

[0143] The decoding module (also referred to as a decoder) can obtain the base layer data, the enhancement layer data and the metadata from the received bitstream. Then, the decoding module can synthesize the base layer data and the enhancement layer data to obtain an HDR image.

[0144] The graphics processing module can process the HDR image and then send the processed HDR image to the display module. The graphics processing module can also process the base layer data and then send the processed base layer data to the display module. The graphics processing module can also process the base layer data and the HDR image and then send the processed HDR image and the processed base layer data to the display module.

[0145] The display module may present an HDR image based on the received data (eg, a processed HDR image, or a processed HDR image and processed base layer data, etc.).

[0146] For the sake of ease of description, the HDR image obtained by the decoding module based on the basic layer data and the enhancement layer data can be called HDR image 1, the HDR image obtained after the graphics processing module processes the HDR image 1 can be called HDR image 2, and the HDR image displayed by the display module can be called HDR image 3.

[0147] The base layer data, which may also be referred to as a base layer image, a base image or a basic image, may be an SDR image or an HDR image with a lower dynamic range.

[0148] The enhanced layer data, which may also be referred to as enhanced data, enhanced layer image or enhanced image, may include some image detail information. Thus, the image detail information contained in the enhanced layer data is used to supplement the base layer data, thereby synthesizing an HDR image with better contrast (i.e., the HDR image 1 mentioned above).

[0149] The dynamic range of the base layer data is smaller than the dynamic range of the HDR image (ie, HDR image 1) determined based on the base layer data and the enhancement layer data.

[0150] For example, in some embodiments, the base layer data may be an SDR image, and HDR image 1 is an HDR image.

[0151] For another example, in some other embodiments, the base layer data may be an HDR image, but the dynamic range of the HDR image is smaller than the dynamic range of the HDR image 1 .

[0152] For example, assuming the dynamic range of the base layer data is DR x1 To DR x2 , the dynamic range of HDR image 1 is DR y1 To DR y2 In some embodiments, DR y1 Less than DR x1 , DR y2 Greater than DR x2 In other embodiments, DR y1Less than DR x1 , DR y2 Equal to DR x2 In other embodiments, DR y1 Equal to DR x1 , DR y2 Greater than DR x2 In other words, the base layer data can be considered to be an image having a smaller dynamic range than the HDR image 1.

[0153] Fig.12 is a schematic flow chart of an encoding method provided according to an embodiment of the present application. Fig.12 The method shown can be performed by an electronic device or a component in an electronic device (such as a chip, a system on chip (SoC), etc.). The electronic device can be as follows Figure 5 The source device 12 shown, Figure 6 The capture device 2102 shown or Fig.10 For ease of description, the following embodiments assume that Fig.12 The method shown is performed by an electronic device.

[0154] 1201. Obtain a first HDR image and N first basic layer data corresponding to the first HDR image.

[0155] The first HDR image can be directly acquired by the device camera, or it can be synthesized by using multiple low dynamic range images with different exposure values. The embodiment of the present application does not limit the method for acquiring the first HDR image. The embodiment of the present application also does not limit the data format of the first HDR image. In terms of color space, the color space of the first HDR image can be YUV or RGB. In terms of the bit width of the data, the bit width of the first HDR image can be 8 bits, 10 bits, or 12 bits, etc. The photoelectric conversion characteristics of the first HDR image can be PQ, gamma, log, HLG, etc.

[0156] The first basic layer data may also be referred to as first basic layer image data. The first basic layer data corresponds to the first HDR image.

[0157] In some embodiments, the camera may simultaneously capture the first HDR image and the corresponding base layer image data.

[0158] In other embodiments, the corresponding basic layer image data can be obtained by processing the first HDR image. For example, the first HDR image is acquired by a camera or synthesized from multiple SDR images. In this case, the first HDR image is processed by tone mapping, neural network, etc. to obtain the basic layer image data corresponding to the first HDR image.

[0159] For the convenience of description, the basic layer image data corresponding to the first HDR image acquired by the camera can be referred to as basic layer acquisition scheme 1, and the basic layer image data corresponding to the first HDR image acquired by processing the first HDR image can be referred to as basic layer acquisition scheme 2. It can be understood that basic layer acquisition scheme 1 and basic layer acquisition scheme 2 are only examples of acquisition schemes that can acquire basic layer image data, rather than limiting the basic layer image data acquisition scheme. For the convenience of description, the basic layer image data acquired by the acquisition schemes such as the above-mentioned basic layer acquisition scheme 1 or basic layer acquisition scheme 2 can be referred to as reference basic layer data.

[0160] In some embodiments, the reference base layer data may be directly used as the first base layer data.

[0161] In other embodiments, the first basic layer data may be obtained by processing the reference basic layer data. The specific process of determining the first basic layer data based on the reference basic layer data is similar to the process of determining the first enhanced layer data based on the intermediate enhanced data. For example, the reference basic layer data may be encoded using PQ encoding, HLG encoding, gamma encoding, log encoding, or arbitrary curve encoding to obtain the first basic layer data. The specific determination method may refer to the description in the following embodiments. It can be understood that in the case where the first basic layer data is obtained after encoding the reference basic layer data, the first metadata may also carry a decoding scheme for decoding the first basic layer data and / or an encoding scheme for encoding the first basic layer data.

[0162] Similarly, the embodiment of the present application does not limit the format of the first basic layer data. For example, the color space of the first basic layer data can be YUV or RGB. The bit width of the first basic layer data can be 8 bits, 10 bits, or 12 bits, etc. The photoelectric conversion characteristics of the first basic layer data can be PQ, gamma, log, HLG, etc.

[0163] N is a positive integer greater than or equal to 1. That is, in some embodiments, a first HDR image and a first base layer data may be acquired. In other embodiments, a first HDR image and a plurality of first base layer data may be acquired.

[0164] 1202. Determine N first enhancement layer data according to the first HDR image, the N first basic layer data, and at least one first encoding scheme.

[0165] As described above, N is a positive integer greater than or equal to 1. The number of first enhancement layer data is the same as the number of first base layer data. For example, if a first HDR image and a first base layer data are acquired in step 1201, then a first enhancement layer data can be determined based on the acquired first HDR image and the acquired first base layer data and at least one first encoding scheme. If a first HDR image and multiple base layer data (for example, two) are acquired in step 1201, then multiple first enhancement layer data can be determined based on the acquired first HDR image, two base layer data, and at least one encoding scheme.

[0166] 1203: Encode the N first enhancement layer data, the N first basic layer data, and the first metadata to obtain a code stream.

[0167] The first metadata includes first indication information, and the first indication information is used to indicate the at least one encoding scheme or a decoding scheme corresponding to the at least one encoding scheme. The decoding end device can decode the received encoded data according to the first indication information. Therefore, in some embodiments, the first indication information can also be referred to as decoding indication information.

[0168] In some embodiments, N intermediate enhancement data may be determined first, and then the N first enhancement layer data may be determined according to the N intermediate enhancement data.

[0169] In some embodiments, the N intermediate enhancement data may be determined based on the first HDR image and the N first base layer data.

[0170] In other embodiments, N data may be first determined based on the first HDR image and the N first basic layer data, and then the N data may be processed to obtain the N intermediate enhanced data. For ease of description, the N data may be referred to as N reference intermediate enhanced data. In other words, in some embodiments, the N intermediate enhanced data are N reference intermediate enhanced data. In other embodiments, the N intermediate enhanced data correspond one-to-one to the N reference intermediate enhanced data, and each of the N intermediate enhanced data is determined based on the corresponding reference intermediate enhanced data.

[0171] First, how to determine the reference intermediate enhancement data is introduced.

[0172] In some embodiments, the first HDR image may have a first base layer data (ie, N is equal to 1). In this case, a reference intermediate enhancement data may be determined based on the first HDR image and the first base layer data. For ease of description, HDRα[j] is used to represent the value of the j-th pixel in the first HDR image, baseα[j] is used to represent the value of the j-th pixel in the first base layer data, and Enhanceα[j] is used to represent the value of the j-th pixel in the reference intermediate enhancement data. Then in some embodiments, HDRα[j], baseα[j] and Enhanceα[j] satisfy the following relationship:

[0173] Enhance α[j]=HDR α[j] / f(base α[j]), (Formula 2.1).

[0174] In some other embodiments, HDRα[j], baseα[j] and Enhanceα[j] satisfy the following relationship:

[0175] Enhance α[j]=HDR α[j]-f(base α[j]), (Formula 2.2).

[0176] The f() in Formula 2.1 and Formula 2.2 can be a default conversion function or a conversion function specified by metadata, which is not limited in the embodiments of the present application. In addition, the embodiments of the present application do not limit the analytical expression of the conversion function. For example, in some embodiments, f(baseα[j])=baseα[j]. For another example, in other embodiments, f(baseα[j])=A×baseα[j], where parameter A can be a preset value or a value determined according to baseα[j]. For another example, in other embodiments, f(baseα[j])=baseα[j]+B, where parameter B can be a preset value or a value determined according to baseα[j].

[0177] In some other embodiments, the first HDR image may have multiple first base layer data (i.e., N is a positive integer greater than or equal to 2). In this case, multiple reference intermediate enhancement data may be determined based on the first HDR image and the multiple first base layer data. For ease of description, HDRα[j] is used to represent the value of the jth pixel in the first HDR image, and baseα is used to represent the value of the jth pixel in the first HDR image. i [j] represents the value of the jth pixel in the i-th first basic layer data among the plurality of first basic layer data, using Enhanceα i[j] represents the value of the jth pixel in the i-th intermediate enhancement data in the multiple reference intermediate enhancement data, where i is a positive integer greater than or equal to 1 and less than or equal to N, that is, i=1, ..., N. HDRα[j], baseα i [j] and Enhance α i [j] may satisfy the above formula 2.1 or formula 2.2. In other words, in some embodiments, HDRα[j], baseα i [j] and Enhance α i [j] satisfies the following relationship:

[0178] Enhanceα i [j] = HDRα[j] / f(baseα i [j]), (Formula 2.3).

[0179] In some other embodiments, HDRα[j], baseα i [j] and Enhance α i [j] satisfies the following relationship:

[0180] Enhanceα i [j] = HDRα[j]-f(baseα i [j]), (Formula 2.4).

[0181] For the description of f(), please refer to the scenario where N is equal to 1, and will not be repeated here.

[0182] As described above, in some embodiments, the reference intermediate enhancement data is equal to the intermediate enhancement data. In this case, determining N reference intermediate enhancement data is actually equivalent to determining N intermediate enhancement data. Therefore, N first enhancement layer data can be directly determined based on the N reference intermediate enhancement data. The method for determining N first enhancement layer data based on N reference intermediate enhancement data is the same as the method for determining N first enhancement layer data based on N intermediate enhancement data.

[0183] The following describes how to determine the intermediate enhancement data based on the reference intermediate enhancement data.

[0184] In some embodiments, a numerical range of reference intermediate enhanced data may be obtained. For example, a minimum value and / or a maximum value of the reference intermediate enhanced data may be obtained. The reference intermediate enhanced data is then mapped according to the acquired data to obtain intermediate enhanced data. For example, the minimum value may be mapped to a first preset value, and / or the maximum value may be mapped to a second preset value. Then, based on the mapping relationship between the minimum value and / or the maximum value, other values ​​of the reference intermediate enhanced data are mapped to obtain intermediate enhanced data. The first preset value is different from the second preset value. The first preset value may be less than the second preset value. For example, the first preset value is 0 and the second preset value is 1.

[0185] In other embodiments, a histogram of the reference intermediate enhanced data may be obtained, and then a mapping relationship TME() may be obtained using a histogram equalization method. The intermediate enhanced data is determined based on the mapping relationship. Enhanceα[j] is used to represent the value of the j-th pixel in the reference intermediate enhanced data, and EnhanceAfterα[j] is used to represent the value of the j-th pixel in the intermediate enhanced data. In some embodiments, Enhanceα[j] and EnhanceAfterα[j] satisfy the following relationship:

[0186] EnhanceAfterα[j]=TME(Enhanceα[j]), (Formula 3.1).

[0187] In some other embodiments, Enhanceα[j] and EnhanceAfterα[j] satisfy the following relationship:

[0188] EnhanceAfterα[j]=TME()×Enhanceα[j], (Formula 3.2).

[0189] In some embodiments, the reference intermediate enhanced data may be multi-channel data. In this case, the single-channel data may be determined based on the multi-channel data to obtain the intermediate enhanced data.

[0190] In some embodiments, the intermediate enhanced data may be data of one channel of the corresponding reference intermediate enhanced data. For example, assuming that the reference intermediate enhanced data includes data of three channels (R channel data, G channel data, and B channel data), then the corresponding intermediate enhanced data may be data of one channel of the three channels. For example, the intermediate enhanced data may be R channel data. In other words, the reference intermediate data and the intermediate enhanced data satisfy the following relationship:

[0191] EnhanceAfter[Single-Channel]=Enhance[Channel 1 / 2 / … / P], (Formula 4.1)

[0192] Among them, EnhanceAfter[Single-Channel] represents the intermediate enhanced data, Enhance[Channel1 / 2 / … / P] represents the data of one channel among channels 1 / 2 / … / P in the reference intermediate enhanced data, and P is a positive integer greater than or equal to 2.

[0193] In other embodiments, the multi-channel data of the reference intermediate enhanced data may be transformed respectively to obtain multiple transformation results, and one of the transformation results is determined to be the intermediate enhanced data corresponding to the reference intermediate enhanced data. For example, assuming that the reference intermediate enhanced data includes data of three channels (R channel data, G channel data, and B channel data), the reference intermediate enhanced data may be transformed using a conversion function to obtain the transformed reference intermediate enhanced data. Then, the value of one channel in the transformed reference intermediate enhanced data is selected as the intermediate enhanced data. The relationship between the first intermediate enhanced data, the second intermediate enhanced data, and the reference intermediate enhanced data may be expressed by Formula 4.2 and Formula 4.3:

[0194] Enhance2[Multi-Channel]=f(Enhance[Multi-Channel]), (Formula 4.2)

[0195] EnhanceAfter[Single-Channel]=Enhance2[Channel 1 / … / P], (Formula 4.3)

[0196] Wherein Enhance2[Multi-Channel] represents the reference intermediate enhancement data after the transformation, f() represents the conversion function, Enhance[Multi-Channel] represents the reference intermediate enhancement data, EnhanceAfter[Single-Channel] represents the intermediate enhancement data, Enhance2[Channel 1 / … / P] represents the data of one channel from channel 1 to channel P in the transformed reference intermediate enhancement data, and P is a positive integer greater than or equal to 2.

[0197] In some embodiments, the reference intermediate enhancement data may be determined according to the relationship function and the reference intermediate enhancement data.

[0198] In some embodiments, the intermediate enhancement data includes values ​​of multiple pixels, each pixel in the multiple pixels corresponds to multiple groups of pixels, the multiple groups of pixels respectively belong to multi-channel data of the corresponding reference intermediate enhancement data, each group of pixels in the multiple groups of pixels includes at least one pixel, and the pixel value of each pixel in the multiple pixels is determined based on the pixel values ​​of the corresponding multiple groups of pixels and the relationship function.

[0199] For example, assuming that the reference intermediate enhancement data includes three channel data, pixel k is the kth pixel in the intermediate data. Then pixel k has three groups of corresponding pixels, and these three groups of pixels correspond one to one to the three channels. If the size of the reference intermediate enhancement data is the same as the size of the reference intermediate data, then each group of pixels in the three groups of pixels includes a pixel value. In other words, the three groups of pixels include three pixel values ​​in total, and these three pixel values ​​belong to the three groups of pixels respectively. These three pixel values ​​are the pixel values ​​of the kth pixel of the reference intermediate enhancement data in the three channels. The pixel value of pixel k can be determined based on these three pixel values ​​and the relationship function. The pixel value of pixel k and these three pixel values ​​can be expressed by the following relationship:

[0200] EnhanceAfter k [Single-Channel] = C (Enhance k [Channel 1], Enhance k [Channel2], Enhance k [Channel3]),

[0201] (Formula 5.1)

[0202] Among them EnhanceAfter k [Single-Channel] represents the pixel value of pixel k, Enhance k [Channel 1] refers to the pixel value of the kth pixel in channel 1 of the reference intermediate enhancement data. k [Channel 2] refers to the pixel value of the kth pixel in channel 2 of the reference intermediate enhancement data. k [Channel 3] represents the pixel value of the k-th pixel in channel 3 of the reference intermediate enhancement data, and C() represents the relationship function.

[0203] In some embodiments, the relationship function may be to take all element eigenvalues, such as maximum value, minimum value, average value, variance, etc. In other embodiments, the relationship function may be to transform or combine the above eigenvalues, such as weighted summation, etc.

[0204] In some embodiments, part of the N intermediate enhancement data may be determined according to corresponding reference intermediate enhancement data, while another part of the intermediate enhancement data may directly use the corresponding reference intermediate enhancement data.

[0205] After the N intermediate enhancement data are determined, the N first enhancement layer data may be determined according to the N intermediate enhancement data. The N first enhancement layer data may be obtained by encoding the N intermediate enhancement data.

[0206] In some embodiments, at least one encoding scheme may be used to encode the N intermediate enhancement data to obtain the N first enhancement layer data. The N first enhancement layer data correspond one-to-one to the N intermediate enhancement data, and each first enhancement layer data is the result of encoding the corresponding intermediate enhancement data.

[0207] In some embodiments, at least one encoding scheme may include only one encoding scheme. In this case, the one encoding scheme may be used to encode the N intermediate enhancement data respectively to obtain the N first enhancement layer data.

[0208] In other embodiments, the number of coding schemes is the same as the number of intermediate enhancement data. In other words, there may be N coding schemes. The N coding schemes correspond one-to-one to the N intermediate enhancement data. Each coding scheme is used to encode the corresponding intermediate enhancement data to obtain the first enhancement layer data. In other words, the i-th coding scheme among the N coding schemes can be used to encode the i-th intermediate enhancement data among the N intermediate enhancement data to obtain the i-th first enhancement layer data among the N first enhancement layer data, i=1, ..., N.

[0209] In some other embodiments, the number of encoding schemes may be less than the number of intermediate enhancement data. In this case, multiple intermediate enhancement data may use the same encoding scheme.

[0210] If N is equal to 1, then the N coding schemes can be any one of the following coding schemes: PQ coding, HLG coding, gamma coding, log coding, or arbitrary curve coding.

[0211] If N is a positive integer greater than or equal to 2, then the N coding schemes can be one or more of the above coding schemes (i.e., PQ coding, HLG coding, gamma coding, log coding, arbitrary curve coding). For example, if N is equal to 2, then the two coding schemes can be PQ coding and log coding, respectively. For another example, if N is equal to 2, then the two coding schemes can both be log coding and the parameters of the two coding schemes are different. For example, one of the two coding schemes is a log coding with a base of 2, and the other coding scheme is a log coding with a base of 3.

[0212] In some embodiments, the encoding scheme and specific parameters used by the encoding scheme (eg, the base in log encoding) may be pre-set.

[0213] In other embodiments, the coding scheme may be determined by the coding side. In this case, the coding scheme used may be indicated to the decoding side through the first indication information, so that the decoding side selects the corresponding decoding scheme to decode the received code stream, or the decoding scheme may be indicated to the decoding side using the first indication information, so that the decoding side can directly use the decoding scheme indicated by the first indication information to decode the received code stream.

[0214] For example, in some embodiments, the first indication information is used to indicate at least one decoding scheme, and the at least one decoding scheme corresponds one-to-one to the at least one encoding scheme. Each decoding scheme in the at least one decoding scheme is used to decode data encoded using the corresponding encoding scheme.

[0215] For another example, in some embodiments, the first indication information is used to indicate at least one decoding function and a decoding parameter of each decoding function in the at least one decoding function. The at least one decoding function corresponds one-to-one to the at least one coding scheme. The i-th decoding function in the at least one decoding function is used to decode the i-th first enhancement layer data in the at least one first enhancement layer data.

[0216] In some embodiments, the coding scheme may be any function curve. In this case, the intermediate enhancement data may be compressed and encoded using the function curve. In some embodiments, different intermediate enhancement data may be compressed and encoded using different curve shapes and parameters. In other embodiments, different intermediate enhancement data may be compressed and encoded using the same curve shape and parameters. The first indication information may indicate the curve shape and parameters used to compress the intermediate enhancement data.

[0217] In some embodiments, the encoding scheme may be a logarithmic (log) or exponential function. Taking the logarithmic function as an example, the logarithmic function may be used to compress and encode the intermediate enhancement data to obtain the first enhancement layer data.

[0218] In some embodiments, the parameters of the function used in the compression coding (e.g., the base of the logarithmic function) are pre-set or pre-negotiated. In this case, the first metadata indicates the type of function used for the compression coding. Exemplarily, it can be indicated that the type of function used for the compression coding is a logarithmic function or an exponential function.

[0219] In some other embodiments, the parameters of the function used in the compression coding are determined during compression. In this case, the first metadata may indicate the function used for the compression coding and the parameters used by the function.

[0220] In other embodiments, the function used for compression coding and the parameters of the function may be pre-set or pre-negotiated.

[0221] In some embodiments, one piece of intermediate enhancement data may be divided into multiple regions, and then the multiple regions may be encoded using multiple encoding schemes respectively.

[0222] In some embodiments, if N is a positive integer greater than or equal to 2, each of the N intermediate enhancement data may be divided into a plurality of regions.

[0223] In some other embodiments, if N is a positive integer greater than or equal to 2, some of the N intermediate enhancement data may be divided into multiple regions, while the remaining intermediate enhancement data may not be divided. The intermediate enhancement data that is not divided may be encoded with reference to the above scheme to obtain corresponding first enhancement layer data.

[0224] For the convenience of description, the following description is given by taking the division of each intermediate enhancement data as an example.

[0225] Assume that the i-th intermediate enhancement data among N intermediate enhancement data can be divided into N i Use N areas respectively. i The coding scheme for this N i The regions are encoded to obtain the i-th first enhancement layer data among the N first enhancement layer data. The i-th first enhancement layer data includes N i For the convenience of description, the area before encoding can be called the first area, and the area after encoding can be called the second area. In other words, the i-th intermediate enhancement data can be divided into N i The i-th first enhancement layer data includes N i Second area. Use N i The coding scheme for this N i The first area code is obtained i In other words, the N i The j-th second region among the second regions is i The jth coding scheme among the N coding schemes is used to i The first region is encoded. i is a positive integer greater than or equal to 2.

[0226] For example, assuming N=3, the first intermediate enhancement data can be divided into N1 first regions. The N1 first regions are encoded using N1 coding schemes to obtain N1 second regions. The first first enhancement layer data includes the N1 second regions. Similarly, the second intermediate enhancement data can be divided into N2 first regions. The N2 first regions are encoded using N2 coding schemes to obtain N2 second regions. The second first enhancement layer data includes the N2 second regions. The third intermediate enhancement data can be divided into N3 first regions. The N3 first regions are encoded using N3 coding schemes to obtain N3 second regions. The third first enhancement layer data includes the N3 second regions.

[0227] In some embodiments, the number of regions divided into any two intermediate enhanced data may be the same. Taking N=3 as an example, in this case, N1=N2=N3.

[0228] In some embodiments, the number of regions divided into any two intermediate enhanced data may be the same or different. Taking N=3 as an example, in this case, the relationship among N1, N2, and N3 may be any of the following: N1=N2≠N3, N1≠N2=N3, and N1≠N2≠N3.

[0229] In some embodiments, the N i Any two of the N coding schemes may be different coding schemes. For example, assuming that coding scheme 1 and coding scheme 2 are i There are two encoding schemes in the encoding schemes, encoding scheme 1 uses function curve encoding, and encoding scheme 2 uses logarithmic function encoding.

[0230] In other embodiments, the N i Any two of the N coding schemes can be the same coding scheme but with different parameters. For example, suppose coding scheme 1 and coding scheme 2 are the N i Two encoding schemes are used in the encoding schemes. Encoding scheme 1 and encoding scheme 2 both use logarithmic functions for encoding, but the base of encoding scheme 1 is 2, and the base of encoding scheme 2 is 3.

[0231] In other embodiments, the N i Any two of the N coding schemes can be the same coding scheme and have the same parameters. For example, suppose coding scheme 1 and coding scheme 2 are the N i Two coding schemes among the N coding schemes. Both coding scheme 1 and coding scheme 2 use logarithmic function encoding, and the base of coding scheme 1 and coding scheme 2 is 2. In this case, it can also be considered that the N i An encoding scheme is an encoding scheme.

[0232] In some embodiments, the first indication information may be used to indicate M groups of coding schemes. Each group of coding schemes in the M groups of coding schemes corresponds to one or more of the N intermediate enhancement data. A group of coding schemes corresponding to the i-th intermediate enhancement data in the N intermediate enhancement data includes N i The N i The coding schemes can be used to encode the N i The regions are encoded to obtain the i-th first enhancement layer data of the N first enhancement layer data. M may be a positive integer greater than 1.

[0233] If M is equal to N, the first indication information may be used to indicate N groups of coding schemes. The N groups of coding schemes correspond one-to-one to the N intermediate enhancement data. The i-th group of coding schemes in the N groups of coding schemes includes N i The N i The coding schemes are used to encode the N i Encode the area.

[0234] In the above-mentioned embodiments, the first indication information indicates a coding scheme. The decoding end device may determine a corresponding decoding scheme according to the coding scheme. Accordingly, in other embodiments, the first indication information may directly indicate a decoding scheme. In this way, the decoding end device may directly decode the corresponding data according to the decoding scheme indicated by the first indication information. For example, the first indication information may be used to indicate M groups of decoding schemes. Each group of decoding schemes in the M groups of decoding schemes corresponds to one or more of the N first enhancement layer enhancement data. A group of decoding schemes corresponding to the i-th first enhancement layer data in the N first enhancement layer data includes N i decoding scheme. i The decoding schemes may be respectively used for the N in the i-th first enhancement layer data. i Encode the area.

[0235] If M is equal to N, the first indication information may be used to indicate N groups of decoding schemes, wherein the i-th group of decoding schemes in the N groups of decoding schemes includes N i decoding scheme, the N i The decoding scheme and the N i There is a one-to-one correspondence between the N i The decoding schemes are respectively used to decode the N i Decode the area.

[0236] In some embodiments, the intermediate enhancement data is not processed and / or encoded, but is directly used as the first enhancement layer data.

[0237] It should be noted that the present application does not limit the first enhancement layer data, the intermediate enhancement data, and the reference intermediate enhancement data in any domain, which may be a linear domain or a PQ domain or a log domain, etc. The present application also does not limit the color space of the first enhancement layer data, the intermediate enhancement data, and the reference intermediate enhancement data, which may be a color space such as YUV, RGB, Lab, or HSV.

[0238] In addition, before and / or after determining the intermediate enhancement data and / or the first enhancement layer data, or after determining the reference intermediate enhancement data, color gamut mapping may be performed so that the data is converted from the current color gamut to the target color gamut. The data of the target color gamut is then processed. The current color gamut and the target color gamut include but are not limited to BT.2020, BT.709, DCI-P3, sRGB, etc.

[0239] The embodiments of the present application do not limit the codec. For example, codecs such as versatile video coding (VVC), joint photographic experts group (JPEG), high efficiency image format (HEIF), advanced video coding (AVC) / H.264, audio video coding standard (AVS), and high efficiency video coding (HEVC) can be used. The embodiments of the present application do not limit the location of the first metadata. For example, the first metadata can be placed in supplemental enhancement information (SEI), network abstraction layer (NAL) units, reserved packet units, application (APP) extension information, etc.

[0240] Each coding scheme in the at least one coding scheme may be a coding function. In some embodiments, the first indication information may indicate at least one coding function. For example, the first indication information may carry the at least one coding function. For another example, in some embodiments, each coding function may have an index. The first indication information may carry at least one index, and the at least one index corresponds to the at least one coding function one-to-one. In this way, the decoding end may determine the correspondence to at least one coding function through the at least one index, and then determine the decoding function according to the coding function.

[0241] In some embodiments, the coding function may also include one or more parameters. The first indication information may indicate the at least one coding function and the parameters of each coding function. Similarly, the first indication information may carry the at least one coding function and the parameters of each coding function. The first indication information also carries the index of each coding function and the parameters of each coding function. In this way, the decoding end can determine the coding function by the index in the first indication information, and determine the parameters of the coding function according to the parameters in the first indication information. In other embodiments, the parameters of the coding function may also have corresponding indexes. In this case, the first indication information may carry at least one first index and at least one second index, the at least one first index corresponds to the at least one coding function one-to-one, and the at least one second index corresponds to the parameters of the at least one coding function one-to-one. In this way, the decoding end device can determine the coding function by the first index and determine the parameters of the coding function by the second index.

[0242] In some embodiments, the first indication information may indicate a decoding function. In this way, the decoding end device may no longer need to determine the corresponding decoding function through the encoding function, thereby reducing the overhead of the decoding end device. For example, in some embodiments, the first indication information may carry at least one decoding function, and the at least one decoding function corresponds one-to-one with the at least one encoding scheme, and each decoding function is used to decode a code stream encoded using the corresponding encoding scheme. Similarly, in some embodiments, the first indication information may carry at least one index, and the at least one index corresponds one-to-one with the at least one decoding function. In this case, the decoding end device can determine the at least one decoding function through the at least one index carried by the first indication information.

[0243] In some embodiments, the decoding function may also include one or more parameters. The first indication information may carry at least one decoding function and the parameters of each decoding function. The first indication information may also carry the index of each decoding function and the parameters of each decoding function. In this way, the decoding end device may determine the decoding function according to the index of the decoding function, and determine the parameters of the decoding function according to the parameters carried in the first indication information. Similarly, in some embodiments, the parameters of each decoding function may also have a corresponding index. In this case, the first indication information may carry at least one third index and at least one fourth index, the at least one third index corresponding to at least one decoding function, and the at least one fourth index corresponding to the parameters of the at least one decoding function. In this way, the decoding end device may determine the decoding function by the third index and determine the parameters of the decoding function by the third index.

[0244] Fig.13 is a schematic flow chart of a decoding method provided according to an embodiment of the present application. Fig.13The method shown can be performed by an electronic device or a component (such as a chip, SoC, etc.) in the electronic device. The electronic device can be as follows Figure 5 The target device 14, such as Figure 6 The terminal device 2106 shown or Fig.10 For ease of description, the following embodiments assume that Fig.13 The method shown is performed by an electronic device.

[0245] 1301, decode the acquired bitstream to obtain N first enhancement layer data, N first base layer data and first metadata, where the first metadata includes first indication information, where the first indication information is used to indicate at least one decoding scheme. N is a positive integer greater than or equal to 1.

[0246] The code stream can be obtained by the encoding end device through Fig.12 For ease of description, assume that Fig.12 The method is performed by a first electronic device. Fig.13 The method shown is executed by a second electronic device.

[0247] In some embodiments, the first electronic device and the second electronic device may be different electronic devices. For example, in some embodiments, after the first electronic device completes the encoding, it may send the code stream to the second electronic device through a transmission link (e.g., optical fiber, WiFI, 3G / 4G / 5G network, etc.). The second electronic device may receive the code stream through the communication link. The first electronic device may also store the code stream in a storage device (e.g., a mobile hard disk, a USB flash drive, etc.). The second electronic device may read the storage device and obtain the code stream stored in the storage device.

[0248] In some other embodiments, the first electronic device and the second electronic device may also be the same electronic device. For example, a smartphone may shoot a video through a camera assembly, and then obtain corresponding first metadata based on the original video file. After encoding the original video file and the first metadata, a code stream is obtained. The code stream is encapsulated into a video file and saved in the built-in memory of the smartphone. The smartphone can read the video file and obtain the code stream. In this case, the first electronic device and the second electronic device are both the smartphone.

[0249] The encapsulation position of the first metadata is determined by the first electronic device during encoding. For example, in some embodiments, if the code stream is obtained by encoding using HEVC or VVC, the second electronic device can obtain the first metadata from the SEI of HEVC or VVC. For another example, in some embodiments, if the code stream is obtained by encoding using AVC / H.264, the second electronic device can obtain the first metadata from the NAL unit or the reserved packet unit. For another example, in some embodiments, the second electronic device can obtain the first metadata from the APP extension information encapsulated in the JPEG file interchange format. For another example, in some embodiments, the second electronic device can obtain the first metadata from the data segment encapsulated by the Moving Picture Experts Group (MPEG)-Part 14 (MPEG-4part14, MP4).

[0250] The first metadata may carry some general information, such as data format, region division information, region traversal order information, or image features, encoding compression format, curve parameters, etc., as well as one or more metadata information units. The metadata information unit may contain data such as coordinate information, image features, curve parameters, etc.

[0251] The first metadata may also include first indication information. The first indication information may indicate the at least one encoding scheme in two ways, directly or indirectly. For example, in some embodiments, the first indication information may include the at least one decoding scheme. In this way, the second electronic device may directly obtain the at least one decoding scheme from the first indication information. For example, in some other embodiments, the first indication information may include at least one encoding scheme, and each encoding scheme has a corresponding decoding scheme. In this way, the second electronic device can determine the at least one decoding scheme based on the at least one encoding scheme.

[0252] The second electronic device can decode the code stream based on the first metadata to obtain the first basic layer data. In some embodiments, the first basic layer data is the basic layer data obtained after decoding. In other words, the second electronic device obtains N basic layer data after decoding the code stream, and the N basic layer data can be used as the N first basic layer data. In other embodiments, the second electronic device can also convert the N basic layer data after decoding the code stream to obtain N basic layer data, and the converted basic layer data is the first basic layer data. The conversion method of the basic layer data can refer to the conversion method of the first enhancement layer data in the following embodiments, and for the sake of brevity, it will not be repeated here.

[0253] The embodiment of the present application does not limit the code stream format of the base layer data and the enhancement layer data (including the first enhancement layer data and the second enhancement layer data mentioned later). In terms of color space, it can be YUV, RGB, Lab, HSV, etc. In terms of data bit width, it can be 8 bits, 10 bits or 12 bits, etc. In terms of value domain, it can be PQ, HLG, gamma or log, etc.

[0254] 1302. Determine a second HDR image based on the N first base layer data and the N first enhancement layer data according to at least one decoding scheme.

[0255] Optionally, in some embodiments, the N first enhancement layer data may be first decoded according to the at least one decoding scheme to obtain N second enhancement layer data. Then, the second HDR image is determined according to the N first basic layer data and the N second enhancement layer data. The N first enhancement layer data correspond to the N second enhancement layer data one by one, and each second enhancement layer data in the N second enhancement layer data is obtained after decoding the corresponding first enhancement layer data.

[0256] In some embodiments, the at least one decoding scheme may include N decoding schemes. The N decoding schemes correspond one-to-one to the N first enhancement layer data. Each decoding scheme in the N decoding schemes is used to decode the corresponding first enhancement layer data to obtain the second enhancement layer data. For example, the i-th decoding scheme in the N decoding schemes can be used to decode the i-th first enhancement layer data in the N first enhancement layer data to obtain the i-th second enhancement layer data of the N second enhancement layer data.

[0257] For example, in some embodiments, the first enhancement layer data may be encoded using PQ / HLG / gamma / log compression. Then the second electronic device may use the corresponding decoding function to decode the first enhancement layer data to obtain the second enhancement layer data. As described above, in some embodiments, the coding scheme and the parameters in the coding scheme (such as the base of the log function) may be pre-agreed or preset by the encoding side and the decoding side. In this case, the second electronic device may directly decode the first enhancement layer data to obtain the second enhancement layer data. In other embodiments, the coding scheme and / or the parameters in the coding scheme are not predetermined or preset. The first metadata may carry the coding scheme and / or the parameters in the coding scheme. The second electronic device may determine the coding scheme and / or the parameters in the coding scheme from the metadata, and then decode the first enhancement layer data based on the determined coding scheme and / or the parameters in the coding scheme to obtain the second enhancement layer data.

[0258] As another example, in some embodiments, the first enhancement layer data may be compressed and encoded using an arbitrary function curve. The first metadata may carry an inverse function of the corresponding function. The second electronic device may decode the first enhancement layer data using the inverse function carried by the first metadata to obtain the corresponding second enhancement layer data. Of course, in other embodiments, the function curve used for compression encoding may also be agreed upon or preset in advance by both parties (i.e., the encoding side and the decoding side). In this case, the second electronic device may directly use the inverse function to decode the first enhancement layer data to obtain the second enhancement layer data.

[0259] For example, in some embodiments, the first enhancement layer data may be encoded and compressed in a format configurable by parameters such as logarithms or exponential functions. In this case, in addition to specifying a specific format, the first metadata also includes function parameter information that is flexibly configured according to different scenarios, such as the base of a logarithmic function, the power of an exponential function, etc. In this case, the second electronic device may determine the format and function parameter information based on the first metadata, and then decode the first enhancement layer data based on the determined format and parameter information to obtain the second enhancement layer data. Of course, in other embodiments, the format and / or function parameters of the function may be pre-agreed or preset. If the format and function parameters of the function are both pre-determined or preset, the second electronic device may directly decode the first enhancement layer data to obtain the second enhancement layer data. If one of the function format or function parameters is pre-agreed or preset, the other is carried by the first metadata. For example, in some embodiments, the function format is pre-agreed or preset. In this case, the second electronic device can obtain function parameters from the first metadata, and then decode the first enhancement layer data based on the implementation agreement or preset function format and the function parameters obtained from the first metadata to obtain the second enhancement layer data.

[0260] In some embodiments, if N is a positive integer greater than or equal to 2, any two decoding schemes among the N decoding schemes may be different decoding schemes. Different decoding schemes may include different schemes and / or different parameters. For example, assume that decoding scheme A and decoding scheme B are two decoding schemes among the N decoding schemes. Decoding scheme A is used to decode enhanced layer data encoded using a logarithmic function; decoding scheme B is used to decode enhanced layer data compressed and encoded using a function curve. For another example, decoding scheme A and decoding scheme B are both used to decode enhanced layer data encoded using a logarithmic function, but the base of the encoded logarithmic function corresponding to decoding scheme A is 2, and the base of the encoded logarithmic function corresponding to decoding scheme B is 3.

[0261] In other embodiments, if N is a positive integer greater than or equal to 2, any two decoding schemes among the N decoding schemes may be the same decoding scheme. For example, decoding scheme A and decoding scheme B are any two decoding schemes among the N decoding schemes. Both decoding schemes are used to decode enhancement layer data encoded using a logarithmic function with a base of 2.

[0262] In some embodiments, the first electronic device divides the data to be encoded into multiple regions when encoding, and then uses multiple encoding schemes to encode the multiple regions respectively. Correspondingly, the second electronic device can use multiple decoding schemes to decode multiple regions of the first enhancement layer data respectively to obtain the second enhancement layer data. The second enhancement layer data also includes multiple regions. For ease of description, the region of the first enhancement layer can be referred to as the third region, and the region of the second enhancement layer data can be the fourth region. In other words, the first enhancement layer data may include multiple third regions, and the second enhancement layer data may include multiple fourth regions. The multiple third regions correspond one-to-one to the multiple fourth regions, and the multiple third regions also correspond one-to-one to the multiple decoding schemes. Each fourth region in the multiple fourth regions is obtained after the corresponding third region is decoded using the corresponding decoding scheme.

[0263] For example, assuming that the i-th first enhancement layer data among N first enhancement layer data includes N i The third area, N i is a positive integer greater than or equal to 2. Then we can use N i The decoding scheme includes N i The third region is decoded to obtain the i-th second enhanced data among the N second enhanced data, the i-th second enhanced data including N i The fourth area. i The fourth area is to use the N i In other words, the N i The j-th fourth region in the fourth regions is i The jth encoding scheme among the N decoding schemes is used to i The third area is decoded. i is a positive integer greater than or equal to 2.

[0264] For example, assuming N=3, the first enhanced layer data can be divided into N1 third regions. The N1 third regions are decoded using N1 decoding schemes to obtain N1 fourth regions. The first second enhanced layer data includes the N1 fourth regions. Similarly, the second first enhanced layer data can be divided into N2 third regions. The N2 third regions are decoded using N2 decoding schemes to obtain N2 fourth regions. The second second enhanced layer data includes the N2 fourth regions. The third first enhanced layer data can be divided into N3 third regions. The N3 third regions are decoded using N3 decoding schemes to obtain N3 fourth regions. The third second enhanced layer data includes the N3 fourth regions.

[0265] In some embodiments, the number of regions divided into any two first enhancement layer data may be the same. Also taking N=3 as an example, in this case, N1=N2=N3.

[0266] In some embodiments, the number of regions divided into any two first enhancement layer data may be the same or different. Taking N=3 as an example, in this case, the relationship among N1, N2, and N3 may be any of the following: N1=N2≠N3, N1≠N2=N3, and N1≠N2≠N3.

[0267] Similarly, in some embodiments, the N i Any two decoding schemes among the N decoding schemes may be different decoding schemes. For example, assuming that decoding scheme 1 and decoding scheme 2 are i The present invention provides two decoding schemes among the decoding schemes, wherein the decoding scheme 1 is to decode the first enhancement layer data encoded by using a function curve, and the decoding scheme 2 is to decode the first enhancement layer data encoded by using a logarithmic function.

[0268] In other embodiments, the N i Any two decoding schemes in the N decoding schemes can be the same decoding scheme but with different parameters. For example, assuming that decoding scheme 1 and decoding scheme 2 are the N i Decoding scheme 1 and decoding scheme 2 are both for decoding the first enhancement layer data encoded using a logarithmic function, but the base of the logarithmic function corresponding to decoding scheme 1 is 2, and the base of the logarithmic function corresponding to decoding scheme 2 is 3.

[0269] In other embodiments, the N i Any two decoding schemes in the N decoding schemes can be the same decoding scheme and have the same parameters. For example, assuming that decoding scheme 1 and decoding scheme 2 are the N decoding schemes, iTwo decoding schemes in the N decoding schemes. Decoding scheme 1 and decoding scheme 2 are both decoding the first enhancement layer data encoded using a logarithmic function, and the base of the logarithmic function corresponding to encoding scheme 1 and encoding scheme 2 is 2. In this case, it can also be considered that the N i An encoding scheme is an encoding scheme.

[0270] In this case, the first indication information may be used to indicate N groups of decoding solutions, wherein the i-th group of decoding solutions in the N groups of decoding solutions includes N i decoding scheme, the N i The decoding scheme and the N i There is a one-to-one correspondence between the N i The decoding schemes are respectively used to decode the N i Decode the area.

[0271] Optionally, in some embodiments, the second HDR image may be directly determined according to the N second enhancement layer data and the N first basic layer data.

[0272] Assume that N is equal to 1, use HDRβ[j] to represent the value of the j-th pixel in the second HDR image, use baseβ[j] to represent the value of the j-th pixel in the first base layer data, and use Enhanceβ[j] to represent the value of the j-th pixel in the second enhancement layer data. Then in some embodiments, HDRβ[j], baseβ[j] and Enhanceβ[j] satisfy the following relationship:

[0273] HDRβ[j]=baseβ[j]×f(Enhanceβ[j]), (Formula 6.1).

[0274] In some other embodiments, HDRβ[j], baseβ[j] and Enhanceβ[j] satisfy the following relationship:

[0275] HDRβ[j]=baseβ[j]+f(Enhanceβ[j]), (Formula 6.2).

[0276] f() in Formula 6.1 and Formula 6.2 can be a default conversion function or a conversion function specified by metadata, which is not limited in the embodiments of the present application. In addition, the embodiments of the present application do not limit the analytical expression of the conversion function. For example, in some embodiments, f(Enhanceβ[j])=Enhanceβ[j]. For example, in other embodiments, f(Enhanceβ[j])=C×Enhanceβ[j], where parameter C can be a preset value or a value determined according to Enhanceβ[j]. For example, in other embodiments, f(Enhanceβ[j])=Enhanceβ[j]+D, where parameter D can be a preset value or a value determined according to Enhanceβ[j].

[0277] If N is a positive integer greater than or equal to 2, then HDRβ[j] can be used to represent the value of the j-th pixel in the second HDR image, and baseβ i [j] represents the value of the jth pixel in the i-th first basic layer data among the N first basic layer data, using Enhanceβ i [j] represents the value of the jth pixel in the i-th second enhancement layer data in the N second enhancement layer data, where i is a positive integer greater than or equal to 1 and less than or equal to N, that is, i=1, ..., N. Then in some embodiments, HDRβ[j], baseβ i [j] and Enhanceβ i [j] satisfies the following relationship:

[0278] HDRβ[j]=A1×baseβ1[j]×f1(Enhanceβ1[j])+A2×baseβ2[j]×f2(Enhanceβ2[j])+…+A N ×baseβ N [j]×

[0279] f N (Enhance N [j]), (Formula 6.3).

[0280] In some other embodiments, HDRβ[j], baseβ i [j] and Enhanceβ i [j] satisfies the following relationship:

[0281] HDRβ[j]=A1×baseβ1[j]+B1×f1(Enhanceβ1[j])+A2×baseβ2[j]+B2×f2(Enhanceβ2[j])+…+A N ×baseβN [j]

[0282] +B N ×f N (Enhance N [j]), (Formula 6.4).

[0283] f1(), f2(), ..., f in Formula 6.3 and Formula 6.4 N () represents a conversion function. In some embodiments, the conversion function may be a default or pre-negotiated one. In other embodiments, the conversion function may be indicated by the first metadata.

[0284] A1, A2, ..., A in Formula 6.3 and Formula 6.4 N The representation may be a constant or a numerical value indicated in the first metadata.

[0285] B1, B2, ..., B in Formula 6.4 N The representation may be a constant or a numerical value indicated in the first metadata.

[0286] Optionally, in some embodiments, determining the second HDR image according to the N second enhancement layer data and the N first basic layer data may include: processing the N second enhancement layer data to obtain N third enhancement layer data; and determining the second HDR image according to the N third enhancement layer data and the N first basic layer data. The manner of determining the second HDR image according to the N third enhancement layer data and the N first basic layer data is the same as the manner of directly determining the second HDR image according to the N second enhancement layer data and the N first basic layer data, and for the sake of brevity, it will not be repeated here.

[0287] Similarly, in some embodiments, before determining the second HDR image according to the N second enhanced layer data and the N first basic layer data, the N first basic layer data may be processed to obtain N second basic layer data, and then the second HDR image may be determined according to the N third enhanced layer data (or N second enhanced layer data) and the N second basic layer data. The manner of determining the second HDR image according to the N third enhanced layer data (or N second enhanced layer data) and the N second basic layer data is the same as the manner of directly determining the second HDR image according to the N second enhanced layer data and the N first basic layer data, and for the sake of brevity, it will not be repeated here. The determination process of the second basic layer data may refer to the determination process of the third enhanced layer data. The following mainly introduces how to determine the N third enhanced layer data according to the N second enhanced layer data.

[0288] The N second enhancement layer data correspond to the N third enhancement layer data one by one, and each third enhancement layer data in the N third enhancement layer data is determined according to the corresponding second enhancement layer data. In other words, the i-th third enhancement layer data in the N third enhancement layer data is determined according to the i-th second enhancement layer data in the N second enhancement layer data, i=1, ..., N.

[0289] The following takes a second enhancement layer as an example to introduce a method for determining the third enhancement layer data. Each third enhancement layer data can be determined by referring to the following method.

[0290] In some embodiments, the first metadata may include a first value and a second value. THH is used to represent the first value, THL is used to represent the second value, Enhanceβ[j] is used to represent the value of the j-th pixel in the second enhancement layer data, and EnhanceAfterβ[j] is used to represent the value of the j-th pixel in the third enhancement layer data. In this case, THH, THL, Enhanceβ[j] and EnhanceAfterβ[j] satisfy the following relationship:

[0291] EnhanceAfterβ[j]=Enhanceβ[j]×THH+(A-Enhanceβ[j])×THL, (Formula 7.1)

[0292] Wherein, A is the maximum value of the normalized interval corresponding to the second enhanced data. For example, if the normalized interval is [0, 1], then A is 1.

[0293] During the encoding process, the intermediate enhanced data can be remapped according to the maximum value and / or the minimum value and the corresponding preset value, and the intermediate enhanced data can be mapped to another numerical range interval (for example, the above-mentioned normalized interval). For the convenience of description, the numerical range interval of the intermediate enhanced data can be referred to as the first interval, and the numerical range interval remapped based on the preset value can be referred to as the second interval. The numerical range of the second enhanced layer data is within the second interval. Through the above technical solution, the second enhanced data can be remapped to the first interval. The above-mentioned first value and second value are the maximum value and minimum value of the intermediate enhanced data in the first interval.

[0294] In some other embodiments, a mapping function TMB() may be determined based on the first metadata. The third enhanced data may be determined based on the mapping function. Enhanceβ[j] is used to represent the second enhanced layer data, and EnhanceAfterβ[j] is used to represent the third enhanced layer data. In this case, Enhanceβ[j] and EnhanceAfterβ[j] satisfy the following relationship:

[0295] EnhanceAfterβ[j]=TMB(Enhanceβ[j]), (Formula 8.1).

[0296] In some other embodiments, Enhanceβ[j] and EnhanceAfterβ[j] satisfy the following relationship:

[0297] EnhanceAfterβ[j]=TMB()×Enhanceβ[j], (Formula 8.2).

[0298] During the encoding process, a mapping relationship can be determined according to the histogram of the intermediate enhanced data, and then the intermediate enhanced data can be mapped to another numerical range interval according to the mapping relationship. The first interval is still used to represent the numerical range interval of the intermediate enhanced data, and the second interval is used to represent the mapped numerical range interval. The numerical range of the second enhanced layer data is within the second interval. Through the above technical solution, the second enhanced data can be remapped to the first interval. TMB() is the inverse function of the mapping relationship. In some embodiments, TMB() can be directly carried in the first metadata. In other embodiments, the first metadata can carry the mapping relationship. The second electronic device can infer TMB() based on the mapping relationship.

[0299] In some other embodiments, the second enhancement layer data may be processed through some signal processing processes to obtain the third enhancement layer data. In this case, Enhanceβ[j] and EnhanceAfterβ[j] satisfy the following relationship:

[0300] EnhanceAfterβ[j]=F(Enhanceβ[j]), (Formula 9.1)

[0301] Wherein F() represents a signal processing process. The signal processing may be filtering (such as bilateral filtering, interpolation filtering, nearest neighbor filtering, etc.), denoising or other image processing methods, which are not limited in this application.

[0302] In other embodiments, the second enhancement layer data may be first remapped back to the first interval using formula 7.1, formula 7.2 or formula 8.2 to obtain third enhancement layer data, and then the third enhancement layer data may be processed using formula 9.1 to obtain fourth enhancement layer data. In this case, the second HDR image is determined based on N fourth enhancement layer data and N of the N first basic layer data. The method of determining the second HDR image based on the N fourth enhancement layer data and the N first basic layer data is the same as the method of directly determining the second HDR image based on the N second enhancement layer data and the N first basic layer data, and for the sake of brevity, it will not be repeated here.

[0303] In some embodiments, after acquiring the second HDR image, the second HDR image may be further transformed to obtain a third HDR image. For example, the second HDR image may be transformed in color space, converted in color gamut, or color compensated to obtain the third HDR image.

[0304] Fig.14 is a schematic structural block diagram of an electronic device provided according to an embodiment of the present application. Fig.14 The electronic device 1400 shown includes an acquisition unit 1401 , a processing unit 1402 and an encoding unit 1403 .

[0305] The acquiring unit 1401 is configured to acquire a first HDR image and N first basic layer data corresponding to the first HDR image.

[0306] Processing unit 1402: A user determines N first enhancement layer data according to the first HDR image, the first basic layer data and at least one encoding scheme, where N is a positive integer greater than or equal to 1.

[0307] The encoding unit 1403 is used to encode the N first enhancement layer data, the N first basic layer data and the first metadata to obtain a code stream, where the first metadata includes first indication information, and the first indication information is used to indicate the at least one encoding scheme or a decoding scheme corresponding to the at least one encoding scheme.

[0308] The specific functions and beneficial effects of the acquisition unit 1401, the processing unit 1402 and the encoding unit 1403 can be referred to the above embodiments, and for the sake of brevity, they will not be repeated here.

[0309] In some embodiments, the acquisition unit 1401 , the processing unit 1402 , and the encoding unit 1403 may be implemented by a processor.

[0310] Fig.15 is a schematic structural block diagram of another electronic device provided according to an embodiment of the present application. Fig.15 The electronic device 1500 shown includes a decoding unit 1501 and a processing unit 1502 .

[0311] The decoding unit 1501 is used to decode the acquired code stream to obtain N first enhancement layer data, N first basic layer data and first metadata, wherein the first metadata includes first indication information, and the first indication information is used to determine at least one decoding scheme, and N is a positive integer greater than or equal to 1.

[0312] The processing unit 1502 is configured to determine a second HDR image according to the at least one decoding scheme, the N first base layer data, and the N first enhancement layer data.

[0313] The specific functions and beneficial effects of the decoding unit 1501 and the processing unit 1502 can be found in the above embodiments, and for the sake of brevity, they will not be repeated here.

[0314] In some embodiments, the decoding unit 1501 and the processing unit 1502 may be implemented by a processor.

[0315] The embodiment of the present application also provides an electronic device. The electronic device can be used to implement the above-mentioned encoding method embodiment. The electronic device includes a processor and a memory. The processor is used to execute the computer program or instructions stored in the memory, or read the data / signaling stored in the memory to execute the methods in the above method embodiments. The memory can be coupled to the processor, or can also be separately provided. The memory can also be used to store the code stream determined by the processor. Optionally, the processor is one or more. Optionally, the memory is one or more. Optionally, the electronic device may also include a transceiver (or communication interface), which is used to receive and / or send signals. For example, the transceiver can be used to send the code stream determined by the processor to another electronic device.

[0316] The embodiment of the present application also provides an electronic device. The electronic device can be used to implement the above decoding method embodiment. The electronic device includes a processor and a memory. The processor is used to execute the computer program or instructions stored in the memory, or read the data / signaling stored in the memory to execute the methods in the above method embodiments. The memory can be coupled to the processor, or can also be separately set. Optionally, the processor is one or more. Optionally, the memory is one or more. Optionally, the electronic device may also include a transceiver (or communication interface), which is used to receive and / or send signals. For example, the transceiver can be used to receive a code stream.

[0317] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0318] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM). For example, RAM can be used as an external cache. By way of example and not limitation, RAM includes the following forms: static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0319] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0320] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0321] Optionally, the above-mentioned electronic device may be a chip.

[0322] The embodiment of the present application also provides a chip system. The chip system (or also referred to as a processing system) includes a logic circuit and an input / output interface.

[0323] Among them, the logic circuit can be a processing circuit in the chip system. The logic circuit can be coupled to the storage unit and call the instructions in the storage unit so that the chip system can implement the methods and functions of each embodiment of the present application. The input / output interface can be an input / output circuit in the chip system, outputting information processed by the chip system, or inputting data or signaling information to be processed into the chip system for processing.

[0324] As a solution, the chip system is used to implement the embodiments of the above encoding methods. For example, the chip system is used to implement the processing-related operations performed by the source device, the capture device or the front-end device in the above method embodiments.

[0325] As a solution, the chip system is used to implement the embodiments of the above decoding methods. For example, the chip system is used to implement the processing-related operations performed by the target device, terminal device or terminal display device in the above method embodiments.

[0326] An embodiment of the present application also provides a computer-readable storage medium on which are stored computer instructions for implementing the methods executed by an encoding end device (eg, a source device, a capture device, or a front-end device) in the above-mentioned method embodiments.

[0327] An embodiment of the present application also provides a computer-readable storage medium on which are stored computer instructions for implementing the methods executed by a decoding end device (eg, a target device, a terminal device, or a terminal display device) in the above-mentioned method embodiments.

[0328] An embodiment of the present application also provides a computer program product, comprising instructions, which, when executed by a computer, implement the methods performed by the encoding end device in the above-mentioned method embodiments.

[0329] The embodiment of the present application also provides a computer program product, which includes instructions, and when the instructions are executed by a computer, the methods performed by the decoding end device in the above-mentioned method embodiments are implemented.

[0330] An embodiment of the present application also provides a communication system, including the aforementioned encoding end device and decoding end device.

[0331] The embodiment of the present application also provides a code stream, which is determined by the above encoding method.

[0332] The embodiment of the present application also provides a device for storing a code stream, which is used to store the code stream determined by the above encoding method.

[0333] The present application also provides a computer device, which includes a memory, and the memory is used to store a code stream determined by the above encoding method.

[0334] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, which will not be repeated here.

[0335] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0336] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. For example, the computer can be a personal computer, a server, or a network device, etc. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)). For example, the aforementioned available medium includes, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0337] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A coding method, characterized in that: The method comprises: Acquire a first HDR image and N first basic layer data corresponding to the first HDR image; Determine N first enhancement layer data according to the first HDR image, the N first base layer data and at least one encoding scheme, where N is a positive integer greater than or equal to 1; The N first enhancement layer data, the N first base layer data, and first metadata are encoded to obtain a code stream, where the first metadata includes first indication information, and the first indication information is used to indicate the at least one encoding scheme or a decoding scheme corresponding to the at least one encoding scheme.

2. The method according to claim 1, characterized in that The determining, according to the first HDR image, the N first base layer data and at least one coding scheme, N first enhancement layer data comprises: Determine N intermediate enhancement data according to the first HDR image and the N first basic layer data; The N intermediate enhancement data are encoded according to the at least one encoding scheme to obtain the N first enhancement layer data.

3. The method according to claim 2, characterized in that The step of encoding the N intermediate enhancement data according to the at least one encoding scheme to obtain the N first enhancement layer data includes: Use N respectively i The i-th intermediate enhancement data in the N intermediate enhancement data is encoded by a coding scheme to obtain the i-th first enhancement layer data in the N first enhancement layer data, wherein the i-th first enhancement layer data includes N i Regions, The use of N i The encoding scheme for encoding the i-th intermediate enhancement data comprises: Use the N i The N encoding schemes for the i-th intermediate enhanced data i Regions are encoded, N i is a positive integer greater than or equal to 2, i=1,…,N.

4. The method according to claim 3, characterized in that The at least one coding scheme includes M groups of coding schemes, and a group of coding schemes corresponding to the i-th intermediate enhanced data in the M groups of coding schemes includes N i A coding scheme, M is a positive integer greater than or equal to 1.

5. The method according to claim 2, characterized in that: The at least one coding scheme comprises M coding schemes, The step of encoding the N intermediate enhancement data according to the at least one encoding scheme to obtain the N first enhancement layer data includes: The i-th intermediate enhancement data is encoded using a coding scheme corresponding to the i-th intermediate enhancement data among the N intermediate enhancement data among M coding schemes to obtain the i-th first enhancement layer data among the N first enhancement layer data, where M is a positive integer greater than or equal to 1, i=1,…,N.

6. The method according to any one of claims 2 to 5, characterized in that The determining N intermediate enhancement data according to the first HDR image and the N first basic layer data includes: Determine N reference intermediate enhanced data according to the first HDR image and the N first base layer data, wherein the reference intermediate enhanced data is multi-channel data; According to the i-th reference intermediate enhancement data among the N reference intermediate enhancement data, the i-th intermediate enhancement data among the N intermediate enhancement data is determined, the intermediate enhancement data is single-channel data, i=1, . . . , N.

7. The method according to claim 6, characterized in that The determining, according to the i-th reference intermediate enhancement data among the N reference intermediate enhancement data, the i-th intermediate enhancement data among the N reference intermediate enhancement data comprises: It is determined that the i-th intermediate enhanced data includes data of one channel of the i-th reference intermediate enhanced data.

8. The method according to claim 6, characterized in that The determining, according to the i-th reference intermediate enhancement data among the N reference intermediate enhancement data, the i-th intermediate enhancement data among the N reference intermediate enhancement data comprises: Transforming the multi-channel data of the i-th reference intermediate enhanced data respectively to obtain a plurality of transformation results, wherein the plurality of transformation results correspond to the multi-channel data one by one, and each of the plurality of transformation results is obtained after the corresponding channel is transformed; It is determined that the i-th intermediate enhanced data is a transformation result among the plurality of transformation results.

9. The method according to claim 6, characterized in that The determining, according to the i-th reference intermediate enhancement data among the N reference intermediate enhancement data, the i-th intermediate enhancement data among the N reference intermediate enhancement data comprises: The i-th intermediate enhanced data is determined according to the relationship function and the multi-channel data of the i-th reference intermediate enhanced data.

10. The method according to claim 9, characterized in that The i-th intermediate enhancement data includes values ​​of multiple pixels, each of the multiple pixels corresponds to multiple groups of pixels, the multiple groups of pixels respectively belong to multi-channel data of the i-th reference intermediate enhancement data, each group of pixels in the multiple groups of pixels includes at least one pixel, and the pixel value of each pixel in the multiple pixels is determined according to the pixel values ​​of the corresponding multiple groups of pixels and the relationship function.

11. The method according to any one of claims 1 to 10, characterized in that The at least one encoding scheme includes one or more of the following encoding schemes: PQ coding, HLG coding, gamma coding, log coding, or any curve coding.

12. The method according to any one of claims 1 to 11, characterized in that The first indication information is used to indicate: at least one encoding function; the at least one encoding function and parameters of each of the at least one encoding function; the at least one decoding function; or The at least one decoding function and parameters of each of the at least one decoding function.

13. A decoding method, characterized in that: The method comprises: Decoding the acquired bitstream to obtain N first enhancement layer data, N first base layer data, and first metadata, wherein the first metadata includes first indication information, and the first indication information is used to determine at least one decoding scheme, where N is a positive integer greater than or equal to 1; According to the at least one decoding scheme, the N first base layer data and the N first enhancement layer data, a second HDR image is determined.

14. The method according to claim 13, characterized in that The determining, according to the at least one decoding scheme, the N first base layer data and the N first enhancement layers, a second HDR image comprises: Decoding the N first enhancement layer data according to the at least one decoding scheme to obtain N second enhancement layer data; The second HDR image is determined according to the N first basic layer data and the N second enhanced layer data.

15. The method according to claim 14, characterized in that The decoding of the N first enhancement layer data according to the at least one decoding scheme to obtain N second enhancement layer data includes: Use N respectively i The i-th first enhancement data in the N first enhancement data is decoded by a decoding scheme to obtain the i-th second enhancement layer data in the N second enhancement layer data, wherein the i-th first enhancement layer data includes N i Regions, The use of N i The decoding scheme for decoding the i-th first enhanced data includes: Use the N i The decoding scheme for the Nth first enhanced data i regions to decode, N i is a positive integer greater than or equal to 2, i=1,…,N.

16. The method according to claim 15, characterized in that The at least one decoding scheme includes M sets of decoding schemes, and a set of decoding schemes corresponding to the i-th first enhancement layer data in the M sets of decoding schemes includes N i decoding schemes, and M is a positive integer greater than or equal to 1.

17. The method according to claim 14, characterized in that The at least one decoding scheme comprises M decoding schemes, According to the at least one decoding scheme, decoding the N first enhancement layer data to obtain N second enhancement layer data includes: Use M decoding schemes and the decoding scheme corresponding to the i-th first enhancement layer data among the N first enhancement layer data to decode the i-th first enhancement layer data to obtain the i-th second enhancement layer data among the N second enhancement layer data, where M is a positive integer greater than or equal to 1, i=1,…,N.

18. The method according to any one of claims 13 to 17, characterized in that The at least one decoding scheme includes a decoding scheme corresponding to one or more encoding schemes of the following encoding schemes: PQ coding, HLG coding, gamma coding, log coding, or any curve coding.

19. The method according to any one of claims 13 to 18, characterized in that The first indication information is used to indicate: at least one encoding function; the at least one encoding function and parameters of each of the at least one encoding function; the at least one decoding function; or The at least one decoding function and parameters of each of the at least one decoding function.

20. An electronic device, characterized in that: The electronic device comprises a unit for implementing the method according to any one of claims 1 to 12, or comprises a unit for implementing the method according to any one of claims 13 to 19.

21. A computer device, characterized in that: include: A processor, the processor being used to be coupled to a memory, read and execute instructions and / or program codes in the memory to execute the method as claimed in any one of claims 1 to 12, or to execute the method as claimed in any one of claims 13 to 19.

22. A chip system, characterized in that: include: A logic circuit, the logic circuit is used to couple with an input / output interface and transmit data through the input / output interface to execute the method as claimed in any one of claims 1 to 12, or to execute the method as claimed in any one of claims 13 to 19.

23. A computer readable medium, characterized in that The computer-readable medium stores a program code, and when the computer program code is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 12, or the computer is caused to execute the method according to any one of claims 13 to 19.

24. A code stream, characterized in that The code stream is determined according to the method according to any one of claims 1 to 12.

Images